CoDiPack  2.2.0
A Code Differentiation Package
SciComp TU Kaiserslautern
Loading...
Searching...
No Matches
codi::DataInterface< T_NestedData, T_InternalPosHandle > Struct Template Reference

Data stream interface for tape data. Encapsulates data that is written e.g. for each statement or argument. More...

#include <dataInterface.hpp>

Inheritance diagram for codi::DataInterface< T_NestedData, T_InternalPosHandle >:

Public Types

using InternalPosHandle = T_InternalPosHandle
 See DataInterface.
 
using NestedData = T_NestedData
 See DataInterface.
 
using Position = EmptyPosition
 Contains position data for this DataInterface and all nested interfaces.
 

Public Member Functions

Adding items
void addDataSize (size_t const &size)
 Add this many items to the data stream, after the data has been manipulated via pointers obtained from getDataPointers().
 
template<typename... Data>
void getDataPointers (Data *&... pointers)
 Get pointers to the data from the storage implementation. The method can only be called after a call to reserveItems() and data can only be accessed from 0 to the number given by reserveItems (excluding). Afterwards, addDataSize() needs to be called with the actual number of elements that have been written.
 
template<typename... Data>
void pushData (Data const &... data)
 Add data to the storage allocated by the implementation. The method can only be called after a call to reserveItems and only as often as the number of reserved items.
 
InternalPosHandle reserveItems (size_t const &items)
 Reserve this many items on the data stream. See pushData for details.
 
Size management
void resize (size_t const &totalSize)
 
void reset ()
 
void resetHard ()
 
void resetTo (Position const &pos)
 
void erase (Position const &start, Position const &end, bool recursive=true)
 
Position functions
size_t getDataSize () const
 
Position getPosition () const
 
size_t getPushedDataCount (InternalPosHandle const &startPos)
 
Position getZeroPosition () const
 
Misc functions
void addToTapeValues (TapeValues &values) const
 Add amount of stored data to the TapeValues object. Not called on the nested vector.
 
template<typename TargetPosition >
TargetPosition extractPosition (Position const &pos) const
 Extract the position of a nested DataInterface from the global position object provide by this interface.
 
void setNested (NestedData *v)
 
void swap (DataInterface &other)
 
Iterator functions
template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void evaluateForward (Position const &start, Position const &end, FunctionObject function, Args &&... args)
 Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.
 
template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void evaluateReverse (Position const &start, Position const &end, FunctionObject function, Args &&... args)
 Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.
 
template<typename FunctionObject , typename... Args>
void forEachChunk (FunctionObject &function, bool recursive, Args &&... args)
 Calls the function object for each continuous segment of data.
 
template<typename FunctionObject , typename... Args>
void forEachForward (Position const &start, Position const &end, FunctionObject function, Args &&... args)
 Calls the function object for each item in the data stream. This call is not recursive.
 
template<typename FunctionObject , typename... Args>
void forEachReverse (Position const &start, Position const &end, FunctionObject function, Args &&... args)
 Calls the function object for each item in the data stream. This call is not recursive.
 

Detailed Description

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
struct codi::DataInterface< T_NestedData, T_InternalPosHandle >

Data stream interface for tape data. Encapsulates data that is written e.g. for each statement or argument.

This interface defines the basic abstraction mechanism of how data is stored in an AD tape. During the recording of an AD tape, different types of data with a varying amount of items need to be stored and associated with each other. There is e.g. data for each statement and data for each argument. Each DataInterface covers one type of data. The management of multiple data streams can become quite cumbersome, therefore the DataInterface is designed in a recursive fashion. Each data stream can be nested with another data stream such that they can exchange position information and synchronize themselves.

Note that the recursive implementation is only that - an implementation. The data itself is not recursive. Think about is as multiple streams of associated data that grow alongside each other in varying speeds.

current state new batch of associated data
stream 1 ========== * <------ |====
stream 2 =============== * <- |=====
stream 3 === * <------------- |===
stream 4 = * <--------------- |=

The * mark the current joint position of the streams (end of the last pushed batch of data). A new batch of associated data is now appended to the streams.

A data item on the data stream can consist of multiple entries, e.g., a data entry can be an int or it can be an int and a double. The underlying implementation defines how this data is stored, e.g. as an array of objects or as an object of arrays. For counting the number of items, each call to pushData() counts as one item regardless of how many entries each item has.

The getPosition() function produces a position for this DataInterface and all nested DataInterfaces. All methods that have the Position type as an argument or modify the position of the DataInterface work recursively on all nested DataInterfaces.

How the data is stored and allocated is determined by the actual implementation of this interface.

See Example tape implementation with CoDiPack for a usage of this interface.

Example usage:

ArgDataInterface argVector = ...;
StmtDataInterface<ArgDataInterface> stmtVector = ...; // argVector is nested into the stmtVector.
stmtVector.setNested(&argVector); // Set the pointer to the nested vector.
// Record some data.
// 1. Request space, from child to parent.
argVector.reserveItems(2);
stmtVector.reserveItems(1);
// 2. Add the data, any order.
argVector.pushData(1.0);
stmtVector.pushData(100);
argVector.pushData(2.0);
// Perform some operations on the data.
// How data is provided to func depends on the interface implementations, let's assume start, end, and a data
// pointer.
auto func = (startStmt, endStmt, stmt, startArg, endArg, arg) {
for (int i = startStmt; i < endStmt; i += 1) { std::cout << stmt[i] << ", ";}
for (int i = startArg; i < endArg; i += 1) { std::cout << arg[i] << ", ";}
};
stmtVector.evaluateForward(stmtVector.getZeroPosition(), stmtVector.getPosition(), func);
// Clear all data vectors.
stmtVector.reset();

For an example on how to use getDataPointers please see the method documentation.

The interface defines methods for adding data to the data stream, getting positional information, resetting the data and iterating over the data.

Adding data:

  • reserveItems(): Needs to be called before a pushData or getDataPointers() call to ensure that enough space is left.
  • pushData(): Add the actual data to the data stream.
  • getDataPointers(): Get pointers to the internally allocated data.
  • addDataSize(): After using pointers obtained by getDataPointers and adding data via these pointers, inform about the number of data items added.

Positional information:

Reset:

  • reset(): Clear data, but do not delete allocated memory.
  • resetHard(): Clear data and free allocated memory.
  • resetTo(): Clear all data up to the specified position.
  • erase(): Erase a specific range of data. Whether memory is freed is implementation dependent.

Iterating:

Template Parameters
T_NestedDataNested data vector needs to implement this DataInterface.
T_InternalPosHandlePosition handle of the the implementing class for internal size computations.

Member Function Documentation

◆ addDataSize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::addDataSize ( size_t const & size)
inline

Add this many items to the data stream, after the data has been manipulated via pointers obtained from getDataPointers().

See getDataPointers for details.

Parameters
[in]sizeNumber of data items that have been written.

◆ addToTapeValues()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::addToTapeValues ( TapeValues & values) const

Add amount of stored data to the TapeValues object. Not called on the nested vector.

Parameters
[in,out]valuesWill only create new data entries and no new section.

◆ erase()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::erase ( Position const & start,
Position const & end,
bool recursive = true )

Erase the given range of data. Implementations may choose to free allocated memory. The parameter recursive controls whether erase is also called on nested interfaces.

◆ evaluateForward()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::evaluateForward ( Position const & start,
Position const & end,
FunctionObject function,
Args &&... args )
inline

Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

func is called for each region of continuous data that is valid for all nested DataInterfaces. func is then called as

function(args...,
start, end, dataEntry1*, dataEntry2*,
startNested, endNested, dataEntry1Nested*, dataEntry2Nested*,
startNestedNested, endNestedNested, dataEntry1NestedNested*,
...);

What kind of data is appended by each DataInterface is implementation dependent. The default is the set (start, end, dataEntry1, dataEntry2, etc.) which is appended.

It has to hold start <= end.

Positions count up in this call.

Parameters
[in]startStarting position.
[in]endEnding position.
[in]functionFunction object called.
[in,out]argsAdditional arguments for the function object.
Template Parameters
selectedDepthSelected depth for the recursion into nested data interfaces. 0 means that only the data from this interface is used. A value of 1 means that the data from the first nested interface is used, etc.. A value of -1 can be used to select the innermost interface, regardless of the number of nested interfaces.
FunctionObjectFunction object which is called.
ArgsArguments for the function object.

◆ evaluateReverse()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::evaluateReverse ( Position const & start,
Position const & end,
FunctionObject function,
Args &&... args )
inline

Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

Same as evaluateForward. It has to hold start >= end. Positions count down in this call.

Parameters
[in]startStarting position.
[in]endEnding position.
[in]functionFunction object called.
[in,out]argsAdditional arguments for the function object.
Template Parameters
selectedDepthSelected depth for the recursion into nested data interfaces. 0 means that only the data from this interface is used. A value of 1 means that the data from the first nested interface is used, etc.. A value of -1 can be used to select the innermost interface, regardless of the number of nested interfaces.
FunctionObjectFunction object which is called.
ArgsArguments for the function object.

◆ extractPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename TargetPosition >
TargetPosition codi::DataInterface< T_NestedData, T_InternalPosHandle >::extractPosition ( Position const & pos) const
inline

Extract the position of a nested DataInterface from the global position object provide by this interface.

Parameters
[in]posPosition of the DataInterface.
Template Parameters
TargetPositionPosition definition of a nested DataInterface.

◆ forEachChunk()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename FunctionObject , typename... Args>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachChunk ( FunctionObject & function,
bool recursive,
Args &&... args )
inline

Calls the function object for each continuous segment of data.

The call is

function(chunk, args...);

Chunk is of the type ChunkBase.

Parameters
[in]functionFunction object called.
[in]recursiveTrue if same call should be performed for all nested DataInterfaces.
[in,out]argsAdditional arguments for the function object.
Template Parameters
FunctionObjectFunction object which is called.
ArgsArguments for the function object.

◆ forEachForward()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename FunctionObject , typename... Args>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachForward ( Position const & start,
Position const & end,
FunctionObject function,
Args &&... args )
inline

Calls the function object for each item in the data stream. This call is not recursive.

The call to function is

function(args..., entry1*, entry2*, ...);

It has to hold start <= end.

Parameters
[in]startStarting position.
[in]endEnding position
[in]functionFunction object called.
[in,out]argsAdditional arguments for the function object.
Template Parameters
FunctionObjectFunction object which is called.
ArgsArguments for the function object.

◆ forEachReverse()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename FunctionObject , typename... Args>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachReverse ( Position const & start,
Position const & end,
FunctionObject function,
Args &&... args )
inline

Calls the function object for each item in the data stream. This call is not recursive.

See forEachForward.

It has to hold start >= end.

Parameters
[in]startStarting position.
[in]endEnding position
[in]functionFunction object called.
[in,out]argsAdditional arguments for the function object.
Template Parameters
FunctionObjectFunction object which is called.
ArgsArguments for the function object.

◆ getDataPointers()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename... Data>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::getDataPointers ( Data *&... pointers)
inline

Get pointers to the data from the storage implementation. The method can only be called after a call to reserveItems() and data can only be accessed from 0 to the number given by reserveItems (excluding). Afterwards, addDataSize() needs to be called with the actual number of elements that have been written.

The call to reserveItems only represents the maximum number of data items that can be accessed safely. It is fine if less data items are accessed.

After all elements have been written to the arrays, addDataSize needs to be called with the final written number of entries.

Example usage:

// 1. Request space.
argVector.reserveItems(10);
// 2. Add the data.
int* dataInt;
double* dataDouble;
argVector.getDataPointers(dataInt, dataDouble);
for(int i = 0; i < 5; i += 1) {
dataInt[i] = i + 1;
dataDouble[i] = i + 11;
}
// 3. Add the number of items.
argVector.addDataSize(5);
Data stream interface for tape data. Encapsulates data that is written e.g. for each statement or arg...
Definition dataInterface.hpp:149
void addDataSize(size_t const &size)
Add this many items to the data stream, after the data has been manipulated via pointers obtained fro...
void getDataPointers(Data *&... pointers)
Get pointers to the data from the storage implementation. The method can only be called after a call ...
InternalPosHandle reserveItems(size_t const &items)
Reserve this many items on the data stream. See pushData for details.
Parameters
[in]pointersThe pointers that are populated with the data from the internal representation.
Template Parameters
DataTypes of the pointers.

◆ getDataSize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
size_t codi::DataInterface< T_NestedData, T_InternalPosHandle >::getDataSize ( ) const
inline
Returns
Total number of data items stored.

◆ getPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
Position codi::DataInterface< T_NestedData, T_InternalPosHandle >::getPosition ( ) const
inline
Returns
The current global position of this DataInterface and all nested interfaces.

◆ getPushedDataCount()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
size_t codi::DataInterface< T_NestedData, T_InternalPosHandle >::getPushedDataCount ( InternalPosHandle const & startPos)
inline

Compute the number of data items stored after a call to reserveItems.

◆ getZeroPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
Position codi::DataInterface< T_NestedData, T_InternalPosHandle >::getZeroPosition ( ) const
inline
Returns
The start position of the DataInterface and all nested interfaces.

◆ pushData()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
template<typename... Data>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::pushData ( Data const &... data)
inline

Add data to the storage allocated by the implementation. The method can only be called after a call to reserveItems and only as often as the number of reserved items.

pushData() can be called less often than indicated with reserveItems(). The call to reserveItems only represents the maximum number of data items that can be pushed safely.

After a new call to reserveItems(), only this many number of data items can be pushed, leftovers will not accumulate.

For an example of how to use pushData, please see the DataInterface documentation.

Parameters
[in]dataThe number of arguments has to match the number of data stores of the implementation.
Template Parameters
DataTypes of the pushed data.

◆ reserveItems()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
InternalPosHandle codi::DataInterface< T_NestedData, T_InternalPosHandle >::reserveItems ( size_t const & items)
inline

Reserve this many items on the data stream. See pushData for details.

Parameters
[in]itemsNumber of data items to reserve.
Returns
Can be used in getPushedDataCount(). Only the newest handle is valid.

◆ reset()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::reset ( )

Reset to the zero position. Data is not deallocated. Also called on nested interfaces.

◆ resetHard()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resetHard ( )

Reset to the zero position. Data is deallocated and the default size is allocated again. Also called on nested interfaces.

◆ resetTo()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resetTo ( Position const & pos)

Reset to the given position. Data is not deallocated. Also called on the nested interfaces.

◆ resize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resize ( size_t const & totalSize)

Allocate the requested number of data items.

◆ setNested()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::setNested ( NestedData * v)

Set the pointer to the nested vector. Needs to be done before any other action and only once.

◆ swap()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
void codi::DataInterface< T_NestedData, T_InternalPosHandle >::swap ( DataInterface< T_NestedData, T_InternalPosHandle > & other)

Swap with other DataInterface of the same type.


The documentation for this struct was generated from the following file: