Tutorial 1 - Forward mode AD

Goal: Introduction to forward mode AD with CoDiPack.

Prerequisite: AD forward mode. See Forward AD Equation

Function: Simple real valued function

Real func(const Real& x) {
  return x * x * x;
}

Full code:

 
#include <codi.hpp>
#include <iostream>
 
using Real = codi::RealForward;
 
Real func(const Real& x) {
  return x * x * x;
}
 
int main(int nargs, char** args) {
  Real x = 4.0;
 
  x.setGradient(1.0);      // Step 1: Set tangent seeding
  Real y = func(x);        // Step 2: Evaluate function
 
  // Step 3: Access gradients
  std::cout << "f(4.0) = " << y << std::endl;
  std::cout << "df/dx(4.0) = " << y.getGradient() << std::endl;
 
  return 0;
}

The derivative computation with the forward mode of CoDiPack is quite simple and needs only three steps:

• Set the direction of the derivative on the input variables.
• Evaluate the function.
• Get the direction of the derivative from the output variables.

The simplicity comes from the implementation. The tangent data is stored in the active types and no tape is recorded. All tangent evaluations are directly done in the statement evaluation. This is what ADOL-C calls the ''tapeless'' mode.

If you need to record a tape and evaluate it in a forward manner, please have a look at example Example 6 - Forward mode tape evaluation.

Step 1: Set the tangent seeding.

In the forward AD equation, the variable describes the vector of input variables. On these values the tangent direction needs to be set. For a single variable this can be done with the functions gradient and setGradient.

Step 2: Tangent function evaluation

In this step, CoDiPack is only indirectly involved. The function needs to be evaluated in the program and CoDiPack needs to evaluate the forward AD mode equation for each statement that are called during the evaluation. It is therefore necessary to write the function such that it uses the CoDiPack type. How this is done depends on the program that is differentiated. The best option is to write the function as a template function such that the calculation type is flexible. The second option is most of the time used when software with a large code base is differentiated. Here, a global typedef like using Real = codi::RealForward is used and all doubles in the program are changed to this typedef. The calculation type can then be changed during compile time and different executables can be generated.

Step 3: Get the directional derivative

In the forward AD equation, the variable describes the vector of output variables. During the function evaluation, CoDiPack computed the directional derivative for these variables which is the vector . For a single variable, the tangent information can be extracted with the functions gradient and getGradient.

Notes on multiple tangent computations

The forward mode is very simple to use and multiple tangents evaluations do not require any additional effort. The only think to keep in mind is that tangent values are only reset by CoDiPack if the value is overwritten. Tangent seedings, that are set on the input values, are not reset and need to be reset by the user.

Left over tangent seedings can also happen if the computational path changes and values from an old evaluation are used. See Example 1 - Old tangent leftovers in forward mode AD for an example.