Example 23 - OpenMP Parallel Codes

Goal: Learn how to differentiate OpenMP parallel codes with CoDiPack and OpDiLib.

Prerequisite: Tutorial 2 - Reverse mode AD

Full code:

#include <codi.hpp>
#include <opdi.hpp>
#include <iostream>
 
#include <opdi/backend/macro/macroBackend.hpp>
#include <opdi.hpp>
 
using Real = codi::RealReverseIndexOpenMP;  // use a CoDiPack type suitable for OpenMP parallel applications
using Tape = typename Real::Tape;
 
int main(int nargs, char** args) {
 
  // initialize OpDiLib
 
  opdi::backend = new opdi::MacroBackend();
  opdi::backend->init();
  opdi::logic = new opdi::OmpLogic;
  opdi::logic->init();
  opdi::tool = new CoDiOpDiLibTool<Real>;
 
  // usual AD workflow in the serial parts of the code
 
  Real x = 4.0;
 
  Tape& tape = Real::getTape();
  tape.setActive();
  tape.registerInput(x);
 
  // parallel computation
 
  Real a[1000];
  Real y = 0.0;
 
  OPDI_PARALLEL()  // this example uses OpDiLib's macro backend, where OpenMP pragmas are replaced by such macros
  {
    OPDI_FOR()
    for (int i = 0; i < 1000; ++i)
    {
      a[i] = sin(x * i);
    }
    OPDI_END_FOR
  }
  OPDI_END_PARALLEL
 
  for (int i = 0; i < 1000; ++i) {
    y += a[i];
  }
 
  // usual AD workflow
 
  tape.registerOutput(y);
  tape.setPassive();
  y.setGradient(1.0);
 
  opdi::logic->prepareEvaluate();  // prepare OpDiLib for evaluation
 
  tape.evaluate();
 
  std::cout << "f(" << x << ") = " << y << std::endl;
  std::cout << "df/dx(" << x << ") = " << x.getGradient() << std::endl;
 
  // finalize OpDiLib
 
  opdi::backend->finalize();
  delete opdi::backend;
  delete opdi::logic;
  delete opdi::tool;
 
  return 0;
}
 
// don't forget to include the OpDiLib source file
#include <opdi.cpp>
 

OpenMP parallel codes are differentiated with the help of the OpDiLib AD tool add-on (https://scicomp.rptu.de/software/opdi). Please refer to the OpDiLib documentation for further examples and modes of operation.

The usual AD workflow is the same as for a serial code, except for the initialization and finalization of OpDiLib. In the example at hand, OpDiLib's macro backend is used, where pragmas such as #pragma omp parallel are replaced by macros such as OPDI_PARALLEL() together with end macros such as OPDI_END_PARALLEL. If you have a compiler with OMPT support, you can use OpDiLib's OMPT backend that supports the usage of unmodified OpenMP pragmas (see https://github.com/SciCompKL/OpDiLib for examples).

OpDiLib can only be used with CoDiPack types that are thread-safe for applications in OpenMP parallel codes. Right now, the dedicated type that supports this is codi::RealReverseIndexOpenMP.