OpDiLib – Open Multiprocessing Differentiation Library
OpDiLib is a universal add-on for reverse mode operator overloading AD tools that enables the differentiation of OpenMP parallel code. OpDiLib is written in C++ and relies on modern OpenMP features around OMPT to achieve differentiation without any additional modifications of the source code.
Features
- interface for the coupling with operator overloading AD tools
- fully automatic differentiation via OMPT
- semi-automatic differentiation via a macro interface without OMPT
- automatic deduction of a parallel reverse pass
- no a priori restrictions of the data access patterns
- tools for the performance optimization of the parallel reverse pass
- AD handling for most features of the OpenMP 2.5 specification
atomic directives are not handledflush directives are not handled
Source Code
OpDiLib is released as open source software under the LGPLv3 license.
The source code is available on GitHub.
Contact: opdi@scicomp.uni-kl.de
Newsletter: opdi-info@scicomp.uni-kl.de
Lead Developer: Johannes Blühdorn
Responsibility: Prof. Dr. Nicolas R. Gauger
Publication
If you use OpDiLib in one of your applications and write a paper, please cite us!
2023
J. Blühdorn, M. Sagebaum, N. R. Gauger
Event-Based Automatic Differentiation of OpenMP with OpDiLib Journal Article
In: ACM Transactions on Mathematical Software, vol. 49, iss. 1, no. 3, pp. 1–31, 2023.
@article{nokey,
title = {Event-Based Automatic Differentiation of OpenMP with OpDiLib},
author = {J. Blühdorn, M. Sagebaum, N. R. Gauger},
doi = {10.1145/3570159},
year = {2023},
date = {2023-03-21},
urldate = {2023-03-21},
journal = {ACM Transactions on Mathematical Software},
volume = {49},
number = {3},
issue = {1},
pages = {1--31},
abstract = {We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.
