Date and Place: Thursdays and hybrid (live in 32-349/online via Zoom). For detailed dates see below!
Content
In the Scientific Computing Seminar we host talks of guests and members of the SciComp team as well as students of mathematics, computer science and engineering. Everybody interested in the topics is welcome.
List of Talks
Event Information:
-
Thu08Feb2024
SC Seminar: Timo Sonnenschein
10:30Hybrid (Room 32-349 and via Zoom)
Timo Sonnenschein, Department of Computer Science, University of Kaiserslautern-Landau (RPTU)
Title: Introduction to Implicit Layers in the context of Deep Equilibrium Models
Abstract:
Training modern neural networks can be challenging with the limited memory of specialized hardware. Reusing parameters and storing only a few intermediate results for backward propagation can reduce memory consumption significantly.
Implicit layers offer a solution by iteratively applying the same function until convergence and calculating the gradients at the converged point without any intermediate results using implicit differentiation. Further, implicit layers allow for various sets of requirements. Due to the separation of problem definition and actual computation, they can utilize several solving methods and adapt the accuracy to the problem. One major shortcoming is the runtime, not only for training but also for inference. Further investigation might solve this problem.
One architectures using implicit layers are the Deep Equilibrium Models. They have comparable accuracy to state-of-the-art models in a variety of fields, including language processing and visual tasks.
In conclusion, this talk explores implicit layers, emphasizing their efficiency, adaptability, and potential contributions to neural network architectures. The discussions on DEQ models aim to unravel their applications in the field.
How to join online
You can join online via Zoom, using the following link:
https://uni-kl-de.zoom.us/j/63123116305?pwd=Yko3WU9ZblpGR3lGUkVTV1kzMCtUUT09