Hyperbolic Optimizer as a Dynamical System
No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
ML Research Press
Abstract
During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.