Recent developments in the practice of numerical programming require optimization problems not only to be solved numerically, but also to be differentiated. This allows to integrate the computational operation of evaluating a solution in larger models, which are themselves trained or optimized using gradient methods. Most well known applications include bilevel optimization and implicit input-output relations in deep network models. Fixed point of contraction mappings provide a natural high level description of these applications. We will briefly describe the interplay between implicit differentiation and derivatives of fixed point iterations in the smooth settings, dating back to early works in automatic differentiation. Motivated by the possibility to differentiate solutions of nonsmooth or constrained problems, we will describe recent extension of these results to the conservative gradient setting, a notion of generalized derivative which is compatible with calculus and gradient type optimization methods. We will also cover explicit examples related to the differentiation of solutions to monotone inclusions. Joint work with Jérôme Bolte, Tony Silvetti-Falls, Tam Le, Samuel Vaiter.

Zoom link