Geometry-dependent matching pursuit: a transition phase for convergence on linear regression and LASSO - ENS - École normale supérieure
Pré-Publication, Document De Travail Année : 2024

Geometry-dependent matching pursuit: a transition phase for convergence on linear regression and LASSO

Résumé

Greedy first-order methods, such as coordinate descent with Gauss-Southwell rule or matching pursuit, have become popular in optimization due to their natural tendency to propose sparse solutions and their refined convergence guarantees. In this work, we propose a principled approach to generating (regularized) matching pursuit algorithms adapted to the geometry of the problem at hand, as well as their convergence guarantees. Building on these results, we derive approximate convergence guarantees and describe a transition phenomenon in the convergence of (regularized) matching pursuit from underparametrized to overparametrized models.
Fichier principal
Vignette du fichier
article.pdf (5.11 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04491840 , version 1 (06-03-2024)

Identifiants

Citer

Céline Moucer, Adrien B Taylor, Francis Bach. Geometry-dependent matching pursuit: a transition phase for convergence on linear regression and LASSO. 2024. ⟨hal-04491840⟩
65 Consultations
28 Téléchargements

Altmetric

Partager

More