The Distribution of Ridgeless Least Squares Interpolators

Дата публикации: 17-08-2026 20:26:00

The Ridgeless minimum $\ell_2$-norm interpolator in overparametrized linear regression has attracted considerable attention in recent years in both machine learning and statistics communities. While it seems to defy conventional wisdom that overfitting leads to poor prediction, recent theoretical research on its $\ell_2$-type risks reveals that its norm minimizing property induces an `implicit regularization' that helps prediction in spite of interpolation.
This paper takes a further step that aims at understanding its precise stochastic behavior as a statistical estimator. Specifically, we characterize the distribution of the Ridgeless interpolator in high dimensions, in terms of a Ridge estimator in an associated Gaussian sequence model with positive regularization, which provides a precise quantification of the prescribed implicit regularization in the most general distributional sense. Our distributional characterizations hold for general non-Gaussian random designs and extend uniformly to positively regularized Ridge estimators.
As a direct application, we obtain a complete characterization for a general class of weighted $\ell_q$ risks of the Ridge(less) estimators that are previously only known for $q=2$ by random matrix methods. These weighted $\ell_q$ risks not only include the standard prediction and estimation errors, but also include the non-standard covariate shift settings. Our uniform characterizations further reveal a surprising feature of the commonly used generalized and $k$-fold cross-validation schemes: tuning the estimated $\ell_2$ prediction risk by these methods alone lead to simultaneous optimal $\ell_2$ in-sample, prediction and estimation risks, as well as the optimal length of debiased confidence intervals.

Схожие новости

#	Наименование новости	Тональность	Информативность	Дата публикации
1	Deconvolution in unlinked linear models	0	4.62	17-08-2026
2	Near-optimal Delta-convex Estimation of Lipschitz Functions	0	9.71	17-08-2026
3	Generalized Resubstitution for Regression Error Estimation	0	4.2	17-08-2026
4	High-dimensional Parameter Transfer With Fused-Regularizer	0	9.38	17-08-2026
5	Inference with non-differentiable surrogate loss in a general high-dimensional classification framework	0	8.38	17-08-2026
6	Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent	0	6.66	17-08-2026
7	Flexible Functional Treatment Effect Estimation	0	5.7	17-08-2026
8	Semi-supervised learning for linear extremile regression	0	6.83	17-08-2026
9	Persistence Diagrams Estimation of Multivariate Piecewise H{\"o}lder-continuous Signals	0	6.3	17-08-2026
10	High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks	0	8.7	17-08-2026

Классификация: . Схожих патентов: 0. Схожих новостей: 10. Тональность: 0. Информативность: 5.78. Источник: jmlr.org.