Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Дата публикации: 17-08-2026 20:26:00

Statistically consistent algorithms have been widely employed for dealing with noisy labels. Their objective functions are designed so that minimizing the expected risk on noisy data leads to the same minimizer as minimizing the expected risk on clean data. From the weak law of large numbers, penalizing the variance of losses would reduce the discrepancy between the average loss and the expected risk on the clean data when there is a finite training sample, and the estimation error in the model's parameters can be reduced. Interestingly, we found that the variance of losses needs to be encouraged for label-noise learning. Specifically, encouraging a large variance of losses would boost the memorization effect and reduce the harmfulness of incorrect labels. Regularizers can be easily designed to encourage a large variance of losses and be plugged into many existing algorithms. Empirically, the proposed method by encouraging a large variance of losses could improve the generalization ability of baselines on both synthetic and real-world datasets.

Схожие новости

#	Наименование новости	Тональность	Информативность	Дата публикации
1	High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks	0	8.7	17-08-2026
2	Investigating the Histogram Loss in Regression	0	5.53	17-08-2026
3	Corruptions of Supervised Learning Problems: Typology and Mitigations	0	5.48	17-08-2026
4	Statistical Learning Theory for Neural Operators	0	10.21	17-08-2026
5	The Sample Complexity of Parameter-Free Stochastic Convex Optimization	0	5.7	17-08-2026
6	Beyond Unconstrained Features: Neural Collapse for Shallow Neural Networks with General Data	0	4.36	17-08-2026
7	Graph-based Clustering Revisited: A Relaxation of Kernel k-Means Perspective	0	10.94	17-08-2026
8	Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss	0	10	17-08-2026
9	Near-optimal Delta-convex Estimation of Lipschitz Functions	0	9.71	17-08-2026

Классификация: . Схожих патентов: 0. Схожих новостей: 9. Тональность: 0. Информативность: 4.9. Источник: jmlr.org.