Autor:

Chen, Hao ; Mo, Zhanfeng ; Yang, Zhouwang

Współtwórca:

Korbicz, Józef (1951- ) - red. ; Uciński, Dariusz - red.

Tytuł:

The generalization error bound for a stochastic gradient descent family via a Gaussian approximation method

Tytuł publikacji grupowej:

AMCS, volume 35 (2025)

Temat i słowa kluczowe:

stochastic gradient descent ; Gaussian approximation ; KL-divergence ; generalization bounds

Abstract:

Recent works have developed model complexity based and algorithm based generalization error bounds to explain how stochastic gradient descent (SGD) methods help over-parameterized models generalize better. However, previous works are limited by their scope of analysis and fail to provide comprehensive explanations. In this paper, we propose a novel Gaussian approximation framework to establish generalization error bounds for the U-SGD family, which is a class of SGD with asymptotically unbiased and uniformly bounded gradient noise. We study U-SGD dynamics, and we show both theoretically and numerically that the limiting model parameter distribution tends to be Gaussian, even when the original gradient noise is non-Gaussian.

Wydawca:

Zielona Góra: Uniwersytet Zielonogórski

Data wydania:

2025

Typ zasobu:

artykuł

DOI:

10.61822/amcs-2025-0018

Strony:

251-266

Źródło:

AMCS, volume 35, number 2 (2025) ; kliknij tutaj, żeby przejść

Jezyk:

eng

Licencja CC BY 4.0:

kliknij tutaj, żeby przejść

Prawa do dysponowania publikacją:

Biblioteka Uniwersytetu Zielonogórskiego