Stationarity Floors and Vanishing Perturbations in Sharpness-Aware Minimization

Abstract

We study a deterministic family of sharpness-aware minimization methods for smooth nonconvex functions. The perturbation is $$ y_k=x_k+\rho\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^\alpha}, \qquad 0\leq\alpha\leq 1, $$ so that its effective radius is $\rho\norm{\nabla f(x_k)}^{1-\alpha}$. For $0<\alpha\leq1$, we give an explicit complexity bound above the stationarity level $(L\rho)^{1/\alpha}$. A one-dimensional quadratic example reaches this level exactly, showing that the bound describes a real limitation of the constant-parameter rule. The unnormalized case $\alpha=0$ is treated separately and requires $L\rho<1$. We then introduce a clipped rule which agrees with the constant-$\rho$ rule away from stationary points and becomes proportional to the gradient near them. The clipped method has $\norm{\nabla f(x_k)}\to0$ and the usual $O(T^{-1/2})$ stationarity bound. Numerical tests on a quadratic function, the Rosenbrock function, and a five-dimensional nonconvex function illustrate the stationarity floor of the unclipped rule and the effect of clipping.