We study the global convergence of discrete algorithms for optimizing non-convex empirical loss functions. Many of these algorithms correspond to the Euler discretization of a diffusion process. One limitation by previous work is that the stationary distribution of the discretized process has been only approximated by the Gibbs’ distribution. In this work we improve the approximation by using weak backward error analysis to construct higher order approximate measures, and prove this type of approximate measures are uniformly stable.