Cs.LG

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in …
Cite arXiv
Inference of Online Newton Methods with Nesterov's Accelerated Sketching
Reliable decision-making with streaming data requires principled uncertainty quantification of online methods. While first-order …
Cite arXiv
A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method
In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving …
Cite arXiv
High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We …
Cite DOI arXiv
Online Covariance Matrix Estimation in Sketched Newton Methods
Given the ubiquity of streaming data, online algorithms have been widely used for parameter estimation, with second-order methods …
Cite DOI arXiv
Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling
Constrained stochastic nonlinear optimization problems have attracted significant attention for their ability to model complex …
Cite arXiv
Online Covariance Estimation in Nonsmooth Stochastic Approximation
We consider applying stochastic approximation (SA) methods to solve nonsmooth variational inclusion problems. Existing studies have …
Cite arXiv
Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programming
We consider online statistical inference of constrained stochastic nonlinear optimization problems. We apply the Stochastic Sequential …
Cite URL arXiv
Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
In this work, we consider solving optimization problems with a stochastic objective and deterministic equality constraints. We propose …
Cite arXiv
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching
We consider solving equality-constrained nonlinear, nonconvex optimization problems. This class of problems appears widely in a variety …
Cite arXiv