Unified Joint Optimization with Nyström Approximation for Scalable Adaptive SVM
ORIGINAL / Revisited Support Vector Machines: A Unified Joint Optimization Framework with Nystr\"om Approximation for Scalable Adaptive Learning
This work couples kernel selection and SVM training as a continuous bi-level joint optimization problem, using Nyström low-rank approximation and block coordinate ascent to reduce training complexity from cubic to O(Nm) while aiming to preserve accuracy, addressing classical SVM bottlenecks in the big-data regime.
01 ABSTRACT
Classical SVMs face two main bottlenecks: cubic training complexity and rigid static kernels, with traditional methods decoupling kernel learning from decision-boundary optimization. The paper proposes a "Revisited SVM" unified framework that recasts learning as a continuous bi-level joint optimization problem and derives an objective J(α, θ) for simultaneously optimizing Lagrange multipliers and kernel hyperparameters. A hybrid block coordinate ascent algorithm integrated with Nyström low-rank approximation is used, and experiments across four heterogeneous domains report accuracy matching or exceeding exhaustive grid search with up to 100x training speedups.
02 KEY FINDINGS
- Kernel hyperparameter learning and SVM dual variable optimization are unified as a continuous bi-level joint optimization problem.
- Block coordinate ascent is combined with Nyström low-rank approximation to reduce computational complexity to O(Nm).
- Experiments span four domains: bioinformatics, finance, cybersecurity, and computer vision.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG