一种赋有BB类步长的新随机方差缩减梯度算法
DOI:
作者:
作者单位:

贵州大学数学与统计学院

作者简介:

通讯作者:

基金项目:

国家自然科学基金项目;贵州省自然科学基金项目


A New Stochastic Variance Reduced Gradient Algorithm with BB-like Step Size
Author:
Affiliation:

School of Mathematics and Statistics, Guizhou University

Fund Project:

The National Natural Science Foundation of China; The Natural Science Foundation of Guizhou Province

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    【目的】为随机方差缩减梯度(SVRG)算法引入自适应步长,并在此基础上进一步提高算法数值性能。【方法】首先利用具有二维二次终止性的BB类步长自适应计算SVRG算法的步长。然后在SVRG算法的内循环中引入停止准则和负动量来加速算法的收敛速度。【结果】通过利用Matlab对提出的新算法进行数值实验,观察算法的数值性能。【结论】通过分析算法的数值实验结果,得出算法性能与在最佳步长调整下的SVRG方法相当,此外新算法对于初始步长的选取不敏感,且具有自动生成最优步长的能力。

    Abstract:

    [Purposes]Adaptive step size is introduced into the Stochastic Variance Reduction Gradient(SVRG) algorithm, and the numerical performance of the algorithm is further improved on this basis. [Methods]Firstly, the step size of the SVRG algorithm is calculated by BB step size with two-dimensional quadratic termination. Then the stopping criterion and negative momentum are introduced into the inner loop of the SVRG algorithm to accelerate the convergence rate. [Results]The proposed algorithm''s numerical experiment was carried out using Matlab, and the numerical performance of the algorithm was observed. [Conclusions]By analysing the numerical experimental results of the algorithm, it is concluded that the algorithm''s performance is comparable to that of the SVRG method with the optimal step size adjustment. In addition, the new algorithm is insensitive to the selection of the initial step size and can automatically generate the optimal step size.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2023-10-14
  • 最后修改日期:2024-04-01
  • 录用日期:2024-06-19
  • 在线发布日期: