Stability of a Regularized Newton Method with Two Potentials

Boushra Abbas; Ramez Koudsieh

doi:doi:10.11648/j.ijtam.20210701.11

| Peer-Reviewed

Stability of a Regularized Newton Method with Two Potentials

Boushra Abbas, Ramez Koudsieh

Published in International Journal of Theoretical and Applied Mathematics (Volume 7, Issue 1)

Received: 14 February 2020 Accepted: 30 November 2020 Published: 22 January 2021

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.

Published in	International Journal of Theoretical and Applied Mathematics (Volume 7, Issue 1)
DOI	10.11648/j.ijtam.20210701.11
Page(s)	1-11
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2021. Published by Science Publishing Group

Keywords

Monotone Inclusions, Newton Method, Levenberg-Marquardt Regularization, Dissipative Dynamical Systems, Lyapunov Analysis, Weak Asymptotic Convergence, Forward-Backward Algorithms, Gradient-Projection Methods

References

[1]	B. Abbas, H. Attouch, Dynamical systems and forward-backward algorithms associated with the sum of a convex subdifferential and a monotone cocoercive operator, Optimization, (2014) http://dx.doi.org/10.1080/02331934.2014.971412.
[2]	B. Abbas, H. Attouch, B. F. Svaiter, Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces, J. Optim. Theory Appl., 161 (2014), No. 2, pp. 331-360.
[3]	Alvarez, F., Attouch, H., Bolte, J., Redont, P.: A second- order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics. J. Math. Pures Appl. 81, 747–779 (2002).
[4]	Antipin, A. S.: Minimization of convex functions on convex sets by means of differential equations. Differential Equations. 30, 1365–1375 (1994).
[5]	Attouch, H., Briceno, L., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48, 3246–3270 (2010).
[6]	H. Attouch, P. Redont, B. F. Svaiter, Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces, J. Optim. Theory Appl., 157 (2013), No. 3, pp. 624–650. International Journal of Theoretical and Applied Mathematics 2020; 7(1): 1-11 11
[7]	Attouch, H., Svaiter, B.F.: A continuous dynamical Newton-like approach to solving monotone inclusions. SIAM J. Control Optim. 49, 574–598 (2011).
[8]	Bauschke, H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory. CMS books in Mathematics, Springer (2011).
[9]	Beck, A., Teboulle, M.: Gradient-based algorithms with applications in signal recovery problems. In: Palomar, D., Eldar, Y. (eds.): Convex Optimization in Signal Processing and Communications, pp. 33–88. Cambridge University Press, (2010).
[10]	Bolte, J.: Continuous gradient projection method in Hilbert spaces. J. Optim. Theory Appl. 119, 235–259 (2003).
[11]	Br´ ezis, H.: Op´ erateurs Maximaux Monotones et Semi- Groupes de Contractions dans les Espaces de Hilbert. North-Holland/Elsevier, New-York (1973).
[12]	Br´ ezis, H.: Analyse Fonctionnelle. Masson, Paris (1983).
[13]	Bruck, R. E.: Asymptotic convergence of nonlinear contraction semigroups in Hilbert spaces. J. Funct. Anal. 18, 15-26 (1975).
[14]	Dennis, J. E., Schnabel, R. B.: Numerical Methods for Unconstrained Minimization. Prentice-Hall, Englewood Cliffs, NJ,1983. ReprintedbySIAMpublications(1993).
[15]	Ferreira, O. P., Svaiter, B. F.: Kantorovich’s theorem on Newton’s method on Riemannian manifolds. Journal of Complexity 18, 304–329 (2002).
[16]	G¨uler, O.: On the convergence of the proximal point algorithm for convex minimization, SIAM J. Control Optim. 29, 403–419 (1991).
[17]	Haraux, A.: Syst` emes dynamiques dissipatifs et applications. RMA 17, Masson, Paris, (1991).
[18]	Nocedal, J., Wright, S.: Numerical Optimization. Springer Series in Operations Research, Springer-Verlag, New-York (1999).
[19]	Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Amer. Math. Soc. 73, 591–597 (1967).
[20]	Peypouquet, J., Sorin, S.: Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time, J. of Convex Analysis 17, 1113-1163 (2010).
[21]	Rockafellar, R. T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976).
[22]	Rockafellar, R. T.: Maximal monotone relations and the second derivatives of nonsmooth functions. Ann. Inst. Henri Poincar? 2, 167–184 (1985).
[23]	Solodov, M. V., Svaiter, B. F.: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M., Qi, L., (eds): Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355-369. Kluwer Academic Publishers, (1999).

Cite This Article

Plain Text BibTeX RIS

APA Style

Boushra Abbas, Ramez Koudsieh. (2021). Stability of a Regularized Newton Method with Two Potentials. International Journal of Theoretical and Applied Mathematics, 7(1), 1-11. https://doi.org/10.11648/j.ijtam.20210701.11

Copy | Download

ACS Style

Boushra Abbas; Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int. J. Theor. Appl. Math. 2021, 7(1), 1-11. doi: 10.11648/j.ijtam.20210701.11

Copy | Download

AMA Style

Boushra Abbas, Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int J Theor Appl Math. 2021;7(1):1-11. doi: 10.11648/j.ijtam.20210701.11

Copy | Download

@article{10.11648/j.ijtam.20210701.11,
  author = {Boushra Abbas and Ramez Koudsieh},
  title = {Stability of a Regularized Newton Method with Two Potentials},
  journal = {International Journal of Theoretical and Applied Mathematics},
  volume = {7},
  number = {1},
  pages = {1-11},
  doi = {10.11648/j.ijtam.20210701.11},
  url = {https://doi.org/10.11648/j.ijtam.20210701.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijtam.20210701.11},
  abstract = {In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.},
 year = {2021}
}

Copy | Download

TY  - JOUR
T1  - Stability of a Regularized Newton Method with Two Potentials
AU  - Boushra Abbas
AU  - Ramez Koudsieh
Y1  - 2021/01/22
PY  - 2021
N1  - https://doi.org/10.11648/j.ijtam.20210701.11
DO  - 10.11648/j.ijtam.20210701.11
T2  - International Journal of Theoretical and Applied Mathematics
JF  - International Journal of Theoretical and Applied Mathematics
JO  - International Journal of Theoretical and Applied Mathematics
SP  - 1
EP  - 11
PB  - Science Publishing Group
SN  - 2575-5080
UR  - https://doi.org/10.11648/j.ijtam.20210701.11
AB  - In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.
VL  - 7
IS  - 1
ER  -

Copy | Download

Author Information

Boushra Abbas

Mathematics Department, Tichreen University, Lattakia, Syria
Ramez Koudsieh

Mechatroncis Department, Manara University, Lattakia, Syria

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Boushra Abbas, Ramez Koudsieh. (2021). Stability of a Regularized Newton Method with Two Potentials. International Journal of Theoretical and Applied Mathematics, 7(1), 1-11. https://doi.org/10.11648/j.ijtam.20210701.11

Copy | Download

ACS Style

Boushra Abbas; Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int. J. Theor. Appl. Math. 2021, 7(1), 1-11. doi: 10.11648/j.ijtam.20210701.11

Copy | Download

AMA Style

Boushra Abbas, Ramez Koudsieh. Stability of a Regularized Newton Method with Two Potentials. Int J Theor Appl Math. 2021;7(1):1-11. doi: 10.11648/j.ijtam.20210701.11

Copy | Download

@article{10.11648/j.ijtam.20210701.11,
  author = {Boushra Abbas and Ramez Koudsieh},
  title = {Stability of a Regularized Newton Method with Two Potentials},
  journal = {International Journal of Theoretical and Applied Mathematics},
  volume = {7},
  number = {1},
  pages = {1-11},
  doi = {10.11648/j.ijtam.20210701.11},
  url = {https://doi.org/10.11648/j.ijtam.20210701.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijtam.20210701.11},
  abstract = {In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.},
 year = {2021}
}

Copy | Download

TY  - JOUR
T1  - Stability of a Regularized Newton Method with Two Potentials
AU  - Boushra Abbas
AU  - Ramez Koudsieh
Y1  - 2021/01/22
PY  - 2021
N1  - https://doi.org/10.11648/j.ijtam.20210701.11
DO  - 10.11648/j.ijtam.20210701.11
T2  - International Journal of Theoretical and Applied Mathematics
JF  - International Journal of Theoretical and Applied Mathematics
JO  - International Journal of Theoretical and Applied Mathematics
SP  - 1
EP  - 11
PB  - Science Publishing Group
SN  - 2575-5080
UR  - https://doi.org/10.11648/j.ijtam.20210701.11
AB  - In a Hilbert space setting, we introduce dynamical systems, which are linked to Newton and Levenberg-Marquardt methods. They are intended to solve, by splitting methods, inclusions governed by structured monotone operators M = A + B, where A is a general maximal monotone operator, and B is monotone and locally Lipschitz continuous. Based on the Minty representation of A as a Lipschitz manifold, we show that these dynamics can be formulated as differential systems, which are relevant to the Cauchy-Lipschitz theorem, and involve separately B and the resolvents of A. In the convex subdifferential case, by using Lyapunov asymptotic analysis, we prove a descent minimizing property, and weak convergence to equilibria of the trajectories. Time discretization of these dynamics gives algorithms combining Newton’s method and forward-backward methods for solving structured monotone inclusions. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton’s methods.
VL  - 7
IS  - 1
ER  -

Copy | Download