Artificial neural networks are widely used to solve various applied problems. For the successful application of artificial neural networks, it is necessary to choose the correct network architecture, to select its parameters, threshold values of the elements, activation functions, etc. The problem of evaluating the neural network parameters, based on a study of the probabilistic behavior of the network is much promising. The study in the direction of developing probabilistic methods for perceptron-type pattern recognition systems is considered in different works. The concept of the characteristic function of the perceptron introduced by S. V Dayan was used by him to prove theorems on the existence of a perceptron solution. At the same time, issues of choosing a network architecture, theoretical assessment, and optimization of neural network parameters remain relevant. In this paper, we propose a mathematical apparatus for studying the relationship between the probability of correct classification of input data and the number of elements of hidden layers of a neural network. To evaluate the network performance and to estimate some parameters of the neural network such as the number of associative elements depending on the number of classification classes the mathematical expectation and variance of weights at the input of the output layer are considered. A theorem on the necessary and sufficient condition for the existence of a solution for a neural network is proved. By a solution of neural networks, the ability to recognize images with a probability other than zero is meant. The results of the proved theorem and its corollaries coincide with the results obtained by F. Rosenblat and S. Dayan for the perceptron in a different way.
| Published in | American Journal of Mathematical and Computer Modelling (Volume 5, Issue 3) |
| DOI | 10.11648/j.ajmcm.20200503.14 |
| Page(s) | 83-88 |
| Creative Commons |
This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited. |
| Copyright |
Copyright © The Author(s), 2020. Published by Science Publishing Group |
Neural Networks, Parameters of Neural Network, Probability of Recognition, Solution in Neural Network, Characteristic Function
| [1] | Rosenblatt F. Principles of Neurodynamics. Perceptrons and Theory of Brain Mechanisms. M., Mir, 1965 (in Russian). |
| [2] | Glushkov V. M. Introduction to cybernetics. Publishing House of the Academy of Sciences of the Ukrainian SSR, K., 1964. (in Russian). |
| [3] | Ivakhnenko A. G. Self-learning recognition systems and automatic control. Technique, K., 1969. (in Russian). |
| [4] | Nielson N. Learning machines. Mir, M., 1967. (in Russian). |
| [5] | Ivakhnenko A. G. Heuristic Self-Organization in Problems of Engineering Cybernetics // Automatica. 1970. Vol. 6, No. 2. P. 207-219. |
| [6] | Ivakhnenko A. G. Polynomial Theory of Complex Systems // IEEE Transactions on Systems, Man and Cybernetics. 1971. Vol. 4. P. 364-378. |
| [7] | Ikeda S., Ochiai M., Sawaragi Y. Sequential GMDH Algorithm and Its Application to River Flow Prediction // IEEE Trans. on Systems, Man and Cybernetics. 1976. Vol. 7. P. 473-479. |
| [8] | Witczak M, Korbicz J, Mrugalski M., et al. A GMDH Neural Network-Based Approach to Robust Fault Diagnosis: Application to the DAMADICS Benchmark Problem // Control Engineering Practice. 2006. Vol. 14, No. 6. P. 671-683. |
| [9] | Kondo T., Ueno J. Multi-Layered GMDH-type Neural Network Self-Selecting Optimum Neural Network Architecture and Its Application to 3-Dimensional Medical Image Recognition of Blood Vessels // International Journal of Innovative Computing, Information and Control. 2008. Vol. 4, No. 1. P. 175-187. |
| [10] | Cybenko G. Approximations by superpositions of a sigmoidal function // Mathematics of control, signals, systems. 1989. Vol. 2. Р. 303–314. |
| [11] | Hornik K., Stinchcombe M., White H. Multilayer feedforward networks are universal approximators // Neural networks. 1989. № 2 P. 359–366. |
| [12] | Golovko, V. A. Krasnoproshin V. V. Neural network data processing technologies: textbook. Minsk: BSU, 2017, 263 p. (in Russian). |
| [13] | Tarkhov D. A. Neural networks. Models and algorithms. Book 18. M. Radio Engineering, 2005, 256 p. (in Russian). |
| [14] | Weng J., Ahuja N., Huang T. S. Cresceptron: a Self-Organizing Neural Network Which Grows Adaptively // International Joint Conf. on Neural Networks. 1992. Vol. 1. P. 576-581. |
| [15] | Weng J. J., Ahuja N., Huang T. S. Learning Recognition and Segmentation Using the Cresceptron // International Journal of Computer Vision. 1997. Vol. 25, No. 2. P. 109-143. |
| [16] | Ranzato M. A., Huang F. J., Boureau Y. L., et al. Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2007. P. 1-8. |
| [17] | Scherer D., Muller A., Behnke S. Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition. Lect, Notes in Comp. Science. 2010. Vol. 6354, P. 92-101. |
| [18] | Ioffe S., Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift // JMLR Workshop and Conference Proceedings. Proceedings of International Conference on Machine Learning, 2015. Vol. 37. P. 448-456. |
| [19] | Szegedy C., Liu W, Jia Y. et al. Going Deeper with Convolutions // IEEE Conference on Computer Vision and Pattern Recognition, 2015. P. 1-9. |
| [20] | Szegedy C., Vanhoucke V., Ioffe S., et al. Rethinking the Inception Architecture for Computer Vision // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. P. 2818-2826. |
| [21] | Szegedy C., Ioffe S., Vanhoucke V., et al. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning // Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), 2017. P. 4278-4284. |
| [22] | LeCun Y., Bottou L., Orr G. B. Efficient Back Prop // Neural Networks: Tricks of the Trade. 1998. P. 9-50. |
| [23] | Streit R. L. Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Networks, V. 5, 199, №5, P. 764-783. |
| [24] | Zaknich A. Introduction to the modified probabilistic neural network for general signal processing applications. IEEE Transactions on Signal Processing, V. 46, 1998, №7, P. 1980-1990. |
| [25] | Romero R. David and Touretzky R. T., Optical Chinese Character Recognition using Probabilistic Neural Networks, 1996. |
| [26] | Bishop C. Neural Networks for Pattern Recognition, Oxford Univ. Press, 1995. |
| [27] | Perceptron - pattern recognition system, Edited by Ivahnenko A. G., Kiev, Naukova Dumka, 1975, p. 426. (in Russian). |
| [28] | S. V. Dayan. The concept of the characteristic function of the perceptron. Proceedings of the scientific and technical conference ErNIIMM (Yerevan Scientific Research Institute of Mathematical Machines). Yerevan, Armenia, 1968. (in Russian). |
| [29] | S. V. Dayan. Optimal learning of perceptron to recognize external situations. Reports of the 24th All-Union Session. Gorky, 1968. (in Russian). |
| [30] | S. V. Dayan. Investigation of the probabilistic properties of the characteristic function of the perceptron. In the book: The problems of bionics. 3. Publishing of Kharkov University. Kharkov. 1970. (in Russian). |
| [31] | Sargsyan S. G., Dayan S. V. Modeling of the learning process for pattern recognition on computers, YSU, Scientific notes, 1984 (in Russian). |
| [32] | Sargsyan S. G. Determination of the probability characteristics of adaptive recognition system, Trans. of Intern. Conf. Adaptable software, Kishinev, 1990. pp. 46-51 (in Russian). |
| [33] | Kharatyan A., Sargsyan S. G., Hovakimyan A. S., Time Series Forecasting Using Artificial Neural Networks, YSU, Faculty of Economics. Yearbook, Yerevan, 2014 (in Russian). |
| [34] | S. Sargsyan, A. Hovakimyan, Probabilistic Methods for Neural Networks Study, Quarterly Journal of Mechanics and Applied Mathematics, Issue 4 (2), Volume 69, Oxford University Press, Nov. 2016, pp. 669-675. |
| [35] | S. Sargsyan, A. Hovakimyan. Statistical Evaluation of the Performance of the Neural Network. London Journal of Research in Computer Science and Technology, Volume 17 | Issue 1 | Compilation 1.0, 2017, pp. 1-6. |
| [36] | S. Sargsyan, A. Hovakimyan, M. Ziroyan. Hybrid Method for the Big Data Analysis Using Neural Networks. Engineering Studies, Issue 3 (2), Volume 10. Taylor & Francis, 2018. P. 519-526. |
APA Style
Sargsyan Siranush, Hovakimyan Anna. (2020). One Approach to the Problem of the Existence of a Solution in Neural Networks. American Journal of Mathematical and Computer Modelling, 5(3), 83-88. https://doi.org/10.11648/j.ajmcm.20200503.14
ACS Style
Sargsyan Siranush; Hovakimyan Anna. One Approach to the Problem of the Existence of a Solution in Neural Networks. Am. J. Math. Comput. Model. 2020, 5(3), 83-88. doi: 10.11648/j.ajmcm.20200503.14
AMA Style
Sargsyan Siranush, Hovakimyan Anna. One Approach to the Problem of the Existence of a Solution in Neural Networks. Am J Math Comput Model. 2020;5(3):83-88. doi: 10.11648/j.ajmcm.20200503.14
Share This Article
Twitter
Linked In
Facebook
Copy
Download@article{10.11648/j.ajmcm.20200503.14,
author = {Sargsyan Siranush and Hovakimyan Anna},
title = {One Approach to the Problem of the Existence of a Solution in Neural Networks},
journal = {American Journal of Mathematical and Computer Modelling},
volume = {5},
number = {3},
pages = {83-88},
doi = {10.11648/j.ajmcm.20200503.14},
url = {https://doi.org/10.11648/j.ajmcm.20200503.14},
eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajmcm.20200503.14},
abstract = {Artificial neural networks are widely used to solve various applied problems. For the successful application of artificial neural networks, it is necessary to choose the correct network architecture, to select its parameters, threshold values of the elements, activation functions, etc. The problem of evaluating the neural network parameters, based on a study of the probabilistic behavior of the network is much promising. The study in the direction of developing probabilistic methods for perceptron-type pattern recognition systems is considered in different works. The concept of the characteristic function of the perceptron introduced by S. V Dayan was used by him to prove theorems on the existence of a perceptron solution. At the same time, issues of choosing a network architecture, theoretical assessment, and optimization of neural network parameters remain relevant. In this paper, we propose a mathematical apparatus for studying the relationship between the probability of correct classification of input data and the number of elements of hidden layers of a neural network. To evaluate the network performance and to estimate some parameters of the neural network such as the number of associative elements depending on the number of classification classes the mathematical expectation and variance of weights at the input of the output layer are considered. A theorem on the necessary and sufficient condition for the existence of a solution for a neural network is proved. By a solution of neural networks, the ability to recognize images with a probability other than zero is meant. The results of the proved theorem and its corollaries coincide with the results obtained by F. Rosenblat and S. Dayan for the perceptron in a different way.},
year = {2020}
}
TY - JOUR T1 - One Approach to the Problem of the Existence of a Solution in Neural Networks AU - Sargsyan Siranush AU - Hovakimyan Anna Y1 - 2020/09/16 PY - 2020 N1 - https://doi.org/10.11648/j.ajmcm.20200503.14 DO - 10.11648/j.ajmcm.20200503.14 T2 - American Journal of Mathematical and Computer Modelling JF - American Journal of Mathematical and Computer Modelling JO - American Journal of Mathematical and Computer Modelling SP - 83 EP - 88 PB - Science Publishing Group SN - 2578-8280 UR - https://doi.org/10.11648/j.ajmcm.20200503.14 AB - Artificial neural networks are widely used to solve various applied problems. For the successful application of artificial neural networks, it is necessary to choose the correct network architecture, to select its parameters, threshold values of the elements, activation functions, etc. The problem of evaluating the neural network parameters, based on a study of the probabilistic behavior of the network is much promising. The study in the direction of developing probabilistic methods for perceptron-type pattern recognition systems is considered in different works. The concept of the characteristic function of the perceptron introduced by S. V Dayan was used by him to prove theorems on the existence of a perceptron solution. At the same time, issues of choosing a network architecture, theoretical assessment, and optimization of neural network parameters remain relevant. In this paper, we propose a mathematical apparatus for studying the relationship between the probability of correct classification of input data and the number of elements of hidden layers of a neural network. To evaluate the network performance and to estimate some parameters of the neural network such as the number of associative elements depending on the number of classification classes the mathematical expectation and variance of weights at the input of the output layer are considered. A theorem on the necessary and sufficient condition for the existence of a solution for a neural network is proved. By a solution of neural networks, the ability to recognize images with a probability other than zero is meant. The results of the proved theorem and its corollaries coincide with the results obtained by F. Rosenblat and S. Dayan for the perceptron in a different way. VL - 5 IS - 3 ER -
Information
Email: