…’JL
 NTT‰ÈŠwŠî‘bŒ¤‹†Š,
 §619-0237‹ž“s•{¸‰Ø’¬Œõ‘ä2-4
 E-mail: shin.mizutani.wb[at]hco.ntt.co.jp

Analyses of a Chaotic Neural Network as Spatiotemporal Chaotic Systems and Studies of the Perturbed System

Research Summary
A lot of experimental results which indicated the importance roles of chaos in biological information processing, are reported. In neural network modeling, a chaotic neural network has been proposed based on the chaotic responses in studies of squid giant axons and the Hodgkin-Huxley equation [1]. This associative dynamics of the network is studied for information processing, especially as fluctuations of escaping from local minima in optimization and for retrieving associative memories [2-5]. However, the dynamics is not analyzed enough until obtaining the understanding. On the other hand, in the field of physics, coupled systems of a nonlinear map have been intensively studied to understand spatiotemporal chaos and the various properties are being understood. We have been analyzing the spatiotemporal chaos in the chaotic neural network and indicated the important relations between the properties and the nerve system such as the brain. First, we focused on 1-dimensional network with homogeneous coupling in the nearest neighbor and analyzed by numerical calculations. In the turbulent phase, the state is similar with the individual chaos of each neuron. We found the exponential decay of correlations in space and time. This properties guarantees the thermodynamic treatment of the system. By the analyses of the subsystem, the density of the extensive quantifiers are obtained and we confirmed a few of them. The distributed whole information in the subsystem is a important property for the mechanism of the biological memory system. In such systems, the information is distributively represented and the whole information can be retrieved from the subsystems with any size. Based on these results of analyses, we proposed a control method of chaos in the high dimensional system and applied to the chaotic neural network. The conventional studies of controlling chaos are mainly regarding to the low dimensional system. Controlling chaos in the high dimensional systems is not studied enough because the controlling itself is considered difficult and the approach to a target is very unusual in such a high dimensional space. We propose the modified improvement for controlling chaos in the high dimensional system based on the method for the low dimensional system. This method uses nonlinear feedback instead of conventional linear feedback and we focused on the exponential feedback to extend the basin for controlling. We also proposed new concept of associative memory model using the controlling chaos in the chaotic neural network. This model does not realize for a difficulty of finding the unstable points in heterogeneous networks such as associative memory whose weight is learned from binary patterns, however, it is important concept. In physiological experiments of the olfactory system, learned stimuli caused the oscillatory response and unlearned stimuli caused the chaotic response. This result support the associative memory by controlling chaotic neural network. Next, we analyze the driven chaotic neuron model by a weak sinusoid. The modulation by a sinusoid is similar to the control feedback with a respect of a perturbation and we can consider it as a meaning of a control. The chaotic neuron model driven by a weak sinusoid shows resonance similar to stochastic resonance. Resonance in chaotic systems which is similar to stochastic resonance, was only reported a few cases. The chaotic neuron model is proposed based on conventional models as the threshold system and the excitable system that show stochastic resonance, however, it is a deterministic chaotic system. We analyzed properties of resonance by numerical calculations and we showed the enhancement of resonance by coupling and summing. We discussed the relations of resonance in the deterministic chaotic system and information processing by the biological system, especially, the relations of enhancement of resonance by coupling and summing and the meaning of parallel processing by the biological system. We discuss meanings of chaos in information processing by the biological system by studying and analyzing the mathematical model and try to propose significant concepts for biological information processing from studies of spatiotemporal chaos in the neural network and the coupled system.

Reference
[1] K. Aihara, T. Takabe, and M. Toyoda, ``Chaotic Neural Networks,'' Phys.Lett. A, vol. 144, no. 6/7, pp.333--340, 1990.
[2] H. Nozawa, ``A Neural Network Model as a Globally Coupled Map and Applications Based on Chaos,'' Chaos, vol. 2, no. 3, pp.377--386, 1992.
[3] T. Yamada and K. Aihara, ``Nonlinear Neurodynamics and Combinatorial Optimization in Chaotic Neural Networks'', J. Intell. Fuzzy Sys.,vol. 5, no. 1, pp.53--68, 1997
[4] M. Hasegawa, T. Ikeguchi, T. Matozaki, and K. Aihara, ``Solving Combinatorial Optimization Problems by Nonlinear Neural Dynamics,'' Proc. Int. Conf. on Neural Net., pp.3140--3145, 1995.
[5] M. Adachi and K. Aihara, ``Associative Dynamics in a Chaotic Neural Network,'' Neural Networks, vol. 10, no. 1, pp.83--98, 1997.
 
 

Topics
An Analysis of the Turbulent Phase in a Chaotic Neural Network
Controlling Chaos in Chaotic Neural Networks
Resonance in a Chaotic Neuron Model Driven by a Weak Sinusoid
Enhanced Resonance by Coupling and Summing in Sinusoidally Driven Chaotic Neural Networks
Ordering in the Mean Field of a Chaotic Neural Network with Quenched Disorder
 
 

Summary
An Analysis of the Turbulent Phase in a Chaotic Neural Network
This paper considers several properties of spatiotemporal chaos in a chaotic neural network. The Lyapunov spectrum, spatiotemporal information, and bit-wise information are calculated to measure properties of spatiotemporal chaos in a 1-dimensional nearest neighbor coupling network with a periodic boundary. The chaotic neural network can qualitatively represents the chaotic response of actual neurons based on biological studies. Many studies try to use chaotic dynamics in the network for escaping local minima in the optimization problem and for retrieving associative memories from embedded patterns. To understand properties of spatiotemporal chaos in the network, we mainly consider fully developed spatiotemporal chaos (FDSTC) phase to compare the properties of logistic coupled map lattice (CML). The Lyapunov spectrum has smooth shape or step-wise structure. The spectrum becomes smooth in FDSTC because the phase is close to the independent chaotic behavior with no interaction between neurons. The decay of spatiotemporal information in FDSTC is roughly exponential in space and time. The distribution of bit-wise information is extended over whole binary places. In FDSTC, the density of thermodynamic extensive quantifiers is assured because of exponential decay of spatiotemporal information. Comparing these results to the analyses in CML, both FDSTC phase has similar properties as spatiotemporal chaotic behavior, that are about information decay in space and time and information distribution in binary places.
Documents available on-line (in English, PostScript)

Controlling Chaos in Chaotic Neural Networks
We propose how to control chaos in chaotic neural networks. The chaotic neural network was proposed by Aihara et al. and is composed by the chaotic neuron model which is based on the study of squid giant axons and Hodgkin-Huxley equations. Therefore, the neuron model has chaotic response which former models do not have. Some studies try to use this chaos to search a memorized pattern in associative memory, but it is difficult to determine when the chaotic dynamics should stop. We considered recently proposed chaos control techniques to avoid this problem. We used exponential feedback control which makes one of the unstable periodic orbits stable using multiplicative exponential feedback on a system parameter. We modified this control to stabilize the unstable periodic orbit in high dimensional systems. For simplicity, we show that chaos in chaotic neural networks can be controlled under either 1-dimensional nearest neighbor or global coupling. The feedback can stabilize unstable periodic points and control the chaos in the networks. If the system include noise, the control is able to achieve.
Documents available on-line (in English, PostScript)

Resonance in a Chaotic Neuron Model Driven by a Weak Sinusoid
We show by numerical calculations that a chaotic neuron model driven by a weak sinusoid has resonance. This resonance phenomenon has a peak at a drive frequency similar to that of noise-induced stochastic resonance (SR). This neuron model was proposed from biological studies and shows a chaotic response when a parameter is varied. SR is a noise induced effect in driven nonlinear dynamical systems. The basic SR mechanism can be understood through synchronization and resonance in a bistable system driven by a subthreshold sinusoid plus noise. Therefore, background noise can boost a weak signal using SR. This effect is found in biological sensory neurons and obviously has some useful sensory function. The signal-to-noise ratio (SNR) of the driven chaotic neuron model is improved depending on the drive frequency; especially at low frequencies, the SNR is remarkably promoted. The resonance mechanism in the model is different from the noise-induced SR mechanism. This paper considers the mechanism and proposes possible explanations. Also, the meaning of chaos in biological systems based on the resonance phenomenon is considered.
Documents available on-line (in English, PostScript)

Enhanced Resonance by Coupling and Summing in Sinusoidally Driven Chaotic Neural Networks
Enhancement of resonance is shown by coupling and summing in sinusoidally driven chaotic neural networks. This resonance phenomenon has a peak at a drive frequency similar to noise-induced stochastic resonance (SR), however, the mechanism is different from noise-induced SR. We numerically study the properties of resonance in chaotic neural networks in the turbulent phase with summing and homogeneous coupling, with particular consideration of enhancement of the signal-to-noise ratio (SNR) by coupling and summing. Summing networks can enhance the SNR of a mean field based on the law of large numbers. Global coupling can enhance the SNR of a mean field and a neuron in the network. However, enhancement is not guaranteed and depends on the parameters. A combination of coupling and summing enhances the SNR, but summing to provide a mean field is more effective than coupling on a neuron level to promote the SNR. The global coupling network has a negative correlation between the SNR of the mean field and the Kolmogorov-Sinai (KS) entropy, and between the SNR of a neuron in the network and the KS entropy. This negative correlation is similar to the results of the driven single neuron model. The SNR is enhanced around a few frequencies and the dependence on frequency is clearer and smoother than the results of the driven single neuron model. The SNR is saturated as an increase in the drive amplitude, and further increases change the state into a nonchaotic one. Such dependence on the drive frequency and amplitude exhibits similarities to the results of the driven single neuron model. The nearest neighbor coupling network with a periodic or free boundary can also enhance the SNR of a neuron depending on the parameters. The network also has a negative correlation between the SNR of a neuron and the KS entropy whenever the boundary is periodic or free. The network with a free boundary does not have a significant effect on the SNR from both edges of the free boundaries.
Documents available on-line (in English, PostScript)

Ordering in the Mean Field of a Chaotic Neural Network with Quenched Disorder
Order of the mean field by quenched disorder is studied in the turbulent phase of a chaotic neural network. Quenched disorder means the distributed randomness of the input in each neuron or the weight in the network. The average power spectrum of the mean field is used to observe the order and to focus on its sharpness of the peak. In the turbulent phase, the sharpness in the power peak increases as disorder ratio except around the phase. We suppose that this ordering effect is important to process information for actual neural networks because of general existence of such heterogeneity.
Documents available on-line (in English, PostScript)
 
 

Bibliography
S. Mizutani, T. Sano, T. Uchiyama, and N. Sonehara, ``Controlling Chaos in Chaotic Neural Networks'', Electron. Comm. Jpn. Pt3, Vol.81, No.8, pp.73-82, 1998.
S. Mizutani, T. Sano, T. Uchiyama, and N. Sonehara, ``Resonance in a Chaotic Neuron Model Driven by a Weak Sinusoid'', IEICE Fundamentals, in press.
S. Mizutani, T. Sano, and K. Shimohara, ``Enhanced Resonance by Coupling and Summing in Sinusoidally Driven Chaotic Neural Networks'', IEICE Fundamentals, submitted.
S. Mizutani, T. Sano, T. Uchiyama, and N. Sonehara, ``An Analysis of the Turbulent Phase in a Chaotic Neural Network'', in preparation.
S. Mizutani, and K. Shimohara, ``Ordering in the Mean Field of a Chaotic Neural Network with Quenched Disorder'', in preparation.