NTT Communication Science Laboratories,

Room D302, 2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0237 Japan

E-mail: shin.mizutani[at]hco.ntt.co.jp

Analyses of a Chaotic Neural Network as Spatiotemporal Chaotic Systems and Studies of the Perturbed System

Research Summary

A lot of experimental results which indicated the importance roles
of chaos in biological information processing, are reported. In neural
network modeling, a chaotic neural network has been proposed based on the
chaotic responses in studies of squid giant axons and the Hodgkin-Huxley
equation [1]. This associative dynamics of the network is studied for information
processing, especially as fluctuations of escaping from local minima in
optimization and for retrieving associative memories [2-5]. However, the
dynamics is not analyzed enough until obtaining the understanding. On the
other hand, in the field of physics, coupled systems of a nonlinear map
have been intensively studied to understand spatiotemporal chaos and the
various properties are being understood. We have been analyzing the spatiotemporal
chaos in the chaotic neural network and indicated the important relations
between the properties and the nerve system such as the brain. First, we
focused on 1-dimensional network with homogeneous coupling in the nearest
neighbor and analyzed by numerical calculations. In the turbulent phase,
the state is similar with the individual chaos of each neuron. We found
the exponential decay of correlations in space and time. This properties
guarantees the thermodynamic treatment of the system. By the analyses of
the subsystem, the density of the extensive quantifiers are obtained and
we confirmed a few of them. The distributed whole information in the subsystem
is a important property for the mechanism of the biological memory system.
In such systems, the information is distributively represented and the
whole information can be retrieved from the subsystems with any size. Based
on these results of analyses, we proposed a control method of chaos in
the high dimensional system and applied to the chaotic neural network.
The conventional studies of controlling chaos are mainly regarding to the
low dimensional system. Controlling chaos in the high dimensional systems
is not studied enough because the controlling itself is considered difficult
and the approach to a target is very unusual in such a high dimensional
space. We propose the modified improvement for controlling chaos in the
high dimensional system based on the method for the low dimensional system.
This method uses nonlinear feedback instead of conventional linear feedback
and we focused on the exponential feedback to extend the basin for controlling.
We also proposed new concept of associative memory model using the controlling
chaos in the chaotic neural network. This model does not realize for a
difficulty of finding the unstable points in heterogeneous networks such
as associative memory whose weight is learned from binary patterns, however,
it is important concept. In physiological experiments of the olfactory
system, learned stimuli caused the oscillatory response and unlearned stimuli
caused the chaotic response. This result support the associative memory
by controlling chaotic neural network. Next, we analyze the driven chaotic
neuron model by a weak sinusoid. The modulation by a sinusoid is similar
to the control feedback with a respect of a perturbation and we can consider
it as a meaning of a control. The chaotic neuron model driven by a weak
sinusoid shows resonance similar to stochastic resonance. Resonance in
chaotic systems which is similar to stochastic resonance, was only reported
a few cases. The chaotic neuron model is proposed based on conventional
models as the threshold system and the excitable system that show stochastic
resonance, however, it is a deterministic chaotic system. We analyzed properties
of resonance by numerical calculations and we showed the enhancement of
resonance by coupling and summing. We discussed the relations of resonance
in the deterministic chaotic system and information processing by the biological
system, especially, the relations of enhancement of resonance by coupling
and summing and the meaning of parallel processing by the biological system.
We discuss meanings of chaos in information processing by the biological
system by studying and analyzing the mathematical model and try to propose
significant concepts for biological information processing from studies
of spatiotemporal chaos in the neural network and the coupled system.

Reference

[1] K. Aihara, T. Takabe, and M. Toyoda, ``Chaotic Neural Networks,''
Phys.Lett. A, vol. 144, no. 6/7, pp.333--340, 1990.

[2] H. Nozawa, ``A Neural Network Model as a Globally Coupled Map and
Applications Based on Chaos,'' Chaos, vol. 2, no. 3, pp.377--386, 1992.

[3] T. Yamada and K. Aihara, ``Nonlinear Neurodynamics and Combinatorial
Optimization in Chaotic Neural Networks'', J. Intell. Fuzzy Sys.,vol. 5,
no. 1, pp.53--68, 1997

[4] M. Hasegawa, T. Ikeguchi, T. Matozaki, and K. Aihara, ``Solving
Combinatorial Optimization Problems by Nonlinear Neural Dynamics,'' Proc.
Int. Conf. on Neural Net., pp.3140--3145, 1995.

[5] M. Adachi and K. Aihara, ``Associative Dynamics in a Chaotic Neural
Network,'' Neural Networks, vol. 10, no. 1, pp.83--98, 1997.

Topics

An
Analysis of the Turbulent Phase in a Chaotic Neural Network

Controlling
Chaos in Chaotic Neural Networks

Resonance
in a Chaotic Neuron Model Driven by a Weak Sinusoid

Enhanced
Resonance by Coupling and Summing in Sinusoidally Driven Chaotic Neural
Networks

Ordering
in the Mean Field of a Chaotic Neural Network with Quenched Disorder

Summary

An
Analysis of the Turbulent Phase in a Chaotic Neural Network

This paper considers several properties of spatiotemporal chaos in
a chaotic neural network. The Lyapunov spectrum, spatiotemporal information,
and bit-wise information are calculated to measure properties of spatiotemporal
chaos in a 1-dimensional nearest neighbor coupling network with a periodic
boundary. The chaotic neural network can qualitatively represents the chaotic
response of actual neurons based on biological studies. Many studies try
to use chaotic dynamics in the network for escaping local minima in the
optimization problem and for retrieving associative memories from embedded
patterns. To understand properties of spatiotemporal chaos in the network,
we mainly consider fully developed spatiotemporal chaos (FDSTC) phase to
compare the properties of logistic coupled map lattice (CML). The Lyapunov
spectrum has smooth shape or step-wise structure. The spectrum becomes
smooth in FDSTC because the phase is close to the independent chaotic behavior
with no interaction between neurons. The decay of spatiotemporal information
in FDSTC is roughly exponential in space and time. The distribution of
bit-wise information is extended over whole binary places. In FDSTC, the
density of thermodynamic extensive quantifiers is assured because of exponential
decay of spatiotemporal information. Comparing these results to the analyses
in CML, both FDSTC phase has similar properties as spatiotemporal chaotic
behavior, that are about information decay in space and time and information
distribution in binary places.

Documents
available on-line (in English, PostScript)

Controlling
Chaos in Chaotic Neural Networks

We propose how to control chaos in chaotic neural networks. The chaotic
neural network was proposed by Aihara et al. and is composed by the chaotic
neuron model which is based on the study of squid giant axons and Hodgkin-Huxley
equations. Therefore, the neuron model has chaotic response which former
models do not have. Some studies try to use this chaos to search a memorized
pattern in associative memory, but it is difficult to determine when the
chaotic dynamics should stop. We considered recently proposed chaos control
techniques to avoid this problem. We used exponential feedback control
which makes one of the unstable periodic orbits stable using multiplicative
exponential feedback on a system parameter. We modified this control to
stabilize the unstable periodic orbit in high dimensional systems. For
simplicity, we show that chaos in chaotic neural networks can be controlled
under either 1-dimensional nearest neighbor or global coupling. The feedback
can stabilize unstable periodic points and control the chaos in the networks.
If the system include noise, the control is able to achieve.

Documents
available on-line (in English, PostScript)

Resonance
in a Chaotic Neuron Model Driven by a Weak Sinusoid

We show by numerical calculations that a chaotic neuron model driven
by a weak sinusoid has resonance. This resonance phenomenon has a peak
at a drive frequency similar to that of noise-induced stochastic resonance
(SR). This neuron model was proposed from biological studies and shows
a chaotic response when a parameter is varied. SR is a noise induced effect
in driven nonlinear dynamical systems. The basic SR mechanism can be understood
through synchronization and resonance in a bistable system driven by a
subthreshold sinusoid plus noise. Therefore, background noise can boost
a weak signal using SR. This effect is found in biological sensory neurons
and obviously has some useful sensory function. The signal-to-noise ratio
(SNR) of the driven chaotic neuron model is improved depending on the drive
frequency; especially at low frequencies, the SNR is remarkably promoted.
The resonance mechanism in the model is different from the noise-induced
SR mechanism. This paper considers the mechanism and proposes possible
explanations. Also, the meaning of chaos in biological systems based on
the resonance phenomenon is considered.

Documents available
on-line (in English, PostScript)

Enhanced
Resonance by Coupling and Summing in Sinusoidally Driven Chaotic Neural
Networks

Enhancement of resonance is shown by coupling and summing in sinusoidally
driven chaotic neural networks. This resonance phenomenon has a peak at
a drive frequency similar to noise-induced stochastic resonance (SR), however,
the mechanism is different from noise-induced SR. We numerically study
the properties of resonance in chaotic neural networks in the turbulent
phase with summing and homogeneous coupling, with particular consideration
of enhancement of the signal-to-noise ratio (SNR) by coupling and summing.
Summing networks can enhance the SNR of a mean field based on the law of
large numbers. Global coupling can enhance the SNR of a mean field and
a neuron in the network. However, enhancement is not guaranteed and depends
on the parameters. A combination of coupling and summing enhances the SNR,
but summing to provide a mean field is more effective than coupling on
a neuron level to promote the SNR. The global coupling network has a negative
correlation between the SNR of the mean field and the Kolmogorov-Sinai
(KS) entropy, and between the SNR of a neuron in the network and the KS
entropy. This negative correlation is similar to the results of the driven
single neuron model. The SNR is enhanced around a few frequencies and the
dependence on frequency is clearer and smoother than the results of the
driven single neuron model. The SNR is saturated as an increase in the
drive amplitude, and further increases change the state into a nonchaotic
one. Such dependence on the drive frequency and amplitude exhibits similarities
to the results of the driven single neuron model. The nearest neighbor
coupling network with a periodic or free boundary can also enhance the
SNR of a neuron depending on the parameters. The network also has a negative
correlation between the SNR of a neuron and the KS entropy whenever the
boundary is periodic or free. The network with a free boundary does not
have a significant effect on the SNR from both edges of the free boundaries.

Documents available
on-line (in English, PostScript)

Ordering
in the Mean Field of a Chaotic Neural Network with Quenched Disorder

Order of the mean field by quenched disorder is studied in the turbulent
phase of a chaotic neural network. Quenched disorder means the distributed
randomness of the input in each neuron or the weight in the network. The
average power spectrum of the mean field is used to observe the order and
to focus on its sharpness of the peak. In the turbulent phase, the sharpness
in the power peak increases as disorder ratio except around the phase.
We suppose that this ordering effect is important to process information
for actual neural networks because of general existence of such heterogeneity.

Documents
available on-line (in English, PostScript)

Bibliography

S. Mizutani,
T. Sano, T. Uchiyama, and N. Sonehara, ``Controlling Chaos in Chaotic Neural
Networks'', Electron. Comm. Jpn. Pt3, Vol.81, No.8, pp.73-82, 1998.

S.
Mizutani, T. Sano, T. Uchiyama, and N. Sonehara, ``Resonance in a Chaotic
Neuron Model Driven by a Weak Sinusoid'', IEICE Fundamentals, in press.

S.
Mizutani, T. Sano, and K. Shimohara, ``Enhanced Resonance by Coupling and
Summing in Sinusoidally Driven Chaotic Neural Networks'', IEICE Fundamentals,
submitted.

S.
Mizutani, T. Sano, T. Uchiyama, and N. Sonehara, ``An Analysis of the Turbulent
Phase in a Chaotic Neural Network'', in preparation.

S.
Mizutani, and K. Shimohara, ``Ordering in the Mean Field of a Chaotic Neural
Network with Quenched Disorder'', in preparation.