Main Areas of Research:
|
Description
[note that there are also single pages for deepening applications one by one: put the mouse on the flag "research" above]
1. Mathematical methods for complex systems
Disordered statistical mechanics, coupled to graph theory, is nowadays one of the main tools -and in some sense a whole novel perspective- for investigating "complexity": I combine them (i.e. replica trick, stochastic stability, cavity fields, etc. from the former and percolation and spectral theory, clustering and modularity analysis, etc. regarding the latter) and mix them with techniques of cybernetics (i.e. transfer functions theory, Laplace transforms, Fourier analysis etc.) to describe the emergent spontaneous features of a huge variety of networks (i.e. immune networks, neural networks, biochemical networks, etc.). To this task, a continuous development of new mathematical models and methods is required and part of my work consists in building these abstract theoretical scaffolds, mainly focusing on their applications to decode biological complexity (from molecular and cellular networks to human interactions). In particular, among the various mathematical techniques, I extensively worked on smooth cavity field, multiple stochastic stability, random overlap structures, non-linear PDE approaches, interpolating replica trick, replica symmetry breaking from novel perspectives, hebbian small world (i.e small worlds without rewiring), continuous time random walks, anomalous diffusions, and more.
Selected publications:
Selected publications:
- Adriano Barra, Aldo Di Biasio, Francesco Guerra,
Replica symmetry breaking in mean field spin glasses trough Hamilton-Jacobi technique,
J. Stat. Mech, (2010).
Here we have shown that it is possible to obtain the full Parisi solution of the SK model without the usage of Statistical Mechanics but only via variational principles of classical mechanics. In particular, within the general scheme where the free energy obeys an Hamilton-Jacobi PDE and the overlap evolves under shocks within a Riemann-Hopf PDE it is possible to prove that each step K of RSB mirrors the motion of a system in a K+1 Euclidean space, while the full Parisi solution has its convergence on an Hilbert space, thus the corresponding order parameter becomes a functional (as in the original framework). Results from this route are in full agreement with those from replica trick or cavity fields.
- A. Barra, A. Di Lorenzo, F. Guerra, A. Moro,
On quantum and relativistic mechanical analogues in mean field spin models,
Proc. Royal Soc. A (2014).
Here we extended the analogy between solving for the free energy of a mean field system and obtaining the explicit motion of a suitable dynamical system to relativistic and quantum mechanics. We have shown that, while the bridge to two-body problems (as i.e. the SK or the CW models) lies within an Euclidean analogy, the quantum relativistic extension (i.e. the Klain-Gordon field) accounts for all the general P-spin models. - Adriano Barra,
The mean field Ising model trough interpolating techniques,
J. Stat. Phys. (2008).
This is a review in which I apply several techniques widely used in Disordered Systems to the simplest Curie-Weiss model: the underlying idea is to show how these techniques do work in very elementary cases. In particular I discuss in this paper: (1) the smooth cavity field approach, (2) the Hamilton-Jacobi approach, (3) Parisi theory and replica trick, (4) stochastic stability. - Adriano Barra, Francesco Guerra, Emanuele Mingione,
Interpolating the Sherrington-Kirkpatrick replica trick,
Philos. Magaz. (2011).
Guerra's interpolating techniques have played a major role in rigorously proving several scenarios dealing with spin glasses (in particular at work with the celebrated Sherrington-Kirkpatrick model). However, for their intrinsic construction, these techniques have been -at the beginning- naturally implemented within the schemes of cavity fields and stochastic stability, while the largest bulk of scientists involved in disordered systems uses the replica trick. In this paper, dedicated to David seventieth birthday, we extended the main concepts of interpolation to the replica trick developing in this way a revised version of the method, that, by the way, allowed to finally prove that the famous limits of zero replicas and infinite volume do commute, as assumed in the pioneering approaches by Sherrington, Kirkpatrick and Parisi (and many more in a cascade fashion). - Peter Sollich, Daniele Tantari, Alessia Annibale, Adriano Barra,
Extensive parallel processing on scale free networks,
Phys. Rev. Lett. (2014).
In this work we develop a novel variant of the cavity field technique that works on systems whose graphs show finite connectivity and loops. In particular our approach has been used here to show that it is possible even to solve neural networks on these graphs and that extensive parallel processing (where N neurons manage simultaneously O(N) patterns) is more robust on homogeneous networks, thus finding a novel balancing principle in physics: scale free are best performing for a large class of demands (i.e. stability under attack, information spreading, etc.), however, are less convenient dealing with information processing (more than information spreading).
2. Theoretical Immunology
The immune system is an amazing underpercolated network built of by several different agents (i.e. B lymphocytes, T helpers, T suppressors, T killers, natural killers, macrophages, neutrofils, and so on) as well as chemical messengers (i.e. kytochines, antibodies), and displays a broad amount of emergent features, which are not implicitly accounted when considering the single agent making the system itself (as low/high dose tolerance, full self/non-self discrimination, systemic anergy, parallel antigenic processing, decisions via quorum sensing, etc): explaining these emerging features of the system, via disordered statistical mechanics and graph theory as well as developing a formulation of a "thermodynamics" for the immune system (whose applications may be useful in clinics) are my long term goals.
I am involved primarily in the regulation and coordination roles played by the helpers (Th1 and Th2 families) and the suppressors, their links with cytokine productions and antigenic binding. I try to answer questions as "which is the computational power of the immune system?, how much information is required for a clonal expansion to start? how pattern recognition and information caption is split among membrane's receptors of a lymphocyte (i.e. between the complex TCR-MHC or BCR-antigen and the dialogue via cytokines)? and where memory is stored in the system and which is the maximal capacity in terms of bits for this memory? what happens once the memory is saturated? how do the highly parallel processing capabilities of the network work? what is an impaired immune system from an information theory perspective and what can be done to make it healthy again? how many antigen recognitions can be handled simultaneously in a lymphnode? How expressed is the mature repertoire? and more..."
Given the remarkable systemic similarities among immune and neural performances (as pattern recognition, decision capabilities, long and short term memory, etc.), the underlying bridges with other cognitive systems as Hopfield neural networks or Hebbian and Boltzmann machines is under development as well. However, beyond evident similarities, even deep differences exist between these two systems. At first the brain (or better several of its modules in the cortex) are built by highly connected neurons, while dialogues among lymphocytes -due to their chemical specificity- are extremely selectively, implying the overall network to be under-percolated (this has enormous consequences at the systemic level, i.e. the braim is mainly a high-definition serial processor while the immune system is mainly a low-definition parallel processor). Further, while there exist several times of neurons (as piramidal, stellar, etc.), mathematical models for neural networks usually consider one toy model of neuron (i.e. the Stein integrate and fire neuron) as, despite too rough, at the end of the day whatever the nature of the neuron, the mechanisms behind its firing (the genesis of a spike) are universal among the various types of real neurons. In Immunology this oversimplification can not be pursued as different types of lymphocytes do really behave in completely different ways (for instance B cells produce antibodies, while helpers and suppressors produce cytokines and killers fagocyte and induce lysis).
Hence, keeping in mind the strong differences, we can still try to search for structural analogies between these systems: in particular, among the results achieved so far, in our group we showed how the (spin-glass like) interactions among B-cells, helpers and suppressors can be mapped into an equivalent, effective, framework for helpers and suppressors alone, where they arrange themselves toward an associative Hebbian network able to instruct several effector clones at once: so to say, these lymphocytes are able to orchestrate their interactions with B-cells involved to cope pathogens time by time invading the host by "deciding" which is the best soldier (the best B-clone or the best ensemble of B-clones) to use for a particular fight (or for an ensemble of infections simultaneously acting on the host).
Remarkably we showed further how dilution in these networks -which is a biological must (becase, as a huge difference between immune and neural networks, immune networks -topologically- are dynamical and interactions among their nodes -clones- are achieved through short range diffusion mainly and not via "long-range spikes on static connections")- implies multitasking capabilities by the immune system as a whole: helpers and regulators are indeed able to elaborate several strategies for fighiting several antigens at the same time, which is a key feature of the immune response and is obtained through signaling toward the effector branch via low-information contents (via kytochiens), while pattern recognition (dense of information) is performed at the level of B and T cell-receptors.
Finally other aspects of my research involve the antigen-antibody affinity maturation process in the secondary adaptive immune response (whose description is inspired by the phenomenology -and techniques- reminiscent of driven trap models and weak ergodicity breaking) and the saturation (exaustion) of pattern recognition capability shown by the B-cell network seen in its relation to the macroscopic breakdown of self-defenses (old immune systems) with respect to the external pathogens or cancer development.
I wanna know more: CLICK HERE!
Selected publications:
I am involved primarily in the regulation and coordination roles played by the helpers (Th1 and Th2 families) and the suppressors, their links with cytokine productions and antigenic binding. I try to answer questions as "which is the computational power of the immune system?, how much information is required for a clonal expansion to start? how pattern recognition and information caption is split among membrane's receptors of a lymphocyte (i.e. between the complex TCR-MHC or BCR-antigen and the dialogue via cytokines)? and where memory is stored in the system and which is the maximal capacity in terms of bits for this memory? what happens once the memory is saturated? how do the highly parallel processing capabilities of the network work? what is an impaired immune system from an information theory perspective and what can be done to make it healthy again? how many antigen recognitions can be handled simultaneously in a lymphnode? How expressed is the mature repertoire? and more..."
Given the remarkable systemic similarities among immune and neural performances (as pattern recognition, decision capabilities, long and short term memory, etc.), the underlying bridges with other cognitive systems as Hopfield neural networks or Hebbian and Boltzmann machines is under development as well. However, beyond evident similarities, even deep differences exist between these two systems. At first the brain (or better several of its modules in the cortex) are built by highly connected neurons, while dialogues among lymphocytes -due to their chemical specificity- are extremely selectively, implying the overall network to be under-percolated (this has enormous consequences at the systemic level, i.e. the braim is mainly a high-definition serial processor while the immune system is mainly a low-definition parallel processor). Further, while there exist several times of neurons (as piramidal, stellar, etc.), mathematical models for neural networks usually consider one toy model of neuron (i.e. the Stein integrate and fire neuron) as, despite too rough, at the end of the day whatever the nature of the neuron, the mechanisms behind its firing (the genesis of a spike) are universal among the various types of real neurons. In Immunology this oversimplification can not be pursued as different types of lymphocytes do really behave in completely different ways (for instance B cells produce antibodies, while helpers and suppressors produce cytokines and killers fagocyte and induce lysis).
Hence, keeping in mind the strong differences, we can still try to search for structural analogies between these systems: in particular, among the results achieved so far, in our group we showed how the (spin-glass like) interactions among B-cells, helpers and suppressors can be mapped into an equivalent, effective, framework for helpers and suppressors alone, where they arrange themselves toward an associative Hebbian network able to instruct several effector clones at once: so to say, these lymphocytes are able to orchestrate their interactions with B-cells involved to cope pathogens time by time invading the host by "deciding" which is the best soldier (the best B-clone or the best ensemble of B-clones) to use for a particular fight (or for an ensemble of infections simultaneously acting on the host).
Remarkably we showed further how dilution in these networks -which is a biological must (becase, as a huge difference between immune and neural networks, immune networks -topologically- are dynamical and interactions among their nodes -clones- are achieved through short range diffusion mainly and not via "long-range spikes on static connections")- implies multitasking capabilities by the immune system as a whole: helpers and regulators are indeed able to elaborate several strategies for fighiting several antigens at the same time, which is a key feature of the immune response and is obtained through signaling toward the effector branch via low-information contents (via kytochiens), while pattern recognition (dense of information) is performed at the level of B and T cell-receptors.
Finally other aspects of my research involve the antigen-antibody affinity maturation process in the secondary adaptive immune response (whose description is inspired by the phenomenology -and techniques- reminiscent of driven trap models and weak ergodicity breaking) and the saturation (exaustion) of pattern recognition capability shown by the B-cell network seen in its relation to the macroscopic breakdown of self-defenses (old immune systems) with respect to the external pathogens or cancer development.
I wanna know more: CLICK HERE!
Selected publications:
- E. Agliari, A. Barra, F. Moauro, F. Guerra,
A thermodynamical perspective of immune capabilities,
J. Theor. Biol. (2011).
In this paper we show the key idea that has generated several other papers on the subject. The key idea is the following: under a microscope the interactions among B and T cells appear as spin-glass like (i.e. helpers and suppressors send/receive both positive and negative signalling molecues to/from the effector branches as B cells). However, crucially, at difference with neural networks -where we deal generically with one type of neuron- these networks are played by different actors (it is not possible to use a "generic lymphocyte" as different white cells perform very different tasks. Thus we are left with multi-partite spin-glasses in the immune network. Remarkably the thermodynamics of multiparties spin-glasses is the same of Hebbian associative networks within one party only. In other words a bipartite spin-glass, whose parties are B and T cells, behaves as a single-party associative network built of by T cells only and whose stored patterns are "instructions to be sent to B cells". This allowed numerous progresses in our undertanding of immune emergent properties. - E. Agliari, A. Barra, G. Del Ferraro, F. Guerra, D. Tantari,
Anergy in self-directed B-cells: a statistical mechanics perspective,
J. Theor. Biol. (2014).
This paper belongs to a special issue of the Journal of Theoretical Biology dedicated to Autoimmune Problems (in mammals).
In this paper we tried to address an ancient controversy regarding the genesis of anergy shown by self-directed B cells. In a nutshell, despite clonal selection (both positive and negative deletion) at ontogenesis, mature self-directed B cells do arrive in the final repertoire, however they are armless, that is, despite the (obvious) presence of self tissues, they do not undergo clonal expansion and do not attack host cells. How this quiescience is induced in these cells? Two theory appeared along the decades. Within the 70 and 80 the main trand for an explanation was a proper orchestration of the idiotypic network, by which self-directed B cells are kept anergic by the influence of all the other B cells forming the B network. In the 90 and 00 the main strand is the two signal model that states that two signals are needed (and must be integrated on the B cell's membrane) before a B cell is allowed to expand. The first signal is the antigen while the second signal is a "consensus" by helper T cells. The latter do not give their consensus to self-directed B cells and the absence of the second signal induces anergy in these cells. However, while there are no empirical doubt on the two signal model (while extensive sampling of the idiotypicnetwork is hopeless as it is made of by O(N^10) nodes), the two signal model does not explain how helpers "know" that a particular B cell is self-reactive. We have shown in this paper that these two routes are actually the same and that helpers "know" that a particular B cell is self-reactive thanks to the idiotypic network. - A. Annibale, E. Agliari, A. Barra, A.C.C. Coolen, D. Tantari, Immune networks:
Multitasking capabilitites near saturation,
J. Phys. A: Math & Theor. (2013).
In this paper we developed the first self-consistent theory for realistic immune networks. For "realistic" I mean the following: the network is at finite connectivity, the amount of B and T cells are comparable and, in principle, a fraction O(1) of the network can be simultaneously working (that means extensive paralle processing of antigens). This implies a far from trivial functional extension of the replica trick, whose solution is of a great complexity already at the replica symmetric level. The mathematics reported in this paper is actually of a very broad range of validity. It could play for instance with little to no modification also for the dual network of genes and proteins. This paper has been selected as a highly scientific sound paper by Institute of Physics Publishing. - E. Agliari, A. Barra, et al.,
Cancer driven dynamics of immune cells in a microfluidic enviroment,
Nature Scientific Reports (2014).
This is a paper where joint efforts by a broad collaboration are reported. Indeed three different institutions collaborated to give rise to this paper. the Department of Physics of Sapienza (with our group), the Institute for Nanophotonics of C.N.R. (with Businaro as head) and the Istituto Superiore di Sanità (with Mattei as head). In a nutshell we reported about the first systemic experiments on lymphocyte dynamics held on a LabOnChip technology. The CNR guys developed the LOC plate, the ISS guys contributed with all the lines of cells (both immune and cancerous cells) and we made our know-how on the expected dynamics.
We performed two kind of experiments: we analyzed how healty cells approach their target (i.e. the cancer) as well as how deranged cells move in the same situation. While -as expected- the latter are found to behave purely diffusively -as in standard random walk theory- the former share spatial jumps that do not respect TLC (almost Levy flights) meanwhile, overall, keep the evolution of the center of mass of the immune system moving of a straight ballistic motion. We are actually working the maximum entropy approach à la Bialek to infer the patterns of interactions that give rise to this highly coordinated motion.
3. Spin glasses and random networks
I work on fully connected spin-glasses since 2003 (when I studied the interpolating cavity field on the Sherrington-Kirkpatrick model for my laurea thesis and thereafter on the Viana-Bray spin glass for my PhD thesis). Among the results achieved, I tried to show how the cavity field and the stochastic stability techniques can be mapped one into the other (in gauge invariant systems only), and I developed a mathematical strategy to evaluate criticality both in fully connected mean field glassy models as well as in broad range of other mean field models (ranging from diluted to multi-partite systems).
I am currently interested in soft spins as they are natural candidates to describe continuous features in applications (i.e. antibody concentrations in blood tests) and in fluctuations theory on diluted networks (where these may play fundamental roles as triggers for non-conventional phase transitions).
Furthermore I am interested in graph theory (and its related termodynamics) when applied to glassy networks: recently we showed a way to generate small world networks trough a shift in the pattern definition of a suitably adapted Hopfield model (hence skipping the original rewiring procedure) and we worked out their equilibrium statistical mechanics, recovering several interesting cases (finitely connectivity regime, extreme dilution regime, fully connected, etc.).
At the end attention is due to bipartite (and in general multipartite) spin systems (and their equivalence with Hebbian and Boltzmann machines and, more generally, with autonomously-learning systems) where interpolating schemes and sum rules for replica symmetric free energy are now handly: in particular, here, we postulated the "ziggurat ansatz" for multipartite mean field spin glasses, an ansatz that extends the Parisi replica symmetry breaking (holding for the one-body case only) to spin glasses with multiple parties and, with the join effort of Dmitry Panchenko (available here), we have shown that this is indeed the real solution for these models. This has further interest in neural networks as the latter can be seen as special cases of multi-partite spin glasses.
Selected Publications:
I am currently interested in soft spins as they are natural candidates to describe continuous features in applications (i.e. antibody concentrations in blood tests) and in fluctuations theory on diluted networks (where these may play fundamental roles as triggers for non-conventional phase transitions).
Furthermore I am interested in graph theory (and its related termodynamics) when applied to glassy networks: recently we showed a way to generate small world networks trough a shift in the pattern definition of a suitably adapted Hopfield model (hence skipping the original rewiring procedure) and we worked out their equilibrium statistical mechanics, recovering several interesting cases (finitely connectivity regime, extreme dilution regime, fully connected, etc.).
At the end attention is due to bipartite (and in general multipartite) spin systems (and their equivalence with Hebbian and Boltzmann machines and, more generally, with autonomously-learning systems) where interpolating schemes and sum rules for replica symmetric free energy are now handly: in particular, here, we postulated the "ziggurat ansatz" for multipartite mean field spin glasses, an ansatz that extends the Parisi replica symmetry breaking (holding for the one-body case only) to spin glasses with multiple parties and, with the join effort of Dmitry Panchenko (available here), we have shown that this is indeed the real solution for these models. This has further interest in neural networks as the latter can be seen as special cases of multi-partite spin glasses.
Selected Publications:
- Adriano Barra,
Irreducible Free energy expansion and overlaps locking in mean field spin glasses,
J. Stat. Phys. (2006).
This is my first paper. In early 2002 ago I asked for a thesis to Professor Francesco Guerra, who became my first collaborator and one of my best friends, and he told me "you have to solve the SK model with a different route w.r.t. Parisi theory and we must learn something by your theory if you want to graduate with me". After I spent one year dealing with the genesis of a variant of the cavity approach nowadays known as "smooth cavity expansion", in middle 2003 I came up with a technique that, despite not able to give the entire solution of the SK model at low temperature, however was able to obtain an irreducible expansion for its free energy as well as the first rigorous calculation of the critical indexes (that have been found to coincide with those replica simmetric) and very famous ultrametric polynomial relations known as Ghirlanda-Guerra identitis. Thus I obtained my master and my first paper. - A. Barra, G. Genovese, F. Guerra, D. Tantari,
A solvable mean field model of a gaussian spin glass,
J. Phys. A (2014).
In this work we decided to investigate in full details a model that is expected (and indeed it is) replica symmetric, namely a spin glass whose couplings as well as spin variables are Gaussians. The motivation behind this inspection is two-fold. From one side, the Gaussian spin glass is the perfect banchmark where ideas and techniques can be easily tested (despite being not too trivial as the CW model). On the other side, as we have shown that associative networks (as neural and immune ones) can always be decomposed as linear combinations of two spin glasses, one with Ising spins and one with Gaussian spins, a clear picture of the Gaussian spin glass was needed even to increase our understanding of biological complexity in general terms. - Adriano Barra, Elena Agliari
Equilibrium statistical mechanics on correlated random graphs,
J. Stat. Mech. (2011).
Beyond the Erdos-Renyi graph, the prototype of a random graph, however in Nature other random or pseudo-random graphs do exist (and sometimes can actually be predominant). Among these novel times of graphs we find the scale free networks (not addressed here) and the small worlds. These have been discovered in late XX century firstly by Watts and Strogatz then extensively everywhere, however, due to their recent genesis in the scientific community, a statistical mechanical analysis of these network were missing and has been developed by various researchers, each one within his/her typical style. Here is our formulation of small world statistical mechanics: we present at first a glance at its topological properties, then we move to analyze the thermodynamics: statistical mechanics has been developed via interpolations on double stochastic stability and allowed a clear picture of the evolution of the first moment of the distribution of the order parameters. Field-theory driven fluctuation theory accounted in the final part of the paper also to give a clear picture of the critical behavior of the system, that undergoes a classical second order phase transition in the space of the tunable parameters. - Adriano Barra, Giuseppe Genovese, Francesco Guerra
Equilibrium statistical mechanics of bipartite spin systems.
J.Phys. A, (2011).
This research is motived by the key observation that associative networks, namely Hebbian systems, can always be decomposed as linear superpositions of two spin glasses. As a consequence, as the structure of the variational principles of thermodynamics -within the canonical ensemble- was missing for systems made of by interacting parties, this paper is devoted to fill this gap. We showed that the free energy of these systems -and the relative self-consistencies for their order parameters- can be derived within a novel minmax principle, a remarkable difference w.r.t. mono-party systems where the minmax leaves the place in favor of a simple sup.
4. Neural Networks and Cognitive Systems
My research interest in neural networks covers different aspects of learning and retrieval. In particular we developed the first neural network able to spontaneously move from serial processing to parallel processing, we shed lights on thermodynamical equivalence among Hofpfield networks and Boltzmann machines and, recently, we gave a complete description of the extensive multitasking processing where N neurons are able to cope with P \sim \alpha N retrieval in parallel without falling into spurious states typical of the underlyng glassy nature of these networks.
As sidelines I am interested in the exact equivalence of dynamics in spin glasses and Pavlovian learning mechanisms in neural networks) in particular in modular neural networks, which are available mixing the Hebb prescription for learning with the Dyson hierarchical topology or with the Barabasi-Albert scale-free graphs.
Lastly, we recently developed a clear scenario for neural networks beyond the mean field paradigm, in particular showing how neural networks embedded on a hierarchical topology (as those introduced by Dyson in ferromagnetic contexts via the renormalization group) may work both as serial processors as well as parallel processors, decreasing a bit the gap between theoretical modeling and biological reality.
I wanna know more: CLICK HERE!
Selected Publications:
As sidelines I am interested in the exact equivalence of dynamics in spin glasses and Pavlovian learning mechanisms in neural networks) in particular in modular neural networks, which are available mixing the Hebb prescription for learning with the Dyson hierarchical topology or with the Barabasi-Albert scale-free graphs.
Lastly, we recently developed a clear scenario for neural networks beyond the mean field paradigm, in particular showing how neural networks embedded on a hierarchical topology (as those introduced by Dyson in ferromagnetic contexts via the renormalization group) may work both as serial processors as well as parallel processors, decreasing a bit the gap between theoretical modeling and biological reality.
I wanna know more: CLICK HERE!
Selected Publications:
- Elena Agliari, Adriano Barra, Andrea Galluzzi, Francesco Guerra, Francesco Moauro,
Multitasking associative networks,
Phys. Rev. Lett, (2012).
This is the first letter where we posed the mathematical basis for a self-consistent statistical mechanical theory of parallel processing. Indeed, while Hopfield networks -harmonic oscillators for neural networks- see their neurons to work in parallel, the overall synergic result of their effort is the serial retrieval of a pattern of information (i.e. an image) per time. Here we show how to modify the kernel of the network in order to allow the system to handle contemporarily several patterns at once. In this pioneering work however only small load were considered, that means that a network composed of N neurons can retrieve simultaneously at most O(logN) patterns. The theory has been largely improved in further works and now we understood how extensive parallel processing is possible (that is retrieving O(N) patterns simultaneously) and we are able to control network's performances of various topologies, ranging from completely homogeneous to scale free.
- Adriano Barra, Giuseppe Genovese, Francesco Guerra,
The replica symmetric approximation of the analogical neural network,
J. Stat. Phys. (2010).
This work has two values. The first is that here is contained the generalization of stochastic stability to several parties: this technique found large usaged in the study of several other glassy systems. The second is that we have finally shown that neural networks whose patterns are analogic are not able to accomplish a satisfactory retrieval. This can be roughly understood as follows: consider the standard Hopfield model in the high load (close to saturation). We know -by a simple TLC argument- that if the amount of stored patterns goes too fast w.r.t. the way the thermodynamic limit on N is taken (that is for $\alpha$ larger than $\alpha_c$), then the Hebbian kernel becomes a i.i.d. N[0,1] and the system behaves very closely to the SK, hence retrieval is no longer achieved. Assuming patterns already analogic is in some sense similar to pushing the Hopfield network in the spin glass phase (beyond the black-out catastrophe): retrieval is lost by the proliferation of metastable states. While this was a controversy credo in the past, since 2010 it is now a simple theorem in Mathematical Physics. - Adriano Barra, Alberto Bernacchia, Pierluigi Contucci, Enrica Santucci,
On the equivalence among Hopfield neural networks and restricted Bolzmann machines,
Neural Net. (2012).
In Artificial Intelligence as well as in Neurobiology learning and retrieval are two separate aspects of cognition. Retrieval is made by the firing neurons, whose timescale ranges in O(milliseconds), while learning is achieved via synaptic rearrangments that take place on timescales O(days/weeks). As a consequence, usually when making retrieval, synapses are assumed quenched (i.e. this is the "adiabatic approximation" in standard Physics) and only neurons are allowed to thermalize. Then free energy minimization automatically encodes for spontaneous retrieval. Otherwise one usually skips details on neurons -when looking at network's learning- and study the evolution of the synapses (i.e. with Fokker-Planck-like approaches just to give a reference frame).
Thus, in disordered statistical mechanics, the harmonic oscillator for retrieval is the Hopfield network, while the harmonic oscillator for learning is the (eventually restricted) Boltzmann machine. In this paper we have shown that these two systems share several deep thermodynamical properties, hence we developed the first representational equivalence suggesting that retrieval and learning are not so disparate phenomena, rather they should be though of as "the two face of the medal of cognition". - Adriano Barra, Daniele Tantari, Giuseppe Genovese, Francesco Guerra,
How glassy are neural networks?
J. Stat. Mech. (2012).
In the past it was well known that neural networks were spin glasses, mainly for two aspects:
at first, more in philosophical sense, because frustration were present in the system and actually sharply due to frustration the system's free energy can be split into several (extensive, that means growing with N) valleys and each of these valleys can then be used to store information.
Then, also because the Hopfield network beyond saturation collapses on the SK model, that is the paradigm of a mean field spin glass.
Here we have shown several completely novel properties of neural networks that deeply change our way to see at these systems. Indeed we have proved that neural nets are always pure spin glasses, even within the retrieval region, but these spin glasses do not share the same Ising spins, rather they are build by Gaussian spins. This further puzzles the ultrametric decomposition of the stored memories as neural nets are mixed ensembles of Ising and Gaussian spin glasses: while the former are full RSB at low noise level, the latter are always RS, hence the validity tout-court of the Parisi scenario is not obvious. - Elena Agliari, Adriano Barra, Andrea Galluzzi, Francesco Guerra, Daniele Tantari, Flavia Tavani,
Retrieval capabilities of hierarchical networks: From Dyson to Hopfield.
Phys. Rev. Lett. (2014).
Neural network theory is developed at the mean field level. However we know that mean field is a (actually quite good) approximation, but real systems work beyond the mean field scheme. Due to recent advances in interpolatin technology, we are finally left with available theorems and methodologies that can start to tackle the complexity of neural processing beyond the mean-field paradigm. In this letter we studied the Hebbian proposal on the Dyson hierarchical structure. Remarkably, this network whose distance to biology is reduced, spontaneously switches from serial to parallel processing of information and is the first -and solely at the moment- network that share this capability with real brain.
5. Quantitative Sociology
Among the most proliferative branches of applied disordered statistical mechanics are those in quantitative sociology and the so called "econophysics". Mixing graph theory, stochastic processes and statistical mechanics with classical approaches as time series and econometrics, I am interested in developing data-driven models whose emergent features are able to capture (part of the) social and economical complexity of modern societies.
Regarding social interactions I started framing the McFadden ecometric theory of discrete choice within a statistical mechanics framework, that I successively extended covering also the Brock and Daurlauf picture on the importance of imitation (and more recent findings on competition): the whole is developed keeping into account topological features of society in agreement with Milgram and Granovetter
scenarios and the more recent findings by Watts and Strogats.
In particular I work on understanding key mechanisms underlying migrant's integration within a host community in line with Horizon2020 program: I studied the way in which their small-world interaction network naturally arises within the host community and how the network effects affect the social dynamics (investigating the way they get married among themselves and with the natives, or the way the get temporary and/or permanent jobs). Further I study their effects on international trading in relation to the theory of Economical Complexity.
I wanna know more: CLICK HERE!
Selected publications:
Regarding social interactions I started framing the McFadden ecometric theory of discrete choice within a statistical mechanics framework, that I successively extended covering also the Brock and Daurlauf picture on the importance of imitation (and more recent findings on competition): the whole is developed keeping into account topological features of society in agreement with Milgram and Granovetter
scenarios and the more recent findings by Watts and Strogats.
In particular I work on understanding key mechanisms underlying migrant's integration within a host community in line with Horizon2020 program: I studied the way in which their small-world interaction network naturally arises within the host community and how the network effects affect the social dynamics (investigating the way they get married among themselves and with the natives, or the way the get temporary and/or permanent jobs). Further I study their effects on international trading in relation to the theory of Economical Complexity.
I wanna know more: CLICK HERE!
Selected publications:
- Adriano Barra, Pierluigi Contucci, Rickard Sandell, Cecilia Vernia,
An analysis of a large database on immigration integration in Spain: The statistical mechanics of Social Action,
Nature Scientific Reports 4, 4174 (2014).
This is one of the first papers where statistical mechanics is extensively used as the main route to explore the "hot theme" of migrant's integration in a host society, that belongs to Social Complexity. The interest in these fields of research is stemmed by the large migratory fluxes (both intranational and international) -i.e. globalization- and has been so highlighted by our Governments that migrant's integration is a priority in the Horizon2020 agenda.
In this paper we studied four classical quantifiers (i.e. the social name for "order parameter") of Migration Sociology, namely marriages among mixed couples (one native and one migrant) and newborns from mixed parents (that is not strictkly the same for several reasons, while it is certainly correlated to the former) and labour (both temporarily and stable) given to migrants by the host country. However instead of using historical series (the classical approach), that analyzes these quantifiers versus time Q(t), we also extracted data on migrantion density evolution versus time d(t) and, with careful protocols (i.e. "data thermalization"), we inverted t(d) so to finally be able to infer the evolution of Q(d), where time no longer appears and that can be tackled from an equilibrium statistical mechanics perspective. Results on the teste case of Spain suggest that, while imitation is ongoing for marriages, this is not the case for works, but, probably more importantly, resuts suggest that the statistical mechanics approach may work even in these contexts. - Adriano Barra, Elena Agliari
A statistical mechanics approach to Granovetter theory,
Physica A 391, 3017-3026, (2011).
Among the milestones in Quantiative Sociology certainly we found Granovetter's papers on weak ties and Milgram's experiments on small worlds (before their formal mathematical characterization by Watts and Strogatz, that happened in recent times only, i.e. 1996): these pioneers have shown that many social networks are small worlds, roughly that they display a small diameter (it is possible to move from a node to another one into a few steps only, i.e. \sim log(N), with N the amount of nodes), high clustering coefficient (roughly they are formed by several communities highly adbundant of links within the communities, if compared with the amount of ties linking different communities) and the presence of weak ties, that is these "long ties" among different clusters of highly connected nodes (that are usually "fragile", once a weight is associated on the adiacenty matrix entries).
In this paper we have shown that all these properties, as well as a numer of others (i.e. the recovery of the "discrete choice with social interactions", namely the Brock and Daurlauf theory), are shared by these systems and opportune variants on theme of the Curie-Weiss model insisting on an Hebbian graph.
Stochastic dynamics, relaxation to equilibrium, equilibrium thermodynamics and graph properties are discussed. - Elena Agliari, Adriano Barra, Pierluigi Contucci, Rickard Sandell, Cecilia Vernia,
A stochastic approch for quantifying migrant's integration,
New Journal of Physics 16, 103034, (2014)
A different route -w.r.t. statistical mechanics of disordered systems- to modeling phenomena regarding migrant's integration within a host society is clearly paved by stochastic processes. In this paper we analyze the same quantifiers of the paper Nature Scientific Reports 4, 4174 (2014), but this time, stemming ideas and techniques from continous time random walk theory. In this approach we are interested in collecting "jumps" both in the temporal and in the spatial dimensions, where here time and space are played rispectively by migrant's density and quantifier in turn to observe (i.e. marriages, sons, temporary works, permanent jobs): once shown that the distributions for these jumps are always exponential for the quantifiers but range from exponential to log-normal (but not scale free) for the migrant' density it is possible to recover the previous behaviors predicting the evolution of the mean of these observable (i.e. there is a phenomenological similarity between the local square-root growh of the magnetization versus the temperature in the Curie-Weiss model and the square-root growth of the mean square displacement versus the time for a walker in classical -not biased and not trapped- random walk theory). Beyond recovering previous results from a different perspective, this route allows also to develop predictive tools as non-Markovian quantities: for instance the mean first passage time at work in this case reveals which will be the best/worst case (w.r.t. the average) in the expected amount of foreign students starting the school when migrant's density in the host society changes from, say 5% to 6% and this can be useful in planning and managing resources. - Adriano Barra, Pierluigi Contucci,
Toward a quantitative approach to migrants integration,
Europhys. Lett. 89, 68001, (2010).
In this paper we walked an analogy between bipartite spin glasses and Hopfield network. We have already shown in a series of papers that bipartite spin glasses behave as Hopfield networks (i.e. the share the same partition function!). In particular in the Amit-Gutfreund-Sompolinsky (AGS) theory of neural networks, a crucial quantity is alpha=P/N, where P is the amount of stored patterns and N the amount of neurons forming the net. AGS theory told us also that if alpha>alpha_c then the network is no longer able to retrieve.
Now we consider a bipartite model in which one party is made of by P migrants and the other by N natives. In general interactions between migrants and natives will be of both positive and negative signs [we generically have both good and bad memories of events in past dealing with migrants -as with any other thing:)-], hence the two parties may interact in a spin-glass way. If this is the case, however, than this system is an associative nework: indeed the N neurons/natives can interact among themselves and discuss about (i.e. retrieve) one of the possible P patterns/migrants. Remarkably this is possible only if \alpha<\alpha_c, or in other words, if the amount of migrants elapses a critical value, than host citizens will no longer able to distinguish them!
Note that in our theory for migrant's integration there is no concept of judgement. Judgements, both negative as well positive, are allowed (and actually appear as natural/emergent properties of the network) only if the ratio P/N < alpha_c, once this threshold is superated, judgments are no longer possible and the migrant becomes "integrated".
6. Biological cybernetics
I am interested in concrete implementations of cognitive paradigms within the organic matter, that is, to translate operatively what is learnt by statistical mechanics into suitably designed experiments (that usually are implemented on LabOnChip technology, that is provided to our group by the Institute for Nanotechnology of CNR and deal with lymphocyte dynamics, the latter usually provided to us by Istituto Superiore di Sanità). To this task a crystal-clear route from order parameters in statistical mechanics to concentrations (or spike recording) in biological matter (the former regarding lymphocyte, the latter for neurons) must be paved and this field of research has exactly this goal, a clear bridge between observables in statistical mechanics with observables in biochemical kinetics, coupled with the translation of the whole adaptive/cognitive statistical mechanical scenario in cybernetical terms.
To accomplish this task, recently, we showed how classical reaction kinetics (from Micaelis-Menten to Hill and Adair) may account for properly describing operational amplifiers, latches and flip-flops in electronics, while allosteric kinetics à la Monòd or à la Koshland handles spontaneously directly stochastic (bio)-logic gates as Yes, Not, Or, Nor, And, Nand (hence forming a logical base for further spontaneous information processing in biological matter). A unique formalism where saturation curves from chemical kinetics, self-consistencies from statistical mechanics and transfer functions from electronics collapse in full consistency has been finally achieved.
Further research steps in this branch aim to provide a clear cybernetical rationale of signal transduction for lymphocytes, e.g. signal cascades resulting from a stimulation of BCR/TCR receptors by the complex MHC-antigen: we plan to report soon further details on this subject.
I wanna know more: CLICK HERE!
Selected publications:
To accomplish this task, recently, we showed how classical reaction kinetics (from Micaelis-Menten to Hill and Adair) may account for properly describing operational amplifiers, latches and flip-flops in electronics, while allosteric kinetics à la Monòd or à la Koshland handles spontaneously directly stochastic (bio)-logic gates as Yes, Not, Or, Nor, And, Nand (hence forming a logical base for further spontaneous information processing in biological matter). A unique formalism where saturation curves from chemical kinetics, self-consistencies from statistical mechanics and transfer functions from electronics collapse in full consistency has been finally achieved.
Further research steps in this branch aim to provide a clear cybernetical rationale of signal transduction for lymphocytes, e.g. signal cascades resulting from a stimulation of BCR/TCR receptors by the complex MHC-antigen: we plan to report soon further details on this subject.
I wanna know more: CLICK HERE!
Selected publications:
- Elena Agliari, Adriano Barra, Raffaella Burioni, Aldo Di Biasio, Guido Uguzzoni,
Collective behaviors: From biochemical kinetics to electronic circuits,
Nature Scientific Reports (2013)
This paper, as the next one, belong to a common project that is a rigorous translation in cybernetical terms of the dialogues inside chemical networks. The idea in a nutshell is the following: Neural networks are made of neurons that interact via spiking, that is electric signals. However, we are showing systematically that we can "learn and grasp" from neural network theory in order to infer and apply techniques even in the world of immune networks. However the latter are not elettric networks, there dialogues are pushed forward by diffusion of signalling molecules, thus it is an example of biochemical network. But if this is the case, we need to understand with is the bridge between order parameters in statistical mechanics (i.e. magnetizations, eventually Mattis one) and order parameters in chemical kinetics (I.e. reactant concentrations). In this work we show that there is a 1:1 structural and formal mapping among these worlds. In particular, for "J>0" (i.e. consider the CW model), we prove that the self-consistency (that links magnetization and external field) plays in statistical mechanics as the saturation curve of a cooperative biochemical reaction (where we plot the fraction of binded sites -magnetization- as a function of the substrate -log(ext.field)- ) and, further, that this extension can reach operational amplifier and in particular its transfer function, where the input voltage plays as the field and the output voltage as the magnetization. The bridge is then extended to ultrasensitive kinetics (and low temperature ferromagnets in statistical mechanics and analog to digital converters in electronics) and finally to anti-cooperative reactions, where the links -not surprisingly- is with antiferromagnets in statistical mechanics and -less trivially- with flip-flops in electronics. Thus we have shown that with ensembles of positive and negative kinetics (that is with positive and negative ferromagnets) I obtain amplifiers and flip-flops, that are the "bricks" of artificial neural networks: indeed with ferro and antiferromagnets, I build a spinglass in statistical mechanics and neural networks are just one kind of spin glass, thus there is overall perfect general merging of concepts. - Elena Agliari, Matteo Altavilla, Adriano Barra, Lorenzo Dello Schiavo, Evgeny Katz
Notes on stochastic (bio)-logic gates: The role of allosteric cooperativity,
Nature Scientific Reports (2015).
This paper extends the previous one in the following sense: there we considered only cases with one single magnetic field in statistical mechanics, or one substrate in biochemistry or one input volgate in electronics, but chemistry is by far richer and several reactions work with two sources of information (i.e. two reactants). Paradigmatic examples of this are enzymes that work as large proteins and usually link two different substrate (to catalyze their reaction). Thus we developed statistical mechanical models with two fields, we have shown that they succesfully code for the relative biochemistry and, even more remarkably, we analyzed the logical properties of these systems, that is their intrinsic capability to work as (bio)-logic stochastic gates. We have shown, beyond the NOT and YES gates that are single field gates, that two-inputs kinetics naturally encode the four gates AND, NAND, OR and NOT thus a logic base is available within each cell and here is presented both the underlying mathematics and a crystal clear exposition of the bridge among various disciplines: the long term goal is to reduce the complexity of the immune system at a circuital equivalence thus allowing ourselves to use on this biologic information processing network tooks developed by engineers for the main strand that has been electronics in modern digital computing. - Elena Agliari, Adriano Barra,
A Hebbian approach to complex network generation,
Europhys. Lett. 94, 10002, (2011).
In this letter we show that simply with a shift in the pattern entries of an Hopfield model, that is instead of having patterns at values +1 and -1, we consider patterns at values +1 and 0, this implies turning frustration into dilution and the resulting network, a diluted ferromagnet, is not a simple Erdos-Renyi network because the couplings are Hebbian in this case. We have shown here that Hebbian networks are small worlds and we solved both the related stochastic dynamics, the equilibrium statistical mechanics and the relative fluctuation theory. This model may work as a toy-model for a number of biological networks. - Elena Agliari, Adriano Barra, Raffaella Burioni, Aldo Di Biasio,
Mean field cooperativity in chemical kinetics,
Theor. Chem. Acc. 131, 1104, (2012).
This is a first paper where we (Elena and myself) decided to "open the Pandora Box", where "the Pandora box" is the last chapter of the book by C.J. Thompson "Mathematical statistical mechanics" where he used a 1D Ising chain to model reaction kinetics of hemoglobin: we extended this scenario to mean fields because 1D do not have phase transitions. This was not a problem at the time of that book (70) because ultrasensitive kinetics were still to be discovered but nowadays we need to deal with quasi-discontinuous reactions ("quasi" because, at difference with physicists, in chemistry they work mainly experimentally and they -obviously- do not consider the thermodynamic limit, thus their plot remain "analytic" despite with obvious discontinuous boundaries whenever the system under consideratio would diverge). In this paper we made the first equivalences that then has driven the next steps discussed above.