(My apologies if you get multiple copies of this.)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
A panel discussion on the question:
"DOES CONNECTIONISM PERMIT READING OF RULES FROM A NETWORK?"
took place this July at IJCNN'2000 (International Joint Conference on Neural Networks) in Como, Italy. Following persons were on the panel:
1) DAN LEVINE; 2) LEE GILES; 3) NOEL SHARKEY; 4) ALESSANDRO SPERDUTI; 5) RON SUN; 6) JOHN TAYLOR
7) STEFAN WERMTER; 8) PAUL WERBOS; 9) ASIM ROY.
This was the fourth panel discussion at these IJCNN conferences on the fundamental ideas of connectionism. The abstract below summarizes the issues/questions that were addressed by this panel.
The fundamental contention was that the basic connectionist framework as outlined by Rumelhart et al. in their many books and papers has no mechanism for rule extraction (reading of weights, etc. from a network) or rule insertion (constructing and embedding rules into a neural network) as is required by many rule-learning mechanisms (both symbolic and fuzzy ones). This is not a dispute about whether humans can and do indeed learn rules from examples or whether such rules can indeed be embedded in a neural network. It is about whether the connectionist framework let's one do that; that is, whether it allows one to create the algorithms required for doing such things as rule insertion and rule extraction. As pointed out in the abstract below, the cell-based distributed control mechanism of connectionism empowers only individual cells (neurons) with the capability to modify/access the connection strengths and other parameters of a network; no other outside agent can do that, as is required by rule insertion and extraction techniques. These rule-learning techniques are, in fact, moving away from the cell-based distributed control notions of connectionism and using broader control theoretic notions where it is assumed that there are parts of the brain that control other parts. In fact, it can be shown that every connectionist algorithm, from back-propagation to ART to SOM, goes beyond the original framework of cell-based distributed control and uses the broader control theoretic notion that there are parts of the brain that control other parts. There is, of course, nothing wrong with this broader control theoretic notion because there is enough neurobiological evidence about neurotransmitters and neuromodulators to support it.
This debate once more points out the limitations of connectionism. Ron Sun notes that "Clearly the death knell of strong connectionism has been sounded." With regard to rule extractors in rule-learning schemes, John Taylor notes that "There do not seem to be there similar rule extractors of the connection strengths." On the same issue, Paul Werbos says: "But I would not call it a "neural network" method exactly (even though neural net learning is used) because I do not believe that real organic brains contain that kind of hardwired readout device." Noel Sarkey says: "Currently there seems little reason (or evidence) to even think about the idea of extracting rules from our neural synapses - otherwise why can we not extract our bicycle riding rules from our brain" and "However, the relationship between symbolic rules and how they emerge from connectionist nets or even whether or not they really exist has never been resolved in connectionism."
For those interested, summaries of prior debates on the basic ideas of connectionism are available at the CompNeuro website at Caltech. Here is a partial list of the debate summaries available there.
www.bbb.caltech.edu/compneuro/cneuro99/0079.html - Some more questions in the search for sources of control in the brain
www.bbb.caltech.edu/compneuro/cneuro98/0088.html - BRAINS INTERNAL MECHANISMS - THE NEED FOR A NEW PARADIGM
www.bbb.caltech.edu/compneuro/cneuro97/0069.html - COULD THERE BE REAL-TIME, INSTANTANEOUS LEARNING IN THE BRAIN?
www.bbb.caltech.edu/compneuro/cneuro97/0043.html - CONNECTIONIST LEARNING: IS IT TIME TO RECONSIDER THE FOUNDATIONS?
www.bbb.caltech.edu/compneuro/cneuro97/0040.html - DOES PLASTICITY IMPLY LOCAL LEARNING? AND OTHER QUESTIONS
www.bbb.caltech.edu/compneuro/cneuro96/0047.html - Connectionist Learning - Some New Ideas/Questions
Some of the summaries are also available at the CONNEC_L website:
[ More results from www.shef.ac.uk </search?hl=en&lr=&safe=off&output=washingtonpost&q=site:www.shef.ac.uk+Asim+Roy> ]
Asim Roy
Arizona State University
---------------------------------------------------------------------------
DOES CONNECTIONISM PERMIT READING OF RULES FROM A NETWORK?
Many scientists believe that the symbolic (crisp) and fuzzy (imprecise and vague) rules learned, used and expressed by humans are embedded in the networks of neurons in the brain - that these rules exist in the connection weights, the node functions and in the structure of the network. It is also believed that when humans verbalize these rules, they simply "read" the rules from the corresponding neural networks in their brains. Thus there is a growing body of work that shows that both fuzzy and symbolic rule systems can be implemented using neural networks. This body of work also shows that these fuzzy and symbolic rules can be retrieved from these networks, once they have been learned, by procedures that generally fall under the category of rule extraction. But the idea of rule extraction from a neural network involves certain procedures - specifically the reading of parameters from a network - that are not allowed by the connectionist framework that these neural networks are based on. Such rule extraction procedures imply a greater freedom and latitude about the internal mechanisms of the brain than is permitted by connectionism, as explained below.
In general, the idea of reading (extracting) rules from a neural network has a fundamental conflict with the ideas of connectionism. This is because the connectionist networks by "themselves" are inherently incapable of producing the "rules," that are embedded in the network, as output, since the "rules" are not supposed to be the outputs of connectionist networks. And in connectionism, there is no provision for an external source (a neuron or a network of neurons), in a sense a third party, to read the rules embedded in a particular connectionist network. Some more clarification perhaps is needed on this point. The connectionist framework, in the use mode, has provision only for providing certain inputs (real, binary) to a network through its input nodes and obtaining certain outputs (real, binary) from the network through its output nodes. That is, in fact, the only "mode of operation" of a connectionist network. In other words, that is all one can get from a connectionist network in terms of output - nothing else is allowed in the connectionist framework. So no symbolic or fuzzy rules can be "output" or "read" by a connectionist network. The connectionist network, in a sense, is a "closed entity" in the use mode; no other type of operation, other than the regular input-output operation, can be performed by or with the network. There is no provision for any "extra or outside procedures" in the connectionist framework to examine and interpret a network, to look into the rules it's using or the internal representation it has learned or created. So, for example, the connectionist framework has no provision for "reading" a weight from a network or for finding out the kind of rule/constraint learned by a node. The existence of any "outside procedure" for such a task, in existence outside of the network where the rules are, would go against the basic connectionist philosophy. Connectionism has never stated that the networks can be "examined and accessed in ways" other than the input-output mode.
So there is nothing in the connectionist framework that lets one develop procedures to read and extract rules from a network. So a rule extraction procedure violates in a major way the principles of connectionism by invoking a means of extracting the weights and rules and other information from a network. There is no provision/mechanism in the connectionist framework for doing that.
So the whole notion of rules existing in a network, that can be accessed and verbalized as necessary, is contradictory to the connectionist philosophy. There is absolutely no provision for "accessing networks/rules" in the connectionist framework. Connectionism forgot about the need to extract rules.
----------------------------------------------------------------------------------------------------------------------------------------------------
LEE GILES
Early Connectionism/NNs
- McCulloch, Pitts (MC) 40's: models that were basically circuit design, suggestive but very primitive.
- Kleene, 50's: MC networks as regular expressions and grammars. Early AI.
- Minsky, 60's: MC networks as logic, automata, sequential machines, design rules. More AI. (not the perceptron work!) Foundations of early high level VLSI design.
Early connectionism/NNs always had rules and logic as part of their philosophy and implementation.
Late 20th Century Connectionism/NNs
- Rules - vital part of AI.
- Empirical & theoretical work on rules extractable, encodeable and trainable in many if not all connectionist systems.
- Most recent work in data mining
- Future work in SVMs
Rules are more important but not essential in some applications; natural language processing, expert systems and speech processing systems used in many applications.
21st Century Connectionism/NNs
- Knowledge & information discovery and extraction
- Knowledge prediction
- New connectionism/NN - challenges
- Applications will continue to be important
- Cheap and plentiful data will be everywhere
- Text
- Nontext - sensor, audio, video, etc.
- Pervasive computing and information access
- New connectionism/NN future?
- Integration philosophically, theoretically and empirically with other areas of AI, computer and engineering science continues (rules will become more important)
- Biology and chips will play a new role
Rules and laws are important
Foundations of Connectionism/NNs
- Rules were always a theoretical and philosophical part of the connectionist/nn models.
- If/then rules, automata, graphs, logic
- Importance?
- Comfort factor
- New knowledge
- Autonomous systems - communication
---------------------------------------------------------------------------------------------------------------------------------------------------
DAN LEVINE
There are really two basic types of problems that involve encoding rules in neural networks. One type involves inferring rules from a series of interactions with the environment that entail some regularity. An example would be a cognitive task (such as the Wisconsin Card Sorting Test used by clinical neuropsychologists) in which a subject is positively reinforced for certain types of actions and needs to discern the general rule guiding the actions which will be rewarded. The other type of problem involves successfully performing cognitive tasks that are guided by an externally given rule. One example is the Rapid Information Processing task, in which the subject is instructed to press a key when he or she sees three odd or three even digits in a row. In other words, the neural network needs to translate the explicit verbal rule into connection strengths that will guide motor performance in a way that accords with that rule.
The first type of problem has already been simulated in a range of neural networks including some by my own group and by John Taylor's group. These networks typically require modulatory transmitters that allow reward signals to bias selective attention or to selectively strengthen or weaken existing rule representations. Modulatory transmitters are also involved in models in progress of the second type of problem. In this case, activation of a rule selectively biases appropriate representations of and connections among working memory representations of objects, object categories, and motor actions relevant to following the particular rule.
The framing of the question, "Does connectionism allow ...," suggests implicitly that "connectionism" refers to a specified class of neural architectures. However, connectionist and neural network models have been in the last several years increasingly less restricted in their structures. Modelers who began working within specific "schools" such as adaptive resonance (like my own group and Stephen Grossberg's) or back propagation (like Jonathan Cohen's group) have developed models that are guided as much, if not more, by known brain physiology and anatomy as by the original "modeling school." Hence the question should be rephrased "What form of connectionism allows ... ."
----------------------------------------------------------------------------------------------------------------------------------------------------
NOEL SHARKEY
One of the main themes of 1980s connectionism was the unconscious application of rules. This comes from Cognitive Psychology (where many of the major players started). There are very many charted behaviors, such as reading or riding a bicycle, where participants can perform extremely well and yet cannot explicitly state the rules that they are using. One of the main goals of Cognitive Psychologists was to find tasks that would enable them to probe at unconscious skills. Such skills were labeled as "automatic" in contrast to the slower, more intensive "controlled" or "strategic" processes. From the limited amount that psychologists talk beyond their data, strategic processes were vaguely considered to have something to do with awareness or conscious processes. This was a hot potato to be avoided.
But there is no single coherent philosophy covering all of connectionism. Researchers in the 1980s, including myself, began to experiment with methods for extracting the rules from connectionist networks. This was partly motivated by psychological considerations but mainly to advance the field in computing and engineering terms. In my lab, the interest in rule extraction was to help to specify the behavior of physical systems, such as engines, which are notoriously difficult to specify in other ways. For example, if a neural network could learn to perform a difficult-to-specify task, then, if rules could be extracted from that net, a crude specification could be begun.
However, the relationship between symbolic rules and how they emerge from connectionist nets or even whether or not they really exist has never been resolved in connectionism. It seems clear that we can propositionalize our rules and pass them on to other people. Simple rules such as, "eating is not allowed in this room" appear to be learned instantly from reading them in linguistic form, yet we have not seen a universally accepted connectionist explanation for this type of phenomenon and we certainly do not extract these rules from our nervous system after the fact. Now imagine we do extract rules directly from our brains, how would this be done. If we follow from the lessons of rule extraction techniques for neural networks, there are two distinctive methods which may be called internal and external. Internal is where the process of extraction operates on network parameters such as weight values or hidden unit activations. External is where the mechanism uses the input and output relation to calculate the rules - it is assumed that the neural network will have filtered out most of the noise. Currently there seems little reason (or evidence) to even think about the idea of extracting rules from our neural synapses - otherwise why can we not extract our bicycle riding rules from our brain. It seems that the only real option would be to "run the net" and calculate the rules from the input output relations. Nonetheless, this is not a well informed answer, nor is there likely to be one at present. This is an issue that needs considerably more research.
----------------------------------------------------------------------------------------------------------------------------------------------------
ALESSANDRO SPERDUTI
Before arguing about the possibility to extract rules from a neural network, the concept itself of "rule" should be clarified. In fact, when
talking about rules in this context, everybody has in mind the concept of "symbolic rule", i.e., a rule that involves discrete entities. Moreover, the semantics of these entities is defined by a subjective "interpretation" function.
However, the concept of rule is far more general and it can involve in general any kind of entity or variable, subject to the constraint that a
finite description (i.e., representation) of the entity exists and can be used. Thus, a rule involving continuous variables and/or entities is as
well legitimate, and in many cases useful. Consequently, the question about the capability of a connectionist system to capture "symbolic rules" in such a form to permit easy reading is conditional to the nature of the learned function: if it is discrete, the posed question is meaningful. Assuming a discrete nature for the learned function, however, there is no guarantee that a trained neural network will encode the function in a way that allows easy reading of "symbolic rules", whose representation, by the way, is in principle arbitrary with respect to the representational primitives (neurons) of the neural network.
----------------------------------------------------------------------------------------------------------------------------------------------------
RON SUN
Many early connectionist models have some significant shortcomings. For example, the limitations due to the regularity of their structures led to, e.g., difficulty in representing and interpreting symbolic structures (despite some limited successes that we have seen). Other limitations are due to learning algorithms used by such models, which led to, e.g., lengthy training (requiring many repeated trials); complete I/O mappings must be known a priori; etc. There are also limitations in terms of biological relevance. For example, these models may bear only remote resemblance to biological processes; they are far less complex than biological NNs, and so on. In coping with these difficulties, two forms of connectionism emerged: Strong connectionism adheres strictly to the precepts of connectionism, which may be unnecessarily restrictive and incur huge cost for some symbolic processing. On the other hand, weak connectionism (or hybrid connectionism) encourages the incorporation of both symbolic and subsymbolic processes: reaping the benefit of connectionism while avoiding its shortcomings. There have been many theoretical and practical arguments for hybrid connectionism; see e.g. Sun (1994).
In light of this background, how do we answer the question of whether there can be ``rule reading" in connectionist models? Here is my
three-fold answer: (1) Psychologically speaking, the answer is yes. For example, Smith, Langston and Nisbet (1992), Hadley (1990), Sun
(1995) presented strong cases for the existence of EXPLICIT rules in psychological processes, based on psychological experimental data,
theoretical arguments, thought experiments, and cognitive modeling. If connectionist models are to become general cognitive models, they
should be able to handle the use and the learning of such explicit rules too. (2) Methodologically speaking, the answer is also yes. Connectionism is merely a methodology, and not an exclusive one --- to be used to the exclusion of other methodologies. Considering
our lack of sufficient neurobiological understanding at present, a dogmatic or strict view on ``neural plausibility" is not warranted. (3) Computationally speaking, the answer is again yes. By now, we know that we can implement ``rule reading" in many ways computationally, e.g., (a) in symbolic forms (which leads to hybrid connectionism), or (b) in connectionist forms (which leads to connectionist implementationalism). Some such implementations may have as good neurobiological plausibility as any other connectionist models.
The key point is: To remove the strait-jacket of strong connectionism: we should advocate (1) methodological connectionism, treating it as
one possible approach, not to the exclusion of others. and (2) weak connectionism (hybrid connectionism), encouraging the incorporation
of non-NN representations and processes. Clearly, the death knell of strong connectionism has been sounded. It's time for a more open-minded framework in which we conduct our research. My own group has been conducting research in this way for more than a decade. For the work by my group along these lines, see http://www.cecs.missouri.edu/~rsun
----------------------------------------------------------------------------------------------------------------------------------------------------
JOHN TAYLOR
Getting a Connectionist Network to Explain its Rules.
JG Taylor, Dept of Mathematics, King's College, Strand, London WC2R2LS, UK. email: [EMAIL PROTECTED]
Accepting that rules can be extracted from trained neural networks by a range of techniques, I first addressed the problem of how this might occur in the brain. There do not seem to be there similar rule extractors of the connection strengths. In the brain are two extremes: implicit and explicit rules. Implicit skills, which implement rules in motor responses, are not based on an explicit knowledge of the rules implemented by the neural networks of the motor cortex. It is in explicit rules, as supported by language and the inductive/deductive process that
rules are created by human experience.
Turning to language, I described what is presently known from brain imaging about the coding of semantics and syntax in sites in the brain. These both make heavy use of the frontal recurrent cortico-thalamo-NRT circuits, and it can be conjectured to be the architecture used to build phrase structure analysers (through suitable recurrence), guided by 'virtual actions'. These are the basis for syntactic rules and also rules for causal inference, as seen in what is called 'predictive coding' in frontal lobes in monkeys. Thus rule development is undoubtedly supported by such architectures and styles of processing. There is no reason why it cannot be ultimately be implemented in a connectionist framework.
Such a methodology would enable a neural system to learn to talk about, and develop, its own explicit rules (although never the implicit ones), and hence solve part of the problem raised by Asim Roy. Implicit rules can be determined by the rule-extraction methods I noted at the beginning.
---------------------------------------------------------------------------------------------------------------------------------------------------
PAUL WERBOS
Asim Roy has asked us to address many very different, though related issues. A condensed response: (1) A completely seamless interface between rule-based "white box" descriptions and neural net learning techniques already exists. I have a patent on "elastic fuzzy logic" (see Gupta and Sinha eds); Fukuda and Yaeger have effectively applied essentially the same method. But I would not call it a "neural network" method exactly (even though neural net learning is used) because I do not believe that real organic brains contain that kind of hardwired readout device. (2) Where, in fact, DOES symbolic reasoning arise in biology? Some of my views are summarized in the book "The Evolution of Human Intelligence" (see www.futurefoundation.org). Curiously, there is a connection to a previous panel Asim organized, addressing memory-based learning. In the most primitive mammal brains, I theorized back in 1977 that there is an interplay between two levels of learning: (1) a slow-learning but powerful system which generalizes from current experience AND from memory; (2) a fast-learning but
poorly-generalizing heteroassociative memory system. (e.g. See my chapter in Roychowdhury et al, Theoretical Advances...). At IJCNN2000, Mike Denham described the "what when where" system of the brain. I theorize that some (or all) primates extended the heteroassociative memory system, to include "who what when where," using mirror neurons to provide an encoding of the experience of other primates. In other words, even monkeys probably have the power to generalize from the (directly observed) experience of OTHER MONKEYS,
which they can reconstruct without any higher reasoning faculties. I theorize that human intelligence is basically an extension of this
underlying capability, based on a biological system to reconstruct experience of others communicated first by dance (as in the Bushman dance), and later by "word movies." Symbolic reasoning ala Aristotle and Plato, and propositional language ala English, are not really biologically based as such, but learned based on modern culture, and rooted in the biology which supports dance and word movies. If there can be such a thing as truly biologically rooted symbolic/semiotic intelligence, we aren't there yet; modern humanity is only a kind of missing link, a halfway house between other primates and that next level. (For more detail, see the last chapter of my book "The Roots of Backpropagation," Wiley 1994, which also includes the first published work on true backpropagation, and the chapter in Kunio Yasue et al eds, ...Consciousness... forthcoming from John Benjamins.)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
STEFAN WERMTER
Linking Neuroscience, Connectionism and Symbolic Processing
Does connectionism permit reading of rules from a network? There are at least two main answers to this question. Researchers from Knowledge Engineering and Representation would argue that it has been done successfully, that it is useful if it helps to understand the networks, or they might not even care whether reading is part of connectionism or external symbolic processes. Connectionist representations can be represented as symbolic knowledge at higher abstraction levels. Symbolic extraction may be not part of connectionism in the strict sense, but symbolic knowledge can emerge from connectionist networks. This may lead to a better understanding and also to the possibility for combining connectionist knowledge with symbolic knowledge sources.
Researchers from Cognitive Science, Neuroscience, on the other hand, would argue that in real neurons in the brain there is no symbolic reading mechanism, that symbolic processing emerges based on dynamics of spreading of activation in cortical cell assemblies and that there may be rule-like behavior emerging from neural elements. It would be useful in the future to explore constraints and principles from cognitive neuroscience for building more plausible neural network architectures since there is a lot of new evidence from fmri, eeg, meg experiments. Furthermore, new computational models of spiking neural networks, pulse neural networks, cell assemblies have been designed which promise to link neuroscience with connectionist and even symbolic processing. We are leading the exploration of such efforts in the EmerNet project www.his.sunderland.ac.uk/emernet/. Computational models can benefit from emerging vertical hybridization and abstraction: 1. The symbolic abstraction level is useful for abstract reasoning but lacks preferences. 2. The connectionist knowledge has preferences but still lacks neuroscience reality. 3. Neuroscience knowledge is biologically plausible but architecture and dynamic processing are computationally extremely complex. Therefore we argue for an integration of all three levels for building neural and intelligent systems in the future.
**********************************************************************************************************
BIOSKETCHES
-----------------------------------------------------------------------------------------------------------------------------------------------------------
DAN LEVINE
Web site: www.uta.edu/psychology/faculty/levine
-----------------------------------------------------------------------------------------------------------------------------------------------------------
LEE GILES
http://www.neci.nj.nec.com/homepages/giles/html/bio.html
--------------------------------------------------------------------------------------------------------------------------------------
NOEL SHARKEY
Noel Sharkey is an interdisciplinary researcher. Currently a full Professor in the department Computer Science at the university of Sheffield, he holds a Doctorate in Experimental Psychology, is a Fellow of the British Computer Society, a Fellow of the Institution of Electrical Engineers, and a member of the British Experimental Psychology Society. He has worked as a research associate in Computer Science at Yale University, USA, with the AI and Cognitive Science groups and as a senior research associate in psychology at Stanford University, USA, where he has also twice served as a visiting assistant professor. His other jobs have included a "new blood" lecturship (English assistant professor) in Language and Linguistics at Essex University, U.K. and a Readership in Computer Science at Exeter. His editorial work includes Editor-in-Chief of the journal Connection Science, editorial board of Robotics and Autonomous Systems, and editorial board of AI Review. He was Chairman of the IEE professional group A4 (AI) and founding chairman of IEE professional group A9 (Evolutionary and Neural Computing). He has edited special issues on modern developments in autonomous robotics for the journals Robotics and Autonomous Systems, Connection Science, and Autonomous Robots. Noel's intellectual pursuits are in the area of biologically inspired adaptive robotics. In recent years Noel has been involved with the public understanding of science, engineering, technology and the arts.
He makes regular appearances on TV as judge and commentator of robot competitions and is director of the Creative Robotics Unit at Magna (CRUM) with projects in flying swarms of robots and in the evolution of cooperation in collective robots.
-------------------------------------------------------------------------------------------------------------------------------------------------------------
ALESSANDRO SPERDUTI
Alessandro Sperduti received his education from the University of Pisa, Italy ("laurea" and Doctoral degrees in 1988 and 1993, respectively, all in Computer Science.) In 1993 he spent a period at the International Computer Science Institute, Berkeley, supported by a postdoctoral
fellowship. In 1994 he moved back to the Computer Science Department, University of Pisa, where he was Assistant Professor, and where he presently is Associate Professor. His research interests include pattern recognition, image processing, neural networks, hybrid systems. In the field of hybrid systems his work has focused on the integration of symbolic and connectionist systems. He contributed to the organization of several workshops on this subject and he served also in the program committee of conferences on Neural Networks. Alessandro Sperduti is the author or co-author of around 70 refereed papers mainly in the areas of Neural Networks, Fuzzy Systems, Pattern Recognition, and Image Processing. Moreover, he gave several tutorials within international schools and conferences, such as IJCAI `97 and IJCAI `99. He acted as Guest Co-Editor of the IEEE Transactions on Knowledge and Data Engineering for a special issue on Connectionist Models for Learning in Structured Domains, and of the journal Cognitive Systems Research for a special issue on Integration
of Symbolic and Connectionist Information Processing Systems.
-------------------------------------------------------------------------------------------------------------------------------------------------------------
RON SUN
Ron Sun is an associate professor of computer engineering and computer science at the University of Missouri-Columbia. He received his
Ph.D in 1991 from Brandeis University. Dr. Sun's research interests center around the study of intellegence and cognition, especially in the areas of hybrid neural networks model, machine learning, and connectionist knowledge representation and reasoning, He is the author of over 100 papers, and has written, edited or contributed to 15 books, including authoring the book {\it Integrating Rules and Connectionism for Robust Commonsense Reasoning}. and co-editing {\it Computational Architectures Integrating Neural and Symbolic Processes}. For his paper on models of human reasoning, he received the 1991 David Marr Award from Cognitive Science Society
He organized and chaired the Workshop on Integrating Neural and Symbolic Processes, 1992, and the Workshop on Connectionist-Symbolic Integration, 1995, as well as co-chairing the Workshop on Cognitive Modeling, 1996 and the Workshop on Hybrid Neural Symbolic Systems, 1998. He has also been on the program committees of the National Conference on Artificial Intelligence (AAAI-93, AAAI-97, AAAI-99), International Joint Conference on Neural Networks (IJCNN-99 and IJCNN-2000), International Two-Stream
Conference on Expert Systems and Neural Networks, and other conferences, and has been an invited/plenary speaker for some of them.
Dr. Sun is the editor-in-chief of Cognitive Systems Research (Elsevier). He also serves on the editorial boards of Connection Science, Applied Intelligence, and Neural Computing Surveys. He was a guest editor of a special issue of the journal Connection Science and a special issue of IEEE Transactions on Neural Networks, both on hybrid intelligent models. He is a senior member of IEEE.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
JOHN TAYLOR
Trained as a theoretical physicist in the Universities of London and Cambridge. Positions in Universities in the UK, USA, Europe in physics and mathematics. Created the Centre for Neural Networks at King's College, London, in 1990, and is still its Director. Appointed Professor of Mathematics, King's College London in 1972, and became Emeritus Professor of Mathematics of London University in 1996. Was Guest Scientist at the Research Centre in Juelich, Germany, 1996-8, working on brain imaging and data analysis. Has been consultant in Neural Networks to several companies. Is presently Director of Research on Global Bond products and Tactical Asset Allocation for a financial investment company involved in time series prediction and European Editor-in-Chief of the journal Neural Networks. He was President of the International Neural Network Society (1995) and the European Neural Network Society (1993/4). He is also editor of the series Perspectives in Neural Computing. Has been on the Advisory Board of the Brain Sciences Institute, RIKEN in Tokyo since 1997.
Has published over 500 scientific papers (in theoretical physics, astronomy, particle physics, pure mathematics, neural networks, higher cognitive processes, brain imaging, consciousness), authored 12 books, edited 13 others, including the titles When the Clock Struck Zero (Picador Press, 1994), Artificial Neural Networks (ed, North-Holland, 1992), The Promise of Neural Networks (Springer, 1993), Mathematical Approaches to Neural Networks (ed, Elsevier, 1994), Neural Networks (ed, A Waller, 1995) and The Race for Consciousness (MIT Press, 1999).
Started research in neural networks in 1969. Present research interests are: financial and industrial applications; dynamics of learning processes and multi-state synapses; stochastic neural chips and their applications (the pRAM chip); brain imaging and its relation to neural networks; neural modelling of higher cognitive brain processes, including consciousness. Has funded research projects from the EC (on building a hybrid symbolic/subsymbolic processor), from British Telecom.(on Intelligent Agents) and from EPSRC on Building a Neural Network Language System to learn syntax and semantics.
-------------------------------------------------------------------------------------------------------------
STEFAN WERMTER
Stefan Wermter is Full Professor of Computer Science and Research Chair in Intelligent Systems at the University of Sunderland, UK. He is an Associate Editor of the journal Connection Science and serves on the Editorial Board of the journals Cognitive Systems Research, and Neural Computing Surveys. He has written or edited three books as well as more than 70 articles. He is also Coordinator of the international EmerNet network for neural architectures based on neuroscience and head of the intelligent system group http://www.his.sunderland.ac.uk/.
He holds a Diplom from Dortmund University, a MSc from the University of Massachusetts, a PhD and Habilitation from Hamburg University.
Stefan Wermters research interests are in Neural Networks, Hybrid Systems, Cognitive Neuroscience, Natural Language Processing, Artificial Intelligence and Bioinformatics. The motivation for this research is twofold: How is it possible to bridge the large gap between real neural networks in the brain and high level cognitive performance? How is it possible to build more effective systems which integrate neural and symbolic technologies in hybrid systems? Based on this motivation Wermter has directed and worked on several projects, e.g. on hybrid neural/symbolic systems for text processing and speech/language integration. Furthermore, he has research interests in Knowledge Extraction from Neural Networks, Interactive Neural Network Agents, Cognitive Neuroscience, Fuzzy Systems as well as the Integration of Speech/Language/Image Processing.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
