EU FET NEUROBOTICS
FP6-IST-001917

The NEUROBOTICS project will produce a strongly co-ordinated, multidisciplinary and interdisciplinary effort by relying on and enhancing the state of the art in three main scientific areas: robotics, with special reference to bio-mimetic, anthropomorphic systems and bionic components, neuroscience, with special reference to sensory-motor coordination; and interfacing technology, with reference to non invasive and invasive interfaces to the peripheral nervous system (PNS) as well as to the central nervous system (CNS). Furthermore, and more importantly, NEUROBOTICS will consolidate the area of "human augmentation" and "hybrid bionic systems", whose state of the art is at present scattered and rather weak (Dario et al. 1993). As stated by E. Von Gierke, considered as the pioneer of this discipline, the primary goal of bionics is to extend mans physical and intellectual capabilities by prosthetic devices in the most general sense, and to replace man by automata and intelligent machines (Von Gierke et al. 1970).

Referring to the present development of the discipline, Hybrid Bionic Systems (HBSs) can be generically defined as systems that contain both technical (artificial) and biological components. They can include:

• artificial systems with biological elements or subsystems. In such a case, the biological system is a complementary or supplementary element to the technical system;
• biological systems with artificial elements or subsystems. The artificial subsystem, e.g. a robotic artefact, is a complementary or supplementary element to the biological system.

In recent years, many scientific and technological efforts have been devoted to create HBSs that link, via neural interfaces, the human nervous system with electronic and/or robotic artefacts. In general, this research has been carried out with various aims: on the one hand, to develop systems for restoring motor and sensory functionalities in injured and disabled people; on the other hand, for exploring the possibility of augmenting sensory-motor capabilities of humans in general, not only of disabled people.

As regards restoring motor capabilities, several technologies have been devised to exploit the residual nervous and/or muscular activities of the (paralysed or amputated) limbs (Craelius 2002). According to Gasson et al. (2002), such technologies can be basically classified into:

• technologies for the control of prosthetic limbs (Prochazka et al. 1997; Lauer et al., 2000; Craelius, 2002; Popovic 2003). New materials, microfabrication technologies, advanced robotics mechanisms, computational algorithms and regenerative electrodes have been explored (Ramachandran et al., 1993; Bogdan et al., 1994; Montelius et al., 1996; Prochazka et al., 1997; Abboudi et al., 1999; Dario et al., 1998; Ceballos et al., 2002);
• the control of external electrically powered systems. For instance, a system allowing to control an electrically powered wheelchair without using the hands has been developed (Felzer et al., 2002), that relies upon electromyogram (EMG) as input signal.

Restoration of lost sensory-motor functions has been pursued through neuroprostheses for subjects with neurological disorders, such as those caused by spinal cord injury (SCI) or stroke/head injury (Stein et al. 1992; Popovic and Sinkjaer 2000; Lauer 2000), or by robotic devices like the RoboWalker, an active exoskeleton which can augment or replace muscular functions of the lower limbs, for example to assist motor-impaired individuals (http://www.yobotics.com/).

As for sensory functionalities, important results have been achieved in restoring hearing and sight capabilities. Some improvements in auditory performance of people with hearing loss can be obtained with cochlear implants (Simmons et al., 1965; Blume 1999; Loizou 1999, Spelman 1999; Marsot-Dupuch et al., 2001). Retinal implants can be realized in the attempt to regain lost visual functionality. Neuroprosthetic solutions can be classified as cortical (Normann et al., 1999; Dobelle 2000; Normann et al., 2001), retinal (Eckmiller 1997; Peyman et al., 1998; Walter 1998; Rizzo et al., 1999; Zrenner et al., 1999; Walter et al., 1999; Chow et al., 2001; Humayun et al., 2001; Meyer 2002), and optic nerve based (Veraart et al., 1998). A thorough review of the state of the art in the fields of epiretinal, subretinal and optic nerve implants can be found in (Margalit et al., 2002). An interesting approach is the design of an ocular prosthesis with an autonartificial eye to move more naturally (Gu, 2000)

As for augmenting capabilities of able-bodied persons, it is worth to mention the US Defence Advanced Project Agency (DARPA) initiative that is currently trying to develop exoskeletons for human performance augmentation (EHPA), although focussed on military application. In particular, four EHPA projects work on the development of small actuators, lightweight structures, and control technology to be integrated in devices 'wearable' by a human and able to augment his physical capabilities (http://www.darpa.mil/dso/thrust/matdev/ehpa.htm).

Neural interfaces connect the nervous system with artefacts. The control of artificial systems by means of direct interfaces to the nervous system, either in animals or in humans, has been recently investigated by a few groups, especially in the US (Chapin et al., 1999, Levine et al., 2000, Donoghue, 2002, Tatlor et al., 2002, Nicolelis 2003). Multielectrode recordings allowed researchers to simultaneously monitoring the extracellular activity of over a hundred single neurons in both anaesthetized and awake animals, and to predict the outcomes of the animal's behaviour during learning of a motor task (Nicolelis 2001, Nicolelis et al., 2002). This has led to the possibility of investigating how information is processed and encoded in living cultured neuronal networks of animals by interfacing them to a computer-generated animal, the Neurally-Controlled Animat, living in a virtual world (Demarse et al., 2001). Researchers at the University of Illinois and at the University of Genoa have jointly fabricated simple hybrid creatures with a mechanical body controlled by the brain of a lamprey (Graham-Rowe 2000; Reger et al., 2000). The robot is the Kephera, and the lamprey brainstem with part of its spinal cord was extracted and maintained in an oxygenated and refrigerated salt solution. Chapin and colleagues (Chapin et al., 1999) demonstrated that simultaneous recordings from ensembles of cortical and thalamic neurons can be decoded in real time to allow a rat to control monodimensional motion of a robotic arm. Large pyramidal neurons in motor cortex (red triangles) send axons to spinal cord, ending on interneurons and motoneurons. Microelectrodes could record neural activity, which is transformed by an artificial neural network into signals required to operate a robotic arm (Fetz 1999). A similar experiment has been carried out on primates. Wessberg and collegues (Wessberg et al., 2000) recorded the simultaneous activity of large populations of neurons, distributed in the premotor, primary motor and posterior parietal cortical areas, as non-human primates performed two distinct motor tasks. Cortically derived signals have been successfully used for real-time three-dimensional control of robotic arms. These results suggest that long-term control of complex prosthetic robot arm movements can be achieved by simple real-time transformations of neuronal population signals derived from multiple cortical areas in primates.

Among the few experiments carried out so far on human subjects, a remarkable example is described in (Kennedy et al., 2000), where humans with brain-implanted chip have learned to drive a cursor on a computer monitor. This system requires implantation of a Neurotrophic Electrode (that uses trophic factors to encourage growth of neural tissue into the hollow electrode tip) into the outer layers of the human neocortex. The recorded signals are transmitted to a nearby receiver and processed in front of the patient. Another recent experiment consisted in implanting a 100 microelectrodes array onto the median nerve of a human subject. A number of experiments have been carried out using the signals detected by the array. The subject was able to control an electric wheelchair and an intelligent artificial hand. In addition to being able to measure the nerve signals, the implant was also able to create artificial sensation by stimulating individual electrodes within the array (Gasson et al., 2002; Warwick et al., 2003). In Europe, the FET-CYBERHAND project has the goal to produce the fundamental knowledge on neural regeneration and sensory motor control of the hand in humans, and the technological means, with the ultimate aim to develop a new cybernetic prosthesis, directly controlled via bi-directional peripheral neural interfaces.

The basic assumption of the NEUROBOTICS challenge is that recent advancements in the field of Neuroscience, and specifically on understanding sensory-motor mechanisms which govern upper limb motion control (e.g. Grillner 1985; Droulez and Berthoz 1991; Lacquaniti 1997; Burnod et al., 1999; Johansson et al., 1999; Mc Intyre 2001; Grillner et al 2002; Ohki et al., 2002 and many more), adequately combined with enabling robotic, mechatronic and microengineering biomimetic/biomorphic technology, which for example already made possible the development of humanoid robots in Japan (Hirai, 1998, Inoue, 2000; Kanehira, 2002; Kaneko, 2002; Sakagami, 2002) and of advance biomechatronic platforms in Europe (Butterfa§, 2001; Schulz et al, 2001; Dario et al. 2002; Carrozza et al., 2002a; Carrozza et al., 2003a), could lead to a break-through in the fields of human augmentation, and specifically of Hybrid Bionic Systems based on robotic artefacts.

Based on all previous considerations, the main objective of NEUROBOTICS will be to generate, in a 5-year time frame, new scientific knowledge and new enabling technologies for the design and development of Hybrid Bionic Systems (HBSs), in response to the FET ProActive Initiative "Beyond Robotics". NEUROBOTICS will systematically explore the area of HBSs, defined as the integration of a robotic artefact with a human being through appropriate physical and cognitive bi-directional interfaces, and will deeply investigate the theme of human augmentation. More specifically, NEUROBOTICS will focus on human augmentation problems related to upper limb sensory-motor functions when a human brain is always present in the control loop. This choice is the optimal solution for scaling the general problem of human augmentation to a level of complexity and risk compatible with NEUROBOTICS resources and, at the same time, for exploiting the knowledge and the technology made available by previous and ongoing FET projects as a start-up impulse. In spite of this reduced domain of investigation, the results of the project are ewhole domain of HBSs for human augmentation.

The NEUROBOTICS strategy is based on a top-down approach which brings together the "best neuroscience knowledge on sensori-motor control" with the "best robotic technology and interfacing technology available" in order to produce a focused joint effort. The NEUROBOTICS consortium considers this approach as the the one which could reduce potential risks to a minimum, and lead to real break-throughs in five years. The starting point for the project is the most advanced state of the art in neuroscience relevant to HBSs, which is directly provided to the project by a highly-qualified group of neuroscientists. In detail, NEUROBOTICS aims at the following Project Objectives (POs):

• PO1. Providing a comprehensive definition of the scientific domain of Hybrid Bionic Systems, including general taxonomy, functional classification and novel metrics for direct comparison of performance of different HBSs in a common domain;
• PO2 Defining and assessing a set of formal methodologies, intended as new design methods for radically innovative HBSs and as novel experimental methods for executing rigorous scientific tests on NEUROBOTICS systems;
• PO3 Investigating and developing new biomorphic actuation, sensing and neural interfacing technologies as the basic enabling components for developing the NEUROBOTICS artefacts. Novel actuators and sensors will be developed according to a strict biomorphic approach and, in any case, following the suggestions given by neuroscientists;
• PO4 Designing and developing new integrated robotic artefacts, as much biomorphic as required to be effectively interfaced with human body and brain. More specifically, three robotic platforms featuring different levels of hybridness (i.e. mechanical coupling with the human body) and of connectivity (to the human nervous systeaugmentation:
  1. The "Beyond Tele-operation" platform. A biomimetic artefact, for investigating problems related to interfacing humans with scalable artefacts not physically coupled with the human body. Research focus is on remote manipulators, ranging from micro/nano grippers for grasping biological cells through micro-manipulators for minimally invasive surgery to classical robotic arm-hand manipulation systems operating in hazardous environments;
  2. The "Beyond Orthesis" platform. An intelligent wearable artefact, e.g. an arm exoskeleton, for investigating problems related to interfacing humans with artefacts which are loosely physically coupled with the body;
  3. The "Beyond Prosthesis" platform. An arm-hand system, for investigating problems related to interfacing humans with artefacts which are tightly physically coupled with the body. The arm-hand system will be devised to be highly modular so that its components could be used for functional substitution in amputees, i.e. as a hand or as an integrated hand/arm prosthesis, or for functional augmentation in able-bodied persons, i.e. as a third additional limb.
  Both invasive and non-invasive neural direct interfaces to the Peripheral Nervous System (PNS) and to the Central Nervous System (CNS) will be applied to the three different NEUROBOTICS platforms, as is needed in order to achieve the connectivity associated with the haugmentation to be experimented.

  In addition, a set of very advanced bio-inspired robotic platforms, to be used as early prototypes, will be made available to the NEUROBOTICS Consortium from the very beginning of the project, thanks to direct links to as many as eight ongoing FET projects participated by NEUROBOTICS partners, and also by exploiting links to, and research partnership with, Japanese leading groups in the field of humanoid robotics;

• PO5 Designing and performing original scientific experiments to be carried out by using the early prototypes and, as soon as available, the novel platforms provided by NEUROBOTICS. This effort will allow not only to assess the viability of the proposed artefacts and interaction/interfacing technology as innovative solutions for HBSs, but also to hopefully disclose further opportunities to understand the functions of human brain by using the NEUROBOTICS platforms for neuroscientific experiments. Special emphasis will be given to analyse the brain capability of controlling and coordinating the motion of artefacts and to process artificial sensory signals, being them alternative, substitutive or additional to the natural ones;
• PO6Developing novel algorithms for sensory-motor shared control of HBSs, deeply based on adaptive paradigms, learning, action recognition and imitation. These innovative control schemes will be validated in all NEUROBOTICS platforms by means of tasks involving reaching, grasping, manipulating, intersecting and catching moving objects. Because of the presence of the human brain in the high level control loop, the level of autonomy of the artefact will be limited to low-level reactive behaviours and to pre-defined sequences of motor programmes triggered by specific events. The level of autonomy (i.e. shared control) of the HBSs shall be adaptable according to the preferences and to the physical and cognitive characteristics of different persons.

The ambition of NEUROBOTICS is not only to pursue all the objectives listed above by gathering a critical mass of leading research groups in Neuroscience and Robotics, but also to systematise the fusion of neuroscience and robotics to the extent of establishing a new scientific discipline. These additional objectives will be pursued by promoting a limited set of accompanying actions which will be carried out in synergy with other ongoing European and national initiatives, such as:

• PO7The definition of structured interaction modalities for roboticists and neuroscientists, who will share the NEUROBOTICS research methods and tools for scientific experiments and technological developments for hybrid bionic systems. This implies the need for merging the quite different styles and traditions of two different communities: Neuroscience and Robotics;
• PO8 Establishing a new scientific community, by targeted dissemination activities and networking activities; strong links will be established and maintained with Networks of Excellence on relevant themes, such as the EURON (European Robotics Research Network), NEURO.IT, and other similar initiatives to be launched in the VI FP;
• PO9 Educating new generations of neuroscientists and roboticists, by pilot actions within the project lifetimand education materials;
• PO10 Identify and implement an initial set of preparatory actions for promoting the creation of an European (Virtual) Institute of Neuro-Robotics, in strict interaction with robotics research European networks and other ongoing related national initiatives in Sweden, France, Italy, and elsewhere. The ultimate aim of this institute would be to integrate the most advanced achievements of robotics and neuroscience research, thus implementing an engineering equivalent of "big science" programs, and launching long-term initiatives at European and worldwide level. The creation of the Institute is outside the scope of NEUROBOTICS, but the project will stand as a building block, helping delineate the framework and aiming to reach the needed critical mass. In this case, too, links with existing NoEs related to Robotics and to Neuroscience and to other consistent programs and initiatives worldwide will be exploited and strengthened all along the project life-cycle.