Roboskin

 Work at UGDIST

During the last part of the project, UGDIST and IIT worked on skin related SW technologies. In particular, joint work has been established to the aim of designing networking and embedded software and firmware solutions, as well as implementing mechanisms for tactile data processing, with a specific emphasis on automated skin spatial calibration.

NETWORKING AND SOFTWARE SOLUTIONS

Figure 1. The structure of the SW framework, from the hardware (low level) to the application (user level). In this representation the layer “Networking” refers to the network of embedded boards responsible for acquiring data from the skin.

Skin SW technologies include algorithms and data structures allowing tactile to travel from the lowest level (i.e., the actual sensors) up to user applications (Figure 1).

Figure 2. The SW framework provides the user with an HW abstraction mechanism in which the skin is organized in logical parts. Each part is made up of a collection of taxels. The SW framework maps user-level taxel representation structures to the corresponding taxels.

The goal of the SW framework is to provide an abstract and HW independent representation of the skin, which is organized in logical units representing different body parts. A generic representation of this idea is represented in Figure 2.

Both general-purposes and iCub-specific tactile data processing architectures have been investigated and experimentally evaluated in terms real-time performance, specifically taking into account bandwidth, jitter and reliability issues.

A substantial part of the work has been devoted to the design and realization of embedded electronic boards and the associated firmware implementing all the needed skin networking aspects. Work proceeded along two paths. On the one hand, we have experimented with general-purpose standard real-time networking architectures like EtherCAT whereas, on the other hand, we have realized an iCub-specific, custom architecture and protocol on top of the standard Ethernet protocol. Experimental evaluation and performance assessment of both solutions has shown that we can achieve very good performance in both cases. The advantage of the custom solution is that it can work on standard HW (supporting Ethernet), whereas the solution based on EtherCAT clearly provides better performance in terms of jitter and latency at the price of dedicated embedded HW.

AUTOMATED SKIN SPATIAL CALIBRATION

Joint work between UGDIST and IIT has been devoted to investigate modelling methodologies and SW algorithms enabling the use of large-scale robot skin. Specific emphasis has been put on the so-called skin spatial calibration problem.

Figure 3. a) Results of the skin calibration procedure. In this case the skin was deployed on one of the iCub covers (from the forearm, see picture b)). Colored dots represent the estimated position of each taxel. For reference we plot the real position of the triangles (solid colored traces).

Skin spatial calibration deals with the problem of estimating the location of tactile elements mounted on a robot body part (Figure 3). This problem arises with the adoption of tactile systems with a large number of sensors, and it is particularly critical in those cases in which the system is made up of flexible material that is deployed on a curved surface. In this scenario the location of each taxel is partially unknown and difficult to determine manually. Placing the device is in fact an inaccurate procedure that is affected by displacements in both position and orientation.

The method we developed is based on the idea that it is possible to automatically infer taxel positions by measuring the interaction forces exchanged with the environment. The location of the contact can be estimated through force/torque (F/T) measures gathered by a sensor mounted on the robot kinematic chain. This method has been implemented on the iCub humanoid robot. We proposed two different techniques. The first one requires the knowledge of the CAD model of the part where the skin is mounted. The second one recovers the position of each sensor just with measurements coming from the F/T device.

Work at IIT

CONTACT FORCE ESTIMATION USING TACTILE AND FORCE/TORQUE SENSORS

The 6 axis force/torque sensor located inside iCub's arm and the tactile sensor network (i.e. ROBOSKIN) covering the robot are exploited to (i) implement a force controller using simulated joint torque sensors (note that at the moment the robot is not equipped with joint torque sensor), and (ii) compute an estimate of the external forces/moments applied on iCub' skin. Bottom-right corner: planar visualization of the tactile sensor network covering the iCub's arm. Top-right corner: 3d visualization of the robot and the external forces applied on its arm.

Work at UGDIBE

 LARGE BODY AREA SKIN

•    Electromechanical characterization of PVDF piezoelectric films

•    Definition of specifications, optimized design, fabrication and experimental assessment of different prototypes of the interface electronics based on single-ended charge amplifier circuit approach

     

•    Development and assessment of the manufacturing technology of piezoelectric functional transducer arrays : design, manufacturing and test of triangular patches based on PVDF arrays (collaboration UniCA/IIT)

•    Development of real-time tactile data processing algorithms (continuum mechanics approach for contact force and contact shape reconstruction + machine learning approach for threshold-based impact detection)

•    Definition of the integration of the “piezoelectric array + interface electronics” block into the overall robot architecture

 
SMALL BODY AREA SKIN (i.e. palm, fingertip, etc.)

•    Design, fabrication and experimental testing of tactile interface electronic sensing systems for POSFET based transducer arrays.

•    Electromechanical test, assessment and evaluation of microelectronic POSFET arrays.

•    Design and implementation of the miniaturized tactile interface electronic sensing system prototype integrated into the palm of the iCub robot.

EXPERIMENTAL RESULTS

•    Impact hammer testing on the tactile sensing chip (12 taxels-4 out of 16 are faulty).

•    Coin rolling over diagonal direction of the POSFET arrays.

Work at UWN

TACTILE GESTURE ANALYSIS AND PRODUCTION

The ROBOSKIN project is developing and understanding of tactile communication between humans and robots.  In this respect, the Cognitive Robotics Research Centre at the University of Wales, Newport undertook a large scale study of basic tactile gestures produced by humans on a skin-enabled Nao humanoid robot. Gesture data was gathered from over 200 participants aged between 12 and 16, producing seven tactile gestures on a Nao humanoid robot covered in a total of 648 tactile capacitors on it upper and lower arms.  Figure 1 illustrates the experimental setup for the gesture data capture experiments.  The seven gestures studied were “tap”, “poke”, “push”, “touch”, “stroke”, “pinch” and “grab” with all participants being asked to produce all seven gestures.

The human gesture data was then used to implement a gesture production capability on the Nao robot for three simple gestures, i.e., gestures that can be distinguished by their force and duration profiles.  In order to distinguish the more complex gestures, further dimensions such as area, change in location and number of touch areas had to be considered.  Figure 2 illustrates the experimental setup for the gesture production.  During production the active robot, i.e., the robot producing the gesture starts from a fixed point 2cm from the upper arm of the passive robot, i.e., the skin-enabled robot perceiving the tactile gesture.

Finally, the quality of the gestures produced by the humans and the Nao were compared.  We concluded that the simple touches human touch data was clearly separable in both the temporal and tactile dimensions and that the gesture production was able to reliably produce the simple gestures similar to the corresponding gesture produced by humans.  Figure 3 presents the raw data from the human and robot gesture production.  Figure 4 presents the basic statistical properties of these data sets.

Figure 1: The experimental setup for capturing human tactile gestures	Figure 2: Experimental setup for robot tactile gesture production

Figure 3: The raw touch data produced by the human                                subjects and the robot	Figure 4: The mean and variance of the tactile data produced                                     by the human subjects and the robot

PROTECTIVE REFLEXES FOR SAFE HUMAN-ROBOT INTERACTION

The ROBOSKIN project is developing a set of protective reflexes that utilise the tactile capabilities of the robot skin to enable humanoid robots keep themselves and any humans in their presence safe.  The reflexes are currently implemented on the Nao humanoid robot using a unique skin sensor developed in the ROBOSKIN project.  Figure 1 depicts the Nao robot with the skin sensor covering the upper and lower arms.

 Figure 5: The Nao robot with the robot skin sensor on the upper and lower arms.

The reflexes use a model based on reflex receptive fields as reported in studies of humans.  The robot was given five separate reflexes, ‘upper arm’, ‘forearm top’, ‘forearm in’ ‘forearm bottom’ and ‘forearm out’, each with a dedicated field.  The size of the reflexes produced varies with the distance of the stimulation point from the centre of the receptive field.  It also varies based on the force of the stimulation. 

 Work at UNICA

The fabrication of artificial robot skin based on Organic Thin Film Transistor (OTFT) technology allowed to detect mechanical stimuli on highly flexible substrates. The activity was focused on the optimization and the calibration of the organic transducers.

Work at UH

SKIN-BASED ROBOT ASSISTED PLAY

Here we furthered the investigation of cognitive and embodied learning in tactile social interaction with children with autism. Trials with the robot KASPAR that was equipped with the ROBOSKIN’s sensor capabilities continued in several schools. Below some example

• Exploring  ‘cause and effect‘ 

• Embodied learning – developing coordination.  

Work at EPFL

In the context of object interaction and manipulation, one characteristic of a robust grasp is its ability to comply with external perturbations applied to the grasped object while still maintaining the grasp. An approach for grasp adaptation was developed. Using a multistep learning procedure, a statistical model is learned to adapt the hand posture solely based on the perceived contact between the object and fingers. The model dataset is built by first demonstrating an initial hand posture, which is then physically corrected by a human teacher pressing on the fingertips, exploiting compliance in the robot hand. The learner then replays the resulting sequence of hand postures, to generate a dataset of posture-contact pairs that are not influenced by the touch of the teacher. The learned model may be further refined by repeating the correction-replay steps. Grasp adaptation is demonstrated in response to changes in contact, and improved adaptation is shown with additional rounds of model refinement.

This approach was validated on the iCub robot, through a user study. The impact of providing different types of robot feedback on the effectiveness of teaching by demonstration is considered as well. The goal is to determine the best way of providing feedback in relation to robot’s tactile sensing in order to improve the teaching by demonstration interaction. We aim to: reduce the teaching time, reduce the rounds of demonstration required and improve the metrics used for learning. Three types of feedback were provided: verbal, graphical display, and facial expressions feedback. Task performance and usability were evaluated in all cases, using both robot objective measurements, as well as post-experiment questionnaires. The study revealed a strong effect of the feedback given by the robot on the subjective usability ratings and task performance. These results lead to a better understanding of how to develop better means of interaction as well as improve efficiency when using tactile sensing.