The first surgical operation in the world involving virtual elements was successfully carried out at the Policlinico Universitario S. Orsola of Bologna. During the operation, in a real operating theatre of course, the surgeon could view the virtual elements in front of him which were able to aid and guide him. This was made possible thanks to a visor integrated with Augmented Reality, called VOSTARS, which the surgeon wore during the operation.
“Until this moment,” explains Vincenzo Ferrari, a biomedical engineer from the Department Of Information Engineering at the University of Pisa and coordinator of the European team who designed VOSTARS, “augmented reality has not been fully exploited in the operating theatre. The visors which are currently available on the market project a few digital contents directly into the line of vision of the surgeon, such as, for example, a 3D image of the organ to be operated on. These virtual images captured by the radiological scanners (like CAT and MRI scans) are normally viewed by the surgeon prior to the operation, to aid him during the preparation leading up to the surgery. A visor has never been used before to guide the actual moment of surgery, due to the difficulty the human eye encounters in bringing into focus both real and virtual objects simultaneously.”
The surgical operation at the Policlinico Universitario S. Orsola of Bologna with the VOSTARS visor
Bringing virtual objects into focus actually means that real objects become blurred as the eye perceives them as being at different distances. Obviously, this should not occur at the moment when the surgeon is holding a scalpel, and therefore, so far, it has been impossible to exploit virtual information to guide surgery.
Potential additional information about the patient and the operation must therefore be provided on an external monitor, obliging the surgeon to look away from the patient constantly transferring his concentration to the monitor, which proves to be tiring and at times ineffective.
The VOSTARS visor was developed to overcome these problems. It is the outcome of a European project coordinated by the University of Pisa, which has seen scientists and technicians from four different countries working for three years with the aim of designing a highly innovative wearable surgical visor capable of projecting the patient’s specific information as well as more general information about the organs involved in the operation directly into the line of vision of the user even during the surgery itself.
“In order to do this,” explains Ferrari, ”we had to solve some very complex problems mainly to do with eye-hand coordination and the coherence between the real and the virtual image linked to temporal and spatial issues and the ability to bring objects into focus. It is obvious that if the surgeon needs to follow a virtual cutting line, this must appear in the right place at the right time during the operation, and yet managing to obtain this is no easy task. Furthermore, the surgeon must be able to bring into focus both the virtual image and the patient in order to follow up with the scalpel.
Thanks to a video camera, VOSTARS combines the images in front of the surgeon with the patient’s radiological images, and ensures that both remain perfectly coherent and in focus. In addition, during the phases of the operation where the precise virtual guide is not necessary (for example at the beginning or at the end), the visor can become transparent allowing the surgeon to see the operating field directly with his own eyes. This possibility of passing from a view aided by the video camera – ‘video see-through’ - to a direct view through the transparent visor – ‘optical see-through’ – is the distinctive characteristic of VOSTARS, which stands for ‘Video-Optical See-Through Augmented Reality System’.”
The experimental operation in maxillofacial surgery consisted in resecting and repositioning the patient’s jawbones to restore the function of mastication.
“Thanks to the VOSTARS visor,” comments Dr. Giovanni Badiali who is head of the project at the Policlinico Universitario S.Orsola of Bologna and the surgeon who performed the operation, “we were able to view the facial bone structure, the jaws and the cutting line using augmented reality before the operation. In the next step, during the operation, the device allowed us to visualize a dotted line in 3D directly on the patient’s bones, indicating the path to follow using the surgical instrument. With the help of the visor we were able to cut the jaw with the necessary precision.”
Further experiments are on the agenda at the Policlinico. When fully operational, the system will bring about a reduction in the length of surgery time and an increase in precision.