The traditional route to knowledge is to read a book from a library. We’re investigating how we can go beyond this and embed knowledge directly into the perception of the user, right where action happens and performance is required.
Wearables thereby act as gateways, mediating between objective reality and its enhancements with visual, auditive, haptic, etc. overlays. When done well, they help turn sensorimotor perception into experience.
This requires two types of world knowledge, i.e., data about the workplace and data about the activity pursued. While the first is rather stable, the latter is dynamic and much more rapidly. We’re researching both representation and implementation, working on both standards as well as development toolkits and frameworks.
This morning, we picked up a retired Laerdel SimMan Classic donated from our Faculty of Health and Life Sciences. We’re working together with the Cognitive Robotics Lab on modifying the patient manikin, with PAL planning to contribute augmented reality overlays for training on imaging technologies in medicine (such as training to read MRI scans).
This week the WEKIT project entered its second phase, moving from requirements engineering forward to the first cycle of design, development, and evaluation. Project members came together in Oxford to discuss ideas about sensors to be used for experience capturing (in the picture: EEG headband), the AR guidance system for re-enactment, and system interoperability. Productive discussions around maintenance training in space, aircraft engine maintenance, and medical imaging translated the technical possibilities directly to their industrial application.
We’ve been featured in the UK’s number one HR magazine, People Management, published by the CIPD with a circulation of more than 132,000 in an article about ‘The Future of Work’.
Within this video, shot when Fridolin was still at the Open University, our principal investigator explains, how augmented reality and wearable computing can change the world as we know it.
Attending ‘Fashion Revolution’, an awareness raising campaign about ethical sourcing of textiles and clothing, Dr. Fridolin Wild had a chance to brief the Lord Mayor of Oxford about the TCBL project in which we lead the knowledge sharing activities with augmented reality tracking and experience extraction.
Supported by the IEEE Standards Association, Epson, Vuzix, Optinvent, Bitstars, and Atheer, we’ve organised an Augmented Reality Hackathon at RWTH Aachen in Germany over the week-end April 15-17. Three days of intense coding saw an amazing range of apps invented, designed, and implemented: two motion interaction apps (using a real-time infrared scanner), a face recognition app with speech to text for people with hearing disabilities, a collaborative graffiti wall written entirely with open web technology, and a game with complex physics simulations.
The code projects are available here.
We hosted a guest talk and visit from Ben Vaughan, the Chief Open Source Evangelist of DAQRI Inc, speaking about “Open Source Augmented Reality with ARToolKit since 1999” on March 1. ARToolKit is probably the oldest Open Source SDK project for Augmented Reality, with its origins rooted already in the late 1990ies. Recently acquired by Daqri Inc, the Los Angeles based foundry of industrial AR solutions, ARToolKit is meant to become the foundation layer for AR systems, similar to that of an operating system or the internet. Ben believes that this foundation layer needs to be in the Public Domain: open, verifiable, and modifiable.
By investing in improving ARToolKit and by publishing that work openly, Ben and DAQRI are making a serious commitment to open AR. It’s not entirely altruistic: DAQRI uses ARToolKit in its products, and good AR engineers from the community are good potential recruits for DAQRI’s commercially focused efforts, but those same benefits also accrue to everyone else who participates in the community, and the same ARToolKit code base used and improved by DAQRI is available to everyone else under the same license terms. The presentation discussed how ARToolKit underpinned the past growth of the AR market, the reasons why it has been open sourced, and its future development agenda.
Dr. Fridolin Wild (Senior Research Fellow and Scientific Director of WEKIT) spoke about Wearables Enhanced Knowledge Intensive Training (WEKIT). Augmented reality and wearables are potential game changers, re-inventing the way we perceive and work with information. In the publicly-funded WEKIT project (Horizon 2020), we explore experience capturing with AR and wearables to support on the one hand observation of a master performing problem-solving tasks and, on the other, to deliver augmented real-time guidance to trainees. In this Research Centre talk, Fridolin introduced the department to the project and outlined the future R&D timeline, and paved the way for further collaboration on AR and wearables within the department.
The new expert working group within the IEEE will elaborate a proposal for augmented reality learning experience models (IEEE ARLEM), with an early draft scheduled for the late summer and a first ballot possible as early as 2016.
The proposed Augmented Reality (AR) learning experience model will specify how to represent learning activities and their according workplace reference models in a standardized interchange format in order to lower entry barriers for authoring of learning experience spanning real world interaction using sensors and computer vision, and web applications.
Purpose of the standards working group is to develop an overarching integrated conceptual model and the according data model specifications for representing activities, learning context and environment (aka ‘workplace’), and potentially other data model components needed for AR-enhanced learning activities.
Separation of slow-changing data for, e.g., environment descriptions from fast-changing data as, e.g., required for step by step guidance is intended and isolation in linked, but physically separate interchange formats is intended to facilitate efficient handling and storage.
The new standards committee (NesCom) of the IEEE has appointed KMi’s Fridolin Wild to chair this working group.
“This new standard will help bring down production costs of augmented reality experiences significantly, turning normal web-designers into augmented reality engineers”, so Wild. He denotes further: “I’m very glad that we’re entering this working group with a strong Open University proposal, that we’ve elaborated with partners across the EU in our TELLME research project”.
Augmented Reality is providing a whole new set of possibilities to improve proficiency and safety on the workplace and reduce costs during training, in particular in manufacturing and automotive industries.
Among the applications developed to support workers on the shop floor, telementoring plays a leading role as a demonstrator of the capabilities of AR in industry. We’re currently exploring the user experience of telementoring with GhostHands – the use of a virtual model of a mentor’s real hands, appearing in a workers full AR view of a work task. The mentor can see the workers view ‘through their eyes’ and can place their GhostHands into that view.
Both worker and mentor appear to cope with this ‘ghost assisted’ experience very well, and recognise a sense of joint accomplishment from the cues that the hands embody in completing a task.
Publication: Giuseppe Scavo, Fridolin Wild, Peter Scott (2015): The GhostHands UX: telementoring with hands-on augmented reality instruction, link