The traditional route to knowledge is to read a book from a library. We’re investigating how we can go beyond this and embed knowledge directly into the perception of the user, right where action happens and performance is required.
Wearables thereby act as gateways, mediating between objective reality and its enhancements with visual, auditive, haptic, etc. overlays. When done well, they help turn sensorimotor perception into experience.
This requires two types of world knowledge, i.e., data about the workplace and data about the activity pursued. While the first is rather stable, the latter is dynamic and much more rapidly. We’re researching both representation and implementation, working on both standards as well as development toolkits and frameworks.
The WEKIT project was selected as ‘Highly Commended’ at the Innovation Excellence Award 2018 of The Stationers’ Company.
“Human brains learn by getting feedback. The judges were impressed with WEKIT, a system to enable more efficient workplace training using various wearable tracking devices and motion sensors, coupled with augmented reality, to give useful and readily assimilated feedback.” (Innovation Excellence Award 2018, The Stationers’ Company).
The paper first-authored by our research visitor Carla Bonato Marcolin won the best-paper award at the 15th International Conference on Information Systems and Technology Management (CONTECSI’18). The paper reports findings of a novel LSA-based tool for text analysis, including capturing, cleaning and formatting publicly available data from consumer reviews. The tool and findings help hotel managers in the analysis of large amounts of text data, thereby facilitating decision-making.
Augmented Reality is the future, everyone agrees, but do you really know what it takes to create Augmented Reality applications? On May 24, 2018, in a one day public event, experts invited by the AR-FOR-EU project provide a briefing about future ‘maker skills’ needed in the field of Augmented Reality. At the event, the Eramus+ strategic partnership will inform about required knowledge, skills, and competences, learning methods, key learning outcomes, performance levels, and assessment criteria, filling the gap between formal education and the workplace.
Dr Fridolin Wild, director of PAL, was invited to attend a Showcase and Networking meeting at the Microsoft Hololens Lounge in London – as one of twelve universities invited, on May 10, 2018. Microsoft shared some details about the Mixed Reality strategy and observations on the importance of academia as enable to industry, including an announcement for two new mixed reality apps (remote assist and layout, both now in the store). The universities shared their research. The building and demos took place in the stylish Hololens Lounge.
We are looking for a Post-Doc to join our dynamic and interdisciplinary team with a strong tradition of excellence in research and outstanding industry links in the professional fields of Augmented Reality and Wearable Technologies. Click here for further particulars.
It is been a great week for PAL, the first two days of the week were devoted to education. Alla spoke at the STORIES 2018: Technology Matter Conference, an annual event organised by the Department of Education, University of Oxford. This year’s topic was the use of technology in education and research, and Alla presented a talk about the use of augmented reality in education.
It this moment we can not provide you with the images from the event, but you can trust us, it was amazing:)
There were talks on engagement through technology, improving performance in the classrooms, and very energetic discussions about methods and data analysis.In particular, we want to highlight the talk by Anna-Maria Ramezanzadeh about “Qualtrics” – an online tool for collecting data during the experiments.
If you’ve missed Alla’s talk, you can read her abstract below.
“A powerful medium is what enables a powerful representation. Over the past decade, researchers have been actively exploring augmented reality and thinking about how we can support users in performing skilled tasks. We look at how technological advancements in augmented reality are increasingly redefining our understanding of spatial interaction, tools, and processes and why augmented reality is a platform for future learning. This talk will focus on the application of augmented reality in education, knowledge transfer and information visualisation. We address this question by conducting an experiment with 142 subjects in three different industries: aviation, medical, and space, looking at different aspects of interaction such as technology acceptance, simulator sickness, system usability and user interface design. The software application was designed for Microsoft HoloLens for experience capturing and experience re-enactment, where experts can create a training sequence, and students can replay it and learn in a totally different way. The idea to create a comprehensible representation of knowledge and its systematization in a knowledge-intensive environment enables intuitive interactions with holographic objects and reduces cognitive load”.
And, still to come:)
In a partnership of five leading new universities, we are building a innovative teaching program for Augmented Reality for Computer Scientists (and students of related disciplines). As a first step, we are now running an open survey to find out which skills need to be developed in the courses we will create over the next three years, generously supported by Erasmus+. If your organization engages in AR, would kindly fill in the following
questionnaire? It takes 7-8 minutes: http://codereality.net/survey/
Our artists in residency, Yann Deval and Marie G. Losseau, presented a preview of ATLAS, the work in progress, on invitation of the European Commission at SxSW in Texas, Austin, from March 10-12. ATLAS is an experience that mixes augmented reality and virtual reality and spectators were invited to engage with it to build virtual cities using the ATLAS seed launcher. Each seed sprouts a house, following some urbanistic rules, adapting to their environment. There are houses in the clouds, uprooted houses, houses on stilts, flying houses, and more. The cities assembled by the audience take on a life of their own, with and without interaction of the users, just like any living organism. The work allows to assemble huge cities, in which you can wander and loose yourself. It provokes a reflection on urbanism, architecture, and their influence on our lifestyle. It gives life to the inanimate.
The big question ATLAS helps to explore is one about the relationship of physical space to virtual space. ATLAS transforms the user’s physical space into a map of the environment, swapping perspectives.
The map serves as a launch pad for sprouting houses, establishing a link between the physical space of the user’s surrounding and the holographic houses.
Technically, this was realised using custom shaders, using CG, the shader programming language by NVidia. Holographic houses can be hidden, when they pass behind a physical object. This was created with ease using Hololens tools, but adds real value and credibility to the experience.
The Hololens, by Microsoft, is a ‘developer kit’, a prototype. The space where holograms are displayed is very limited, and that can often disrupt the feeling of immersion for the user. ATLAS uses this limitation as a creative constraint: user gaze unveils the virtual overlays organically, making the limitations appear as if by choice.
Though not an essential technical change, from the user perspective this was a significant one. Frustration decreased and the number of participants mentioning the limitation of the field of view decreased eminently.
ATLAS will offer different chapters, each submerging the user into another layer of augmented reality. Dividing the experience into chapters is perhaps an influence of literature on our work.
To give the houses life, an animation was created with Houdini, a 3D software for procedural animations and simulations. Houdini is a procedural tool, which helped to speed up development of the organic animation, compared with when using other tools such as MeshLab or Cinema4d.
Several different houses were 3D-scanned and integrated, some of them built by artist Marie Loseau, others by 9-year-old kids during the MCCS series of workshops in Molenbeek, Brussels.
To build bridges between realities, windows were created that let the spectator view into a VR world.
The outcome of the residency at this point is a functional prototype, a five minute MR (Mixed Reality) experience for a single user.
The prototype was also presented at MCCS Molenbeek, where 100 kids participated in the project Classes Urbaines. A lot of ideas for the next chapters of the work are coming together during discussions with the WEKIT project team and our Oxford Brookes University students. These will be integrated in September during the next Oxford residency!
ATLAS (prototype): An experience by Yann Deval & Marie G. Losseau
In collaboration with WEKIT / Dr. Fridolin Wild (Oxford Brookes University) for the Mixed Reality part.
Kids from Molenbeek’s schools 1, 5 and 10 for contributing to the building of the cities.
With the support of VERTIGO STARTS program of the European Commission with IRCAM-Centre Pompidou and EPFL
Maison des Cultures et de la Cohésion Sociale de Molenbeek Saint-Jean
Programming, graphic design and music by
Model making and scenography by
Marie G. Losseau
WEKIT (Oxford Brookes University / Performance Augmentation Laboratory)
Dr. Fridolin Wild
External contributions (Mixed Reality)
TriplanarWorld Shader by Robert Yang
SpatialMappingRendererWithNormals by Matt@DeckTwelve
Professor Anu Ojha from the National Space Academy guided us through the day of an astronaut, explaining how the pressure works and what are the pressure difference effects on the body. With him, our team experienced the Sokol space suit, a type of Russian spacesuit, worn by all who fly on the Soyuz spacecraft! We were trying to understand the effect of a mechanical pressure suit on a body of astronauts and think how we can integrate different sensors to measure physiological parameters alongside augmented reality glasses. The suit consists of an inner pressure layer of rubberised polycaprolactam and an outer layer of green nylon canvas. Boots are integrated with the suit but gloves are removable and attach by means of blue anodised aluminium wrist couplings. We must say the full suit is quite heavy to wear, especially when you think about it being on your body for more than 10 hours per day, but it’s a unique feeling when your body is squeezed by the air.
The R package ‘mpia’ provides a computational model to understand, analyse, and advise on human learning.
Meaningful Purposive Interaction Analysis (MPIA) combines the ‘best of’ from social network analysis (SNA) with latent semantic analysis (LSA) to help create and analyse a meaningful learning spaces from the digital traces left by a learning community in the co-construction of knowledge.
The hybrid algorithm is implemented in the statistical programming language and environment R, introducing packages which capture – through matrix algebra – elements of learners’ work with more knowledgeable others and resourceful content artefacts. This building block application allows to use and build analytics to guide students and support decision-making in learning.
The package can be downloaded here:
Photo: Sonia Bernac