The traditional route to knowledge is to read a book from a library. We’re investigating how we can go beyond this and embed knowledge directly into the perception of the user, right where action happens and performance is required.
Wearables thereby act as gateways, mediating between objective reality and its enhancements with visual, auditive, haptic, etc. overlays. When done well, they help turn sensorimotor perception into experience.
This requires two types of world knowledge, i.e., data about the workplace and data about the activity pursued. While the first is rather stable, the latter is dynamic and much more rapidly. We’re researching both representation and implementation, working on both standards as well as development toolkits and frameworks.
In June Dr Fridolin Wild and I attended the BMW innovation fair. This boasted an array of inventions surrounding the concept of industry 4.0 (or the 4th version of the industrial revolution).
Armed with our trusty Hololens we demonstrated the ATLAS project created by PALs artist in residence; a visually stimulating augmented experience in which the user throws augmented “balls of paper” in the spatially mapped environment. The balls plant into the real-world environment, like seeds, they then grow into a buildings. We also showed videos of trials from the WEKIT project. We showed the WEKIT aircraft maintenance trial video in which you see an ‘expert’ using WEKIT on a Hololens to record how to perform a maintenance task on a plane which is then played back by a ‘novice’ who is able to complete the task without supervision. This allowed us to discuss, with anyone who would listen, how AR based workplace learning will be a great thing. The feedback was impressive. What struck me is how enthusiastic many of the ‘higher ups’ were in discussions about how AR could be used to train new recruits. It was pointed out to me that employing as many people as BMW do, even with a good retention rate, it’s impossible not to retrain hundreds of staff every year. I can see this being a great place for AR to integrate itself within the workforce.
It was a busy day with many stands and talks. I met an old classmate of mine, now a software developer in BMW. He proudly showed me an application he had developed with his team. The application allowed data to be pushed directly to the employees who need it around the factory. There were a few other AR people and another stand covered in Hololens . We stood next to a friend of Fridolins, an enthusiastic Scrum master who was encouraging passers-by to compete in activities designed to explain why agile is better. I enjoyed a talk by a company who created a wrist scanner. Small form and easy to carry, this simple solution demonstrated performance augmentation very well, it allowed employees who worked with barcodes to easily scan them without carrying around a predated heavy hand scanner..
Finally, the topic that really, really stuck with me was on the forklift trucks. BMW are in the process of rolling out new automated forklifts which, as I understand it, will replace all manually operated forklifts within a year. This is a great example of industry 4.0. The engineer in me applauds, what a magnificent creation – the future is here! Soon we will all have our own R2-D2 to help us with everyday tasks like cleaning the carpets or fixing our X-Wings. There is also another side of me that worries though. Technology is moving so fast and pushing the job market, so many industries will transform in the next couple decades. Can we find a fairway to share this world when factories only needs one person and a dog to run?
AWE (Augmented World Expo), the world’s largest and longest running conference and expo dedicated to augmented and virtual reality, today announced the list of finalists for the ninth annual Auggie Awards. Celebrating the best-of-the-best in AR+VR, winners will be presented with the prestigious Auggie Award on Thursday, May 31 at the Auggie Awards Ceremony live on stage at AWE USA 2018. AWE USA 2018 run May 30-June 1, 2018 at the Santa Clara Convention Center in Silicon Valley.
“For almost a decade, AWE has recognized hundreds of exceptionally talented and profoundly influential AR and VR technologists whose work is changing the world,” said Ori Inbar, co-founder and executive producer of AWE. “This year’s submissions—the largest number in AWE history including many by giant companies — represent an incredibly diverse and competitive assortment of achievements. These finalists illustrate the XR technologies and use cases with the greatest potential to impact our lives.”
This year Alla Vovk gave a talk at AWE USA about Spatiality of Augmented User Interface. Her talk was focused on what is a spatial interface and how people interact with the physical-digital environment in real workplaces. Intuitive things that are adoptions from the physical world and their combination with basic principles of human perception are implemented in AR training applications.
Additionally, the annual Auggie Awards have been the most recognized industry AR & VR awards in the world since 2010. Now in its 9th year, the Auggies continue to showcase the best-of-the-best in XR (AR/VR/MR) with this year seeing over 250 nominations in 10 active categories. Past winners include successful companies such as Spin Master, Lowe’s, Leap Motion, Meta, Microsoft HoloLens, Zappar, PTC Vuforia and more.
Sixty-four companies and innovators were announced as Auggie Award finalists. The 2018 finalists by category (and in alphabetical order) are:
Best Art or Film sponsored by IEEE Standards Association
AURORA by Pink Kong Studios
CAQUETÁ: Alianza con la Naturaleza (CAQUETÁ: Alliance with Nature) by VR Americas LLC
Charlie Fink’s Metaverse, An AR Enabled Guide to VR & AR by Charlie Fink
Karim & Noor by Blink Studios
Parabola @ SF Design Week by Heavy Projects
Best Campaign sponsored by PTC Vuforia
Acura AR Race by Current Studios
Augmented Passion: Russia World Cup 2018 by VR Americas LLC
BK Dinos by bizAR Reality
Fox Movies VR by Creote Studio Limited
I Caught It AR Cannon NY Jets Campaign by Xperiel
Johnson Controls HoloLens Product Launch by Vaylian Studios
Living Wine Labels, Treasury Wine Estates by Tactic
Best Consumer App
IKEA Place by IKEA
Markilux 3D – the most sophisticated AR-3D-configurator by ViewAR
Meo: Your real AR avatar by Meo
My Austrian Premium Economy by ViewAR
Pulse-Tech by Pharmaconsult
Skrite – Social Sky Messages by Skrite Labs Inc.
Streem by Streem
Best Creator & Authoring Tool sponsored by Microsoft
AR Designer by Envrmnt by Verizon
Augmania by Augmanian
awe (Augmented Web Experiences) by awe.media
Gravity Sketch by Gravity Sketch
Lens Studio by Snap Inc. by Snap Inc.
Studio by PTC
Best Developer Tool sponsored by DAQRI
AR Navigation by Insider Navigation
Navisens MotionDNA by Navisens
Unity by Unity
ViewAR SDK by ViewAR GmbH
Vuforia Engine by Vuforia
Wikitude SDK 7 by Wikitude
Best Enterprise Solutions sponsored by Kopin
Ethical Self Brain Hacking by VR Americas LLC
HiAR Industry by HiScene Information Technology CO., Ltd
Holoroom Test Drive by Lowe’s Innovation Labs
Proceedix by Proceedix
Skylight by Upskill
Volkswagen AR indoor navigation by Volkswagen / Insider Navigation
VR/AR in Surgery by ARIS MD
Best Game or Toy
DroneTopolis AR by CircleSquare Entertainment
HoloTats by Balti Virtual
Little Hippo AR Books by Little Hippo Books
MERGE Cube by MERGE
PuzzlAR: World Tour by Bica Studios
Red Matter by Vertical Robot
Star Wars: Jedi Challenge by Lenovo
Best Headworn Device sponsored by Qualcomm
Action One by Shadow Creator Information Technology Co.,Ltd.
DigiLens MonoHUD by DigiLens Inc.
HiAR G100 by HiScene Information Technology Co., Ltd
HMT-1 by RealWear
IQbuds by Nuheara
PlayStation VR by Sony Interactive Entertainment America
Toshiba dynaEdge AR Smart Glasses by Toshiba
Vive Pro by HTC Vive
Best Input & Output Hardware sponsored by Bosch
Haptics Digital Signage by Ultrahaptics
Holo-Stylus by Holo-Light
Mudra Inspire by Wearable Devices LTD
R-VISTA70 by RAONTECH
Sense Glove by Sense Glove
Simulated Reality by Dimenco
Best Interaction Software Tool
Future of VR Storytelling by Quantum Interface
GeoCV Smart Phone Scanner by GeoCV
Lumo Play by Lumo interactive
ManoMotion by ManoMotion
Superb Reality XR Gesture Control & Hand Tracking software by Superb Reality
For information, visit https://augmentedworldexpo.com or follow @ARealityEvent on Twitter.
Augmented Reality is the future, everyone agrees, but do you really know what it takes to create Augmented Reality applications? On May 24, 2018, in a one day public event, experts invited by the AR-FOR-EU project provide a briefing about future ‘maker skills’ needed in the field of Augmented Reality. At the event, the Eramus+ strategic partnership will inform about required knowledge, skills, and competences, learning methods, key learning outcomes, performance levels, and assessment criteria, filling the gap between formal education and the workplace.
Dr Fridolin Wild, director of PAL, was invited to attend a Showcase and Networking meeting at the Microsoft Hololens Lounge in London – as one of twelve universities invited, on May 10, 2018. Microsoft shared some details about the Mixed Reality strategy and observations on the importance of academia as enable to industry, including an announcement for two new mixed reality apps (remote assist and layout, both now in the store). The universities shared their research. The building and demos took place in the stylish Hololens Lounge.
Alla Vovk from Performance Augmentation Lab is unveiling a series of new research concepts at this year’s ACM CHI Conference, held between 21st-26th April in Montreal, Canada, which have the potential to revolutionise how we interact with technology.
The ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction, where researchers and practitioners gather from across the world to discuss the latest in interactive technology. As part of the conference, Alla is presenting a paper about new ways for people to interact with emerging technology to delegates from worldwide academic institutions and high profile companies such as Microsoft, IBM Research, Google and Facebook.
The ‘Simulator Sickness in Augmented Reality Training Using the Microsoft HoloLens’ paper explores the effects of using head-worn augmented reality displays such as motion sickness.
Augmented Reality is on the rise with consumer-grade smart glasses becoming available in recent years. Those interested in deploying these head-mounted displays need to understand better the effect technology has on the end user. One key aspect potentially hindering the use is motion sickness, a known problem inherited from virtual reality, which so far remains under-explored. In this paper, we address this problem by conducting an experiment with 142 subjects in three different industries: aviation, medical, and space. We evaluate whether the Microsoft HoloLens, an augmented reality head-mounted display, causes simulator sickness and how different symptom groups contribute to it (nausea, oculomotor and disorientation). Our findings suggest that the Microsoft HoloLens causes across all participants only negligible symptoms of simulator sickness. Most consumers who use it will face no symptoms while only a few experience minimal discomfort in the training environments we tested it in.
The full paper can be found here:
We are looking for a Post-Doc to join our dynamic and interdisciplinary team with a strong tradition of excellence in research and outstanding industry links in the professional fields of Augmented Reality and Wearable Technologies. Click here for further particulars.
It is been a great week for PAL, the first two days of the week were devoted to education. Alla spoke at the STORIES 2018: Technology Matter Conference, an annual event organised by the Department of Education, University of Oxford. This year’s topic was the use of technology in education and research, and Alla presented a talk about the use of augmented reality in education.
It this moment we can not provide you with the images from the event, but you can trust us, it was amazing:)
There were talks on engagement through technology, improving performance in the classrooms, and very energetic discussions about methods and data analysis.In particular, we want to highlight the talk by Anna-Maria Ramezanzadeh about “Qualtrics” – an online tool for collecting data during the experiments.
If you’ve missed Alla’s talk, you can read her abstract below.
“A powerful medium is what enables a powerful representation. Over the past decade, researchers have been actively exploring augmented reality and thinking about how we can support users in performing skilled tasks. We look at how technological advancements in augmented reality are increasingly redefining our understanding of spatial interaction, tools, and processes and why augmented reality is a platform for future learning. This talk will focus on the application of augmented reality in education, knowledge transfer and information visualisation. We address this question by conducting an experiment with 142 subjects in three different industries: aviation, medical, and space, looking at different aspects of interaction such as technology acceptance, simulator sickness, system usability and user interface design. The software application was designed for Microsoft HoloLens for experience capturing and experience re-enactment, where experts can create a training sequence, and students can replay it and learn in a totally different way. The idea to create a comprehensible representation of knowledge and its systematization in a knowledge-intensive environment enables intuitive interactions with holographic objects and reduces cognitive load”.
And, still to come:)
In a partnership of five leading new universities, we are building a innovative teaching program for Augmented Reality for Computer Scientists (and students of related disciplines). As a first step, we are now running an open survey to find out which skills need to be developed in the courses we will create over the next three years, generously supported by Erasmus+. If your organization engages in AR, would kindly fill in the following
questionnaire? It takes 7-8 minutes: http://codereality.net/survey/
Our artists in residency, Yann Deval and Marie G. Losseau, presented a preview of ATLAS, the work in progress, on invitation of the European Commission at SxSW in Texas, Austin, from March 10-12. ATLAS is an experience that mixes augmented reality and virtual reality and spectators were invited to engage with it to build virtual cities using the ATLAS seed launcher. Each seed sprouts a house, following some urbanistic rules, adapting to their environment. There are houses in the clouds, uprooted houses, houses on stilts, flying houses, and more. The cities assembled by the audience take on a life of their own, with and without interaction of the users, just like any living organism. The work allows to assemble huge cities, in which you can wander and loose yourself. It provokes a reflection on urbanism, architecture, and their influence on our lifestyle. It gives life to the inanimate.
The big question ATLAS helps to explore is one about the relationship of physical space to virtual space. ATLAS transforms the user’s physical space into a map of the environment, swapping perspectives.
The map serves as a launch pad for sprouting houses, establishing a link between the physical space of the user’s surrounding and the holographic houses.
Technically, this was realised using custom shaders, using CG, the shader programming language by NVidia. Holographic houses can be hidden, when they pass behind a physical object. This was created with ease using Hololens tools, but adds real value and credibility to the experience.
The Hololens, by Microsoft, is a ‘developer kit’, a prototype. The space where holograms are displayed is very limited, and that can often disrupt the feeling of immersion for the user. ATLAS uses this limitation as a creative constraint: user gaze unveils the virtual overlays organically, making the limitations appear as if by choice.
Though not an essential technical change, from the user perspective this was a significant one. Frustration decreased and the number of participants mentioning the limitation of the field of view decreased eminently.
ATLAS will offer different chapters, each submerging the user into another layer of augmented reality. Dividing the experience into chapters is perhaps an influence of literature on our work.
To give the houses life, an animation was created with Houdini, a 3D software for procedural animations and simulations. Houdini is a procedural tool, which helped to speed up development of the organic animation, compared with when using other tools such as MeshLab or Cinema4d.
Several different houses were 3D-scanned and integrated, some of them built by artist Marie Loseau, others by 9-year-old kids during the MCCS series of workshops in Molenbeek, Brussels.
To build bridges between realities, windows were created that let the spectator view into a VR world.
The outcome of the residency at this point is a functional prototype, a five minute MR (Mixed Reality) experience for a single user.
The prototype was also presented at MCCS Molenbeek, where 100 kids participated in the project Classes Urbaines. A lot of ideas for the next chapters of the work are coming together during discussions with the WEKIT project team and our Oxford Brookes University students. These will be integrated in September during the next Oxford residency!
ATLAS (prototype): An experience by Yann Deval & Marie G. Losseau
In collaboration with WEKIT / Dr. Fridolin Wild (Oxford Brookes University) for the Mixed Reality part.
Kids from Molenbeek’s schools 1, 5 and 10 for contributing to the building of the cities.
With the support of VERTIGO STARTS program of the European Commission with IRCAM-Centre Pompidou and EPFL
Maison des Cultures et de la Cohésion Sociale de Molenbeek Saint-Jean
Programming, graphic design and music by
Model making and scenography by
Marie G. Losseau
WEKIT (Oxford Brookes University / Performance Augmentation Laboratory)
Dr. Fridolin Wild
External contributions (Mixed Reality)
TriplanarWorld Shader by Robert Yang
SpatialMappingRendererWithNormals by Matt@DeckTwelve
Professor Anu Ojha from the National Space Academy guided us through the day of an astronaut, explaining how the pressure works and what are the pressure difference effects on the body. With him, our team experienced the Sokol space suit, a type of Russian spacesuit, worn by all who fly on the Soyuz spacecraft! We were trying to understand the effect of a mechanical pressure suit on a body of astronauts and think how we can integrate different sensors to measure physiological parameters alongside augmented reality glasses. The suit consists of an inner pressure layer of rubberised polycaprolactam and an outer layer of green nylon canvas. Boots are integrated with the suit but gloves are removable and attach by means of blue anodised aluminium wrist couplings. We must say the full suit is quite heavy to wear, especially when you think about it being on your body for more than 10 hours per day, but it’s a unique feeling when your body is squeezed by the air.