Augmented Reality: The Blend of Fantasy and Reality
In recent years, there have been many games which went viral and were downloaded millions of times. Some like Candy Crush were loved for good-enough reasons while some like Blue Whale grabbed headlines for their harmful nature. But one game stood out was Pokémon Go because it ushered in a new era in gaming. The game required its player to move around and find different Pokémon characters at different locations. A cute little Pikachu could be seen hiding behind your sofa or a ferocious Electrode would have captured your bed. This blending of the real and the fantasy worlds was made possible by Augmented Reality (AR). With the announcement of Metaverse, AR and VR (Virtual Reality) have gained more significance. Let us understand what AR is and its types.
What is Augmented Reality?
For example, when you open AR Zone app on your Samsung mobile, it makes use of your phone camera to capture your surroundings, which then are augmented with a picture or animation of a dinosaur. And voila just like that, your living room turns into a Jurassic Park!
Though the visual content dominates AR, the tech can also support other sensory modalities such as olfactory (relating to smell), auditory (concerned with hearing) and tactility (associated with touch).
AR – A Historical Perspective
The invention of AR, which sometimes is also referred as Mixed Reality, goes back to 1968 when Ivan Sutherland, a Harvard professor and computer scientist, created the first head-mounted display called ‘The Sword of Damocles’. Sutherland succeeded in providing the user with an enhanced sensory perception through the use of computer graphics. This was the time when augmented and virtual reality were essentially the same concept. Later, as time and tech progressed, the two split and advanced further.
In 1990, Tom Caudell, a researcher at Boeing coined the term ‘Augmented Reality’ for the technique that was used to overlay artificial graphics on the real-world. 1992 was the year when a ‘true’ AR system, as it is defined today, came into being. Louis Rosenburg, a researcher for the US Air Force, created a fully functional AR system called ‘Virtual Fixtures’ that allowed military personnel to virtually control and guide machinery to perform tasks such as training pilots on safe flying practices. Since the 90s till today, AR has come a long way with applications ranging from e-commerce to archaeology.
Types of AR
- Marker-based AR
This type of AR is also known as recognition-based AR because it depends on triggers in the form of marker images to activate. These markers could be user-defined images or distinct patterns that are easily recognised or processed by the camera. Though, a black and white square with a bold border is preferred as a marker, certain objects in a person’s surroundings can also be used as triggers. For example, the Lenskart app uses marker-based AR technology to show how a particular pair of spectacles or sunglasses would look on you. In the app, as soon as your face, especially your eyes and nose, is detected, a pair of glasses are strapped on your live camera feed. Thus, your face acts as a trigger for the AR to initialize.
The image set as a marker has to be unique so stock photos from the web are avoided. For marker recognition, Computer Vision (CV) algorithms are used. A repository of markers is created either locally (i.e. on the mobile device) or sourced from a cloud-based database. Image recognition and augmentation is faster and occurs without needing a network in the former system rather than the latter. On the other hand, access to external marker database requires a network (internet) connection. Marker-based AR faces challenges such as requirement of huge contrast within the marker (bold black and white colours) and loss of augmentation in case the marker is not visible.
Another interesting application of marker-based AR is in museums. While visiting a museum or a historical site, we often fail to read every plaque that describes an object. In such a case, an AR-enabled headset or a smartphone app that scans the marker near the artifact. This shall start an audio clip describing the object and stating related historical facts. The Museum of Modern Art in New York has created an exhibit in their art gallery that showcases AR features which viewers can watch using a mobile app.
- Marker-less AR
Because there are limitations on the situations where marker-based AR can be used, Marker-less AR was developed. It removes the need for an outside prompt in order for the AR to initiate. It is highly flexible in allowing users to decide the position of an augmented object, animation, image, etc. This flexibility further extends to the object’s size too. To illustrate, if you desire to see whether a wardrobe fits in your bedroom, you can use the Ikea mobile app to place the furniture piece in the designated spot, along with other items. Thus, you have a whole sample bedroom look, all done virtually! Hence apps or devices that make use of Marker-less AR require highly sophisticated computer vision and image recognition as well as positioning softwares.
AR applications that use marker-less type for augmentation, rely on sensors (accelerometer, gyroscope, etc.), camera and processor of the device, for example, a smartphone. Complex mathematical formulae and algorithms are utilized to showcase the precise location and size of the augmented object. Due to the suitability to a narrow pool of specialized applications, marker-less AR may not make sense in certain situations – such as in the aforementioned case of Lenskart AR app.
- Location-based AR
This type of AR is activated at a predetermined location, so it involves mapping of images, audio, etc. onto specific locations. The AR kicks in after the prescribed location has been detected using data from the device camera and sensors such as GPS, digital compass and accelerometer. These sensory inputs are analysed to estimate where the device is focussed. Post this detection, relevant information is juxtaposed over the real camera footage and shown to the user. Furthermore, it allows users to place objects at any designated location. This type of AR continuously detects location for allowing dynamic pairing of the objects in the real-world with the AR information.
As this type does not deploy any marker, it is flexible than its marker-based counterpart. It also remains unhindered by weather and ambient conditions such as fog and light reflections that make detection almost impossible in case of recognition AR. It is most suitable for applications at tourist sites but suffers from fallacies in location-detection due to faulty or inaccurate sensors, thus causing an error in AR overlay.
An excellent example of location-based AR is the game Pokémon Go, which as discussed earlier, plants various characters at different locations and guides players in retrieving them.
- Superimposition-based AR
This type of AR is used in Snapchat and Instagram filters wherein your face or your surroundings are replaced either fully or partially by an augmented view. It involves recognition and identification of an object first, followed by the superimposition of an augmented animation over the original image. Hence it requires highly sophisticated computer vision and position detection softwares.
An AR Society
According to the report by ‘Global Future Council on AR & VR’ by World Economic Forum, the two technologies have seen an upsurge in demand in the post pandemic world. They are predicted to be a key component of Metaverse, a virtual world conjectured to be the future iteration of the internet. Uptick in consumer adoption of AR devices in the realms of gaming and fitness, has driven up sales while businesses continue to expand their usage of mixed reality devices for maintenance, design as well as training. The report further states that these immersive technologies have a huge potential to drive positive societal changes in high-impact fields of healthcare, education and arts.
With a market size of 6.12 billion USD in 2021, AR is projected to rise at a CAGR of 48.6% to a whopping 97.76 billion USD till 2028. PricewaterhouseCoopers (PwC), a reputed analyst firm, predicts AR and VR to boost global GDP by up to $1.5 trillion. With huge investments in men and money pouring in and the rise in consumer appetite for newer and better devices, it is impossible to ignore immersive technologies as the world steps into the combined forces of high 5G network speeds and the innovation-driven Industry 4.0.
Vishvali Deo is an E&TC (Electronics and Telecommunication) Engineer by education and Software Engineer by Profession. She believes that 'Technology is a Great Democratising and Equalising Force' and hence is on a mission to make the general public understand seemingly complex technologies in a simple manner.
She is convinced that the root of today's world problems lie in the past, hence she has also pursued post-graduation in History. She has a keen interest and a good grip over Economics, Political Science and Environmental Engineering. She has a penchant for working with Women and spreading Digital Literacy amongst them, with the aim of their empowerment. She also strives to provide Free Quality Education to children and counsels young adults. Besides, she is also skilled at Public Speaking, having won many awards in Elocution & Debate Competitions and Technical Paper Presentations.