Augmented reality refers to the utilization of computers to modify reality, usually to provide additional assistance when a human user is interacting with the real environment. The real world around humans provides a wealth of information that the human user must absorb and process through their senses. The most useful and informative of all human senses is the sense of vision and a huge amount of information about the ambiance is required to be sensed as well as processed by the human visual system. Computers are useful because they can provide an overlay of information to assist with the human processing of the information which they perceive through their senses, mostly the visual sense.
As an example, a human who is walking around an urban area can be provided additional information through a head-mounted display which superimposes textual information about the identification of buildings and other important landmarks so that the human is provided additional guidance about the identification of buildings and landmarks or other useful information about what they are viewing. The computer has to sense the environment to provide any additional information and this is done through a camera that captures the same view as the human. The camera provides images that are identified by a computer and virtual scenes are created in the virtual world generated by a computer. The computer program then generates additional assistive information that is presented to the human user to assist them in their interactions with their environment.
The augmented reality software program is required to process information that a computer senses through sensors and the real space is transformed into a virtual space within the computer, with computer software identifying, recognizing, and transforming the external inputs from the real world to transform these inputs into assistive outputs for the user of an augmented reality system. Augmented reality can, therefore, be considered to be a variation of virtual reality which completely immerses a user inside a synthetic environment. Augmented reality makes it possible for a human user to view the world with virtual objects that are generated by a computer and then superimposed or merged with it. The characteristics of virtual reality systems include a combination of the real and the virtual, interactions in real-time, and registration in 3 – D.
 |
| Augmented Reality System Applications Essay; Image by Pexels from Pixabay. |
Augmented reality has proven itself to be useful in a wide variety of applications. From assisting surgeons to perform delicate surgery to assisting soldiers on the battlefield or helping with education as well as assisting archaeologists to gather field data on a site, virtual reality has proven to be of great benefit to humans in a very wide range of applications. However, virtual reality technology is still evolving and with advances in computer science, even greater sophistication is likely to be possible in a large number of applications. Advances in wearable computing, software, miniaturization, display technologies as well as sensors, and radio networking have had a profound impact on the application of augmented reality.
Augmented reality systems are now far less bulky, portable and can be worn by a human user who may want to be assisted in a variety of ways to perform a task. Systems making use of augmented reality are now often used outdoors in harsh weather conditions. Despite the possibilities, virtual reality implementations do have some limitations as well as difficulties with implementation. Registration errors or problems associated with sensing or bandwidth of the display devices can cause deterioration in the performance of augmented reality systems.
Objects in the real and the virtual world have to be aligned correctly about each other otherwise the illusion which is created as a result of the two worlds coming together will not be comfortable for humans. Sensing devices that provide data inputs for the augmented reality computer have to be able to track the field of vision of a user. Sensing errors, mechanical alignments, incorrect viewing parameters such as field of view or tracker –to- eye position or orientation as well as optical distortion in the display system are some of the problems which can affect the performance of an augmented reality system.
In this essay, an attempt has been made to discuss various aspects of augmented reality systems, including perspectives associated with their construction, portability, user benefits for those using augmented reality, outdoor use of augmented reality systems, and the problem of errors in such systems. It is hoped that the readers will benefit from such a discussion and gain a broader understanding of the current state of augmented reality.
A Description of the Operation of an Augmented Reality System;
Augmented reality systems usually add to the information that is received from the real world. In augmented reality vision systems, objects from the virtual world generated by a computer can be added to the scene from the real world. However, augmented reality systems can also remove sensations from the real world, and as an example; it is possible to remove a table that a human using the augmented reality system may be viewing as a result of the computer painting it over. Although augmented reality usually refers to a computer modifying the real information that is presented to the human vision system, vision is not the only sense that a computer can interfere with.
A system in which multi-sensory input is provided may turn on a heat lamp when a user approaches a spot in the virtual environment that is exposed to the sun or the smell of roses may be directed to a user when they approach a rose garden in a mixed environment. A computer can also direct that a fan be switched on to provide a flow of air to a user when they approach a spot in the virtual environment that is exposed to wind. Thus, augmented reality involves modification of reality that is presented to a user by a computer that also senses reality and which has created a virtual model based on what has been sensed. Examples of the application of augmented reality include the superimposition of internal information over external surfaces or the augmentation of the viewed environment with informational labels.
When thinking about augmented reality, it is appropriate to think about the physical space, the virtual space, and the measured space. The concept of physical and virtual space should be clear, however, the concept of measured space refers to the representation of the physical space inside a computer and how sensory data is used to present the real space to a user. The generation of augmented and virtual spaces requires the joining together or overlapping of the space maps associated with the physical and the virtual spaces, with the interaction being managed by computers and people using their sensory capabilities.
Virtual space does not necessarily represent the physical space and this space is a model of the physical space that is created the real world and its views as perceived by sensors. Virtual space contains inputs from the physical space which are incorporated into the electronic space by the computer. The generation of augmented reality involves superimposing electronically generated space onto real objects and space. The virtual space inside a computer must correspond to the real space which has to be mapped into the computer using some sort of a global reference such as a global positioning system.
Relative measurements of objects in physical space are transferred to the computer so that the electronic space may be generated, even though it may not be known where the physical space is concerning the rest of the universe. Thus, a car’s interior dimensions are relative to each other, although it may not be known where the car is in the universe. A close mapping may exist between the physical and the virtual space if the electronic space is required to be a close representation of the physical space. It may be required to represent different parameters from the real space into a virtual space, such as the variation of light intensity or temperature.
Some events in the physical space may also be required to generate a trigger and somehow alter the electronic virtual space. Different types of sensors may be used to provide information about the physical space such as the dimensions of the physical space being measured by ultrasound or sonar. Sensor inputs are required to be fed into a computer, processed, and then used to present a reflection in the electronic space. Triggers from the physical space are also measured by sensors and as an example; the presence of a human object in an area of the physical space may be registered by a capacitive sensor that will transmit this information into the computer.
The computer that is being used may then take some action to process this trigger and generate some outputs or variations in the electronic space. Virtual spaces that are generated inside a computer after some processing of the data related to physical spaces are translated and made available in the real world by projection involving video, audio, tactile or hepatic devices or even by using the sense of smell. The virtual world of the computer is projected at some point or location in the physical world which is known as the point of projection. Devices such as a screen, a virtual reality goggle, PDA, etc are used for such projections.
Projections of the virtual world into the real world through devices present users with an illusion of occupying some part of the physical space such as the space behind a projection screen or in front of a holographic plate. In purely virtual reality environments, the sensing of an object from the physical space may have an impact on the projection of the virtual space into the physical space, however, the object that is being sensed will not be a part of the virtual space. In mixed or augmented spaces, however, the sensed object will be incorporated into the virtual space and hence the link between the object that is being sensed and the projection is important.
Location in space is usually measured in terms of some sort of coordinates which could be Euclidean coordinates such as polar, spherical, or ordinance grid coordinates, or the location may be relative to a zone with objects being detected as being in a zone such as a room or a part of a room. Information about the position may also be relational with some objects being detected as being close to another object. The level of accuracy with which measurements in the physical space need to be made and information in the virtual space need to be displayed will vary for different applications, but the requirements related to accuracy will determine the amount of data that needs to be exchanged between the physical and virtual spaces.
As an example, head-mounted displays and cameras that are mounted on these displays can be used to present computer-modified reality to a user. The cameras sense reality and feed a video signal to a computer which is then modified by the computer according to the programmed algorithms for the generation of virtual space involving the identification of objects, the addition of data to images, image manipulation, or object cancellation, etc. The desired additions or modifications to the human view of reality are then projected onto the human eye through head-mounted monitors or optical diversion and mixing of the real and virtual space. This is illustrated in the diagrams below.
The Generation of Visual Augmented Reality using Head-Mounted Displays, Cameras, and Diverters for the Mixing of Real and Virtual Spaces; Managing Multiple Spaces for Generation of Augmented Reality;
Apart from optical see-through augmented reality displays, it is also possible to generate augmented reality using monitors and video see-through displays. Optical see-through displays in which the real-world images are mixed with the virtual reality images that are generated by a computer using optical mixing are different from video see-through displays which project images that are desired to be seen by a human user onto the eye using a video display without any optical mixing from the real world.
The User Benefits of an Augmented Reality System;
Augmented reality systems attempt to present a world that consists of a combination of computer-generated imagery or other sensory inputs and real-world perceptions that are available to a human user. Thus, augmented reality systems can present an enhanced view of the world to the users and the enhancements provide additional information which can be of benefit to the user in a variety of ways. The additional information which can be superimposed on the perception of a user can take a variety of forms depending on the application or the intended use of the augmented reality application.
As an example, an augmented reality system may be designed to superimpose a pipeline system that has been designed for an industrial setting, or the system may assist a surgeon by superimposing internal imagery of a patient’s diseased organs on the image of their exterior anatomy, assisting the surgeon to perform surgery. All augmented reality systems assist humans to perform a task by enabling the human user to visualize, readily access additional information, or superimpose objects that are not visible onto real views.
Augmented reality systems can enhance human understanding of what they can perceive and thus humans are assisted in solving problems that may be difficult if they were not provided any additional pieces of information. However, the applications of augmented reality are very broad and this technology has proven to be useful in very many applications. Thus, augmented reality may be used for entertainment or gaming, providing additional input on views of sporting events as well as assisting humans in more serious endeavors such as the battlefield, archaeology or architecture or urban design, etc.
Objects that are superimposed on real world views using augmented reality may be required to be presented in correct perspective depth as well as being accurately positioned concerning other real or virtual objects and this can assist in human users being able to perform delicate work due to the additional understanding that they can acquire as a result of using augmented reality systems. Miniaturization of computing elements and advances in wireless as well as general technological advances on a broad front have made it possible for augmented reality systems to be miniaturized and to be made wearable or portable, adding to their general usefulness both indoors and outdoors. Thus, augmented reality systems can assist, entertain, inform or aid humans by enhancing their perceptions in a wide variety of applications.
Portable Augmented Reality Systems in Wearable Computers;
Portability is required if an augmented reality user has to change their position in the world. Unlike virtual reality, augmented reality users cannot change their position by changing their location in the computer-generated virtual world and have to be able to move around in the physical world. Portability is essential for a wide variety of augmented reality applications because users may be required to roam around in large areas. However, such requirement means that the augmented reality equipment has to be self-contained as well as portable and that it should be possible to track users outdoors.
The requirements for being able to track a user means that markers or communication relays will have to be utilized, especially if the computational requirements associated with augmented reality cannot be fully catered for on the portable end of the equipment and signals have to be relayed to a desktop, mainframe or networked computer. Because augmented reality does not replace the real world as virtual reality does, therefore, the resolution requirements associated with display devices or sensors are generally less stringent than those for virtual reality applications.
Full-color displays are usually required for virtual reality applications, but monochrome displays may be sufficient for augmented reality. Requirements associated with rendering are also not a problem with augmented reality applications because images are only required to add to the real-world visual inputs. However, tracking and sensing requirements for augmented reality applications are far more stringent than those for virtual reality applications because objects in the virtual space have to be correctly aligned with objects in the real world.
As a result of the requirement to accurately track a user’s head and objects in the surrounding area, a large volume of tracking data is likely to be generated which must be processed in real-time. If a portable or wearable computer is incapable of catering to these processing requirements, then reliable communications links must carry the tracking or associated data to remote computers. Thus, trackers or sensors are required to have a longer range, greater input variety, and bandwidth as well as better accuracy.
Advances in portable augmented reality systems have been made possible as a result of faster, more rugged, and better computing devices that consume low power. Wearable computers are required to be comfortable, safe, and adjustable for an individual’s personal preferences. The user interfaces for such systems have to be user-friendly as well as being capable of operating correctly in a harsh and noisy environment. Gesture and hand motion-based interfaces have shown a promise in wearable computing, but such systems are required to be able to interface with a user reliably. Software for augmented reality substantially increases in complexity with complex applications.
Short-range wireless communication technologies such as Bluetooth have made it possible for sensors to be readily placed on all parts of the human body. Thus, it may be concluded that portable augmented reality systems are required and useful but their widespread use and acceptance will depend on the reliability, ruggedness, comfort as well as computing power that can be packed into such systems. However, technological advances on a broad front have resulted in many portable augmented reality systems being developed including systems for military, field archaeology, or navigational assistance to name a few.
The human factors requirements associated with wearable computing demand that aspects related to safety, ergonomics, anthropometry, and ease of use or usability be considered in the design of portable or wearable augmented reality systems. Human users should be able to comprehend information that is being presented to them without any perceptual interference. The design of the wearable system should be based on ergonomic studies with a good visually coupled display design. The user interface design should promote reliability in interactions with the system and usability for the human user. The augmented reality system is required to minimize the cognitive load on the human user and not add to the problems. It should be possible to indulge in team collaboration through the wearable augmented reality system.
The design of user interfaces for human interactions is of the greatest importance in the design of wearable augmented reality systems design. Many usability guidelines should be considered for designs and the quality of the design for usability can also be used to evaluate an augmented reality system. The design of a user interface should take into consideration factors including the level of user experience, support for interactions with other users, number and location of users, ease of calibration, support for body-centered interactions, reduced relative latency, and provide the user with a tracking system that is accurate to a small fraction of a degree in orientation and a few millimeters in position.
The overall system processing delay should be minimized and there should be a level of consistency in the visual or sensory cues that are presented to the user. The errors which affect the performance of an augmented reality system and which are discussed in section 6 should be minimized. The considerations associated with a good design of a wearable augmented reality system are too numerous to be tabulated here, but a detailed discussion about these issues is presented in Gabbard, Joseph. L. “Researching Usability Design and Evaluation Guidelines for Augmented Reality (AR) Systems”. Virginia Tech. 2001.
The Outdoor Use of an Augmented Reality System;
Augmented reality systems are increasingly being used outdoors. Whereas it is obvious that the use of augmented reality is gaining greater acceptance because there is a need for applications to assist humans with various activities outdoors, it is also important to understand that the outdoor use of augmented reality presents some technical challenges. Examples of the outdoor use of augmented reality include assistance with rescue operations after a natural disaster such as an earthquake or assisting with military operations in an urban environment.
Both of the previously mentioned applications require augmented reality equipment to operate in a harsh environment in which any existing communications infrastructure is likely to have been obliterated or in danger of being destroyed. The military application can assist soldiers in identifying buildings, getting a feel for their interior, and being informed about potential threats such as the location of snipers from battlefield intelligence networks, with such information being conveniently displayed on their field of vision. However, communication links are important because sophisticated computations on data cannot be performed on portable augmented reality units.
Disaster relief can be aided by augmented reality as a result of rescuers being provided with on-the-spot information about buildings, terrain, and the likely results of any planned rescue operations such as the likely impact of clearing a path through rubble. Although the additional information that is presented by outdoor augmented reality systems is likely to be of great benefit, it has to be realized that the generation of such information is only possible after extensive digital surveys of the areas for which augmented reality is being used are available. The likely design of such augmented reality systems will also mean that not all computation or data storage is capable of being performed on the portable units which are provided to the field users.
The need to establish and maintain communication links as well as sophisticated computer equipment with which portable units are connected through communication links as well as the requirements to collect detailed data such as airborne laser scanning data under conditions of war or immediately after a natural disaster are some of the difficulties associated with the previously outdoor augmented reality applications. The equipment has to be extremely rugged and reliable as well as capable of being worn over long periods.
In military applications, the communication system which consists of fixed transmitters can be vulnerable and there is a requirement to maintain the large bandwidth communication links between the portable units and a central computer so that the augmented reality system can superimpose virtual scenes in real-world information with accuracy as well as reliability. It will be undesirable to have augmented reality system malfunctions in situations that require that users concentrate on other critical tasks rather than fixing any idiosyncrasies associated with the augmented reality equipment. Generally speaking, it is difficult for the existing tracking technology to track a user with sufficient accuracy when a portable augmented reality system is being used outdoors.
The Potential Sources of Error in an Augmented Reality System;
An important problem that has been observed in augmented reality systems is that objects which have been generated by the computer in a virtual environment for superimposition onto the view of the real environment do not remain correctly aligned or locked onto the real-world objects. The degree of misalignment can vary as the user changes their field of view and thus, with a changing field of vision due to a user moving their head, objects that are being viewed through the augmented reality displays can appear to swim as a user moves their head.
The errors which create problems in augmented reality systems can be broadly classified as static or dynamic errors. Static errors refer to those sources of error which can cause problems with the presentation of augmented reality when the user and the objects within the environment are completely still. Dynamic errors do not have an impact until there is a movement of the viewpoint or the objects. Some of the sources of error which can have an impact on the visual performance of augmented reality systems are as follows:
Static Errors;
- Optical distortion is caused by distortions or imperfections in the camera and lens systems, especially in objects which are away from the optical axis.
- Errors in the tracking and sensing system result from a distorted output from such systems.
- Mechanical misalignments exist within the components of the augmented reality system such as combiners, optics, or monitors due to factors such as not having a sufficiently rigid mechanical frame.
- Incorrect viewing parameters due to a lack of calibration that has an impact on how the reported head or camera locations are translated into viewing matrices which are used by scene generators for drawing images that are required to be superimposed on to the real-world inputs.
Dynamic Errors;
- End–to–end system delays cause problems because each component in the augmented reality system requires some time to produce an output for a given input. Any change in the position or orientation of a user will take some time before being processed and registered on the augmented reality picture that is presented to a viewer. Delays associated with the tracking subsystem, communication links, and scene generation as well as time delays that are associated with frame buffers will prevent an immediate updating of the augmented reality picture that is presented to the viewer. A certain level of delay or lag is acceptable and these delays will not cause a serious deterioration in the augmented reality presentation.