Invented by Gary R. Bradski, Samuel A. Miller, Rony Abovitz, Magic Leap Inc

The market for methods for creating virtual and enhanced reality has been growing rapidly in recent years. With advancements in technology and increased demand for immersive experiences, businesses and individuals are seeking innovative ways to create virtual and enhanced reality content. Virtual reality (VR) refers to a computer-generated simulation that immerses users in a three-dimensional environment. Enhanced reality (ER), on the other hand, combines elements of the real world with virtual objects, creating an augmented experience. Both VR and ER have gained popularity across various industries, including gaming, entertainment, education, healthcare, and even marketing. One of the primary methods for creating virtual and enhanced reality is through the use of specialized hardware and software. VR headsets, such as the Oculus Rift and HTC Vive, provide users with a fully immersive experience by tracking their head movements and displaying virtual content accordingly. These headsets are often accompanied by motion controllers, allowing users to interact with the virtual environment. In terms of software, there are several platforms available for creating virtual and enhanced reality content. Unity and Unreal Engine are two popular game development engines that support VR and ER. These platforms provide developers with the tools and resources needed to create realistic and interactive virtual worlds. Another method for creating virtual and enhanced reality is through the use of 360-degree cameras. These cameras capture footage from all angles, allowing users to explore the environment in a more immersive way. This method is particularly popular in the entertainment industry, where filmmakers and content creators are using 360-degree cameras to create immersive videos and experiences. Additionally, advancements in computer vision and artificial intelligence (AI) have contributed to the development of methods for creating virtual and enhanced reality. AI algorithms can analyze and interpret real-world data, allowing virtual objects to interact with the environment in a more realistic manner. This technology has applications in various fields, including gaming, training simulations, and even virtual shopping experiences. The market for methods for creating virtual and enhanced reality is expected to continue growing in the coming years. According to a report by MarketsandMarkets, the global augmented reality and virtual reality market is projected to reach $571.42 billion by 2025, with a compound annual growth rate of 63.3% from 2020 to 2025. This growth can be attributed to the increasing adoption of VR and ER across industries and the continuous advancements in technology. In conclusion, the market for methods for creating virtual and enhanced reality is expanding rapidly, driven by advancements in technology and increasing demand for immersive experiences. Specialized hardware and software, 360-degree cameras, and AI algorithms are among the key methods used to create virtual and enhanced reality content. As the market continues to grow, we can expect to see more innovative solutions and applications in various industries.

The Magic Leap Inc invention works as follows

The configurations disclosed are for presenting virtual and augmented realities to users. The system can comprise an image-capturing device that captures one or multiple images. These images correspond to a user’s field of view when using a head mounted augmented reality device. A processor is communicatively connected to the image-capturing device and extracts a map point set from the images. It then identifies a sparse set and a dense set from the extracted map points.

Background for Methods for creating virtual and enhanced reality

Modern computing and display technology has facilitated the development systems for so-called?virtual realities? “Modern computing and display technologies have facilitated the development of systems for so-called ‘virtual reality’ Digitally reproduced images, or portions thereof, are presented in such a way that they appear to be real, or can be perceived to be so. Virtual reality (VR) scenarios typically involve the presentation of digital image data without any transparency to the real-world input. Augmented reality (AR) scenarios typically present digital image data as an enhancement to the visualization of the world around the user. An augmented reality scene, for example, may allow the user to see virtual objects superimposed over or in between real world objects.

The human visual perception system can be complex. It is difficult to create a VR/AR technology that allows for a rich, comfortable presentation of virtual elements alongside other real-world or virtual imagery elements. The traditional stereoscopic glasses feature two displays, which are set up to display slightly different images. This allows the human visual system to perceive a three-dimensional perspective. Many users have found that these configurations are uncomfortable due to the mismatch between vergence, and accommodation. This can be overcome in order to perceive images in three dimensions. Some users cannot tolerate stereoscopic setups.

The human eye is a complex organ that consists of a cornea and iris. It also has a macula and retina. There are also optic nerve pathways from the retina to the brain. The macula, located in the middle of the retina and used to view moderate detail, is the macula. The fovea, a part of the retina at the center of macula, is used to see the finer details in a scene. It contains 120 cones of photoreceptors per visual degree, more than any other area of the retina.

The human visual system does not function as a passive sensor system. It actively scans its environment. The photoreceptors in the eye respond to stimulation changes, not a constant stimulation. This is similar to using a flatbed to scan an image or a finger for Braille reading. “Motion is needed to transmit photoreceptor data to the brain.

In fact, experiments using substances like cobra venom to paralyze eye muscles have shown that blindness will occur if the subject is positioned with their eyes open and viewing a static image with the paralysis caused by the venom. The photoreceptors cannot provide information to the brain without any changes in stimulation. Blindness results. This is thought to be at least one of the reasons why normal human eyes have been observed moving back and forth or dithering in a side-to-side movement, also called “microsaccades”.

The brain’s visualization center also receives valuable information about perception from the movement of both eyes, and their components. The vergence (e.g. rolling movements to bring the eyes together to fixate on an object, or to diverge the lines of vision) of both eyes in relation to each other is closely related to the focusing (or “accommodation?”) The lenses of the eye are closely associated with focusing (or?accommodation?) Under normal conditions, changing the focus of the lenses of the eyes, or accommodating the eyes, to focus upon an object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the ?accommodation-vergence reflex.? Normal conditions will also trigger a change of vergence. It is well known that working against this reflex can cause eye fatigue, headaches or other types of discomfort for users.

The movement of the head which houses the eyeballs also has an important impact on the visualization of objects. People tend to move their head to see the world. They are constantly repositioning or reorienting it in relation to an object. Most people will move their heads if their gaze must be moved more than 20 degrees to focus on an object. (For example, they do not like to view things “from the corner” of their eyes). People also move or scan their heads when listening to sounds. This is to improve the audio signal and to take advantage of the geometry between the ears and the head. Head motion parallax, which is a relationship between the relative motions of objects at various distances and head motion as a function eye vergence, provides powerful depth cues to the human visual system. If a person fixes his gaze on an object and moves their head side-to-side, objects farther away from the object will move the same way as the head. Items in front of the object will move the opposite direction. These cues can be extremely useful in determining the spatial location of objects relative to a person. “Of course, head motion is also used to look at objects.

Further head and eye movements are coordinated with a’vestibulo ocular reflex,’ which stabilizes the image information relative the retina during head rotates, keeping the object information about centered on retina. The eyes will rotate in the opposite direction as a reflexive and proportionate response to the head rotation. This is done to maintain a stable fixation. This compensatory relationship allows many people to read a book even while moving their head from side to side. The same is generally not true if the book moves back and forth with the head essentially stationary. This reflex is primarily developed to coordinate head and eye movements, but is not usually used for hand motion. This paradigm is important for AR systems because the head movements of users may be directly associated with eye motions. An ideal system will preferably be ready to work in this relationship.

With world-centric configurations it may be desirable for inputs to include accurate head pose measurements, accurate representation or measurement of real-world objects and geometries surrounding the user, low latency dynamic rendering of the augmented reality as a function head pose, as well as a general low-latency display.

The U.S. Patent Applications listed below present systems and methods to work with a typical person’s visual configuration to address different challenges in virtual and augmented realities applications. These virtual reality systems and/or AR present a number of challenges. They include the speed and quality of the virtual content delivered, the eye relief for the user, the size and portability, as well as other system and optical issues.

The systems and techniques described in this document are designed to work with the visual layout of a typical human being to address these challenges.

Embodiments” of the present invention relate to devices, methods and systems that facilitate virtual reality or augmented reality interactions for one or multiple users. A system for displaying digital content is disclosed in one aspect.

The augmented-reality system in one aspect comprises an image-capturing device that captures one or multiple images. These images correspond to the field of view of a person wearing a head mounted augmented reality device. A processor is communicatively connected to the image-capturing device and extracts a map point set from the images. It then identifies a sparse point set and a dense point set from the extracted map points. Finally, it performs a normalization of the map points set.

The detail description, figures, and claims describe “Additional and Other Objects, Features, and Advantages of the Invention.

We will now describe in detail various embodiments with reference to the illustrations, which are provided to allow those who have the knowledge to apply the invention. The figures and examples are not intended to limit the scope or application of the invention. In cases where certain elements of the invention can be implemented partially or completely using known components, methods, or processes, only the portions that are essential to understanding the present invention (or the methods or process) will be described. The detailed descriptions of any other portions will be omitted in order not to obscure it. Further, different embodiments include present and future equivalents of the components that are referred to in this document by way of example.

In this specification, specific embodiments of the invention have been described. You will see that there are many ways to modify and change the invention without departing from its spirit and scope. The process flows described above, for example, are described in relation to a specific ordering of the process actions. The order of some of the described processes actions can be altered without affecting the scope of the invention. “The specification and drawings should be considered in a more illustrative than restrictive manner.

Disclosed are systems and methods for generating virtual or augmented reality. Virtual content can be delivered strategically to the eyes of the user in order to create a realistic virtual reality or augmented reality experience. This is done while respecting the physiology and limitations of the human eye. This disclosure provides various embodiments of optical systems that can be integrated into AR systems. The disclosures will primarily be in the context AR systems. However, the technologies can be applied to VR systems as well.

Click here to view the patent on Google Patents.