Invented by Gary R. Bradski, Samuel A. Miller, Rony Abovitz, Magic Leap Inc
The Magic Leap Inc invention works as follows
The configurations disclosed are for presenting virtual and augmented realities to users. The system can comprise an image-capturing device that captures one or multiple images. These images correspond to a user’s field of view when using a head mounted augmented reality device. A processor is communicatively connected to the image-capturing device and extracts a map point set from the images. It then identifies a sparse set and a dense set from the extracted map points.
Background for Methods for creating virtual and enhanced reality
Modern computing and display technology has facilitated the development systems for so-called?virtual realities? “Modern computing and display technologies have facilitated the development of systems for so-called ‘virtual reality’ Digitally reproduced images, or portions thereof, are presented in such a way that they appear to be real, or can be perceived to be so. Virtual reality (VR) scenarios typically involve the presentation of digital image data without any transparency to the real-world input. Augmented reality (AR) scenarios typically present digital image data as an enhancement to the visualization of the world around the user. An augmented reality scene, for example, may allow the user to see virtual objects superimposed over or in between real world objects.
The human visual perception system can be complex. It is difficult to create a VR/AR technology that allows for a rich, comfortable presentation of virtual elements alongside other real-world or virtual imagery elements. The traditional stereoscopic glasses feature two displays, which are set up to display slightly different images. This allows the human visual system to perceive a three-dimensional perspective. Many users have found that these configurations are uncomfortable due to the mismatch between vergence, and accommodation. This can be overcome in order to perceive images in three dimensions. Some users cannot tolerate stereoscopic setups.
The human eye is a complex organ that consists of a cornea and iris. It also has a macula and retina. There are also optic nerve pathways from the retina to the brain. The macula, located in the middle of the retina and used to view moderate detail, is the macula. The fovea, a part of the retina at the center of macula, is used to see the finer details in a scene. It contains 120 cones of photoreceptors per visual degree, more than any other area of the retina.
The human visual system does not function as a passive sensor system. It actively scans its environment. The photoreceptors in the eye respond to stimulation changes, not a constant stimulation. This is similar to using a flatbed to scan an image or a finger for Braille reading. “Motion is needed to transmit photoreceptor data to the brain.
In fact, experiments using substances like cobra venom to paralyze eye muscles have shown that blindness will occur if the subject is positioned with their eyes open and viewing a static image with the paralysis caused by the venom. The photoreceptors cannot provide information to the brain without any changes in stimulation. Blindness results. This is thought to be at least one of the reasons why normal human eyes have been observed moving back and forth or dithering in a side-to-side movement, also called “microsaccades”.
The brain’s visualization center also receives valuable information about perception from the movement of both eyes, and their components. The vergence (e.g. rolling movements to bring the eyes together to fixate on an object, or to diverge the lines of vision) of both eyes in relation to each other is closely related to the focusing (or “accommodation?”) The lenses of the eye are closely associated with focusing (or?accommodation?) Under normal conditions, changing the focus of the lenses of the eyes, or accommodating the eyes, to focus upon an object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the ?accommodation-vergence reflex.? Normal conditions will also trigger a change of vergence. It is well known that working against this reflex can cause eye fatigue, headaches or other types of discomfort for users.
The movement of the head which houses the eyeballs also has an important impact on the visualization of objects. People tend to move their head to see the world. They are constantly repositioning or reorienting it in relation to an object. Most people will move their heads if their gaze must be moved more than 20 degrees to focus on an object. (For example, they do not like to view things “from the corner” of their eyes). People also move or scan their heads when listening to sounds. This is to improve the audio signal and to take advantage of the geometry between the ears and the head. Head motion parallax, which is a relationship between the relative motions of objects at various distances and head motion as a function eye vergence, provides powerful depth cues to the human visual system. If a person fixes his gaze on an object and moves their head side-to-side, objects farther away from the object will move the same way as the head. Items in front of the object will move the opposite direction. These cues can be extremely useful in determining the spatial location of objects relative to a person. “Of course, head motion is also used to look at objects.
Further head and eye movements are coordinated with a’vestibulo ocular reflex,’ which stabilizes the image information relative the retina during head rotates, keeping the object information about centered on retina. The eyes will rotate in the opposite direction as a reflexive and proportionate response to the head rotation. This is done to maintain a stable fixation. This compensatory relationship allows many people to read a book even while moving their head from side to side. The same is generally not true if the book moves back and forth with the head essentially stationary. This reflex is primarily developed to coordinate head and eye movements, but is not usually used for hand motion. This paradigm is important for AR systems because the head movements of users may be directly associated with eye motions. An ideal system will preferably be ready to work in this relationship.
With world-centric configurations it may be desirable for inputs to include accurate head pose measurements, accurate representation or measurement of real-world objects and geometries surrounding the user, low latency dynamic rendering of the augmented reality as a function head pose, as well as a general low-latency display.
The U.S. Patent Applications listed below present systems and methods to work with a typical person’s visual configuration to address different challenges in virtual and augmented realities applications. These virtual reality systems and/or AR present a number of challenges. They include the speed and quality of the virtual content delivered, the eye relief for the user, the size and portability, as well as other system and optical issues.
The systems and techniques described in this document are designed to work with the visual layout of a typical human being to address these challenges.
Embodiments” of the present invention relate to devices, methods and systems that facilitate virtual reality or augmented reality interactions for one or multiple users. A system for displaying digital content is disclosed in one aspect.
The augmented-reality system in one aspect comprises an image-capturing device that captures one or multiple images. These images correspond to the field of view of a person wearing a head mounted augmented reality device. A processor is communicatively connected to the image-capturing device and extracts a map point set from the images. It then identifies a sparse point set and a dense point set from the extracted map points. Finally, it performs a normalization of the map points set.
The detail description, figures, and claims describe “Additional and Other Objects, Features, and Advantages of the Invention.
We will now describe in detail various embodiments with reference to the illustrations, which are provided to allow those who have the knowledge to apply the invention. The figures and examples are not intended to limit the scope or application of the invention. In cases where certain elements of the invention can be implemented partially or completely using known components, methods, or processes, only the portions that are essential to understanding the present invention (or the methods or process) will be described. The detailed descriptions of any other portions will be omitted in order not to obscure it. Further, different embodiments include present and future equivalents of the components that are referred to in this document by way of example.
In this specification, specific embodiments of the invention have been described. You will see that there are many ways to modify and change the invention without departing from its spirit and scope. The process flows described above, for example, are described in relation to a specific ordering of the process actions. The order of some of the described processes actions can be altered without affecting the scope of the invention. “The specification and drawings should be considered in a more illustrative than restrictive manner.
Disclosed are systems and methods for generating virtual or augmented reality. Virtual content can be delivered strategically to the eyes of the user in order to create a realistic virtual reality or augmented reality experience. This is done while respecting the physiology and limitations of the human eye. This disclosure provides various embodiments of optical systems that can be integrated into AR systems. The disclosures will primarily be in the context AR systems. However, the technologies can be applied to VR systems as well.
Click here to view the patent on Google Patents.