Invented by Aydogan Ozcan, Bingen Cortazar, Hatice Ceylan Koydemir, Derek Tseng, Steve Feng, University of California
The University of California invention works as followsA system for determining chlorophyll in a sample leaf includes a device that holds the leaf and has a main body with a power supply, multiple switchable light sources (e.g. red and white from a broadband source) and a cap attachable to the main frame using one or more fastening mechanisms. The leaf sample is held between the main body of the device and the cap during imaging. The system comprises a mobile device with a camera that can capture an image of a leaf illuminated by a plurality of switchable lights, and a network-connected mobile device. An application on the mobile device is configured to send the images via the network to a remote server or computer for data processing. The transferred images are used to calculate a final value of the chlorophyll indices.
Background for Method for Quantification of Plant Chlorophyll Content
Large scale industrialization has resulted in a wide range of environmental impacts over the last century (e.g. air, soil, and water pollution, as well as deforestation and desertification). This has caused significant concerns about human-driven changes to our planet, both in urban and rural areas. Climate change includes changes in the distribution of precipitation and an increase in global average temperatures. These changes have a significant impact on plant and animal ecologies, e.g. plant growth rates and soil mineralization. Plant health and growth rate have been used to indicate various environmental factors due to their resilience and ubiquitous nature. Rapid plant monitoring is important for agriculture applications. It helps maintain plant health, identify emerging diseases and improve crop production efficiency.
In one aspect, a small and cost-effective leaf-holder device with a multispectral illuminator is paired or used in conjunction with a mobile device that has camera functionality and wireless connection as part of a measurement system for chlorophyll content of plant leaves. The mobile electronic device includes a program that can be used to capture images of an illuminated plant leaf in the leaf holder and transmit them to a remote computer. In a preferred aspect of this invention, the leaf can be illuminated by a red light (e.g. from a LED red) and a separate white light (e.g. from a LED white). After the images are acquired and sent, they are processed to identify an area of interest (ROI). The red channel is extracted from each image after identifying ROI. The ROI intensity values are averaged for each image. In the red/white lighting scheme, for example, the ROI obtained by the white light source ROIwhite has an average intensity and the ROI obtained by the red light source ROIred has an average intensities. The two average intensities are correlated to a calibration curve, or an equivalent, in order to produce an index value. In one embodiment, an index value is generated that corresponds to the commonly used index generated by the SPAD 502Plus. The calibration curves, or their equivalents, can be used to produce other indices that are commonly used to measure the chlorophyll contents. “Another example is the chlorophyll-content index (CCI).
The calculated index value can be sent to the mobile device after it has been calculated (or the final index can be calculated locally on the mobile device). The calculated index, along with other information, (e.g. image, text or gps coordinates) can be presented to the user on the mobile electronic device. The mobile electronic device may present the calculated index to the user. The mobile electronic device can be a wearable imager (e.g. glasses-based imaging devices) in one aspect, but the invention isn’t limited to this. Mobile electronic devices can also be mobile phones (e.g. smartphones), tablet-based devices, or webcams.
In one embodiment, the device for measuring chlorophyll in a plant leaf includes a leaf holding illuminator. The leaf-holding illumination device includes a power supply (e.g. batteries) and two light sources that can be switched (red and/or white). The voltage regulator regulates the power. The electronic components of the leaf-holding illumination device are enclosed in a main housing or body that protects them and creates a uniform pattern of light internally. A light-isolating cap is included on the device to minimize light variations in the leaf illumination area. The leaf is placed between the main body or housing and the cap with the cap being held to the main body using magnets or other fasteners for easy attachment/detachment. The leaf is illuminated by light coming from two different light sources. Illumination light can be either direct or reflected from a surface that contains the main body. The light is diffused before it reaches the leaf’s surface to create a uniform illumination pattern. “In some embodiments the light is transmitted by the leaf, i.e. transmission mode, while in other embodiments the light is reflected from the leaf, i.e. reflection mode.
The system or platform includes an electronic mobile device which, in one aspect, is a wearable imager such as Google Glass. Google Glass, which is worn on the user’s face, contains an application or software designed to capture images from the leaf-holding illumination device. The leaf is photographed in both white and red light. The images are uploaded using Google Glass’ wireless connection (e.g. WiFi) to a remote computer. The images are digitally processed at the remote server (e.g. cropped) in order to identify ROI. The ROI’s red channel (for images with red and white illumination) is extracted, and the average intensity is calculated for each ROI. The average intensity values of white and red are converted to index values, such as SPAD values, that represent chlorophyll. The remote server returns the average of the two index values. The calculated chlorophyll may take the form of an SPAD index. Other information may include the image of ROI as well as date and time captured, validity of ROI area, GPS coordinates and similar.
In one embodiment, the system for measuring the chlorophyll content in a sample leaf includes a device that holds the sample leaf and has a number of light sources with different spectrums. The diffuser is interposed inside an optical path between the switchable light source and the leaf. The cap can be attached to the main body in a variety of ways. The system comprises a mobile device with a camera that can capture images of a leaf lit by the switchable light sources.
The remote computer (which may be a server) performs the data processing. The ROI in the image is cropped and the intensity measurements of the red channel are extracted (average intensity values). The average intensities are then compared to corresponding calibration functions or curves (or their equivalents) in order to produce a SPAD value. According to one embodiment, the SPAD values for different illumination wavelengths are averaged. In other embodiments different functions can be used to calculate a final value based on the index value of each database. The remote computer sends this final SPAD value to the mobile device. “In an alternative embodiment the final SPAD value is calculated by the mobile electronic device, instead of the remote computing system.
In a second embodiment, an illuminator for holding leaves is described. It includes a main frame with a cavity interior and a surface that supports the leaf. On or within the main body portion, a power source can be disposed. In the cavity, a plurality of different light sources are located. The device is equipped with a switch that connects the light sources and the power source. The device has a cap that consists of a base with an aperture and an extension at the aperture. The cap is detachably attached to the surface supporting the leaf on the main body. The cap and the main body can be fastened together with any number of fasteners.
In another embodiment, the method for measuring the chlorophyll content in a sample of a plant leaf includes loading the sample into an illuminator that holds the sample and illuminating it with the illuminator at a spectrum of illumination with the first illumination source. The second illumination spectrum is then illuminated with the second illumination source. The images of the illuminated sample are captured by a camera on a mobile device, while it is illuminated with the first and second illumination spectrums. The mobile device sends the images captured at the first and second illumination spectrums to a server. The remote server computes the average intensity values for ROIs obtained by each illumination source. The remote server calculates the chlorophyll value of the first illumination based upon a comparison between the average intensity of ROI for the first lighting source and a calibration curvature. It also computes the chlorophyll value of the second illumination based upon a calculation of the chlorophyll value of ROI for the second lighting source. The remote server calculates the final chlorophyll value using the chlorophyll value of the first illumination and the chlorophyll value of the second illumination and transfers that value to the mobile device.
In another embodiment, the method for measuring the chlorophyll content in a sample of a plant leaf includes loading the sample into an illuminator that holds the sample and then illuminating it with the illuminator at a spectrum of illumination with the first illumination source. The second illumination spectrum is illuminated with the second illumination source. The images of the illuminated sample are captured by a camera on a mobile device, while it is illuminated with the first and second illumination spectrums. The mobile device sends the images captured at the first and second illumination spectra to a remote server. The remote server computes the average intensity values for ROIs obtained by each illumination source. The remote server calculates the chlorophyll value of the first illumination based upon a comparison between the average intensity of ROI for the first lighting source and a calibration curvature. It also computes the chlorophyll value of the second illumination based upon a calculation of the ROI average intensity for the second lighting source. The remote server transfers to the mobile device the chlorophyll value of the first illumination and the value of the second illumination. The mobile device then calculates the final chlorophyll value using the chlorophyll value from the first illumination and the value for second illumination.
Experiments on leaves were conducted using a handheld leaf holder that featured two-color illumination, red/white. This was paired up with Google Glass which is a cloud connected wearable computer with a camera as well as various spatiotemporal sensors. Google Glass, running a specially-developed Android app, was used to demonstrate a non-destructive, rapid and accurate leaf chlorophyll measuring platform. The SPAD-502 Plus meter was compared to the hand-held external leaf holder device using its standard SPAD index value which maps directly to chlorophyll in plants. The Google Glass-based chlorophyll estimater was calibrated using different plant species and a range SPAD indexes to match the sensitivity of the commercial SPAD-502 meters (for both indoor and outdoor conditions). The same system, after this calibration, was used to estimate (blindly), the chlorophyll indexes of 15 different plant species from the UCLA Mildred Mathias Garden under indoor and outdoor lighting. Google Glass-based non-destructive and rapid chlorophyll measurements can be useful for urban plant monitoring and indirect climate change measurement, as well for early detection of air, water and soil quality degradation.
FIGS. According to one embodiment, 1A and1B show an embodiment of a method for measuring the chlorophyll content in a leaf. As explained below, the chlorophyll content is measured by returning an index value of chlorophyll that can be used as a proxy or indicator for chlorophyll. This can be a SPAD or a CCI value. The system comprises a leaf-holding illumination device 10 which is used to hold the leaf 12 and sequentially illuminates it with different illumination spectrums from separate illumination sources. The transmitted light from the leaf 12 is captured by a mobile electronic unit 50, which includes a camera. The leaf-holding illumination device 10 consists of a main housing or body with a leaf-supporting surface 16. This supports the leaf 12, and is covered by a removable cap 18. The leaf-supporting surfaces 16 have an aperture so that the light can pass through the main body 14, and the leaf 12, (in transmission mode). As shown, the aperture can be left open or an optically clear window can be formed on the leaf-supporting surfaces 16. The leaf 12 is sandwiched between the leaf-supporting surfaces 16 and 18 of the main body 14. In one embodiment, both the main 14 and removable cap 18 include at least one magnetic element 20 (e.g. a permanent magnet, or magnetically sensitive material) mounted around the perimeter of the main 14 and/or removable 18 cap in corresponding positions such that magnetic attraction keeps the cap 18 firmly against the main 14 with the leaves 12 pinched between these two components.
As an alternative to magnetic components 20, various other mechanical attachment schemes can be used to secure removable cap 18 to main body 14. To secure the two parts together, you could use clasps, clips or tabs. For the same purpose, one or more fasteners (e.g. screws, bolts and nuts) could be used to secure these components together. You could also use an elastic band to hold the pieces together. The two pieces could also be fixed together using friction between the removable cap and main body 14. The cap 18 could, for example, have edges that can fit into slots or other forms of the main body 14. “Fastening means” refers to these alternative mechanical attachment schemes.
The main body 14 or the housing can be made of an optically opaque polymer or plastic material. The main body or housing has an interior portion which houses an electronic board 22 that contains electronics for the leaf holding illuminator 10. There is a first and second light source, each emitting a different spectrum of light. There may be some overlap between the spectra of each light source 24 and 26. The first light source 24, and the second source 26, may both include light emitting diodes. In one aspect, the first LEDs 24 (645 nm peak) and the LEDs 26 (i.e. broadband) are two different colors. The first and second light sources 24 and 26 are connected via a switch 30, located on the mainbody 14, to a power supply 28 (e.g. three alkaline battery) that is stored in the battery compartment 29, on or within the body 14. This switch 30 can be selectively selected to turn on the first or second light sources 24 (or both off). The electronic board 22 contains a voltage regulator that regulates power to both the first and second lights sources 24 and 26.
Referring to FIG. The light from the two light sources (first light source 24) and optical diffuser 32 are passed first through the optical diffuser 32 in order to uniformly light the leaf 12 as shown in FIG. As shown in FIG. In FIG. 1A, first light source 24, and second light source 26, are arranged in reverse orientation. The light is transmitted into the cavity of the main 14 body by the respective light sources 24 and 26. This light is then reflected in the opposite direction with a reflective surface (not illustrated) located in the body 14. It could be a reflective surface, such as a mirror or aluminum foil. In this way, an optical pathway is created from the respective light source 24, 26, to the optical diffuser, then onto the reflective surface and finally back toward the direction of leaf 12, which is sandwiched in between the main body 14, and the cap 18. The optical diffuser may also be used to diffuse the light after it has been reflected from the reflective surface. In a further alternative embodiment, each of the light sources 24, 26 may be oriented forward to illuminate the leaf 12, after it has passed through an optical path between the respective sources 24, 26, optical diffuser 32 and the leaf 12. The leaf-holding illumination device 10 is operated in transmission mode, whereby the light passes through the leaves 12 and images.
In a further alternative embodiment, the leaf holding illuminator 10 can operate in a mode of reflection where light reflected from the surface of leaf 12 is imaged. In this embodiment the light sources 24, 26, illuminate the surface of a leaf 12 by using an optical diffuser, such as 32. The reflected light is then imaged. The light sources 24, 26, may have different spectra depending on whether or not the leaf-holding illumination device 10 is operating in transmission mode. “In either mode, only a portion is imaged of the illuminated leaf surface 12.
As seen in FIGS. The cap 18 has a base portion (34), which includes an aperture 36. The aperture 36 has a circular shape, although it could have been made in any other shape. The base portion 34 extends into a frustoconical extension 38 that surrounds the aperture. The extension 38 prevents the external lighting from interfering in the process of obtaining a light image transmitted through the leaf 12. “The leaf 12 can be placed in position without causing any damage to it by removing the cap 18, placing the leaf on the mainbody 14, and then placing cap 18 over leaf 12. The cap 18 then secures the leaf to the mainbody 14 via magnetic elements.
During use, the user should place the leaf 12 as shown above between the cap 18 & the main body 14. The leaf-holding illumination device 10 is used with a mobile device 50 which has a built-in camera. The mobile electronic 50 can include, for instance, a wearable device, such as Google Glass, made by Alphabet, Inc., with imaging capabilities, although other “wearables” are also possible. Other devices, such as headsets or other accessories, may be used. The mobile electronic device 50 can also include a smartphone or similar mobile phone that includes a camera, such as the Smartphone shown in FIG. 1C. “The mobile electronic device 50 can also include tablet, Tablet PC, or similar devices with camera capability (e.g. webcam).
The mobile electronic devices 50 include a program or an application 51 (e.g.?app?) The mobile electronic device 50 runs the program or application 51 (e.g.,?app?) The application or program can run on any operating system, although the example herein uses the Android operating System. FIG. 1C shows an icon that represents the application 51 displayed on the display of the mobile device 50. The icon can be activated by simply touching it. FIG. FIG. 2 shows the typical steps used to determine the chlorophyll level in a leaf 12, according to an embodiment of the invention. The leaf 12 is first loaded into the leaf holding illuminator 10 device. To begin the measurement of chlorophyll, the user launches the program or application 51 on the mobile device 50. As seen in operation 100 the application 51 asks the user to choose the lighting conditions, which in this case is whether or not the image was taken indoors. There are other lighting conditions that can be used, such as bright sunlight, dusk or overcast. As shown in screen shot 102, the user can swipe to the right for outdoor lighting or swipe to their left for indoor lighting. The user then adjusts the switch (FIGS). The red light source 24, which is located on the leaf-holding illumination device 10, can be activated by pressing 1A or 1B. However, the red source 24 may already be on when the user launches the application on his mobile electronic device 50. In operation 110 the user uses the red light 24 to obtain an image of transmitted light from the leaf 12. The mobile electronic device may give the user text instructions, such as “take a photo of the leaf using the red light.” The user may see a template 53 or overlay on the display or graphic user interface of the mobile electronic device 50 to help align or match the aperture 26. The overlay or template 53 can ensure the correct distance and alignment before obtaining the image.
The user can then use the white light source to obtain an image of light transmitted through the leaf 12. Switching the switch on the leaf holding illuminator 10 turns off the red source 24 and on the white source 26. The mobile electronic device may again prompt the user with text instructions, such as “take a photo of the leaf using the white light.” On the screen, display or graphical interface of the mobile electronic device, the user may again be shown a template 53 that can help align or match the aperture 26. The user confirms data (e.g. answering a question on the mobile device 50), and then the images (obtained using the red and white illumination lights) are sent to the remote server 40. Images can be compressed automatically using a compression format such as JPEG. Images can be sent over a WiFi connection to an Internet-connected wide area network or another network, such as a cell phone network. In this embodiment, data processing is performed remotely in the cloud. Images are obtained locally. Images are processed 132 by the remote server 40. Some data processing 132 can be performed locally on the mobile device 50 in alternative embodiments.
Click here to view the patent on Google Patents.