Invented by Jim Ostrowski, Luis Goncalves, Michael Cremean, Alex Simonini, Alec Hudnut, Datalogic ADC Inc
The Datalogic ADC Inc invention works as followsSystems and Methods for Recognizing and Identifying Items Located on the Lower Shelf of a Shopping Cart in a Checkout Lane of a Retail Store Environment for the Purpose of Reducing or Preventing Loss or Fraud and Increasing the Efficiency of a Checkout Process.” The system includes one of more visual sensor that can take pictures of items, and a computer that receives images from one or more sensors and automatically identifies items. Images of items can be used to train the system to recognize them. The system matches visual features in training images with features extracted from images captured at the checkout. The system compares the visual features in the images with the features in the database, using the scale-invariant features transformation (SIFT), to find one match or more. These matches can then be used to identify items.
Background for Systems and Methods for Merchandise Checkout
The present invention relates in general to visual pattern recognition and, more specifically, to systems and method for automatically recognizing products at retailer checkout stations based on ViPR.
In many retail environments, including grocery stores and department stores as well as office supply stores, home improvements stores, etc., shoppers use shopping carts for carrying merchandise. The basket is used to store the consumer’s goods and the shelf below the basket. Sometimes, a customer will use the lower shelves as extra storage space for large or bulky items.
When a customer uses the lower shelves to store merchandise, they may leave the shop without paying. It may be that the consumer forgets or intentionally steals the merchandise. Cashiers may also fail to notice or not see bottom of basket merchandise (BoB), allowing customers to leave without paying. In the retail industry, cashiers are known to sometimes collude with customers. The collusion may range from allowing the customer to fraudulently take an item on BoB without paying or ringing up items at a lower price. Cashier fraud has traditionally been estimated to be around 35% of the total grocery retailer “shrink”. According to the National Supermarket Research Group 2003/2004 Survey on supermarket shrinkage.
Collectively, this type of loss is known in the retail industry as ?bottom-of-the-basket? (BoB) loss. According to estimates, a typical supermarket could experience bottom-of-the basket revenue losses of $3,000 to $5,000 per lane each year. This loss is equivalent to $30,000-$50,000 of unaccounted revenue for a modern grocery store that has 10 checkout lanes. The potential revenue recovery for a large grocery chain with 1,000 locations can be in excess of 50 million dollars per year.
Several attempts have been made to reduce or minimize bottom-of-the basket losses. These efforts fall into three main categories: process changes and training, lane configuration changes, and supplemental detector devices.
The training and process change are aimed at getting the cashier and bagger inspecting the cart for BOB in every transaction. This approach was not effective due to high staff turnover, constant training requirements, low skill levels of the employees, lack of mechanisms for enforcing new behaviors, and lack of initiative in encouraging tracking and preventing colusion.
Lane Configuration Change” is intended to make the bottom of a basket more visible for the cashier. This can be done by moving the cart away from the customer and onto a different side of the lane (referred to as ‘lane splitting’). Cart swapping is a method of using a second trolley that requires customers to unload their carts and then reload the items on the new cart. It is costly to change the lane configuration, it does not solve the problem of collusion and it is usually a less convenient and efficient way to check out and scan items.
Supplemental Devices include mirrors placed opposite the lane so that the cashier can see BoB without having to lean over or walk around the lane, infrared sensors to alert the Cashier of BoB Items and video surveillance devices which display an image to the Cashier for them to see the BoB. Kart Saver, Inc., for example, sells infrared detection devices.
VerifEye Technologies is marketing a supplementary device to reduce BoB losses.
As can be seen, a need exists for an improved apparatus that can automatically detect and checkout merchandise without the intervention of a cashier, such as when the items are placed on the lower shelf in a shopping cart at the checkout of a retail environment.
The present invention presents systems and methods that allow one or more visual sensor systems, operatively connected to a computer, to view and recognize objects, such as those on the lower shelf in a shopping cart at the checkout of a retail environment. This can not only prevent or reduce loss or fraud but also increase revenue for the store. One or more visual detectors are fixed in the checkout register lane so that, when a cart enters the lane, the visual sensor will recognize and associate one or several objects with instructions, commands, or actions. This can be done without personnel having to see the objects.
In one aspect, the invention includes a system to check out merchandise that includes: a visual sensor to capture an image of an item on a moving structure; and a separate subsystem to analyze the image and detect the object.
The system includes a server that receives analyzed visual data and recognizes the object, sending match data to checkout subsystem.
The system includes a checkout module, a computer that receives visual data, sends match data to it, and receives transaction data. It also has a server which provides database information and log data.
The system includes a Checkout Subsystem, a Log Data Storage, an Object Recognition Table and a Feature Table.
The system includes a checkout lane, a checkout subsystem that receives visual data and analyzes it, a server that receives analyzed visual information from the checkout system and recognizes the merchandise, and sends match data to the subsystem. It also includes an Object Database, which is coupled to the server, and configured to store at least one object to be recognized.
In another aspect of the invention, a database contains a Feature Table that includes an object ID, a View ID, a Feature ID, a Feature Coordinates Field, an Object Name field, A View field and a Detailed Feature field.
In another aspect of the invention, a data base includes an output table that contains an object identification field (ID), a view field ID, a camera field ID, an image field field, and a timestamp.
The method for checking out merchandise in another aspect of this invention includes the following steps: receiving visual images of an item; comparing them with data stored on a database to determine a set match; determining whether the set match is found; and then sending a recognition notification.
The computer-readable medium that contains the program code and instructions for recognizing an item includes: code to receive visual image data; code to compare the visual image with data in a database, to find a matching set; code to determine if the matching set is found; code for determining whether the matching set is found; code for sending recognition alert.
The following steps are included in another aspect of this invention: “A method of checking out merchandise consists of (a), receiving visual image information of an object, (b), comparing that data to data stored in a data base to find a matching set; (c), determining whether the matching set is found; and (d), if not, repeating steps (a), (c), (e), checking each element of a match set is reliable, (f), if none of the elements of a match set is reliable, (a)
Click here to view the patent on Google Patents.