Automated Visual Repository Building

Automated Visual Repository Building

The state-of-the-art in visual object recognition has matured to a level where many objects can be recognized in real-time in video streams. In order to recognize objects in pictures taken from camera phones, 3G video streams, webcams, or any other video recording device, there is a need for a database of images of the objects to be recognized (along with accompanying data such as product ratings, prices, reviews, comparisons or any relevant metadata ). Instead of manually taking pictures to build such a database we made use of the (increasingly) high number of pictures available on-line by downloading images from the web (Amazon, Google image search, Flickr…) with a spider to retrieve a larger, more diverse, and continuously increasingly growing set of images, all with just a simple click of a button. Once the images have been downloaded they need to be filtered, clustered, and cleaned after which feature points are extracted and stored in a database. It is this feature point database that is used to recognize the objects in images. In order to filter out irrelevant images we are planning on using the object recognition itself to create better databases to be used for object recognition!

Automated Visual Repository Building

The objective of the Automated Visual Repository Building project is to develop new algorithms and methods to (semi)automatically build image databases from online disparate image sources with the goal of supporting a variety of visual object recognition and image acquisition applications.

The image database, a.k.a. Visual Repository, needs to contain in vitro images (i.e., containing no background/clutter around the object in the image) under a variety of angles and lighting conditions and ideally need to be created with a couple of keywords and a simple push of a button. We developed methods and algorithms to help build such systems by crawling the web for images. For example, if a database is required to recognize a brand of car, then the first step is to automatically generate a more detailed list of the correlated keywords, such as all of the different models of the car brand. After that all of the related images are pulled in from Google images, FlickR, Bing, and Amazon, as well as by crawling pre-specified websites (e.g, a sales site). The images can then be added and removed from the database through a simple drag and drop manipulation.

The power behind this project is that it provides the back-end for any object recognition application. Thus deployment becomes quick to do and no longer requires technical people.

Useful for all object recognition applications

All object recognition applications involve the following steps :

  1. Collecting digital images of the products to be recognized
  2. Tagging these images with metadata and putting them into a special feature-point database
  3. Loading the database onto a server
  4. Implementing an object recognition algorithm on the server
  5. Building a mobile front-end that communicates with the server

After having implemented numerous applications using object recognition on various different media (mobile devices, computer applications, web services, web pages), we realized that :

  • Steps 1 and 2 require software built in-house, there are no products on the market that do this
  • Steps 3 and 4 never had to get redone, simple copying the code from a previous project was enough
  • Step 5, the front-end, can be done by a team of developers with no prior knowledge of object recognition

With steps 1 and 2 covered by this project, future applications became very quick to develop. For example, developing mobile object recognition for retail took only a few days.

Scientific Advisor

As the Scientific Advisor for object recognition technologies, the following questions always had to get answered :

  • Which class of objects can get recognized?
  • How fast and scalable is it?
  • How reliable are the estimations of the recognized object?

We invite you to contact us if you’re interested in knowing the answers to these questions. We love this area and sharing lessons learned on this tricky and very potential technology!

In addition to back-end support for mobile image taking devices, this automatic recognition of objects in images can be applied to any image and therefore also has applications in image retrieval, content filtering and automated image labeling.


Leave a Comment