Latest Posts

Understanding the Computer Vision Technology

Technology / By

The early 1970s introduced the world to the idea of computer vision, a promising technology automating tasks that would otherwise take humans years to do. In 45 years, computer vision would become an integral component in artificial intelligence (AI) applications across industries. After years of exponential growth, through breakthroughs in AI technology and an explosion of image data, computer vision has become a core component of AI.

So, what is computer vision?

Computer vision is a branch of artificial intelligence. The technology helps to automate visual understanding from a sequence of images, videos, PDFs, or text images with the help of AI and Machine Learning (ML) algorithms. In other words, computer vision replicates certain functions of human vision, but much faster and sometimes more accurately.

Today, the volume of image data is growing exponentially, therefore, identifying and analyzing images is increasingly crucial for generating insights. Computer vision harnesses software and robots to analyze thousands of images, videos, and documents, including PDFs, in order to gather significant information from them. It also enables functions such as object recognition, image restoration, and scene reconstruction. Real-world applications for these technologies have been found in healthcare to identify tumors, and in security services to recognize potential hazards.

How does computer vision work?

Computer vision is an interdisciplinary field. It applies relevant concepts from neurobiology, signal processing, as well as information engineering to process images. Computer vision involves the development of either a theoretical basis and/or algorithmic basis to achieve automatic visual understanding.

The symbolic information represented in image data is understood using theories of statistics, geometry, physics, and learning. Natural Language Processing, a branch of artificial intelligence, applied together with the image recognition techniques of computer vision, help in extracting data from images and PDFs. NLP is used to search specific words in the text algorithms trained in entity recognition. This helps in extraction of relevant data from a vast pool of unstructured data.

The concept of computer vision has been successfully applied to facial recognition. Facebook uses the technology for tagging friends in photos and Snapchat uses it to recognize a person’s face when applying filters. More recently, the concept of computer vision was applied to self-driving cars. The technology empowered driverless cars along with sensor technology to help identify people, other cars, objects, motorcyclists, pedestrians, etc. during driving. Nowadays, it is widely being used to extract data from PDFs and images.

Challenges of Data Extraction using Computer Vision

The data available on the web comes in various formats. It becomes nearly impossible to extract all the relevant information this data due to the inability of computers to understand documents and images as we do. Although computer vision combined with other AI techniques can handle vast amounts of structured and unstructured data, the challenges of extracting relevant information go hand in hand.

Some of the major challenges can arise from the following:

  • Infographics with pie charts or graphs
  • Different font styles, font sizes and orientation
  • Complex structure and style of PDFs, double column vs single column
  • Addition of footnotes, bullet points, special characters
  • Use of foreign languages or symbols
  • Similar colors and patterns of different objects
  • Use of superscripts and subscripts, underlines etc.
computer vision challenge
Picture credit: teenybiscuit

In the case of photo analysis, the content of many images can be confused. However, computer vision must be able to differentiate a sheepdog from a mop. This challenge requires vast amounts of rich labelled data, which can be used to train algorithms to differentiate between images. In addition, the computer vision technology must be taught to understand the correlation between things that exist together or a part from a single object. For instance, a tree trunk cannot be considered a wood log. Training the computer to understand these differences is necessary for efficiency.

In the case of PDFs data extraction, the challenge can be even more complex. When we read a PDF, we take into account the headings, subheadings, colors and size of the text to determine how important it is or in what context it is put there. Footnotes, which are an important part of PDFs and mentions links, sources, or meanings, are often ignored by many computer vision software during extraction. Also, many a time, PDF text extraction follows omitting of unrecognized characters, exclusion of graphs etc.

To extract information such that it can be understood by humans, it is important for the computer to understand the positions of the objects, recognize special characters, take into account different fonts, colored text, footnotes, bullet points, as well as difference in font sizes etc. This is the key to accurately extract information that makes sense, is human-error free, and can be analyzed for decision making or other uses. To achieve this, the use of Optical Character recognition is required. It helps the computer to differentiate between symbols, formats, fonts, etc. in order to read these scanned images efficiently for data extraction.

Why is Computer Vision so popular nowadays?

Computer vision is nowadays being applied to a variety of purposes. It is being used to avoid fraud at stores through the detection of items a person picks up. It has also been applied to the Amazon Go cashier-less stores to identify items and bill the person for them. Many applications, such as Pinterest’s Lens and Amazon’s EchoLook, have found its use in the lifestyle and fashion industry. Lens scans the image for what someone in a picture is wearing and suggests where to buy a similar item, whereas EchoLook gives ratings based on outfit and style. It is also being used by banks to scan cheques, provide home and office security etc. In life sciences, it facilitates research by extracting data in structured format from medical publications and other PDFs.

The reason why computer vision gained so much of popularity from the end of 2016 is its use in both customer-oriented products and apps, as well as its application in various business applications across many different industries. The fact is, prior to computer vision, it was not possible to analyze pictures from multiple angles together to get a full understanding of the image, read vast amounts of PDFs for useful data extraction, or even identify similar objects in a sequence of images. In simple terms, artificial intelligence gave computers and robots a “vision”. This enabled the extraction of important data accurately without misinterpreting different languages, characters, symbols, sizes, or colors. On the other hand, it can help recreate deformed parts of a picture, identify objects in a blurred out image, or even analyze videos or series of images for similar patterns etc. Enabling such kind of extraction of information has led to many innovative ideas.

Key Points:

  • Computer vision is an AI technology that enables the extraction of data from images, PDFs, videos etc. by applying interdisciplinary concepts from geometry, physics, neurobiology, information engineering etc.
  • Computer vision, combined with other AI technologies such as Natural Language Processing can help to efficiently extract information from PDFs.
  • It is being used in mobile applications, business tools, IoT, and across various industries such as lifestyle, health, social media, finance, automobile, robotics, and others.
  • Its uses and purposes range from fraud detection, facial recognition, image restoration, object detection, PDF data extraction, video tracking, and motion estimation to many more.
  • More and more industries are relying on this technology to solve problems in innovative and more effective ways. Therefore, the technology along with other AI branches will grow to provide a solution to many hidden challenges in the future.

Technology blog banner

Featured Blogs