Skip to Main Content

Digital Video and Image Analytics

Analyzing videos and images using digital tools

Introduction

So far, there are three types of digital video visualization patterns that have been widely used; namely, color wheel visualization, bar code visualization, and R coded visualization with statistical data. Belows is a basic introduction to and comparision of the tools and methods for each type of visualization respectively, so that you could pick the one that serves you best.

Color Wheel Visualization

Graphic designer Frederic Brodbeck, creator of Cinemetrics has a number of digital media files and formed excellent color wheel visualization that the user can easily use to visualize not only the major theme color of the movie, but also the frequency of the shots. And even more amazingly, he presents the data not only in still image visualization, but also via dynamic visualization which vividly displays the major theme color, the frequency of sample movie shots as well as the length of each shot. 

              Still Image Visualization (Screen Shot)

             Dynamic Visualization (Screen Shot)

However, since he didn't document his code completely, it is thus hard for other users to restore the code and apply it on other media files for personalized usage. Then another contributor @suite22 posted modifications of his code on Github and made it easier to use, and more importantly the output contains both visualized images which are ready to use and the raw data that provides user the flexibility for customized usage and analysis.  The code can be run on Linux system via VirtualBox, through which you can acquire basic shot length and motion index data. More information could be accessed at Temple University Paley Library Digital Scholarship Center

Virtual Machine Terminal Display(Screen Shot)

        Nightingale Rose Diagram(Sample)

Bar Code Visualization

The Colors of Motion (website):

The bar code visualization for each movie is well defined in detail, and the website explains the basic notions of how the visualization is formed, but the code detail is not disclosed and the bar code visualization is made for commercial purpose. However, you can still pick desired movies' bar code visualization from their database, though their total sample size is very small, or you can inquire specific visualizations based on paid services. 

Movie Bar Code (website):

The bar code visualization for each movie is also well defined. The difference is, instead of showing it vertically, it is presented horizontally. In addition, it provides social media sharing functions and far more movie bar code samples than the Color of Motion website. However, it doesn't disclose code either as it is mainly for commercial purpose, so the application is still quite limited. 

Python Code: 

It is originally applied  in game videos, however, it can also be applied on movies and digital video files for no problem. The website has already disclosed the steps and python codes for how to do it in detail.  Although it requires minimum python coding knowledge, this resource is highly recommended in terms of visualization. 

In addition, @suit22's Github code also provides an option of barcode visualization in the output automatically.

Barcode Sample

R code visualization

Mike Baxter's notebook Notes on Cinemetric Analysis so far has been the most complete reference material specialized in cinematic quantitative data analysis and visualization based on R code.  The greatest advantage of this method is it is highly flexible. It allows you to process not only large data set but also visualize it in various ways, including Kernel Density Estimator(KDE), histogram, scatter plot, color-coded wallpaper(linear/polynomial distribution against cutting point), and barcode visualization as well. It provides terrific data analysis and graphic display, though basic R coding knowledge is required.

Kernel Density Estimator:

Histogram:

Scatter Plot:

S

Wallpaper:

Barcode:

 

Digital Images/pictures visualization

In addition to digital video visualization, there are also a number of innovators with projects devoted to large data set of digital image/picture visualization. Among them, the most prominent one is Software Studies Initiative, a website and lab launched by professor Lev Manovich along with other software developers and humanities scholars. The website presents not only a variety of prominent projects that have been conducted regarding culture analytics, but also some useful tools for digital visualization in assisting the studies. 

Selfie City of London and On Broadway of NYC are two most prominent projects which are trying to explore city dynamics and people emotions by integrating digital visualization of image data from multiple channels including Google Street View, social media image photos, taxi statistics and so on. 

Image Plot is one of the most popular and useful tools developed by the lab for large set of digital image visualization. There are detailed instructions regarding the download and usage of tool on the websites, along with other useful digital image visualization tools listed on the website. 

Here is an example of the visualization of Van Gogh's paintings with Image Plot: