The Rittenhouse trial has put video evidence into the forefront of many discussions and debates around the United States and beyond (checkout our AMA on Reddit). One of the most common questions is, “does pinch and zoom actually alter the video evidence?”
While some people think it is ridiculous to say that zooming in alters the video, the issue is more nuanced and complicated. Authentication, the question of whether an exhibit accurately represents original data, is a legitimate legal debate with real legal consequences.
In this article, we will explore what happens when software zooms in on digital video evidence, explain several key terms, and discuss why it is so important for those who testify in court to understand how their software works.
Before we get started, it is important to note that we are not involved in the Rittenhouse case in any way and have not reviewed the evidence in that trial.
Let’s get started.
Why Resize an Image?
Every image starts with a finite number of pixels (small square shaped color values). Pixels do not have any more information in them other than a single color, and they cannot be resized without creating additional pixels or deleting pixels.
We live during an era where high-definition videos and photos are available at our fingertips. For instance, if you were to take a family photo with a modern iPhone, assuming proper focus/environmental conditions then you could likely have millions of pixels per person in the image.
This level of resolution is impressive, and EXPONENTIALLY more than we typically encounter in the video evidence world where “enhancements” are often requested by investigators.
Even during the day, typical digital video compression more often reveals that ‘enhancements’ are a Hollywood myth and are only valuable in limited scenarios. For example, it is very common for investigators to request enhancements of faces, license plates, and weapons where the entire object may only be reproduced with a few pixels. To make things worse, the recording that needs to be enhanced is often created at night on a low frame rate surveillance system with infrared illumination where the data is heavily compressed.
When producing a sports game replay, the program’s director can zoom in with remarkable clarity because the production is using high-definition cameras in well-lit environments, all the while using live optical zoom features. With most surveillance video evidence, we simply don’t have that technical visual infrastructure.
Because of this limitation, when investigators resize an image that already has so few pixels, interpolation can have a significantly negative impact on the result.
What is an Interpolation Algorithm?
When someone uses software to resize an image, they will be using an interpolation algorithm (method).
Images have limited pixels, and if you resize an image to make it larger, you cannot get extra data from within a single pixel. Instead, you must add more pixels.
For example, if I start with an image that is 352×240 and I double the size to 704×480, then I inherently must make new pixels. The “interpolation” algorithm decides what color to make those new pixels.
The most common interpolation methods are: Nearest Neighbor, Bicubic, and Bilinear.
Nearest Neighbor: This algorithm will duplicate all the original pixels and create identical pixel values, provided the resizing is done to an even integer, such as 200%, 300%, 400%, etc.. It does not create any color values that were not in the original image. This algorithm exaggerates the hard edges in the original image, but is an excellent method for highlighting exactly what the recording device captured. It is also a forensically sound interpolation method and appropriate for answering questions about suspect height, distances, etc.
Bicubic Interpolation: This algorithm will add new pixels with different colors based on a weighted average influenced by a grouping of pixels around each pixel target. It creates smoother looking images that may appear more pleasing to the naked eye. This interpolation method is perfectly fine for demonstrative exhibits, but it is important to understand that the evidence has arguably been modified or changed.
Bilinear Interpolation: Like Bicubic, Bilinear also introduces new colors to newly created pixels with the weighted averages of the pixels around them. Unlike bicubic, the algorithm can be reversed and creates less rounding than bicubic does. It is best used for aspect ratio adjustments.
If you want to learn more about the different interpolation methods and see a specific example of why this matters, then you should watch part of the training video below:
How Algorithm’s Impact What We See
Depending on the interpolation algorithm used, specific questions such as “in what direction was the gun pointed?” could certainly be impacted by using the wrong method.
One of our training classes contains an example of this exact scenario. In that example, the original image of a suspect contained only a few pixels around his hand area:
One of the experts resized the image using bicubic interpolation (image on the left, below) which added dozens of extra pixels and rounded out the shape of the hand area – giving it the appearance of a gun in the individual’s hand. That “fabrication” of shape is something done entirely by the interpolation algorithm:
Originally, the DVR only recorded a jagged block of a few pixels. From a forensic perspective, we should not be able to say anything about that area. The individual could have a gun, or he might not. From the few pixels we have, there is simply no way to know.
How does Pinch to Zoom work?
During the Rittenhouse trial, the judge questioned, “whether using the pinch-and-zoom feature on an iPad would alter an image and he barred prosecutors from using that specific method while presenting video evidence to jurors.”
While some people expressed a concern that the judge’s ruling was unwarranted, Grant Fredericks of Forensic Video Solutions (who was not involved in the case), explained in an interview with the Associated Press that the basic problem with using the pinch and zoom method in court is that it does not produce a permanent record of what the jury was shown.
“Any demonstrative exhibit presented during a trial must be reproduced for the trial record. A copy of exactly what was presented to the jury must be available for potential future review by an appellant court.” Additionally, he describes that the pinch and zoom method employs an unknown interpolation algorithm, making it very difficult to quantify how much change to the image has occurred.
It’s important to understand the technical descriptions for the resizing process and to articulate how they can affect the evidence – because as we saw during the trial, different interpolation methods can in fact change the appearance of what is being resized.
Fredericks adds, “Careful thought must be given prior to trial for each and every video exhibit, including how and on what devices it should be displayed.” For more information about courtroom display of video evidence, check out his article in Police Chief Magazine.
Which Interpolation Method is Best for Playback?
Every video related case is different. There are many different interpolation algorithms, and many of them have a place in the toolbox for different reasons.
As described above, the three most popular interpolation methods (Nearest Neighbor, Bicubic, and Bilinear) are available on the workflow tab of iNPUT-ACE within the resize node when converting files or exporting clips for court.
In addition, the iNPUT-ACE software has a simple toggle on the player of the Interrogate tab that can switch between a “Nearest Neighbor” rendering mode or a smoother rendering mode, depending on the question being asked during the examination. The simple toggle is highlighted below:
Are you looking to answer a forensic question, such as
- What is the size?
- What is the color?
- How many pixels?
- What is the distance?
- Did something move?
If so, then we recommend using Nearest Neighbor rendering.
Are you simply tracking activity and performing a general investigation of what happened? Then we recommend using Smooth rendering for a prettier picture.
All interpolation processes are easy to track and replicate inside iNPUT-ACE, ensuring the evidence is reproduceable and can be clearly communicated to the trier of fact.
Conclusion
There are several key lessons about video evidence from the Rittenhouse trial.
One important takeaway is that experts should be able to describe the technical actions they perform within forensic or investigative software, including being able to explain how their resizing or enhancement filters may have affected the visual evidence.
Dr. John R. Black, who utilized iNPUT-ACE to play the video evidence during the Rittenhouse trial, did an excellent job of explaining why and how he was using iNPUT-ACE. He specifically mentioned that he had completed the iNPUT-ACE Operator and Examiner certifications, and he demonstrated how the courses gave him the knowledge to testify with confidence on a global stage.
If you are interested in learning more about video evidence, we strongly recommend signing up for one of our online courses today.
