One of the most fundamental challenges of video investigation arises from CCTV video containing proprietary structures that generally can’t be played in standard video players. When we receive original surveillance video in formats such as .dat or .drv files, we can’t simply double-click and play in standard media players like VLC, Windows Media Player, or Quicktime.
An obvious solution to this challenge is to convert the video from its proprietary structure into a new playable format. But with this solution comes limitations and complications.
In this webinar and accompanying post, we cover what happens during conversion, how to ensure the converted output is actually playable in standard players, how to validate that no quality is lost, and how to understand complications with various technologies involved in this process.
Understanding the Underlying Data in Video Files
In order to understand some of the pitfalls of the video conversion solution, it’s important to understand what underlies the video itself.
Video files, like all computer files, are binary. In order to play a file, video players identify a necessary codec, which acts like a language that reads the 0s and 1s of the binary video data, interpreting and getting meaning from the data in order to play it. While the correct codec can interpret the images accurately, the wrong codec produces problems with playback.
Using INPUT-ACE, it’s possible to quickly snap to the exact binary (or HEX) position of an exact frame of your video by selecting the File Metadata dropdown on the Interrogate tab and choosing Hex at Current Position.
Viewing hex data with iNPUT-ACE
Viewing this hex reader provides a wealth of information; if you highlight a hex value of a piece of data, you can cross-reference with its binary 8-bit value, its integer value, its ASCII representation, and more.
Another important element in reading and validating video files is identifying the file’s file hash. A hash is a unique signature for each file, like a fingerprint.
If two videos have the same file hash, then they are the same file. If no 1s and 0s change in the video file, regardless of where the file is sent, what processes it goes through, what it is named, and so on, then the file hash will not change. For example, if a file is emailed to someone, then uploaded to Dropbox, then downloaded and onto a thumb drive, and opened on another computer years later, as long as the underlying 1s and 0s of the file haven’t changed, the exact same file hash will be maintained from the original file. In other words, simply moving the location or changing those external details about the file doesn’t change the underlying binary data.
Conversely, if we change any bit of data—any single 1 or 0—the entire file hash will change. This is important in forensic analysis, because if we can show that the file being presented to the court is the exact file that was recorded by the DVR (via identical file hash), we can then authenticate the video and not be overly concerned with questions of authenticity at trial.
However, since proprietary video files often need to be converted, and this process results in a changed file hash, how can we reliably convert the video while ensuring and proving the authenticity of the results?
Implications of Converting Video Files
Forensically speaking, when you alter the underlying data of a file by converting it, the file has been “changed”, and that terminology can have implications when testifying. That said, in video analysis, we should be more concerned with not changing the actual image data (visual information), and not as concerned with ensuring the binary data for the entire file.
In other words, forensic video analysts care more about ensuring that evidentiary pictures don’t change during a conversion than the file itself not changing. Viewing images in an uncompressed format is just as good as viewing the images in an original file (at least as far as the image data is concerned).
For example, we could have a .264 file that we can’t double-click to launch in a standard player, but iNPUT-ACE can read the file and play it back correctly (Learn more about containers, codecs, and raw video here.) If we need to convert this file into a standard format, one solution is to give the file a container via a process called rewrapping. Using an iNPUT-ACE workflow, we can re-wrap the file to produce an .mp4 output easily by selecting an MP4 output node, and selecting the Transcode Video option “Same as Input.”
Rewrapping a file with iNPUT-ACE
When we look at our rewrapped file, all the characteristics look the same: frame rate, pixel format, and so on. But the file hash has changed in our new .mp4 file, showing that we have in fact changed the 1s and 0s from the original file. The rewrapped file has different binary data than the original and will have a different file hash.
Rather than focusing on the changed file hash, what’s actually important is proving that the wrapping process didn’t change the underlying image data or any visual information in the file. We can do this in a few ways:
- By comparing frames in both files and checking that the hex data is the same. In other words, even though the overall binary data has changed, we haven’t changed the underlying 1s and 0s in the frames themselves.
- By writing both files to a bitmap image sequence via iNPUT-ACE workflows, and comparing individual bitmap images from each output. The file hash of the individual bitmap images will be exactly the same, and we can prove this for each image. This means that the original file has the exact same image information as the rewrapped file.
- By calculating the hash of every single frame of data in both files. In other words, instead of calculating the hash of the entire file, we could calculate the hash on each decoded frame—and compare that to frames in another file. iNPUT-ACE makes this process easy by simply selecting the “frame hash CSV” option from the Workflow tab and connecting 2 or more files to it. If every frame from one file matches the hash from every frame in another file, we can prove that the files contain the same exact image data.
As this process shows, rewrapping is easy, fast, and creates a lossless version of the frames of your video, so why not do it every time?
Rewrapping does have a serious limitation. This rewrapping process removes metadata from the original file, and only works when we send a video file directly to a rewrapped output, without performing any other actions during the process. In other words, if we want to resize, crop, or otherwise change the images in any way, re-wrapping is no longer a valid option. Any action that changes the images in the original file inhibits the rewrap process from working.
For these situations, have another option: performing a lossless transcoding process.
Unlike rewrapping, lossless transcoding is a process by which the file is decoded, file info is read and processed, and then the file is reencoded in a new way. In this process, we are changing the files at a binary level. But even though we’re changing the codec—or to use our earlier metaphor, to change the language in which the file is going to be read—we can ensure that we’re always going to visually display the same frames.
To understand the underlying information, consider this example:
Decoding the first set of “image data” with the right codec that can understand the equation produces a resulting value of 10 (or “what the image looks like”). Another way of encoding the same value is 5 multiplied by 2. We need a codec that can understand that equation, and we can produce the exact same result: 10. Or there could be an image that simply has the underlying data of 10 and displays 10 on screen.
In this example, we have 3 different files, all with different data in them – but once decoded, they all produce the value “10”. This is analogous to different video files, all containing different codecs and data, but when decoded, the images are exactly the same. If we can convert a file from one data type into a new data type without changing the underlying images, this process is called “lossless transcoding”.
This is different from “lossy transcoding” where the conversion process changes the underlying images.
Three possible “lossless transcoding” options are: re-wrapping, uncompressed, and lossless H.264.
The uncompressed transcoding process can be performed in iNPUT-ACE by creating an Output to AVI workflow, which prefills the settings with “uncompressed” transcoding options.
The lossless MP4 transcoding process can be performed in iNPUT-ACE by creating an Output to MP4 workflow, which prefills the settings with “lossless h264” transcoding options.
Unlike rewrapping, creating an uncompressed .AVI or a lossless H.264—work even if we perform actions on the file before the output, e.g. if we crop images, apply enhancements, etc. Both uncompressed and lossless h.264 transcoding options produce video output of exactly the same quality, but they do produce larger files than rewrapping does.
Which Transcoding Option is Best?
Which transcoding option is best? It depends. Consider five factors when choosing a transcoding option: file size, speed of conversion, playability, image quality, and if the option is even possible.
As a good rule of thumb:
Rewrapping generally produces the smallest file size, is the fastest process, and is easy to play in most cases. But it has the significant limitation of not allowing for anything but direct conversions (i.e. no filters, or advanced processes can be applied to the video images during the conversion)
Uncompressed transcoding produces the largest file size, will generally be playable in VLC and Quicktime but not Windows Media Player, and is theoretically always possible. If you are able to decode the file, you should be able to transcode it using this method in all cases.
Lossless H.264 transcoding produces a moderate file size, and like the uncompressed option is generally playable in some widely available players, but not in Windows Media Player or Quicktime. Also like uncompressed, it’s theoretically always possible to use this method.
Evaluate these benefits and disadvantages of each method based on your own situation: what type of original video you have, which criteria are most important to you, which player will you be using, and so on.
Finally, make one last consideration: do you always need to have 100% lossless video?
There are also “lossy” transcoding options that doesn’t produce an output of the exact same image quality, but still may be useful.
For example, in iNPUT-ACE, select an MP4 output node, and change the compression quality from 0 (i.e. “lossless”), to something around 20. The value of “20” will introduce some quality loss, but it may not be noticeable to the human eye, and may visually look exactly the same as the lossless version.
Comparing a lossless output and a compressed MP4 in Canvas Editor
Comparing the two side-by-side in the Canvas Editor shows that producing a slightly compressed MP4 may not produce a visually detectable difference. The possible benefits of this option are that the results are easy to play in practically all video players, maintain small file size, and are quick to perform. And if the limitation of the slightly reduced image quality is problematic, you will still have the original file to return to and perform one of the other transcoding options: rewrapping, uncompressed, or lossless H.264.
In summary, the most common transcoding options are:
- Re-wrapping using the default AVI output settings (lossless)
- H.264 using the default MP4 output settings (lossless)
- Uncompressed using the default AVI output settings (lossless)
- H.264 by selecting MP4 and sliding the Compression Quality slider to the right (lossy – with increased compression the higher the Compression Quality is set. Using numbers around 20 will still produce “visually lossless” quality)
iNPUT-ACE reads hundreds of proprietary video formats, and solves issues common to proprietary players with its simple drag-and-drop interface that protects evidence from the start. Ready to begin? Contact us for an iNPUT-ACE trial.
If you’re interested in advanced iNPUT-ACE training, sign up for one of our two-day classes here.