Video Converter. Step by step.

Let me explain what exactly happens when encoding job is triggered. These are major steps and they may be modified in the future. For now program works as single entity, but probably it will be much more efficient to break it down into two components: queue and conversion as separate console apps.

  • Initial Step includes parsing xml files that store application preferences.
  • Scanning Recorded TV folder for newly added files. We utilize LINQ queries to do the job. Files that are currently being recorded are ignored. These files do not contain complete metadata yet and that allows us to identify them first.
  • Extracting metadata from recorded WTV files and writing information into separate xml files with corresponding original file name. I will write separate post on how to get metadata from TV recordings.
  • Cleaning up orphaned metadata xml files. Some recordings may be already deleted.

These steps lead us to building our queue:

  • Parsing existing metadata xml files
  • Parse xml file called job.process.xml that contains names of the items that we want to convert. Concept is similar to Season Pass. We also include our preferences for final product like file format MKV/MP4 and encoding profile.
  • We use LINQ queries to identify recordings that were not processed yet. We can also specify some additional criteria here like channels that need to be excluded from processing.
  • Our findings are written to job.queue.xml file

Conversion steps:

  • Parsing queue xml file
  • Parsing encoding profiles.
  • Queue items should be sorted out based on defined priorities
  • Getting first item in a queue for processing
  • Converting WTV to DVR-MS using WTVConverter.exe
  • Converting DVR-MS to MPEG using FFMPEG
  • Converting MPEG to MKV/MP4 using either FFMPEG or Handbrake

Post-processing steps:

  • Renaming completed files and organizing them into proper folder structure. Separating movies and TV Series items.
  • Creating Kodi type nfo metadata files. Our priority is to use metadata that we already extracted from WTV files. However we will also add ability to retrieve metadata from services like thetvdb.com and themoviesdb.org

As a final step we should check queue to see if there is any additional pending item.

This is a rough idea what should be done when we want to convert a video recording. As you can see we have 3 parts:

  • Building queue
  • Conversion
  • Post-processing

If we combine all of them into single process we are risking some unexpected outcomes. For examples if something fails during post-processing, we may not be able to continue with converting another file in a queue. Therefore it may be better to separate these tasks. On the other hand it would make more sense to isolate building queue first and leave conversion and post-processing together.

That pesky WTV file format.

I spent whole week trying to put something together that would do a job. Soccer tournament Euro 2016 is around a corner and I wanted to record games and test encoding files with Handbrake.

I have to say that this whole experience is really frustrating. Handbrake supports WTV files recorded with Windows Media Center, but every once a while Handbrake desktop app stops conversion process for no apparent reasons. It would seem that there might be issue with wtv itself. So out of 10 files, it is most likely that at least one of them would fail to complete. Similar things were happening while I was testing HandbrakeCLI on my HTPC computer. Console window would just stay open for hours and job would refuse to finish.

I tried different approach and converted WTV file to MPEG format using FFMPEG and setup a job for HandbrakeCLI. Again result is unpredictable. Some files would complete and some simply would refuse to continue.

Since Handbrake was giving me so much trouble, I decided to do conversion in FFMPEG only. First change WTV file to MPEG and then convert MPEG to MKV format. It seemed in the beginning that FFMPEG does a better job, but at some point it failed too. Encoding process stalled for hours.

At this point I had to revert to converting WTV file to DVR-MS format first using Microsoft built-in WTVConverter.exe . This actually reduced errors significantly. I believe that it happened only once or twice and after reboot of HTPC job started and continued until it was finish.

Both Handbrake and FFMPEG support WTV format, however it is better to use following path for unattended video conversion:

  • WTV to DVR-MS using WTVConverter.exe
  • DVR-MS to MPEG using FFMPEG
  • MPEG to MKV/MP4 using FFMPEG or HandbrakeCLI

Encoding is fairly fast. I would say about 1:1, but with additional conversion steps whole process takes longer. Initially I was copying WTV file to temp folder and then do conversion. Once I eliminated that step, process is little bit faster. Recording that is 2-3 hours long can take between 20GB to 30GB. When comes to HTPC performance while doing encoding job, I did not notice major impact on either watching Live or recorded TV show. My HTPC is not the most powerful PC. It uses quad core AMD processor of older generation. I would provide specs later.

 

EDIT: I did about 50 jobs of recorded games and so far I did not have any single failure.

Project: video converter

One of the problems with HTPC is that hard drives fill ups with recordings at much faster pace than we can watch it. Some shows can be deleted almost immediately. Other ones could be archived for later viewing. Unfortunately due to a file size of average recording, is not really practical to store entire season of some TV show in its original format.

The best approach is to remove commercials and convert them to h.264 using tools like Handbrake, VideoReDo or MCEBuddy.  That process is time consuming and it requires user interaction of setting up all available options and starting manually encoding process. Exception is MCEBuddy that does encoding automatically. However, from my personal experience I was not happy with final product.  It uses Handbrake as its encoding engine, but I was getting mixed results and selecting right encoding options is not straight forward.

There is another program called DVRMSToobox that can take care of removing commercial using Comskip or ShowAnalyzer. I believe it can also do some encoding. I used it in the past but at some point removing commercials is not that important anymore.  Skip button works pretty well. That feature is becoming less relevant. Most of the shows will be deleted anyway. If I have to remove commercial I do it manually using VideoReDo that way I have full control of parts that I want to delete.

So here comes converter that I was thinking of. These are the things that I want to achieve:

  • Create queue of items for conversion based on certain criteria
  • Select items based on a name
  • Exclude recordings for channels that are copy protected
  • Extract recordings metadata from WTV file and store it for later use
  • Encode files using HandbrakeCLI
  • Encode files using FFMPEG
  • Utilize existing handbrake and FFMPEG encoding profiles
  • Create custom encoding profiles
  • Apply different encoding profiles for different recordings based on user selection
  • Rename encoded files and create standard folder structures for movies and TV series
  • Create nfo metadata files for Kodi
  • Retrieve metadata from thetvdb.com
  • Move files to a server
  • Rename recorded files without encoding
  • Create RSS feeds
  • This should be a console application that can be started in task manager
  • This could also be triggered as part of watch folder scenario.

 

FFMPEG is a command line program however Handbrake provides 2 different ways of interaction. CLI interface seems to be better suited for my needs and I was able to find more documentation on how to actually use it.

At this point I’m mostly concern on figuring out all the necessary pieces that would make my program work.  Basically I want to take a recorded file from point A to point B that means convert it first in the most predictable way.  I want to get results first.