PlexDbFix. Moving database between computers.

This is probably one of the most annoying things for anybody who has Plex or Kodi database setup on their computer. At some point of time, something goes wrong and you have to wipe everything clean. As a result, all your past history including play status and bookmarks are gone.

I went through this process couple times.  It’s not fun, therefore I’m always cautious about losing ability to restore database to its previous state.

Recently, I had a problem with Plex Server that was residing on one the shares on my server. I decided to install Plex docker instead of plugin too. I also wanted to move it cache drive. It would seem like trivial thing. Install docker and map the paths to existing library. But for some reason Plex server refused to start. Changing file permissions did not fix the issue. Of course everything was working great when I started from scratch with a new database.

After multiple trial and errors I was able to figure out that problem was caused by additional files that are in the same folder as database. Basically, when SQLite database is initiated, it creates 2 files that are like a fingerprint for database. In order to substitute database you have to delete them first.

Once I figure out what is causing the problem I decided to try to move database to different PC. I knew exactly what I have to do, but to my surprise media files were not playing.  With a help of great tool SQLite Spy I was able to inspect file paths. Clearly in order to move database from one location to another, these paths need to be corrected.

I was able to do it in fairly quick. I had some parts of the code from different applications I was working on. All I had to do is to reuse it in the way that it made sense for this particular problem.

At the end standalone application PlexDBFix is a result of the initial idea.

 

plexDbFixSample

 

First it connects to Plex database. It retrieves media parts paths and puts the in the top ListView. Now you have 4 options where do you want to move database. In addition, you need to specify which part of the string you want to replace and what you want to replace it with. Pressing preview button will show changes. Apply button will actually update media_parts and sections tables.

 

Important! Before you proceed always backup your original database file and stop Plex Server.

When making changes you need to specify where you are moving database file. There are four options. You can transfer your database from Server to PC or vice versa. Also, you have choice selecting PC to PC or Server to Server when changing in between a folders only.

Next, you have decide what part of file path you want to substitute. Hitting Preview button will show you how it will look.

Finally you need to press Apply for actual changes to take place. There is no undo button, so as I said before backup your original database file first.

Why do you need to make this changes?

Plex Server database contains information about file paths of your media files. Therefore, when you move it form one location to another, Plex server will not be able to locate them unless you make actual changes. As you probably noticed, we are changing information for video files. We do not touch anything related to artwork. It’s not necessary to do it. Database holds only reference to these files. Therefore you have two options:

  • Copy/Paste folder containing artwork
  • Start from scratch. Create sections that match sections already created in existing database. Scan media folder and allow Plex download necessary metadata. Finally, substitute new created database with the saved one.

Where is Plex Server database located?

Depending on your platform and server setup, you should be able to navigate to Plex Media Server folder and find Plug-in Support. There is a Database folder containing a file that we are looking for. This is an example showing location on my home PC

“\Plex Media Server\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db”

Final step

Before we complete moving database to a new location, we have to do delete two files. Do not worry they will be recreated automatically when Plex Server starts again. If we fail to do so, Plex Server will simply not recognize our new database. These files are in the same folder as our database and they are:

com.plexapp.plugins.library.db-shm

com.plexapp.plugins.library.db-wal

com.plexapp.dlna.db-shm

com.plexapp.dlna.db-wal

Video Converter. Step by step.

Let me explain what exactly happens when encoding job is triggered. These are major steps and they may be modified in the future. For now program works as single entity, but probably it will be much more efficient to break it down into two components: queue and conversion as separate console apps.

  • Initial Step includes parsing xml files that store application preferences.
  • Scanning Recorded TV folder for newly added files. We utilize LINQ queries to do the job. Files that are currently being recorded are ignored. These files do not contain complete metadata yet and that allows us to identify them first.
  • Extracting metadata from recorded WTV files and writing information into separate xml files with corresponding original file name. I will write separate post on how to get metadata from TV recordings.
  • Cleaning up orphaned metadata xml files. Some recordings may be already deleted.

These steps lead us to building our queue:

  • Parsing existing metadata xml files
  • Parse xml file called job.process.xml that contains names of the items that we want to convert. Concept is similar to Season Pass. We also include our preferences for final product like file format MKV/MP4 and encoding profile.
  • We use LINQ queries to identify recordings that were not processed yet. We can also specify some additional criteria here like channels that need to be excluded from processing.
  • Our findings are written to job.queue.xml file

Conversion steps:

  • Parsing queue xml file
  • Parsing encoding profiles.
  • Queue items should be sorted out based on defined priorities
  • Getting first item in a queue for processing
  • Converting WTV to DVR-MS using WTVConverter.exe
  • Converting DVR-MS to MPEG using FFMPEG
  • Converting MPEG to MKV/MP4 using either FFMPEG or Handbrake

Post-processing steps:

  • Renaming completed files and organizing them into proper folder structure. Separating movies and TV Series items.
  • Creating Kodi type nfo metadata files. Our priority is to use metadata that we already extracted from WTV files. However we will also add ability to retrieve metadata from services like thetvdb.com and themoviesdb.org

As a final step we should check queue to see if there is any additional pending item.

This is a rough idea what should be done when we want to convert a video recording. As you can see we have 3 parts:

  • Building queue
  • Conversion
  • Post-processing

If we combine all of them into single process we are risking some unexpected outcomes. For examples if something fails during post-processing, we may not be able to continue with converting another file in a queue. Therefore it may be better to separate these tasks. On the other hand it would make more sense to isolate building queue first and leave conversion and post-processing together.

Project: video converter

One of the problems with HTPC is that hard drives fill ups with recordings at much faster pace than we can watch it. Some shows can be deleted almost immediately. Other ones could be archived for later viewing. Unfortunately due to a file size of average recording, is not really practical to store entire season of some TV show in its original format.

The best approach is to remove commercials and convert them to h.264 using tools like Handbrake, VideoReDo or MCEBuddy.  That process is time consuming and it requires user interaction of setting up all available options and starting manually encoding process. Exception is MCEBuddy that does encoding automatically. However, from my personal experience I was not happy with final product.  It uses Handbrake as its encoding engine, but I was getting mixed results and selecting right encoding options is not straight forward.

There is another program called DVRMSToobox that can take care of removing commercial using Comskip or ShowAnalyzer. I believe it can also do some encoding. I used it in the past but at some point removing commercials is not that important anymore.  Skip button works pretty well. That feature is becoming less relevant. Most of the shows will be deleted anyway. If I have to remove commercial I do it manually using VideoReDo that way I have full control of parts that I want to delete.

So here comes converter that I was thinking of. These are the things that I want to achieve:

  • Create queue of items for conversion based on certain criteria
  • Select items based on a name
  • Exclude recordings for channels that are copy protected
  • Extract recordings metadata from WTV file and store it for later use
  • Encode files using HandbrakeCLI
  • Encode files using FFMPEG
  • Utilize existing handbrake and FFMPEG encoding profiles
  • Create custom encoding profiles
  • Apply different encoding profiles for different recordings based on user selection
  • Rename encoded files and create standard folder structures for movies and TV series
  • Create nfo metadata files for Kodi
  • Retrieve metadata from thetvdb.com
  • Move files to a server
  • Rename recorded files without encoding
  • Create RSS feeds
  • This should be a console application that can be started in task manager
  • This could also be triggered as part of watch folder scenario.

 

FFMPEG is a command line program however Handbrake provides 2 different ways of interaction. CLI interface seems to be better suited for my needs and I was able to find more documentation on how to actually use it.

At this point I’m mostly concern on figuring out all the necessary pieces that would make my program work.  Basically I want to take a recorded file from point A to point B that means convert it first in the most predictable way.  I want to get results first.