History File to DB

BUG / NEW FEATURE

Version 2.5.7 64-bit

Windows 10 x64

It would be nice if there was an option to store the history information in a sqlite db or something of the sorts. My history file gets fairly large, and in turn after about 6 months or so, MCEBuddy stops picking up files during scan, and I have to manually add them or as of recently I have a powershell script that adds them via the CLI.

Once I clear the history file, the scan completes in the matter of seconds, and in turn if I haven’t cleared out some of the adhoc folders, it will add them to the conversion tasks. I feel that it is due to the amount of records in the history file.

If the option to record to a DB is present I feel the search will be faster, and in the event of corruption, or recreation, the previous records could possibly be re-added from a backup.

Just clear the history periodically. Is there a reason you need to know MCEBuddy processed a file instead of something else? If the file is a duplicate (i.e. already there), MCEbuddy won’t process the file. Not being in the history doesn’t matter.

Or am I missing something?

The history is used within a monitoring location to not re-process a file already converted unless the “Re-monitor recorded videos” option is on.
image

There are times when I need to reprocess a file so I check this option. If you are in the same situation @Nick_Skoy then you can try and check this option to see if it picks up the new files. It seems like it should bypass parsing the history.

Sorry on the delayed response. Was out of town this weekend, and busy yesterday.

I have a HDHomeRun Prime, and am using NextPVR. Not to out of whack from what I gather. I have some shows setup to record “ALL Episodes” and others to record “Only New Episodes”. So I believe the problem is with shows that are in the “ALL Episodes” category. I’ll just use “Friends” as an example. It’s on multiple channels, and when I record I don’t care if the show gets recorded multiple times. Or from what I gather it will only record the same episode if on a different channel.

I’ve watched it when I re-add the episodes to MCEBuddy via command line, or drag and drop. It picks up the data, recognizes that it previously converted the file based on the history, and then deletes the file, and moves on to the next item in the queue. So it does the housekeeping as well. When my history file has gotten large or corrupt (even though I can still read the file fine, and don’t notice any visible corruption). The search doesn’t find the recorded shows, and in turn MCEBuddy is idle. Those files then build up, and there have been times where there are 100+ shows/movies that need to be converted, but they don’t. I add the files to MCEBuddy via command line, or manually drag and dropping, and over the course of time, it’ll add them to the queue, process them – whether converting or detecting it already converted, and deletes the source file from the recording folder.

I hope that kinda/makes sense/helps out…

It sounds like there could be 2 kinds of “duplicate/history removal” going on.

  1. The file is in the history DB/log (i.e. same name from the recording DVR) and so it is skipped before processing further.
  2. The file is initially processed as far as resolving what the destination filename would be, and if that file is already there, the file will be skipped before processing further (or maybe after processing, I’m not sure on that detail - I hope MCEBuddy determines it can skip the file before doing the work of processing the file).

Scenario #1 depends on the input file having the same name as a previous recording (i.e. per history rules).

Scenario #2 depends on the output file having the same name as a previously processed recording (i.e. per destination filenaming rules).

For example, when I set my HDHR to “record everything” (i.e. the whole series - it’s not (yet?) as smart as to use the “new” flag in the guide data like my Tivo is), the tuner puts the start-time HHMM and end-time HHMM in the filename. In addtion to the channel it was recorded on, so each broadcast will end up in a separate file, regardless if the guide data indicates any episode information or other metadata.

Notably, almost all the shows on PBS sub-channels (CreateTV, I’m looking at you) have no episode information or even show IDs, so they always end up as one-offs in my “Specials” destination instead of into a TV series destination (with seasons and episodes). Whether they are different episodes or simply rebroadcasts of the same episode at a different time.

This is because for my “Specials” job profiles I have to include that “start time” in the output filename so that I can tell there are potentially mutliple episodes and duplicate episodes. Otherwise, if they all resolve to an output filename of “Showname-SE-RecordDate”, the first one processed will block the others and they’ll get removed from the job queue as a duplicate output file under Scenario #2.

For TV Shows, I don’t want that to happen, and I do want a “first recording wins” kind of processing. So my output filename for TV series (i.e. have an episode # in the metadata) only includes the “FirstAirDate”, and not the “RecordDate”.

Sports events are typically live, and only relevant to the date of the event, so their filename renaming rule has the “RecordDate” not the “FirstAirDate” since some guide data has the “FirstAirDate” set to the date the entire sports show started, e.g. Monday Night Football (not that that is accurate, I just use it as an example to recognize). Also, in a game series show, they don’t always setup the metadata of the Game # in the series as an “episode #”, e.g. Game 3 of the “World Series 2022” show might show up as “World Series 2022 Game 3” in the show title (making recording the whole series just a cluster fart in the DVR) or as Episode 3 in the “World Series 2022” show. Always including the “RecordDate” in the filename for Sports shows solves that problem, no matter what the guide data / metadata says.

I hope that helps understand what MCEBuddy might be doing in your case. You can also search the forums for my posts with my file renaming rules that send each type of show (TV, Movie, Sports, and “Other/everything else”) to different places and have different output filename rules that play nice with my Plex setup.

The problem shows are the PBS shows that end up in my Specials folder and I have to manually de-dupe them and move them to where they go in the proper TV Show and rename them to the proper season and episode. I usually keep the record time as a suffix and then de-dupe keeping the best one MCEBuddy processed.

Can you attach or send me in a PM your history file (or let me know how many lines/entries it has)? We’ve tested it with up to 100,000 entries with no issues. We’ve also overhauled the INI database engine in the latest 2.5.8 beta version to allow it even larger databases.