GTX 1660 ti /NVENC was fast, but the file sizes were terrible!

My rig is a Ryzen 7-1700 w/ 16GB ram, a bunch of hard drives with a SSD for the OS drive- Windows 10 Professional. My video card was/is a GT 730 2GB. I use the donator version 2.4 v11 of MCEB and have HW encoding enabled.

Earlier this year I began capturing some old half hour TV shows via my Roku and a capture box. I use OBS Studio version 23.2 (latest) with basically default settings (1920 x 1080). I did stretch the screen since the shows are in an older format. The end result picture looks great on a HDTV! After the box captures the file, it goes to my HTPC, where MCEB picks it up & processes it using the HEVC MP4 codec. I created a special setting for these shows since they have no ads- no need for Comskip.

Under the GT 730 setup, the rough file was around 470 MB. MCEB knocked it down to around 220 MB. I need these to be as small as possible since there are like 40 episodes/season and I plan on archiving all of them.

After doing a lot of reading, I decided to buy a better graphics card. I got a good deal on the GTX 1660 ti and installed it. I updated the drivers and used the H265 NVENC codec under OBS. After the below results I also used the H264 codec (presumably the same one the 730 used?) and got similar results.

Processing times in MCEB dropped dramatically- like from 40 minutes to 9 minutes, and this took the load off the CPU, which before when processing 2 streams would hit 100%. I was disappointed to see that the end result file sizes were almost as large as the original capture file (well over 400 MB).

I experimented with a number of settings such as bitrate but could not get the same level of compression. I ended up returning the 1660 and reinstalled the 730. My first file conversion and things were back to normal for processing time and file sizes. Since it’s a dedicated machine, I can run the conversions off-hours, but I was expecting far better results, especially since Nvidia and OBS worked together on the latest OBS version. If I was doing something wrong, I’m not sure what it was.

I’m curious if anyone else has seen something similar using an Nvidia card? I’m curious if a more basic card such as a 1050 may work better.

I’ve found file sizes in NVENC to be sub-optimal. I suggest that you crank up the compression to get the files back down. Going back to a 1050 won’t make the files any smaller - just slow the conversion.

Thanks. How do I crank up the compression? I tried reducing the bitrate from 2500 to 2000 and 1500 and the first did not make much of a size difference while the latter was poor quality video.

I suggest you look at profile. Look for this line:

ffmpeg-video=-ss 9 -c:v hevc_nvenc -cq 26 -rc vbr -map 0:v

The CQ part refers to “Constant Quality” - Higher numbers are lower quality and of course, the h265 scale is different from the h264, A CQ of 23 in H265 is about the same as a quality rating of 20 in H264.

Using the above will reduce the file sizes by about 60% vs the uncompressed .TS file. The scale is logarithmic, so small tweaks can make a big impact.

Good luck! Will.

1 Like

Thanks for your time & advice. I found the profile settings, and here is what is on that line:
ffmpeg-video=-ss 0 -vf yadif=0:-1:1,hqdn3d -vcodec libx265 -preset medium -crf 26 -map 0:v -sn

I’m presuming it is different since the GTX/NVENC is gone.

Do you have any advice for a better H265 capable video card to run this operation? I’d rather not spend $200+ again if a lesser card will give decent results and take a load off the CPU. I know the 1050 is a popular card which is why I had mentioned it.

Once I have a HVEC capable card If I can find the similar ffmpeg setting in OBS I may be able to take a load off MCEB as well.

Hi. The profile you excerpted is using x265 which is all software. If you change libx265 to hevc_nvenc, it will start to use your video card for encoding. The syntax for ffmpeg is a little arcane - I posted the profile I use below:

[MKV HVEC Beta Profile Constant Quality]
Description=WARNING: Handbrake Constant Quality encoding (26) with Nvidia HVEC.
order=ffmpeg, handbrake
ffmpeg-general=-threads 0 -hwaccel auto
ffmpeg-video=-ss 9 -c:v hevc_nvenc -cq 26 -rc vbr -map 0:v
ffmpeg-audio=-acodec ac3 -ab 192k -map 0:a
ffmpeg-audioac3=-acodec ac3 -ab 384k -map 0:a
handbrake-general=--decomb --denoise="weak" --loose-anamorphic --verbose=2 -T -O
handbrake-video=--start-at duration:0 --encoder nvenc_h265 --encoder-preset slow --quality 23 --vfr
handbrake-audio=-E ffac3 -R auto -B 192 -D 0 -a 1,2,3,4,5
handbrake-audioac3=-E ffac3 -R auto -B 384 -D 0 -a 1,2,3,4,5

The card you have is more than fine - it’s an updated version of the GTX 10xx line - I bought a RTX 2060 for my Plex / MCEBuddy box, but your 16xx wasn’t out when I bought mine. Good luck! Will.

One other thing: if you’re on a recent build of Windows 10, you can look at the task manager to see where the work is being done as the file is converting. If things are working correctly, you should see the video card spike to 99% when the conversion is actually running. It’s pretty cool. Thanks! Will.

Thank you for the post. I have removed & returned the GTX 1660 ti, so I’m back to the GT 730 which NVIDIA says does not support H265 on their matrix.

I use both HW monitor and GPU monitor programs to see what the CPU and GPU are doing when MCEB is running.

I’ll try to find a budget 10 series card that will do so. Thanks again.

I’ve been happy with my 2060 - the 2050 will also work and is cheaper…

Using your info, I played around in OBS Studio, as my goal is to get a livable file size with the initial capture and just use MCEB for show data and to move it to Plex. I’m not really sure how ffmpeg works, but know how to input the code. Using the below process, I did try to add a couple of dll files to OBS based on a post there so I could use libx265, but the recorder crashed both times I tried it. I’ve read OBS does not support x265.

Using the ffmpeg text from the MCEB profile less the last line where it deals with commercials and using the H264_Nvenc codec, I saw a big reduction in 5 minute long test files. The ones I had captured using H265 Nvenc were about 190 MB while the one I did today was 103 MB (this ratio seems to about what I’m getting using MCEB). The neat thing was I was looking at the OBS data bar and it showed like 14% CPU use while I was capturing. I played it on another machine and it looked good.

I still have some of my half-hour format files (about 24:30 in length) so I’ll try one of those and see what I get. If this works the same way, I can hold off on another video card.

Thanks again for all your help.

Well back to the drawing board. My 24 min file using custom settings was like 503 MB. I realized I was looking at the largest H265 file I had made when I commented before, others using H264 were smaller. The same show using the prior setup was around 487MB, so this made it larger.

I’ll keep trying different settings. If MCEB can do this on the same machine using ffmpeg, there must be a way to get OBS to do so.