How best to rip/compress Blu-ray to MKV etc.?

Discussion in 'High Definition Software' started by BLKMGK, Mar 9, 2008.

  1. BLKMGK

    BLKMGK Well-Known Member

    Okay, HD-DVD proved to be fairly straightforward once I talked to enough people and got some help. I've got my entire collection ripped and compressed to a server and they play terrific! However HD-DVD is pretty much toast now and I'm not even finding any good sales.

    So, I have purchased a 4x BluRay drive. I figured surely this wouldn't be SO much different, right? Well, not so much! I've made SOME progress after reading this thread. However the focus of that thread appears to be different, thus my posting here.

    I have managed to identify the two files that made up my movie (The Patriot). I have joined them together with TSsplitter. I have remuxed the the result with TsRemux such that I have just one audio file and one video file - full resolution. These are in a .M2TS format. While XBMC will probably play that it's 32+Gigs and a bit "bulky". My HD-DVD weigh in at about 10-13Gigs which is more like it - I'm not looking to burn to media.

    With an HD-DVD I'd split out the audio and video using eac3to, use the calculated frames in an AVIsynth script blah blah, compress the video with meGUI using x264, mux them back with MKVmerge, and call it a day. Unfortunately I cannot find (yet) anything nearly as helpful as eac3to for m2ts files although it looks like some audio conversion and channel flipping can be done with that tool - somehow.:confused:

    So, how do I get to my end goal from here? I considered building a graph with Haali and FFMPEG with just the video and trying to compress it but I've not got the frames calculated for a script so I worry about synch. I also have no idea how to pull out the audio:bang: I'm stuck!

    I'm NO pro at this but with help I've managed HD-DVD. Surely others are looking to do the same thing, has anyone come up with a work flow that consistently works? Surround sound, x264 compression, and reasonable file size with no noticable degradation in visual quality is what I'm after. Using PDVD isn't an option as Linux and XBMC is my target platform.:(

    P.S. I know better than to buy anything FOX not to worry! A shame too, Independance Day would've rocked:mad:
     
  2. GeeForce11

    GeeForce11 Well-Known Member

  3. BLKMGK

    BLKMGK Well-Known Member

    Thank you for the response! The last method you mentioned is actually sort of how I was considering doing this - or trying anyway. So far I've been trying to follow along with this http://www.avsforum.com/avs-vb/showthread.php?p=13318652#post13318652 and that has helped but it's slow going. I have the two m2ts files I need, have joined them, and have stripped all but the single audio and video track out. Using xport I've split those two apart and am now trying to use eac3to to put the video into a MKV container to compress as I would an HD-DVD. eac3to is hanging though partway through - doh! I've yet to touch the audio.:bang:

    What you've posted should help - my thanks! All in all this seems WAY harder than HD-DVD dammit. I am worried a great deal about audio synch and maintaining surround sound, have you found synch to be an issue? I'm hoping I can puzzle out a consistent workpath and not hit too many dead ends. I do not want to have to become a full on expert at this just to be able to watch my movies:mad:

    I appreciate the pointers, those were docs I'd not seen previously. If I can come up with something that layman can follow that's any different than what others have done I'll write it up. fingers crossed.... <sigh>

    Edit: Madshi, author of eac3to, says he will be upgrading his tool in the coming weeks to handle m2ts format files. That will be huge IMO.
     
    Last edited: Mar 9, 2008
  4. GeeForce11

    GeeForce11 Well-Known Member

    From what I noticed from the different BD discs I encoded, that sometimes one encoding process will not work properly with one disc, but it does with the other. :bang: The way I see it so far everything is trial and error.

    My goal is a bit different, I made XviD/AC3 encoded AVI backups of the movies with one click encoding as described in the first link of the previous post. The encoding(total work) per movie takes 8-10hrs on my 2.7GHz AMD X2. The backups are 4-5GB each, 6.4Mbit/s XviD @ 1920x1080 and 640kbit/s AC3 5.1.

    Luck with your project :)
     
  5. BLKMGK

    BLKMGK Well-Known Member

    Well, am still fooling around with this one disk. I've manged to rip it again using TSmuxeR which leaves me with an AC3 and a .264 file. What are you doing at that point? One click encoding in meGUI is laughing at me and VLC won't play the file. When you have multiple .m2ts files what are you doing with them exactly and what does the output look like? I'm going to have to read over those posts carefully, this is driving me nutz!

    Heh, the transcoding of just one of the Planet Earth HD-DVDs I just did took 19 hours and shaved 10Gigs off the original file - down to 11Gigs. That's on an overclocked C2D E8400 @3.8Ghz.:rock: My old AMD was taking as much as 80 hours to process some of the early files I did:disagree: Processing this stuff is NO joke and will seriously test the stability and cooling of a system. I can play games all day at 4Ghz but cannot always process a full video at that speed and there's nothing like throwing away 10hours of work to make you back a clock down to get through it next time.
     
  6. GeeForce11

    GeeForce11 Well-Known Member

    I didn't have a case of multiple m2ts files yet, but this is what I would do:
    - BDEdit would list the order of the files the whole movie is put together
    - I would join all these m2ts files accordingly with TSMuxer to get the whole movie in one m2ts file for encoding.
    - for example with Jake Ludington mod of VDubMod is one click encoding of the m2ts to avi or mkv. I encode the video with XviD and direct stream copy of the master audio track. XviD seems like 3x faster than H264 encoding, one of the reasons I went with it. I could of kept the original uncompressed DTS-HD master track (approx 3GB), but to further reduce the size of the avi file for storage I encoded the audio to AC3. It happened once that for some reason the master track was not processed like this, but the next audio track was processed automatically, and I had to demux the master audio track from the m2ts with xport and encode with EAC3to to AC3 and then mux it with the XviD video.

    It looks that if you don't want avi, but mp4, video and audio needs to be muxed out of the m2ts and encoded separately and then muxed back. This seemed for me more complicated and with more steps, another reason I went with avi.
     
    Last edited: Mar 10, 2008
  7. BLKMGK

    BLKMGK Well-Known Member

    Okay, tomorrow evening I will checkout the VDubMod code for sure along with the mods. AC3 is fine by me and it does save a ton of space! I'm shooting for a MKV container, 5.1 soundtrack, and high bitrate H.264 via x264. That's what I've done with HD-DVD and it has worked out perfectly. If I can do the same with BD than I'll be very happy.

    Thanks!
     
  8. BLKMGK

    BLKMGK Well-Known Member

    Okay, Tsmux'd the file to my drive, loaded up the virtualDubMod code, and boom - stuck! VirtualDubMod doesn't accept M2TS format files. Are you doing something to them prior to bringing them in?:bang:
     
  9. GeeForce11

    GeeForce11 Well-Known Member

    I just had the Combined Community Codec Pack (CCCP) with haali splitter and AviSynth installed, and the modded VDubMod imports the m2ts like this:
    [​IMG]
    Make sure to select on the bottom AviSynth Template: "m2ts editing".

    You won't need to create .avs AviSynth file to be able to import the m2ts, it will be created automatically first time you import the file. Once the .avs is created I also imported the m2ts.avs straight to MEGui, but didn't do the encoding/conversion there, but in VDubMod.
     
    Last edited: Mar 11, 2008
  10. BLKMGK

    BLKMGK Well-Known Member

    Ah! Now we're cooking with gas - I have a job queued up now :bowdown: I'm a bit confused on how you took this into meGUI though, did you simply take the template from this program and use it with meGUI or did you do something that would have to be done for each video converted? I see the m2ts profile in avst format but am not familiar with that.

    As soon as my current 1st pass job in meGUI finishes up I'll fire this one off and see hwo it goes. I took the x264 commandline from meGUI and just pasted it into the x264 config for this one so it ought to be interesting to see how it processes:D

    Thanks!

    P.S. Hrm, looks like I now have access to all sorts of filters too! Not sure I want to mess with those too much just yet but that could prove interesting when I have some time - maybe try them on a trailer.
     
    Last edited: Mar 11, 2008
  11. GeeForce11

    GeeForce11 Well-Known Member

    You don't have to do anything, just have the haali splitter and AVISynth installed already before you try to open the m2ts with VDubMod(the version i posted the link for in the first post has all the mods and templates included). It opens up the m2ts and creates the .avs file automatically, what you can import straight to MEGui. Once you open the m2ts in this VDubMod you can do the editing/converting, or close it and continue to work with the m2ts in MEGui.

    http://www.jakeludington.com/camcor...d_m2ts_files_from_sony_hdrsr1_camcorders.html
     
    Last edited: Mar 11, 2008
  12. BLKMGK

    BLKMGK Well-Known Member

    Ah okay, I hadn't noticed the AVI script created when I opened the file. What's odd is what's IN that script - it simply has an entry in it about using DirectShow, I'll post it when I'm at home. I'm FAR from an expert with AVIsynth scripts but this seems pretty simplistic! What I was using with HD-DVD was even a little more complicated than this <shrug>

    Last night I ran the 32ig file I've got through VDubMod and the result was a 5Gig file - this with max bitrate - that completed with "warnings" but I've yet to find what those were. My attempt to use the commandline generated by the meGUI settings failed, I think the x264 program versions are out of synch just for starters. In any case I've got this setup to run through meGUI including the audio track so we'll see what the two look like when it's all done. I've got a job ahead of it though that looks like it will take an entire day so it will be a bit before I've got anything to look at :eek: When done I ought to have a MKV video and an AC3 track to mux together with MKVmerge, audio synch is a worry so we'll see how it goes.

    Thank you for further clarifying!:bowdown:
     
  13. GeeForce11

    GeeForce11 Well-Known Member

    I know that avs thing is as simple as it could be, you could create that file yourself.

    Since you have the video already from VDubMod, I'd extract the wanted audio track from the m2ts with xport and encode it with EAC3to to AC3. This should take a few minutes. Then muxing the audio and video also should take a few minutes, and you're done. :)
     
  14. BLKMGK

    BLKMGK Well-Known Member

    Well, the compressed video from VDubMod fails to play correctly in XBMC. I'll see how it goes when the same video comes out of meGUI - fingers crossed! At least I feel like I'm making some progress, will know sometime late tomorrow maybe.:rock:
     
  15. GeeForce11

    GeeForce11 Well-Known Member

    I would be curious how would a third XviD compression AVI movie made with VDubMod compare quality wise to the MP4 you make.
     
  16. BLKMGK

    BLKMGK Well-Known Member

    That might be interesting to try - particularly if it doesn't take 34hours to compress:bang: I admit I'm using some settings that are a bit nutz but HD-DVD weren't nearly so objectionable time wise. I'll explore using XVID when this is done and the two HD-DVD following it that are stacked in the queue.
     
  17. GeeForce11

    GeeForce11 Well-Known Member

    I would experiment on a few minute clip instead of working with a whole movie.
    What speed do you get encoding to H264? 2-3fps?
     
  18. BLKMGK

    BLKMGK Well-Known Member

    Aww, where's the fun in a few minute clip?:p I'll see what kind of encoding rate I get from it - if we're talking a few hours no biggie but if it's going to take nearly as long then I'll be forced to use shorter pieces.

    Currently using x264 I'm seeing encodes of just about 1.9 frames per second:bang: That's on an E8400 C2D cranked to 3.8Ghz. I have to build a system to replace my slower AMD box which has just puked badly, I'm pondering a Q6600 or another E8400 so that I can offload some of this to it. I do not think I can wait any longer for the next 45nm quads coming out nor am I sure they would buy me anything over an overclocked Q6600. <sigh>
     
  19. GeeForce11

    GeeForce11 Well-Known Member

    From what I see during encoding, these programs don't use multiple cores, CPU usage is always 50-60% on a dual core machine, so there won't be any benefits on the Quad core, at least for the time being until multi threaded software comes out. :(
     
  20. BLKMGK

    BLKMGK Well-Known Member

    Umm which programs? X264 most certainly DOES use multiple cores on my machine and in fact on playback the H.264 decoder on Linux XBMC uses multiple cores too. As it happens when I was trying to decode audio for this BD media it died, I aborted and kicked off the video transcoding. I just noticed while doing another HD-DVD that framerate was slow. Looking I found one core consumed by a zombied program from that audio experience. Killed it and performance is back. On my machine at least, using meGUI, I'm seeing improvements with multiple cores. A quad, even at slower clock, ought to be able to beat my high clocked dual for work like this. Am making a purchase decision today or tomorrow, wish I knew when the new 45nm Intels were going to hit the street :bang: I do not, yet, know if XVID will use multiple cores. If it doesn't I'll kick off a second job:D