On 02 Oct 2014, at 18:06, Ryan M <rymerr...@gmail.com> wrote: > "The sum of several single threaded encodings will result in more total-fps > then 1 multithreaded encoding." > This is interesting. So whatever route I take (many separate boxes or one > many core system) feeding it with several jobs at once and limiting it to > one core is the way to go. > The issue with what your suggesting then is that if there is only one or > two jobs to run, a lot of the boxes are going to be sitting there doing > nothing. Maximizing the throughput on your system requires it always being > feed the maximum # of jobs. With some clever thinking you can divide your source-file in amount of pieces = # cores-1. This way all cores create 1 audio and # cores -1 video parts with can be catted together and muxed to 1 new file. Catting and muxing doesn’t cost any time in relation to the time needed for encoding. Your source file ProResHQ is afaik already a I-frames only file so dividing by # cores -1 is very easy. > The dual Xeon (or lesser but still many core) > is going to be more flexible in that regard (so the high price tag is for > that flexibility). Yes if the one time investment in thinking is more costly. > > On Wed, Oct 1, 2014 at 12:21 PM, Henk D. Schoneveld <belca...@zonnet.nl> > wrote: > >> Some additional things to consider. >> >> Perofrm some short tests, with a 1000 frames file, try with threads 1, >> threads 2, threads 3, etc >> Threads 1 gives you 100% of a single core say 4fps. Threads 2 doesn’t >> perform 8fps, because of overhead, and adding another thread adds less then >> the 2nd core. The sum of several single threaded encodings will result in >> more total-fps then 1 multithreaded encoding. >> >> If you want 40fps you can easily add as many DualCelerons as mentioned >> before to get the desired result, you only need to add the ‘missing’ fps to >> the CPU you already have. >> >> If you could send a 1000frames file, I’m happy to test te results for the >> mentioned Celeron. >> >> Henk >> >> On 01 Oct 2014, at 04:07, Ryan M <rymerr...@gmail.com> wrote: >> >>> It is definitely something to think of/consider. Thanks for the input. >>> >>> On Mon, Sep 29, 2014 at 2:14 PM, Henk D. Schoneveld <belca...@zonnet.nl> >>> wrote: >>> >>>> Size of 1 system 8 * 8.5 * 2.25 inches, so they are very small just a >>>> little bigger then an Apple Mini. >>>> So 8 of them doen’t take that much space, you can stack them. >>>> >>>> On 29 Sep 2014, at 18:37, Ryan M <rymerr...@gmail.com> wrote: >>>> >>>>> Yeah really the actual space/room is the issue there, not sure where >> i'd >>>>> put that. >>>>> With PPSS, it looks like it distributes the encoding of different files >>>>> over the available resources, it wouldn't spread a single encoding job >>>> over >>>>> them correct? >>>> Yes that’s correct. If 1 resource becomes free, job done, it takes the >>>> next file to be done. >>>>> I do need a way to automate ffmpeg instances so more jobs >>>>> can run at the same time. >>>> You can run as many jobs as you do have CPU cores, with 8 systems being >>>> DualCore, 16 jobs at once. >>>>> My jobs aren't just a bunch of files sitting in >>>>> a directory, it is querying a db for files that need to be encoded in >>>>> various ways. >>>> You could run multiple instances of PPSS, with different encoding >>>> parameters. You could query your DB for files with ffmpeg parameters X, >>>> make symlinks to Directory X >>>> and >>>> another query for files with parameters Y and make symlinks to >> directory Y >>>> One PPSS instance runs the X and another instance runs the Y symlinks >>>> directory. >>>>> >>>>> >>>>> On Mon, Sep 29, 2014 at 11:15 AM, Henk D. Schoneveld < >> belca...@zonnet.nl >>>>> >>>>> wrote: >>>>> >>>>>> Why wouldn’t it be realistic ? >>>>>> Have a look at https://code.google.com/p/ppss/ to distribute jobs >> over >>>>>> the amount of systems you have and works like a charm. >>>>>> On 29 Sep 2014, at 17:05, Ryan M <rymerr...@gmail.com> wrote: >>>>>> >>>>>>> I don't know that it is realistic to have 8 systems running though. >>>> If I >>>>>>> can pay more and have the approx same amount of throughput with one >>>>>> system, >>>>>>> that would be better. >>>>>> In what way would it be better, less room, yes but that’s very >> expensive >>>>>> room you pay for in my opinion. >>>>>>> I just don't want to pay like $4k more (for the >>>>>>> Xeon vs Haswell-EP) and only get a few more fps out of it. If >> stepping >>>>>> up >>>>>>> to Xeon allows ~40fps like Sean said then it is probably worth it. >>>>>> Thank >>>>>>> you very much for the input. >>>>>>> >>>>>>> On Sun, Sep 28, 2014 at 7:23 AM, Henk D. Schoneveld < >>>> belca...@zonnet.nl> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> On 28 Sep 2014, at 13:13, Henk D. Schoneveld <belca...@zonnet.nl> >>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> On 26 Sep 2014, at 18:36, Ryan M <rymerr...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> I am putting together hardware to encode a large and ever growing >>>>>>>> catalog >>>>>>>>>> of video using ffmpeg/x264. Much of the source video is 1080p >>>>>> ProResHQ. >>>>>>>>>> Currently I have a box with a Haswell 4770k CPU which gets around >>>>>> 12-15 >>>>>>>>>> fps, I am of course looking to increase that as much as possible. >>>>>>>>> Have a look at >>>>>>>> >>>>>> >>>> >> http://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-5960X+%40+3.00GHz&id=2332 >>>>>>>>> and compare with >>>>>>>>> >>>>>>>> >>>>>> >>>> >> http://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-4770K+%40+3.50GHz&id=1919 >>>>>>>>> or in 1 view >>>>>>>>> http://www.cpubenchmark.net/compare.php?cmp[]=1919&cmp[]=2364 >>>>>>>>> 7 times the price for less then 2 times the performance. >>>>>>>>> Spreading the task over more much cheaper CPU’s gives you more bang >>>> for >>>>>>>> the buck if realtime encoding isn’t needed. >>>>>>>>> You could use 8 Dual Celeron systems to get the same performance >> as 1 >>>>>>>> E5-2690 where CPU cost would be 8x52.45 vs. 1x2299.99 Of course >>>>>>>> motherboards etc are needed and housing but a low cost MB and >> Housing >>>>>> can >>>>>>>> be bought at < 100.00 a piece. 8*(100+52.45) ~ 1220.00 >>>>>>>>> Saved more then $1.000 And the Xeon still needs a motherboard and >>>> case >>>>>>>> of course. >>>>>>>> The CPU Mark and single thread performance are good indicators for >>>>>>>> relative ffmpeg encoding speeds. I do have i5 i7 and the named >>>> Celerons >>>>>>>> which I compared extensively with the same source files and encoding >>>>>>>> parameters. >>>>>>>> Disk I/O isn’t any problem, think of it, 5 times BluRay bitrates >>>> results >>>>>>>> in 90Kb/s < 12MB/s. Every modern disk has no problem with that and >>>> even >>>>>>>> old-fashioned nics, 100Mb/s can cope with that. >>>>>>>>>> >>>>>>>>>> Trying to determine if getting the latest dual CPU Xeon (such as >>>>>> E5-2690 >>>>>>>>>> v3) setup is going to be worth the significant additional cost >> over >>>>>> the >>>>>>>>>> best Haswell-E (Core i7-5960X). >>>>>>>>>> >>>>>>>>>> I've read lots of articles/posts and it is not clear to me. I >> know >>>>>>>> worth >>>>>>>>>> is subjective but looking to know if there'll be significant >>>> increase >>>>>> in >>>>>>>>>> fps using 2 Xeons. I need to justify the $6-7k price tag. >>>>>>>>>> >>>>>>>>>> Any insight/experience would be appreciated. >>>>>>>>>> >>>>>>>>>> thanks >>>>>>>>>> Ryan >>>>>>>>>> _______________________________________________ >>>>>>>>>> ffmpeg-user mailing list >>>>>>>>>> ffmpeg-user@ffmpeg.org >>>>>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> ffmpeg-user mailing list >>>>>>>>> ffmpeg-user@ffmpeg.org >>>>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> ffmpeg-user mailing list >>>>>>>> ffmpeg-user@ffmpeg.org >>>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> ffmpeg-user mailing list >>>>>>> ffmpeg-user@ffmpeg.org >>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>>>> >>>>>> _______________________________________________ >>>>>> ffmpeg-user mailing list >>>>>> ffmpeg-user@ffmpeg.org >>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>>>> >>>>> _______________________________________________ >>>>> ffmpeg-user mailing list >>>>> ffmpeg-user@ffmpeg.org >>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>> >>>> _______________________________________________ >>>> ffmpeg-user mailing list >>>> ffmpeg-user@ffmpeg.org >>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >>>> >>> _______________________________________________ >>> ffmpeg-user mailing list >>> ffmpeg-user@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >> >> _______________________________________________ >> ffmpeg-user mailing list >> ffmpeg-user@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-user >> > _______________________________________________ > ffmpeg-user mailing list > ffmpeg-user@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-user
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user
