[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Jan Gerber jger...@wikimedia.org changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #14 from Andre Klapper aklap...@wikimedia.org --- So is more work needed here (if so: what?) or can this be closed as RESOLVED FIXED? -- You are receiving this mail because: You are on the CC list for the bug.

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Rob Lanphier ro...@wikimedia.org changed: What|Removed |Added CC||ro...@wikimedia.org

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #10 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84493 merged by jenkins-bot: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493 -- You are receiving this mail because: You are

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Gerrit Notification Bot gerritad...@wikimedia.org changed: What|Removed |Added Status|RESOLVED

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Sam Reed (reedy) s...@reedyboy.net changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #11 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84775 had a related patch set uploaded by Reedy: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775 -- You are receiving this mail

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #12 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84775 merged by Akosiaris: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775 -- You are receiving this mail because: You are on the

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #13 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84539 merged by jenkins-bot: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539 -- You are receiving this mail because: You

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #6 from Jan Gerber jger...@wikimedia.org --- right now we run 10 concurrent jobs(puppet manifests/role/applicationserver.pp) with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot of this is mmapped this is

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Gerrit Notification Bot gerritad...@wikimedia.org changed: What|Removed |Added Status|NEW

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #7 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84493 had a related patch set uploaded by J: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493 -- You are receiving this mail

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #8 from Sam Reed (reedy) s...@reedyboy.net --- (In reply to comment #6) right now we run 10 concurrent jobs(puppet manifests/role/applicationserver.pp) with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #9 from Gerrit Notification Bot gerritad...@wikimedia.org --- Change 84539 had a related patch set uploaded by J: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539 -- You are receiving this mail

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #1 from Jan Gerber jger...@wikimedia.org --- problem with using multiple threads is that the memory use per encode multiplies too. whats the amount of ram available in the box, we might be able to use 2 threads per encode now, the

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #2 from Brion Vibber br...@wikimedia.org --- If y'all have to buy more RAM, please buy more RAM. :) Leaving CPUs idle while transcoding a 40m-minute 1080p video makes no sense at all. -- You are receiving this mail because: You

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #3 from Sam Reed (reedy) s...@reedyboy.net --- reedy@tmh1001:~$ cat /proc/meminfo MemTotal: 16419920 kB so 16GB See also http://ganglia.wikimedia.org/latest/?r=hourcs=ce=s=by+namec=Video%2520scalers%2520eqiadtab=mvn= Looks

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #4 from Sam Reed (reedy) s...@reedyboy.net --- I suspect we could at least double that one to a 2. The jobs are done synchronously, right? ie one at a time (per job runner)? In which case, we don't care so much how much memory is

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #5 from Sam Reed (reedy) s...@reedyboy.net --- Thinking about it, the simplest way to fix this bug might be to just make a global. It can then be set for WMF usage as appropriate - even (number of cores - 1) if the boxes only do one

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Sam Reed (reedy) s...@reedyboy.net changed: What|Removed |Added Priority|Unprioritized |High