Rüdiger Thank you for the reply. The system (Zebi - NAS) that we are using has solid state drives for caching reads and writes and is setup to be very fast. I don't have benchmark data from our other servers, so I'm not sure whether the Zebi - NAS is faster or slower.
Here are some of the stats that I have from the Zebi - NAS: Configured: Double parity RAID - can withstand 3 drive failures Performance: Maximum theoretical NFS Input/Output operations per second = 2,000 IOPS (I haven't confirmed this theoretical limit) ZEBI NFS Input/Output operations per second = 1,500 IOPS Maximum theoretical network throughput = 850 Mb/sec (I haven't confirmed this theoretical limit) ZEBI network throughput = 750 Mb/sec Disk throughput = 90 MB/sec Deduplication is around 2:1. Although video cannot be deduped, the multiple instances of a given video translate into only a single copy residing on the Zebi - NAS. I did change the value for the following in the config.properties file: composer.threads = 8 videosegmenter.threads = 8 textanalyzer.threads = 8 archive.threads = 8 I'm not sure if all of these thread values are used in a capture using a standard workflow, but I now have more than one ffmpeg process running on the Worker server when multiple captures are being processed. In all, it seems that the change in the thread values has sped up the processing time of a 1 minute capture from three minutes to one. However, this is all anecdotal, and stems primarily from me running lots of one minute test captures in a classroom and then watching the Admin UI as the job goes from Upcoming to Failed (yes, the job briefly goes into the failed section), then to Capturing, Processing, and then Finished. Brian On Tue, Jan 11, 2011 at 4:48 PM, Ruediger Rolf <[email protected]>wrote: > Hi Brian, > > from what I was told by our server adminstrators and what I can see in the > hobbit trend charts is that the bottleneck for a fast processing is not the > CPU load alone but also the disc access speed to the shared medium. maybe > you can have a look at this to if you want to speed up your environment. > > Rüdiger > > Am 31.12.2010 05:54, schrieb Brian Bolt: > > I have a distributed install running 1.0.1. The Admin and Engage servers > each have 16 cores (Intel). The Worker server has 24 cores (AMD). My first > one minute capture took about five minutes to process. My second capture is > an hour long, and at 1 hour into processing it is currently encoding the > presentation (screen) for preview. > > I expected that the Worker server would endure the greatest processing > load, and 'top' shows that ffmpeg is using 95 - 110% cpu. However, when I > run mpstat -P ALL, all processors are greater than 95% idle. Based on what > I saw in my VM test environment, I anticipated that a 1 hour capture would > take well under 15 minutes to process. > > On another note, the amount of space consumed is very large, 345 MB for > the one minute capture an 15.6 GB for the one hour capture. Note that I > don't yet have the workflow yet configured for streaming, so I anticipate > that these numbers would be even larger if streaming were in the mix. > > [r...@server matterhorn]# du -h --max-depth=1 > 25M ./streams > 8.8G ./files > 6.5G ./workspace > 68K ./server1 > 2.5K ./inbox > 78K ./searchindex > 22M ./downloads > 16G . > > Thoughts and ideas are greatly appreciated. > > Thanks, > Brian > > > _______________________________________________ > Matterhorn-users mailing list > [email protected]http://lists.opencastproject.org/mailman/listinfo/matterhorn-users > > > > -- > > ________________________________________________ > Rüdiger Rolf, M.A. > Universität Osnabrück - Zentrum virtUOS > Heger-Tor-Wall 12, 49069 Osnabrück > Telefon: (0541) 969-6511 - Fax: (0541) 969-16511 > E-Mail: [email protected] > Internet: www.virtuos.uni-osnabrueck.de > >
_______________________________________________ Matterhorn-users mailing list [email protected] http://lists.opencastproject.org/mailman/listinfo/matterhorn-users
