David, As you probably know, there are two ways of using the "workspace" module, needed by the admins and workers to share the files processed: if you compile your machine with -Pworkspace, then Matterhorn assumes that the files are local to the machine; in this case, local may also mean "mounted in the system via a distributed file system", such as NFS, but accessible as if it was local. If, instead, you use -Pworkspace-stub, then the file is requested via REST service to a machine which does have a local workspace, and is copied via HTTP. That's where the 2GB limitation comes, since (apparently) you can not send big files via HTTP, at least not directly. As most people does not use workspace-stub, this issue has not been fixed yet.
If you can put your files in a shared folder and mount it in your admins/workers, so that they can use "workspace" rather than "workspace-stub", then you're good to go. Otherwise, you can use your recently gained new committer status to put some pressure on this issue being fixed :P Best regards Rubén 2012/2/16 <[email protected]> > Sorry David, I can't really tell a lot, I'm not so deep into all this and > rather the trial-and-error type ;) > > The other thing our admin did was to upgrade Java from version 6U15 to > 6U30. But that didn't really help here. > > What helped was the indicated line in the config on the *worker* node. > > Maybe just check if you get some log lines like > 2012-02-02 14:31:37 INFO (WorkspaceImpl:245) - Downloading http://server.... > on your worker node, just before your mentioned error message starts. > > While it should not download the file (and hits the 2GB limit there) but > rather get it from the shared storage. > > Andreas > > David Horwitz schrieb am Thu, 16 Feb 2012 betreff "Re: [Opencast...": > > Hmm ok that didn't work :-( Might be that I didn't quite understand what >> was needed. I now have: >> >> # The path to the repository of files used during media processing. >>> org.opencastproject.file.repo.**path=${org.opencastproject.** >>> storage.dir}/files >>> >>> # The interval in milliseconds between two rounds of dispatching in the >>> service registry. The default value is 5s, and >>> # a mimimum value of 1s is enforced due to performance reasons. Set to 0 >>> to disable dispatching from this service >>> # registry. >>> #org.opencastproject.**serviceregistry.**dispatchinterval=5000 >>> >>> # The base URL of the file server. When using a shared filesystem >>> between servers, set all servers to use the same URL. >>> # If this is set to any value other than >>> ${org.opencastproject.admin.**ui.url}, >>> the admin and file servers must use a >>> # single sign on (SSO) solution such as CAS to avoid requiring users to >>> perform multiple logins. >>> org.opencastproject.file.repo.**url=${org.opencastproject.** >>> admin.ui.url} >>> >> >> But it still fails :-( Did I get something wrong? >> >> David >> >> >> On 02/16/2012 04:35 PM, David Horwitz wrote: >> >>> Thanks Andreas, >>> >>> In my test one the encoded file was 2.1G so this could be the cause. >>> I've made the change in our dev environment and will report back once the >>> test has run. >>> >>> D >>> >>> On 02/16/2012 04:27 PM, [email protected] wrote: >>> >>>> Hi David, >>>> >>>> we have been having the same issue for some videos. As you already >>>> said, this occurs only with some files, and the reason we found here for >>>> the failure is 2fold: >>>> >>>> 1st the configuration admin-worker: as you already said it only happens >>>> there >>>> 2nd the limit of 2GB for Jetty downloads: for some reason the worker >>>> *downloads* the prepared files, and if they are of a size > 2GB it will cut >>>> them at exactly 2GB wich causes the "moov atom not found" error. >>>> >>>> Our admin found this to fix the issue: >>>> in >>>> /opt/matterhorn/felix/conf/**config.properties >>>> set >>>> org.opencastproject.file.repo.**url=${org.opencastproject.** >>>> admin.ui.url} >>>> >>>> as suggested in >>>> http://opencast.jira.com/**browse/MH-8032<http://opencast.jira.com/browse/MH-8032> >>>> >>>> Hope this helps you also, regards, Andreas >>>> >>>> David Horwitz schrieb am Thu, 16 Feb 2012 betreff "[Opencast >>>> Matterhorn]...": >>>> >>>>> Hi All, >>>>> >>>>> We're seeing failures on our production instance in encoding certain >>>>> files. The failure is consistent and always happens at the encoding of the >>>>> presentation for the hold state preview. The log message looks as follows: >>>>> >>>>> 2012-02-16 15:44:28 INFO (AbstractCmdlineEncoderEngine:**233) - >>>>>> Executing encoding command: ffmpeg -strict unofficial -i >>>>>> /data/matterhorn/workspace/**mediapackage/eeaa4715-e6c5-** >>>>>> 444e-88eb-4c0835263a25/**632f74bf-0dfe-4323-9a5a-**ae6f44fbf619/Presentation.mp4 >>>>>> -r 10 -s 320x200 -vcodec flv -b 256000 -ar 11025 >>>>>> /data/matterhorn/workspace/ >>>>>> **mediapackage/eeaa4715-e6c5-**444e-88eb-4c0835263a25/** >>>>>> 632f74bf-0dfe-4323-9a5a-**ae6f44fbf619/Presentation_** >>>>>> b36b560a-3e58-4ee1-a17c-**5d21f077b3ce-10fps-preview.flv 2012-02-16 >>>>>> 15:44:30 INFO (FFmpegEncoderEngine:174) - ffmpeg version 0.8.10, >>>>>> Copyright >>>>>> (c) 2000-2011 the FFmpeg developers >>>>>> 2012-02-16 15:44:30 INFO (FFmpegEncoderEngine:174) - >>>>>> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x13c5760] moov atom not found >>>>>> 2012-02-16 15:44:30 INFO (FFmpegEncoderEngine:174) - >>>>>> /data/matterhorn/workspace/**mediapackage/eeaa4715-e6c5-** >>>>>> 444e-88eb-4c0835263a25/**632f74bf-0dfe-4323-9a5a-**ae6f44fbf619/Presentation.mp4: >>>>>> Operation not permitted >>>>>> 2012-02-16 15:44:30 WARN (AbstractCmdlineEncoderEngine:**269) - >>>>>> Error while encoding video track Presentation.mp4 using >>>>>> 'flash-preview.http': Encoder exited abnormally with status 1 >>>>>> 2012-02-16 15:44:30 WARN (ComposerServiceImpl:249) - Error encoding >>>>>> http://mediadev.cet.uct.ac.za/**files/mediapackage/eeaa4715-** >>>>>> e6c5-444e-88eb-4c0835263a25/**632f74bf-0dfe-4323-9a5a-** >>>>>> ae6f44fbf619/presentation.mp4<http://mediadev.cet.uct.ac.za/files/mediapackage/eeaa4715-e6c5-444e-88eb-4c0835263a25/632f74bf-0dfe-4323-9a5a-ae6f44fbf619/presentation.mp4>and >>>>>> null >>>>>> >>>>> >>>>> >>>>> The error seems to happen when some or all of the presentation is the >>>>> epiphan capture card (seen when no source was connected). I can reliably >>>>> reproduce this with certain recordings on both our production and >>>>> development environments (separate admin/worker) but not my local build >>>>> (all in one). This, with the error "Operation not permitted" suggests some >>>>> kind of workflow/file repository error - the ffmpeg docs lean towards this >>>>> being 1 of the files doesn't exist. However this doesn't explain its >>>>> reproducibility with certain packages and not others. >>>>> >>>>> So far I have verified: >>>>> >>>>> 1) The profile creates a valid working copy (i.e ffmpeg isn't encoding >>>>> to to 0b file) >>>>> 2) the file doesn't exist at: >>>>> /data/matterhorn/workspace/**mediapackage/eeaa4715-e6c5-** >>>>> 444e-88eb-4c0835263a25/**632f74bf-0dfe-4323-9a5a-**ae6f44fbf619/Presentation.mp4 >>>>> But I don't know if one expects it to be cleaned up after the failure ... >>>>> >>>>> >>>>> Any help in solving this is apreciated >>>>> >>>>> David >>>>> >>>>> >>>>> >>>>> ______________________________**_________________ >>>>> Matterhorn mailing list >>>>> [email protected] >>>>> http://lists.opencastproject.**org/mailman/listinfo/**matterhorn<http://lists.opencastproject.org/mailman/listinfo/matterhorn> >>>>> >>>>> >>>>> To unsubscribe please email >>>>> matterhorn-unsubscribe@**opencastproject.org<[email protected]> >>>>> ______________________________**_________________ >>>>> >>>>> >>>> ----------------------- >>>> [email protected] >>>> 01/58801 DW 41523 >>>> mobil: 0664/60 588 4523 >>>> TU Wien >>>> DVR-Nummer: 0005886 >>>> ----------------------- >>>> ______________________________**_________________ >>>> Matterhorn mailing list >>>> [email protected] >>>> http://lists.opencastproject.**org/mailman/listinfo/**matterhorn<http://lists.opencastproject.org/mailman/listinfo/matterhorn> >>>> >>>> >>>> To unsubscribe please email >>>> matterhorn-unsubscribe@**opencastproject.org<[email protected]> >>>> ______________________________**_________________ >>>> >>>> >>> ______________________________**_________________ >>> Matterhorn mailing list >>> [email protected] >>> http://lists.opencastproject.**org/mailman/listinfo/**matterhorn<http://lists.opencastproject.org/mailman/listinfo/matterhorn> >>> >>> >>> To unsubscribe please email >>> matterhorn-unsubscribe@**opencastproject.org<[email protected]> >>> ______________________________**_________________ >>> >>> >> ______________________________**_________________ >> Matterhorn mailing list >> [email protected] >> http://lists.opencastproject.**org/mailman/listinfo/**matterhorn<http://lists.opencastproject.org/mailman/listinfo/matterhorn> >> >> >> To unsubscribe please email >> matterhorn-unsubscribe@**opencastproject.org<[email protected]> >> ______________________________**_________________ >> >> > ----------------------- > [email protected] > 01/58801 DW 41523 > mobil: 0664/60 588 4523 > TU Wien > DVR-Nummer: 0005886 > ----------------------- > ______________________________**_________________ > Matterhorn mailing list > [email protected] > http://lists.opencastproject.**org/mailman/listinfo/**matterhorn<http://lists.opencastproject.org/mailman/listinfo/matterhorn> > > > To unsubscribe please email > matterhorn-unsubscribe@**opencastproject.org<[email protected]> > ______________________________**_________________ >
_______________________________________________ Matterhorn mailing list [email protected] http://lists.opencastproject.org/mailman/listinfo/matterhorn To unsubscribe please email [email protected] _______________________________________________
