Adam, How do you manage your inventory of videos outside of Fedora? As others have recently pointed out, there are some duplication of effort costs associated with using external datastreams: you lose the checksumming, the audit paths and backups when a video changes, etc., all functions that Fedora provides for managed datastreams, but that need to be handled elsewhere when opting for external datastreams. Not that I'm questioning your approach; I'm just curious how you manage it. We're embarking on a path of ingesting thousands of RAW TIFFs now, and we want to make sure we don't make wrong decisions concerning storage and Fedora datastream management, decisions that will come back to haunt us later.
My thinking has evolved over the years concerning disseminators. When I first set up Fedora, back in the 2.2 days, I loved 'em, and went so far as to use disseminators as our sole API between Fedora and consuming applications. Now I avoid them like the plague. They're exceedingly difficult to configure, buggy and hard to debug, and prone to performance problems; and if you decide to go down the path of using XACML/FeSL to control access to your objects, the backdoor nature of disseminators introduces another level of complexity to your policies. I am now of the mind that Fedora should do what it does best: store, track, and serve up datastreams. We've started moving towards handling transformations of our datastreams with third party webservices that simply ask Fedora for a datastream then does the transformation itself, and it looks to make our lives much, much simpler. Paul: my inclination would be to test out configuring Wowza to consume the video datastream using the Fedora RESTful API. As others with more experience have discovered, Fedora can run into problems serving up very large objects; others may have some experience tuning Fedora and the OS to deal with those problems. As an aside, you should be aware of one nasty bug I ran into a few years ago concerning response headers, which prevented some players from streaming some media: https://jira.duraspace.org/browse/FCREPO-182 Concerning authentication and authorization, you could use Fedora's AuthNZ machinery to control access to the videos, but that may introduce more hassle than it's worth, particularly if you need to inject attributes and roles into the Fedora policy contexts. Depending on how Wowza is set up, and whether or not you have Apache/Tomcat as a frontend, you may be better off simply allowing access to the videos with a basic Fedora default policy that allows connections from localhost, perhaps some staff workstation IPs, and whatever host your Wowza server sits on, then use Apache/Tomcat/Wowza to do more fine-grained access control for users. In this respect, it's useful to think of Fedora as occupying the same niche a RDBMS does in an application stack: only consuming applications have direct access to the data store. -- Scott On 03/07/2013 09:39 AM, Adam Wead wrote: > Paul, > > On Thu, Mar 7, 2013 at 9:44 AM, Grotevant, Paul F <p...@utexas.edu > <mailto:p...@utexas.edu>> wrote: > > Good morning, > > We are currently exploring architecture options for storing and > serving up > streaming video from our Fedora Commons repository. We run our own Wowza > streaming server, and the option to require authentication for > viewing the > streamed video is a requirement. > > - How to avoid duplication of storage? Ideally, we would like to > store the > video files as managed data streams in Fedora and feed those to the > Wowza > server, rather than storing separate copies for Fedora and Wowza. One > option for this would be to use Wowza's API to consume the video > directly > from a Fedora URL. Has anyone ever experimented with this? Thoughts on > co-locating Fedora and Wowza in the same Tomcat container or at least on > the same hardware, to reduce the network latency issue? > > > I can speak to this above question because we're using Fedora with > Wowza. I use external datastreams and have all our video files > available via http with Apache as well as NFS. > > Video files reside on server, attached to a SAN, that's running our HSM > software. Large preservation video files are moved off to tape and can > be recalled manually, and access-level video files (H264) stay on disk. > This server then shares all these files via a NFS share. > > The NFS share can then be mounted by a Wowza server to stream the videos > directly, and can also be mounted by another server which serves out the > files via Apache to Fedora. When ingesting new video, the files are > written to the NFS share and then add to the Fedora by means of an > external datastream that points to the files via HTTP. What makes it > all tick is a logical construction of file names and folders. This is > accomplished in part by using the BagIt strategy where digital content > has a prescribed structure on your filesystem. The rest is building > your NFS shares and Apache directories coherently and configuring your > applications to use the correct paths and follow your naming conventions. > > The advantage to this is I don't have to monkey with Wowza at all, or > write any special disseminators in Fedora... neither of which I really > know how to do anyway, so this all plays to my particular technical > strengths. If you want to use managed datastreams, you certainly should > be able to get something working that enables Wowza to grab the video > from fedora and stream it out. I can't say how exactly how you would do > that, but I'm sure it's possible. However, the one technical limitation > you may run up against is storing large video files as managed > datastreams. There as been a thread going on about this recently and > the general consensus it seems is that you should be okay with files in > the 1-2 GB range, but 10 or more and you run into problems. I have > video files in the 100's of GB, so I had to use external datastreams and > built the technical structure I have now based on that requirement. > > I would note that if you can use managed datastreams, I would, but you'd > need to investigate the Wowza integration issue and datastream size > issue first. I'm sure there are other folks here that could speak more > directly to that. > > ...adam > ____________________________________________ > Adam Wead > Systems and Digital Collections Librarian > Rock and Roll Hall of Fame and Museum > 216.515.1960 (t) > 215.515.1964 (f) > > > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > > > > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > -- Scott Prater Shared Development Group General Library System University of Wisconsin - Madison pra...@wisc.edu 5-5415 ------------------------------------------------------------------------------ Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
