On Wed, 2008-08-20 at 16:16 -0400, Daniel Davis wrote: > I am concerned about using NFS for Fedora's mount point. I need to > check with Chris Wilper because I seem to recall some problems when > the network goes flaky (not to mention that NFS is a security mess > unless it is behind a well controlled firewall). Plus NFS is not > really fast. NFS is slowing dying off in most data centers except for > backup or very fault tolerant cheap NAS storage. My tests were with > high performance connections such as dedicated direct connection > storage networks, iSCSI, and Fibrechannel. Unless you use a > connection that runs close to direct connection speeds you will really > slow down disseminations (and ingest).
I was thinking more along the lines of iSCSI or fiber, I just said NFS for dev work, but those are good points, NFS is pretty much unneeded esp with storage sizes today. > The new Akubra project will likely take up how to store over less > reliable networks. > > ALL the bitstreams and all the FOXML are stored as files. This lets > you rebuild if the DBMS or Triplestore fails. The DBMS (and the > Triplestore if you use one) is there to speed up operations or enable > queries. Only a very limited amount of data is stored in the DBMS > (metadata kind of stuff extracted from the FOXML). This is what I thought, but didn't completely understand, thanks for the clarification. > It is perfectly OK to have the DBMS and FC run on the same server in > all but the very largest installations. Most installations run in > your proposed configuration. In a very large repository you may > consider running the DBMS on its own server which is tuned just for > that purpose. > > While you can store just the bitstreams on your SAN/Fabric, I would > keep the objects there too. If you have a well performing SAN/Fabric > connection you can store everything on it except I would keep the > operating system on a locally attached disk for bootstrap convenience > and the swap partition. This is the arch I'm going to setup then, this is great, I appreciate it! This setup won't be huge (70k digital objects) and will focus on the back end for archival and a base for harvesters to hit. P > > -- Dan > > Phil Cryer wrote: > > On Wed, 2008-08-20 at 14:57 -0400, Daniel Davis wrote: > > > > > It should work fine. I have set the mount point within a Brocade Fabric > > > partition and it worked perfectly. The one thing to remember is that > > > only one Fedora instance should control the files. It is OK if the > > > mount point is shared (though you may lose performance) but the specific > > > storage directories must not be shared between two Fedora instances. I > > > have had no problems using a well configured SAN/Fabric for DBMS or > > > Triplestore persistence. It is generally better to dedicate the > > > partition to a specific server and not share it when using it for > > > Fedora, DBMS or Triplestore persistence. > > > > > > > Thanks for the reply - so would I be able to run the mount point over a > > NFS connection to the SAN, or would it have to be hard connected? I was > > thinking of having the DBMS on the same server as FC, and then utilize > > the NFS/SAN mount as the storage for the digital objects in place of the > > way I currently do in FEDORA_HOME/data - is this right? Are the objects > > (jpgs, sid images in this case) stored within the RDMS or are they just > > on the filesystem in the data/objects directory, or am I missing the > > basics on how this functions? > > > > Thanks > > > > P > > > > > > > > > I you dedicate the partition you still have a great failover mechanism. > > > Should the server die you can fail over to a second server and let that > > > server grab the SAN/Fabric partition. You can run Fedora without a > > > rebuild if the failure of the first server was clean with respect to the > > > file store, database, and triplestore. Otherwise you can run the > > > rebuilder but still gain an advantage in that the content bitstreams and > > > foxml files need not be moved. > > > > > > Finally, you can cluster the DBMS if you are using one with that > > > capability and/or use Fedora Journaling if you want a more loosely > > > coupled installation. > > > > > > -- Dan Davis > > > > > > Phil Cryer wrote: > > > > > > > Is there any documentation about having Fedora-commons store objects to > > > > a distributed filesystem - HDFS or the like, or just a local SAN? I > > > > spoke with Sandy last week at RIRI about an S3 module, but I'm wondering > > > > if there's something to map to a local DFS/SAN for now. Would it be a > > > > case of just remapping FEDORA_HOME/data to mount the remote storage? > > > > Does FC even have to know about it, or is it system level at that point? > > > > Does it have something to do with: > > > > <param name="file_system" > > > > value="fedora.server.storage.lowlevel.GenericFileSystem"> > > > > > > > > Thanks > > > > > > > > P > > > > > > > > > > -- > Daniel W. Davis > Chief Software Architect, Fedora Commons > Researcher, Cornell Information Science > http://www.fedora-commons.org > [EMAIL PROTECTED] > [EMAIL PROTECTED] > (607) 255-6090 (Office) > -- Phil Cryer | Open Source Development | Missouri Botanical Garden www: http://mobot.org | latitude, longitude: 38.613877, -90.257943 email: phildotcryeratmobotdotorg | im googletalk/skype: phil.cryer ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Fedora-commons-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
