Re: [fcrepo-dev] Fedora with a lot of data

2011-09-08 Thread Chris Wilper
Hi Jiri, Fedora can certainly handle tens of millions of objects. But I don't know about deployed instances that do that. Part of the main push behind 3.6 is to get our testing house in order so we can say more than just it can handle lots of objects, but with this many objects in this type of

Re: [fcrepo-dev] Fedora with a lot of data

2011-09-08 Thread ajs6f
If you partition across full repository instances as described, you have set yourself one set of scaling problems. If you instead partition your storage layers (Low Level Store, RI, SQL DB, etc.) and provide one Fedora web application (one suite of web services) over them, you can set yourself

Re: [fcrepo-dev] Uploading large media files

2011-09-08 Thread Chris Wilper
The error is coming from Fedora's API-A SOAP implementation when it catches an OutOfMemoryError. The datastream is too large to be delivered via SOAP[1], but if it's retrieved via REST it should be fine. - Chris [1] The GSoC project that Jiri Kremser worked on should help with this in the future

Re: [fcrepo-dev] Fedora with a lot of data

2011-09-08 Thread Aaron Birkland
If you partition across full repository instances as described, you have set yourself one set of scaling problems. If you instead partition your storage layers (Low Level Store, RI, SQL DB, etc.) and provide one Fedora web application (one suite of web services) over them, you can set