Hi
I just thought of fault tolerance in this system. The WritableStore
interface should probably have a rollback method.
interface WritableStore {
void add(DigitalObject o);
void update(DigitalObject oldVers,
DigitalObject newVers);
void remove(DigitalObject o);
void rollback(DigitalObject oldVers,
DigitalObject newVers);
}
This way, if one of the synchronous stores fail an update, the rest of
them can roll back to before the update.
Regards
Asger
On Wed, 2010-01-27 at 15:05 +0100, Asger Askov Blekinge wrote:
> Hi Aron and all
>
> I have read your proposal with great interest. As we could establish on
> the committers meetings, I have been thinking along the same lines. You
> have been doing most of the work for me, which makes me glad. I hope you
> too will be glad, when I piggyback what you started.
>
> You correctly highlight one shortcoming of Fedora, but fail to highlight
> another. Your excellent paper only concerned itself with storage, but
> caching is equally important.
>
>
> Here is my counter proposal:
>
> Fedora has a number of storage systems. Each of these are represented by
> a class implementing an interface like this
> public interface WritableStore {
>
> void add(DigitalObject object);
>
> void update(DigitalObject oldVers,
> DigitalObject newVers);
>
> void remove(DigitalObject object);
> }
> I am sure you can see where I got my inspiration ;) I used your version
> (delete takes an object, rather than a pid for no particular reason).
> But I removed the read() method. All fedora stores should be able to
> digest fedora objects. But not all of them should be able to put an
> object back together again.
>
> Fedora maintains two lists of such stores, one for synchronous updates,
> and one for asynch updates.
>
> Whenever fedora receives a request to change something, it sends the
> request to each of the synchronous stores, queue it for the asynchronous
> stores and return.
>
> It will have one ReadableStore with a interface
>
> interface ReadableStore {
>
> DigitalObject read(String pid);
> }
>
> Whenever objects need to be read, this is the store that is read from.
> Interestingly, this does not need to be any one of the WritableStores.
> You could have a number of Writable stores doing journaling of each
> other, and having the WritableStore implementation alternating between
> them when objects should be read.
>
>
> Fedora, in the current incarnation, uses these stores
>
> Synchronous
> * Lowlevelstore
> * Database
> * Messaging
> * Gsearch (by DOManager)
> * Lodewijks custom database integration
>
> Asynchronous
> * Triple store
> * Search engines (by messaging)
>
> By making all these stores adhere to at standard interface, we could
> easily change which are synchronous and which are not from the config
> file. We could easily add new stores and caches and indexes. Store
> integration could be shared as binary files, with should just be
> installed in fedora, and declared in fedora.fcfg.
>
> We could have multiple redundant stores, and having the ReadableStore
> implementation alternate between them.
>
> With this approach, Fedora becomes
>
> * A webservice interface
> * An object model
> * A system for object access policies
> * Glue for binding stores together
>
> I think I imagine these stores to live alongside the HighlevelStorage
> system, in that the HighlevelStorage is just one implementation. But
> they might also live below the HighlevelStorage.
>
>
> So, what do you think?
>
> Regards
> Asgger
>
>
> On Wed, 2010-01-27 at 07:12 +0100, Aaron Birkland wrote:
> > Hi all,
> >
> > As you may be aware, there is a face-to-face meeting in London where
> > various committers and (other interested parties) will be discussing
> > future directions for Fedora. One topic slated for discussion is that
> > of Fedora's storage architecture. The ideas of multiplexing storage (so
> > that managed content can be sent to different storage devices based on
> > some criteria), hierarchical storage, alternate object representations,
> > etc have been discussed in the past. I would like to propose a few
> > architectural changes to the Fedora core that could make these tasks
> > easier to implement.
> >
> > I made a skeleton of a page on the wiki:
> > http://fedora-commons.org/confluence/x/syTS
> >
> > A draft is currently written up in a pdf doc (which is linked on the
> > wiki page):
> > http://www.fedora-commons.org/confluence/download/attachments/13771955/fedora-highlevel-storage_draft-proposal.pdf
> >
> > The main purpose of the proposal is to spur conversation on the topic..
> > the proposal may or may not represent an ideal approach, but it seems to
> > represent a logical next step that would remove some of the existing
> > architectural barriers to implementing more sophisticated storage
> > strategies.
> >
> > I decided to put this up quickly so that people can have some time to
> > read it over, ask questions, discuss, think about how to improve it, or
> > propose alternatives. Hopefully, this is a topic that people would like
> > to discuss.
> >
> > -Aaron
> >
> >
> > ------------------------------------------------------------------------------
> > The Planet: dedicated and managed hosting, cloud storage, colocation
> > Stay online with enterprise data centers and the best network in the
> > business
> > Choose flexible plans and management services without long-term contracts
> > Personal 24x7 support from experience hosting pros just a phone call away.
> > http://p.sf.net/sfu/theplanet-com
> > _______________________________________________
> > Fedora-commons-developers mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> Fedora-commons-developers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Fedora-commons-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
