Once I've addressed the non trivial lucene update ill move in to this as I am interested in it. Thanks for reply.
On Sunday, March 24, 2013, Mattmann, Chris A (388J) < chris.a.mattm...@jpl.nasa.gov> wrote: > Lewis, > > Since I still have not officially received this email from dev@oodt, > (grr) and since I saw it on the mail archives, I'm going to copy your > email below, with the same subject, and hope it gets threaded right :) > > Comments below: > > > On 3/20/13 5:55 PM, Lewis John Mcgibbney (lewi...@gmail.com) wrote: > >> >>All, >> >>I picked up OODT today and immediately thought about an implementation of >>Apache Gora [0] for abstracting persistence within the CAS metadata >>catalogue. > > +1, I've wanted this for a while. A GoraCatalog implementation of the FM > Catalog > Interface. > >>Right now, for me, the persistence of my metadata catalogue to Lucene or >>MySQL is sufficient and I have no immediate justification for using some >>alternative storage mechanism however I noticed that there are a few areas >>where OODT could generally benefit from the Gora implementation. >>It is natural that product discovery via daemon driven CAS crawler (for >>example) will fire product streams of varying nature towards the catalogue >>storage mechanism. Lucene or MySQL my not be best best option to store >>such >>streams of data and/or the best way to later retrieve that data. Gora >>would >>enable a much more comprehensive variety of data stores to be available >>for >>persistence of catalogue metadata and would also provide a much more >>flexible model specifically geared towards better solutions for metadata >>cataloguing. Currently we support Amazon DynamoDB, Accumulo, Cassandra, >>HBase, HDFS, HSQLDB and MySQL. We have patches for Solr, MongoDB and >>various file based stores. There is also interest to implement an Oracle >>NoSQL DB.... don't ask. > > Haha! > >>I notice that the SolrIndexer tool implemented by Paul provides an >>expressive number of options for indexing to your Solr HTTP server. The >>gora-solr module would provide all these plus more. >>I suppose this entirely depends on the requirements for expanding metadata >>catalogues within the File Manager. >>Is it envisaged that such an implementation is required for some use cases >>or would be required? > > Yes, please, help! :) > >>As Gora builds on Hadoop principles, I suppose it would also enable folks >>use their metadata catalogues in different, possibly useful, use-case >>adaptable ways. >>Just an initial thought. > > A great one at that, I would be super +1 for a GoraCatalog to help in these > situations and would be keen to work on it with you. > > Cheers, > Chris > >>Thanks >>Lewis >> >> >>[0] http://gora.apache.org <http://gora.apache.org/> >>-- >>*Lewis* >> >> > > > > -- *Lewis*
