Re: lucene and mongodb
Thanks you Hendrik, I'm new with Apache Lucene, the problem that arises is like starting with lucene-solr ASAP, in order to move quickly in my project. Do you know a guide or ejemple code about how does lucene-solr works ( storing index in solr as you say) ? Kind regards, Rider On Wed, May 15, 2013 at 6:46 AM, Hendrik Lücke-Tieke h...@outlook.comwrote: Hi mate, we did that (w/ lucene 3.6) and reconsidered it as very bad idea afterwards. Why? (a) out of the box, mongodb does only 16-mb files. Lucene files grow (much) larger than that. (b) lucene indices seem highly optimized to create good performance when reading them from disk. A layer like gridfs is likely to destroy all that handcrafted performance benefits. So you can strore it in mongodb, but there are better ways (e.g. using solr as store instead of mongo) Kind regards, Hendrik -Ursprüngliche Nachricht- Von: Rider Carrion Cleger [mailto:rider.carr...@gmail.com] Gesendet: Dienstag, 14. Mai 2013 10:36 An: java-user-i...@lucene.apache.org; java-user-...@lucene.apache.org; java-user@lucene.apache.org Betreff: lucene and mongodb Hi team, I'm working with apache lucene 4.2.1 and I would like to store lucene index in a NoSql database. So my questions are, - Can I store the lucene index in a mongodb database ? thanks you team! - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lucene and mongodb
there is also elasticsearch (elasticsearch.org) build on top of lucene that might feel more natural if you come from mongo simon On Wed, May 15, 2013 at 11:38 AM, Rider Carrion Cleger rider.carr...@gmail.com wrote: Thanks you Hendrik, I'm new with Apache Lucene, the problem that arises is like starting with lucene-solr ASAP, in order to move quickly in my project. Do you know a guide or ejemple code about how does lucene-solr works ( storing index in solr as you say) ? Kind regards, Rider On Wed, May 15, 2013 at 6:46 AM, Hendrik Lücke-Tieke h...@outlook.comwrote: Hi mate, we did that (w/ lucene 3.6) and reconsidered it as very bad idea afterwards. Why? (a) out of the box, mongodb does only 16-mb files. Lucene files grow (much) larger than that. (b) lucene indices seem highly optimized to create good performance when reading them from disk. A layer like gridfs is likely to destroy all that handcrafted performance benefits. So you can strore it in mongodb, but there are better ways (e.g. using solr as store instead of mongo) Kind regards, Hendrik -Ursprüngliche Nachricht- Von: Rider Carrion Cleger [mailto:rider.carr...@gmail.com] Gesendet: Dienstag, 14. Mai 2013 10:36 An: java-user-i...@lucene.apache.org; java-user-...@lucene.apache.org; java-user@lucene.apache.org Betreff: lucene and mongodb Hi team, I'm working with apache lucene 4.2.1 and I would like to store lucene index in a NoSql database. So my questions are, - Can I store the lucene index in a mongodb database ? thanks you team! - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lucene and mongodb
Hi, On Tue, May 14, 2013 at 10:35 AM, Rider Carrion Cleger rider.carr...@gmail.com wrote: - Can I store the lucene index in a mongodb database ? I don't know whether it's possible, but even if it was, I would not recommend it. Lucene works best on local filesystems, and even better if the disk is an SSD. If your intention was to rely on MongoDB to scale the index, it is better to handle distribution on top of Lucene, similarly to Solr and Elasticsearch, rather than writing distributed implementations of org.apache.lucene.store.Directory. -- Adrien - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lucene and mongodb
Thank you Adrien. I did not think about index's scalability but a safe place to store them (like a SGBD relation... or NoSql). So, can I have for sure scalability and safety with a distribution on top of Lucene like Solr ? On Tue, May 14, 2013 at 11:08 AM, Adrien Grand jpou...@gmail.com wrote: Hi, On Tue, May 14, 2013 at 10:35 AM, Rider Carrion Cleger rider.carr...@gmail.com wrote: - Can I store the lucene index in a mongodb database ? I don't know whether it's possible, but even if it was, I would not recommend it. Lucene works best on local filesystems, and even better if the disk is an SSD. If your intention was to rely on MongoDB to scale the index, it is better to handle distribution on top of Lucene, similarly to Solr and Elasticsearch, rather than writing distributed implementations of org.apache.lucene.store.Directory. -- Adrien - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lucene and mongodb
Hi, On Tue, May 14, 2013 at 1:34 PM, Rider Carrion Cleger rider.carr...@gmail.com wrote: So, can I have for sure scalability and safety with a distribution on top of Lucene like Solr ? Yes, Solr can help you shard your index and add replicas, see http://wiki.apache.org/solr/SolrCloud. -- Adrien - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: lucene and mongodb
That was tried with Lucandra/Solandra, which stored the Lucene index in Cassandra, but was less than optimal, so that model was discarded in favor of indexing Cassandra data directly into Solr/Lucene, side-by-side in each Cassandra node, but in native Lucene. The latter approach is now available from DataStax as DataStax Enterprise (DSE), which also integrates Hadoop. DSE provides the best of Cassandra integrated tightly with the best of Solr. In DSE, Cassandra takes care of all the cluster management, with Solr indexing the local Cassandra data, handing off incoming updates to Cassandra for normal Cassandra storage, and using a variation of the normal Solr code for distributing queries and merging results from other nodes, but depending on Cassandra for information about cluster configuration. (Note: I had proposed to talk in more detail about the above at Lucene Revolution, but my proposal was not accepted.) See: http://www.datastax.com/what-we-offer/products-services/datastax-enterprise As it says, DataStax Enterprise is completely free for development work. -- Jack Krupansky -Original Message- From: Rider Carrion Cleger Sent: Tuesday, May 14, 2013 4:35 AM To: java-user-i...@lucene.apache.org ; java-user-...@lucene.apache.org ; java-user@lucene.apache.org Subject: lucene and mongodb Hi team, I'm working with apache lucene 4.2.1 and I would like to store lucene index in a NoSql database. So my questions are, - Can I store the lucene index in a mongodb database ? thanks you team! - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
