Dear Erick, I remembered some times ago, somebody asked about what is the point of modify Solr to use HDFS for storing indexes. As far as I remember somebody told him integrating Solr with HDFS has two advantages. 1) having hadoop replication and HA. 2) using indexes and Solr documents for other purposes such as Analysis. So why we go for HDFS in the case of analysis if we want to use SolrJ for this purpose? What is the point? Regards.
On Wed, Aug 6, 2014 at 8:59 AM, Ali Nazemian <alinazem...@gmail.com> wrote: > Dear Erick, > Hi, > Thank you for you reply. Yeah I am aware that SolrJ is my last option. I > was thinking about raw I/O operation. So according to your reply probably > it is not applicable somehow. What about the Lily project that Michael > mentioned? Is that consider SolrJ too? Are you aware of Cloudera search? I > know they provide an integrated Hadoop ecosystem. Do you know what is their > suggestion? > Best regards. > > > > On Wed, Aug 6, 2014 at 12:28 AM, Erick Erickson <erickerick...@gmail.com> > wrote: > >> What you haven't told us is what you mean by "modify the >> index outside Solr". SolrJ? Using raw Lucene? Trying to modify >> things by writing your own codec? Standard Java I/O operations? >> Other? >> >> You could use SolrJ to connect to an existing Solr server and >> both read and modify at will form your M/R jobs. But if you're >> thinking of trying to write/modify the segment files by raw I/O >> operations, good luck! I'm 99.99% certain that's going to cause >> you endless grief. >> >> Best, >> Erick >> >> >> On Tue, Aug 5, 2014 at 9:55 AM, Ali Nazemian <alinazem...@gmail.com> >> wrote: >> >> > Actually I am going to do some analysis on the solr data using map >> reduce. >> > For this purpose it might be needed to change some part of data or add >> new >> > fields from outside solr. >> > >> > >> > On Tue, Aug 5, 2014 at 5:51 PM, Shawn Heisey <s...@elyograg.org> wrote: >> > >> > > On 8/5/2014 7:04 AM, Ali Nazemian wrote: >> > > > I changed solr 4.9 to write index and data on hdfs. Now I am going >> to >> > > > connect to those data from the outside of solr for changing some of >> the >> > > > values. Could somebody please tell me how that is possible? Suppose >> I >> > am >> > > > using Hbase over hdfs for do these changes. >> > > >> > > I don't know how you could safely modify the index without a Lucene >> > > application or another instance of Solr, but if you do manage to >> modify >> > > the index, simply reloading the core or restarting Solr should cause >> it >> > > to pick up the changes. Either you would need to make sure that Solr >> > > never modifies the index, or you would need some way of coordinating >> > > updates so that Solr and the other application would never try to >> modify >> > > the index at the same time. >> > > >> > > Thanks, >> > > Shawn >> > > >> > > >> > >> > >> > -- >> > A.Nazemian >> > >> > > > > -- > A.Nazemian > -- A.Nazemian
