SolrJ does not yet have partitioning aware document routing but will likely
support it in the next few months. Since 4.3 SolrJ does send to master nodes
only but that's all. The only cloud aware part indexing currently has is that
if the node your sending documents to is going down, it will fail-over to the
next as long as there are live nodes for a shard.
-----Original message-----
> From:[email protected] <[email protected]>
> Sent: Monday 8th July 2013 19:38
> To: [email protected]
> Subject: Re: Indexing from nutch 1.6 to solr 4.3.1 cloud
>
> Hi,
>
> Currently, I index to solrCloud using nutch-1.6 and nutch-2.1, without any
> additional patches. I run solrindex and point it to one of the solr instances
> in the cloud.
>
> In doing this it seems to me that the solr instance that the documents are
> sent gets overloaded. Let say solrindex command sends 1000 documents per sec
> to two shards. In case without partitoning all 1000 docs is sent to one
> shard, then partitoned by solr and send to other shards. However, in case if
> partitioning is used before sending to solr then each shard gets only 500
> docs, which avoids overloading of solr servers.
>
> Any thoughts about this are welcome.
>
> Thanks.
> Alex.
>
>
>
>
>
>
>
> -----Original Message-----
> From: Tuğcem Oral <[email protected]>
> To: user <[email protected]>
> Sent: Mon, Jul 8, 2013 3:16 am
> Subject: Re: Indexing from nutch 1.6 to solr 4.3.1 cloud
>
>
> OK then. I generated the corresponding patch. If someone also needs it till
> nutch 1.8 is released, I'd be happy to share.
>
> Best,
>
> Tugcem.
>
>
> On Mon, Jul 8, 2013 at 12:10 PM, Markus Jelsma
> <[email protected]>wrote:
>
> > First we need to upgrade to Solr >= 4.3 (NUTCH-1486). Then we'll have to
> > add an option to index via CloudSolrServer (NUTCH-1377) where you input
> > your Zookeeper ensemble vs. a target host. Then we can do NUTCH-1480 and
> > write to multiple individual servers and/or multiple cloud clusters.
> >
> > The upgrade to 4.3 is almost finished. The rest isn't very hard and will
> > be available in Nutch 1.8 if i can help it :)
> >
> > -----Original message-----
> > > From:Tuğcem Oral <[email protected]>
> > > Sent: Monday 8th July 2013 11:06
> > > To: [email protected]
> > > Subject: Re: Indexing from nutch 1.6 to solr 4.3.1 cloud
> > >
> > > You're right, but nutch 1.6 comes along with solrj v3.4 which doesn't
> > > include CloudSolrServer. That's why we wrote such a patch. We already use
> > > CloudSolrServer for querying solr shards.
> > >
> > > BTW: I changed the solrj version of nutch 1.6 to 4.3.1 but they're not
> > > working well together while indexing
> > >
> > > So do you prefer any other solution for partitioning and indexing from
> > > nutch 1.6 to solr cloud?
> > >
> > > Best.
> > >
> >
>
>
>
> --
> TO
>
>
>
