My point is this - *everyone *using shards needs to handle that uniqueness. Why not make it as a mechanism in Solr, with the ability to plugin our own implementation of the hashing method?
Or is that what Solr Cloud is all about? SJ On Wed, Jan 11, 2012 at 4:33 PM, Hector Castro <hectcas...@gmail.com> wrote: > In our setup, we handle the document distribution and uniqueness across > cores outside of Solr. > > -- > Hector > > On Jan 11, 2012, at 1:53 AM, shlomi java wrote: > > > Straying a bit from the subject, > > > > don't you think it will be useful to have the shards parameter used also > in > > the index, in order to maintain document uniqueness? > > I mean as an out of the box feature of Solr. > > > > Because the situation today is that a Solr's client working with a > sharded > > Solr is responsible for keeping a document uniqueness across all shards. > > > > *Solution *- let Solr decide in which shard to index a document, using a > > plugable hashing method. > > > > What do you think? > > > > ShlomiJ > > > > On Tue, Jan 10, 2012 at 6:15 PM, Shawn Heisey <s...@elyograg.org> wrote: > > > >> On 1/9/2012 5:15 PM, Hector Castro wrote: > >> > >>> Hi, > >>> > >>> Has anyone had success with multicore single node Solr configurations > >>> that have one core acting solely as a dispatcher for the other cores? > For > >>> example, say you had 4 populated Solr cores – configure a 5th to be the > >>> definitive endpoint with `shards` containing cores 1-4. > >>> > >>> Is there any advantage to this setup over simply having requests > >>> distributed randomly across the 4 populated cores (all with `shards` > equal > >>> to cores 1-4)? Is it even worth distributing requests across the cores > >>> over always hitting the same one? > >>> > >> > >> I've got a setup where a single index chain consists of seven cores > across > >> two servers. Those seven cores do not have the shards parameter in > them. > >> I have what you call a dispatcher core (I call it a broker core) that > >> contains the shards parameter, but has no index data. > >> > >> If you use a dispatcher core, your application does not need to be > >> concerned with the makeup of your index, so you don't need to include a > >> shards parameter with your request. You can change the index > distribution > >> and not have to worry about your application configuration. This is the > >> major advantage to doing it this way. Distributed search has some > overhead > >> and not all Solr features work with it, so if your application already > >> knows which core will contain the data it is trying to find, it is > better > >> to query the right core directly. > >> > >> Be careful that you aren't adding a shards parameter to a core > >> configuration that points at itself. Solr will, as of the last time I > >> checked, try to complete the recursion and will fail. > >> > >> Thanks, > >> Shawn > >> > >> > >
