I don't know of anyone who's tried and failed to combine transient cores and SolrCloud. I also don't know of anyone who's tried and succeeded.
I'm saying that the transient core stuff has been thoroughly tested in non-cloud mode. And people have been working with it for a couple of releases now. I know of no a-priori reason it wouldn't work in SolrCloud. But I haven't personally done it, nor do I know of anyone who has. It might "just work", but the proof is in the pudding. I've heard some scuttlebutt that the combination of SolrCloud and transient cores is being, or will be soon, investigated. As in testing and writing test cases. Being a pessimist by nature on these things, I suspect (but don't know) that something will come up. For instance, SolrCloud tries to keep track of all the states of all the nodes. I _think_ (but don't know for sure) that this is just keeping contact with the JVM, not particular cores. But what if there's something I don't know about that pings the individual cores? That would keep them constantly loading/unloading, which might crop up in unexpected ways. I've got to emphasize that this is an unknown (at least to me), but an example of something that could crop up. I'm sure there are other possibilities. Or distributed updates. For that, every core on every node for a shard in collectionX must process the update. So for updates, each and every core in each and every shard might have to be loaded for the update to succeed if the core is transient. Does this happen fast enough in all cases so a timeout doesn't cause the update to fail? Or the node to be marked as down? What about combining that with a heavy query load? I just don't know. It's uncharted territory is all. I'd love it for you to volunteer to be the first :). There's certainly committer interest in making this case work so you wouldn't be left hanging all alone. If I were planning a product though, I'd either treat the combination of transient cores and SolrCloud as a R&D project or go with non-cloud mode until I had some reassurance that transient cores and SolrCloud played nicely together. All that said, I don't want to paint too bleak a picture. All the transient core stuff is local to a particular node. SolrCloud and ZooKeeper shouldn't be interested in the details. It _should_ "just work". It's just that I can't point to any examples where that's been tried.... Best, Erick On Wed, Dec 4, 2013 at 5:08 PM, hank williams <hank...@gmail.com> wrote: > Oh my... when you say "I don't know anyone who's combined the two." do you > mean that those that have tried have failed or that no one has gotten > around to trying? It sounds like you are saying you have some specific > knowledge that right now these wont work, otherwise you wouldnt say > "committers > will be addressing this sometime soon", right? > > I'm worried as we need to make a practical decision here and it sounds like > maybe we should stick with solr for now... is that what you are saying? > > > On Wed, Dec 4, 2013 at 5:01 PM, Erick Erickson <erickerick...@gmail.com > >wrote: > > > Hank: > > > > I should add that lots of cores and SolrCloud aren't guaranteed to play > > nice together. I think some of the committers will be addressing this > > sometime soon. > > > > I'm not saying that this will certainly fail, OTOH I don't know anyone > > who's combined the two. > > > > Erick > > > > > > On Wed, Dec 4, 2013 at 3:18 PM, hank williams <hank...@gmail.com> wrote: > > > > > Super helpful. Thanks. > > > > > > > > > On Wed, Dec 4, 2013 at 2:53 PM, Shawn Heisey <s...@elyograg.org> > wrote: > > > > > > > On 12/4/2013 12:34 PM, hank williams wrote: > > > > > > > >> Ok one more simple question. We just upgraded to 4.6 from 4.2. In > 4.2 > > we > > > >> were *trying* to use the rest API function "create" to create cores > > > >> without > > > >> having to manually mess with files on the server. Is this what > > "create" > > > >> was > > > >> supposed to do? If so it was borken or we werent using it right. In > > any > > > >> case in 4.6 is that the right way to programmatically add cores in > > > >> discovery mode? > > > >> > > > > > > > > If you are NOT in SolrCloud mode, in order to create new cores, the > > > config > > > > files need to already exist on the disk. This is the case with all > > > > versions of Solr. > > > > > > > > If you're running in SolrCloud mode, the core is associated with a > > > > collection. Collections have a link to aconfig in zookeeper. The > > config > > > > is not stored with the core on the disk. > > > > > > > > Thanks, > > > > Shawn > > > > > > > > > > > > > > > > > -- > > > blog: whydoeseverythingsuck.com > > > > > > > > > -- > blog: whydoeseverythingsuck.com >
