> Or in a separate Solr core/collection. :) That support is available with this, I think: https://cwiki.apache.org/confluence/display/solr/Blob+Store+API
On Thu, Nov 12, 2015 at 2:31 PM, Erik Hatcher <erik.hatc...@gmail.com> wrote: > Or in a separate Solr core/collection. :) > > On Nov 11, 2015, at 19:05, Walter Underwood <wun...@wunderwood.org> wrote: > > Depending on how fast the access needs to be, you could put that big map > in memcache. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Nov 11, 2015, at 4:04 PM, Gus Heck <gus.h...@gmail.com> wrote: > > P.S. I posted the original message concurrently with the chat session's > occurance I beleive, certainly before I had read it, so no I haven't > actually tried what you suggest yet. > > On Wed, Nov 11, 2015 at 7:02 PM, Gus Heck <gus.h...@gmail.com> wrote: > >> Yes asked by a colleague :). The chat session is now in our jira ticket >> :). >> >> However, my take on it is that this seems like a pretty broad brush to >> paint with to move *all* our classes up and out of the normal core loading >> process. I assume there are good reasons for segregating this stuff into >> separate class loaders to begin with. It would also be fairly burdensom to >> make a separate jar file to break out this one component... >> >> I really just want a way to stash the map in a place where other cores >> can see it (and thus I can appropriately synchronize things so that the >> loading only happens once). I'm asking because it seems like surely this >> must be a solved problem... if not, it might be easiest to just solve it by >> adding some sort of shared resources facility to CoreContainer? >> >> -Gus >> >> On Wed, Nov 11, 2015 at 6:54 PM, Shawn Heisey <apa...@elyograg.org> >> wrote: >> >>> On 11/11/2015 4:11 PM, Gus Heck wrote: >>> > I have a case where a component loads up a large CSV file (2.5 million >>> > lines) to build a map. This worked ok in a case where we had a single >>> > core, but it isn't working so well with 40 cores because each core >>> loads >>> > a new copy of the component in a new classloader and I get 40 new >>> > versions of the same class each holding it's own private static final >>> > map (one for each core). Each line is small, but a billion of anything >>> > gets kinda heavy. Is this the intended class loading behavior? >>> > >>> > Is there some where that one can cause a class to be loaded in a parent >>> > classloader above the core so that it's loaded just once? I want to >>> load >>> > it in some way that leverages standard solr resource loading, so that >>> > I'm not hard coding or setting sysprops just to be able to find it. >>> > >>> > This is in a copy of trunk from about a month ago... so 6.x stuff is >>> > mostly available. >>> >>> This sounds like a question that I just recently answered on IRC. >>> >>> If you remove all <lib> elements from your solrconfig.xml files and >>> place all extra jars for Solr into ${solr.solr.home}/lib ... Solr will >>> load those jars before any cores are created and they will be available >>> to all cores. >>> >>> There is a minor bug with this that will be fixed in Solr 5.4.0. It is >>> unlikely that this will affect third-party components, but be aware that >>> until 5.4, jars in that lib directory will be loaded twice by older 5.x >>> versions. >>> >>> https://issues.apache.org/jira/browse/SOLR-6188 >>> >>> Thanks, >>> Shawn >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >> >> >> -- >> http://www.the111shift.com >> > > > > -- > http://www.the111shift.com > > >
