I'm trying to stay within the realm of solr's resource loader, so that I
don't need to tweak startup parameters (e.g. classpath, sysprops) or rely
on hardcoded stuff. The data must be usable by a query SearchComponent, and
those are loaded by the core using the resource loader...
One question I have about Shawn's solution, after a little thought is: If I
do move the jar, how do I get the class to load before I get to my
SeachComponent... if it gets loaded by reference there it will still be
under the Core's classloader I think...
On Wed, Nov 11, 2015 at 7:08 PM, Benson Margulies <bimargul...@gmail.com>
wrote:
> What is the connection of a blob of data and a class in a class
> loader? Is it a class of your own that you're using to store the data?
>
> Solr can't change fundamental facts about class loader; if an object
> of a class needs to be shared across class loaders, it has to be
> loaded into a common parent. If you don't want to do that broadly,
> you'll need indeed to factor out a jar for the job.
>
> If it isn't a special class, but rather just an instance of some
> boring ordinary class and your problem is sharing the _reference_,
> consider JNDI.
>
>
>
> On Wed, Nov 11, 2015 at 7:02 PM, Gus Heck <gus.h...@gmail.com> wrote:
> > Yes asked by a colleague :). The chat session is now in our jira ticket
> :).
> >
> > However, my take on it is that this seems like a pretty broad brush to
> paint
> > with to move *all* our classes up and out of the normal core loading
> > process. I assume there are good reasons for segregating this stuff into
> > separate class loaders to begin with. It would also be fairly burdensom
> to
> > make a separate jar file to break out this one component...
> >
> > I really just want a way to stash the map in a place where other cores
> can
> > see it (and thus I can appropriately synchronize things so that the
> loading
> > only happens once). I'm asking because it seems like surely this must be
> a
> > solved problem... if not, it might be easiest to just solve it by adding
> > some sort of shared resources facility to CoreContainer?
> >
> > -Gus
> >
> > On Wed, Nov 11, 2015 at 6:54 PM, Shawn Heisey <apa...@elyograg.org>
> wrote:
> >>
> >> On 11/11/2015 4:11 PM, Gus Heck wrote:
> >> > I have a case where a component loads up a large CSV file (2.5 million
> >> > lines) to build a map. This worked ok in a case where we had a single
> >> > core, but it isn't working so well with 40 cores because each core
> loads
> >> > a new copy of the component in a new classloader and I get 40 new
> >> > versions of the same class each holding it's own private static final
> >> > map (one for each core). Each line is small, but a billion of anything
> >> > gets kinda heavy. Is this the intended class loading behavior?
> >> >
> >> > Is there some where that one can cause a class to be loaded in a
> parent
> >> > classloader above the core so that it's loaded just once? I want to
> load
> >> > it in some way that leverages standard solr resource loading, so that
> >> > I'm not hard coding or setting sysprops just to be able to find it.
> >> >
> >> > This is in a copy of trunk from about a month ago... so 6.x stuff is
> >> > mostly available.
> >>
> >> This sounds like a question that I just recently answered on IRC.
> >>
> >> If you remove all <lib> elements from your solrconfig.xml files and
> >> place all extra jars for Solr into ${solr.solr.home}/lib ... Solr will
> >> load those jars before any cores are created and they will be available
> >> to all cores.
> >>
> >> There is a minor bug with this that will be fixed in Solr 5.4.0. It is
> >> unlikely that this will affect third-party components, but be aware that
> >> until 5.4, jars in that lib directory will be loaded twice by older 5.x
> >> versions.
> >>
> >> https://issues.apache.org/jira/browse/SOLR-6188
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
> >
> >
> > --
> > http://www.the111shift.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
--
http://www.the111shift.com
