Some that come to mind, SOLR-5944 SOLR-13350 LUCENE-9302 / SOLR-14381
On Mon, May 3, 2021 at 11:31 PM Jan Høydahl <[email protected]> wrote: > Can you give an example of a recent solr feature that needed such a > workflow - I.e where it would be very annoying to have to do the Lucene > part first and then the solr part, using a local snapshot version? > > Jan Høydahl > > 3. mai 2021 kl. 17:32 skrev Ishan Chattopadhyaya < > [email protected]>: > > > One of the other problems I have with maven, besides where will the > snapshots go, is the local development workflow. If I were to work on a > feature that needs changes to both Lucene and Solr, I would need to make > Lucene changes in a separate git repo, push them to local Maven, have solr > build against that and so on. This is considerably more complicated, to my > mind, than working on a single repo (with the other repo being a submodule) > and single project inside the IDE, which git submodule can allow us to do. > > On Mon, 3 May, 2021, 8:27 pm Houston Putman, <[email protected]> > wrote: > >> I agree, Cassandra. The nightlies are a very good option that is very >> easy to get started with. >> >> On Mon, May 3, 2021 at 9:55 AM Cassandra Targett <[email protected]> >> wrote: >> >>> I’m sort of surprised no one has mentioned the >>> https://nightlies.apache.org/ server, which could be used for this >>> purpose. Jenkins can push to it, and nothing gets deleted until we decide >>> to delete it (or overwrite it). Solr Operator builds go there as do drafts >>> of the Ref Guide pre-publication (in different directories now, but we’ll >>> fix that eventually). >>> >>> Cassandra >>> On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, wrote: >>> >>> RE "an external system like Maven" -- we're merely talking about adding >>> another JAR repo to the list of repos we already have. Heck, for this >>> limited purpose, we could even use http://home.apache.org/~dsmiley/ >>> Note that the Lucene project already uses home dirs of some users for >>> benchmark data. If we were talking about adding a Maven *build* then I >>> would totally appreciate your concern. >>> >>> I volunteer to use my space for this purpose. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya < >>> [email protected]> wrote: >>> >>>> > Let’s not complicate things. >>>> By bringing in external systems like Maven, I think we're complicating >>>> things even though a straight forward way (git submodules) exists. >>>> >>>> On Sat, May 1, 2021 at 5:00 PM Jan Høydahl <[email protected]> >>>> wrote: >>>> >>>>> Sub modules is for organizing internal repos, not for pulling in >>>>> external deps. Let’s not complicate things. And once we switch main to >>>>> 10.x >>>>> we’d need to use pure jar dependencies in branch_9x to depend upon >>>>> actually >>>>> released and voted Lucene binaries. >>>>> >>>>> We need some tooling to smoothly work with bleeding edge Lucene >>>>> including local snapshot builds with not yet pushed changes to Lucene. I’d >>>>> rather put in some work to establish such tooling or procedures. >>>>> >>>>> Jan Høydahl >>>>> >>>>> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya < >>>>> [email protected]>: >>>>> >>>>> >>>>> > What submodules don't solve is releases - if you're >>>>> > on a >>>>> particular unreleased Lucene version then releasing >>>>> > Solr would still mean you need some kind of >>>>> > "public" pinned Lucene release for the >>>>> > Mavenworld. >>>>> >>>>> >>>>> Can we then update the submodule to point to the release tag or sha >>>>> that Lucene got released from? >>>>> >>>>> On Sat, 1 May, 2021, 11:45 am Dawid Weiss, <[email protected]> >>>>> wrote: >>>>> >>>>>> > Other than literally adding the git submodule, would we do anything >>>>>> else to modify the gradle build so that or do we (and Jenkinsfile) have >>>>>> to >>>>>> manually do a step to install Lucene before proceeding? >>>>>> >>>>>> Technically you add a submodule and then make a composite build from >>>>>> Solr side. I can provide a PR that does it if you wish... This is >>>>>> simple. What submodules don't solve is releases - if you're on a >>>>>> particular unreleased Lucene version then releasing Solr would still >>>>>> mean you need some kind of "public" pinned Lucene release for the >>>>>> Maven world. If you distribute the entire thing as binaries you can >>>>>> just publish a snapshot of Lucene code, of course. >>>>>> >>>>>> > I think adding git submodule means that we have to add back in all >>>>>> the build code for it. >>>>>> >>>>>> No. The submodule is just a reference to a particular repository (and >>>>>> commit) - the build code and remains nearly identical as it was with >>>>>> Lucene. Composite builds in gradle are quite nice in that they're >>>>>> (almost) transparent compared to dependency references. >>>>>> >>>>>> > At which point, I'd rather just copy and commit the >>>>>> > code they have so we don't have to learn another git system. >>>>>> >>>>>> It's not really a different git system - it's a mechanism of >>>>>> interacting with multiple repositories at once. Yes, it does add some >>>>>> additional complexity but I think sooner or later you'll have to >>>>>> interact with it anyway - many people favor monorepos these days and >>>>>> this is a common way to assemble them from fragmented >>>>>> sub-repositories. >>>>>> >>>>>> > I've >>>>>> > heard submodules don't play nice with Jenkins, but don't have any >>>>>> > direct experience with them. >>>>>> >>>>>> Some tools may require an additional flag to initialize and clone >>>>>> sub-modules on the initial pull. Or manually. >>>>>> >>>>>> git submodule init >>>>>> git submodule update --recursive. >>>>>> >>>>>> To summarize - I am not pushing for submodules, I do think they're a >>>>>> viable alternative but pinned Maven references are simpler. The >>>>>> problem with maven references is that we'd need a long(er) term maven >>>>>> repository where such pinned references would live (without being >>>>>> wiped out). Perhaps infra can help here and it'd solve the problem. >>>>>> >>>>>> Dawid >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>>
