Can you give an example of a recent solr feature that needed such a workflow - I.e where it would be very annoying to have to do the Lucene part first and then the solr part, using a local snapshot version?
Jan Høydahl > 3. mai 2021 kl. 17:32 skrev Ishan Chattopadhyaya <[email protected]>: > > > One of the other problems I have with maven, besides where will the snapshots > go, is the local development workflow. If I were to work on a feature that > needs changes to both Lucene and Solr, I would need to make Lucene changes in > a separate git repo, push them to local Maven, have solr build against that > and so on. This is considerably more complicated, to my mind, than working on > a single repo (with the other repo being a submodule) and single project > inside the IDE, which git submodule can allow us to do. > >> On Mon, 3 May, 2021, 8:27 pm Houston Putman, <[email protected]> wrote: >> I agree, Cassandra. The nightlies are a very good option that is very easy >> to get started with. >> >>> On Mon, May 3, 2021 at 9:55 AM Cassandra Targett <[email protected]> >>> wrote: >>> I’m sort of surprised no one has mentioned the >>> https://nightlies.apache.org/ server, which could be used for this purpose. >>> Jenkins can push to it, and nothing gets deleted until we decide to delete >>> it (or overwrite it). Solr Operator builds go there as do drafts of the Ref >>> Guide pre-publication (in different directories now, but we’ll fix that >>> eventually). >>> >>> Cassandra >>>> On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, wrote: >>>> RE "an external system like Maven" -- we're merely talking about adding >>>> another JAR repo to the list of repos we already have. Heck, for this >>>> limited purpose, we could even use http://home.apache.org/~dsmiley/ Note >>>> that the Lucene project already uses home dirs of some users for benchmark >>>> data. If we were talking about adding a Maven *build* then I would >>>> totally appreciate your concern. >>>> >>>> I volunteer to use my space for this purpose. >>>> >>>> ~ David Smiley >>>> Apache Lucene/Solr Search Developer >>>> http://www.linkedin.com/in/davidwsmiley >>>> >>>> >>>>> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya >>>>> <[email protected]> wrote: >>>>> > Let’s not complicate things. >>>>> By bringing in external systems like Maven, I think we're complicating >>>>> things even though a straight forward way (git submodules) exists. >>>>> >>>>>> On Sat, May 1, 2021 at 5:00 PM Jan Høydahl <[email protected]> >>>>>> wrote: >>>>>> Sub modules is for organizing internal repos, not for pulling in >>>>>> external deps. Let’s not complicate things. And once we switch main to >>>>>> 10.x we’d need to use pure jar dependencies in branch_9x to depend upon >>>>>> actually released and voted Lucene binaries. >>>>>> >>>>>> We need some tooling to smoothly work with bleeding edge Lucene >>>>>> including local snapshot builds with not yet pushed changes to Lucene. >>>>>> I’d rather put in some work to establish such tooling or procedures. >>>>>> >>>>>> Jan Høydahl >>>>>> >>>>>>> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya >>>>>>> <[email protected]>: >>>>>>> >>>>>>> >>>>>>> > What submodules don't solve is releases - if you're >>>>>>> > on a >>>>>>> particular unreleased Lucene version then releasing >>>>>>> > Solr would still mean you need some kind of >>>>>>> > "public" pinned Lucene release for the >>>>>>> > Mavenworld. >>>>>>> >>>>>>> >>>>>>> Can we then update the submodule to point to the release tag or sha >>>>>>> that Lucene got released from? >>>>>>> >>>>>>>> On Sat, 1 May, 2021, 11:45 am Dawid Weiss, <[email protected]> >>>>>>>> wrote: >>>>>>>> > Other than literally adding the git submodule, would we do anything >>>>>>>> > else to modify the gradle build so that or do we (and Jenkinsfile) >>>>>>>> > have to manually do a step to install Lucene before proceeding? >>>>>>>> >>>>>>>> Technically you add a submodule and then make a composite build from >>>>>>>> Solr side. I can provide a PR that does it if you wish... This is >>>>>>>> simple. What submodules don't solve is releases - if you're on a >>>>>>>> particular unreleased Lucene version then releasing Solr would still >>>>>>>> mean you need some kind of "public" pinned Lucene release for the >>>>>>>> Maven world. If you distribute the entire thing as binaries you can >>>>>>>> just publish a snapshot of Lucene code, of course. >>>>>>>> >>>>>>>> > I think adding git submodule means that we have to add back in all >>>>>>>> > the build code for it. >>>>>>>> >>>>>>>> No. The submodule is just a reference to a particular repository (and >>>>>>>> commit) - the build code and remains nearly identical as it was with >>>>>>>> Lucene. Composite builds in gradle are quite nice in that they're >>>>>>>> (almost) transparent compared to dependency references. >>>>>>>> >>>>>>>> > At which point, I'd rather just copy and commit the >>>>>>>> > code they have so we don't have to learn another git system. >>>>>>>> >>>>>>>> It's not really a different git system - it's a mechanism of >>>>>>>> interacting with multiple repositories at once. Yes, it does add some >>>>>>>> additional complexity but I think sooner or later you'll have to >>>>>>>> interact with it anyway - many people favor monorepos these days and >>>>>>>> this is a common way to assemble them from fragmented >>>>>>>> sub-repositories. >>>>>>>> >>>>>>>> > I've >>>>>>>> > heard submodules don't play nice with Jenkins, but don't have any >>>>>>>> > direct experience with them. >>>>>>>> >>>>>>>> Some tools may require an additional flag to initialize and clone >>>>>>>> sub-modules on the initial pull. Or manually. >>>>>>>> >>>>>>>> git submodule init >>>>>>>> git submodule update --recursive. >>>>>>>> >>>>>>>> To summarize - I am not pushing for submodules, I do think they're a >>>>>>>> viable alternative but pinned Maven references are simpler. The >>>>>>>> problem with maven references is that we'd need a long(er) term maven >>>>>>>> repository where such pinned references would live (without being >>>>>>>> wiped out). Perhaps infra can help here and it'd solve the problem. >>>>>>>> >>>>>>>> Dawid >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>
