Sounds great Uwe! Can you do it please? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley
On Fri, May 7, 2021 at 1:55 AM Uwe Schindler <[email protected]> wrote: > Hi Cassandra. Sure, We can create a job on ASF Jenkins that is only > manually triggered. > It builds the maven artifacts into a local file system repo. As post build > job we publish the output folder on nighties repo like the ref guide. > > The version number is passed as -DreleaseVersion system property (e.g. > populated by Jenkins from the build number), like we do on artifacts builds > on jenkins anyways. It's no snapshot anymore, but like an half official > build with some custom version internal to our process. > > If you want to upgrade Solr's Lucene, just press "build" on jenkins, wait > for it to finish, and finally copy-paste the version number from Jenkins > build description into gradle dependency version file. > > Uwe > > Uwe > > Am May 6, 2021 8:13:00 PM UTC schrieb Cassandra Targett < > [email protected]>: >> >> It could be a manual process if that’s what you think is easiest and it >> could still be on nightlies. It’s just a curl command to push something up: >> https://nightlies.apache.org/authoring.html. If it is a manual process, >> I think it would be better for it to be hosted in a place any of us could >> access, in case you’re busy/something happens to you and we need to pin a >> different snapshot. >> >> I’m not sure I see a reason why Solr couldn’t have our own Jenkins job >> that built Lucene once a week (or on demand, or whatever cadence that >> works) and pushes the snapshot to nightlies - we don’t have to use one of >> Lucene’s jobs, do we? >> >> I feel like we could also cascade a new Lucene snapshot build into a >> “Lucene validation” Solr build that verifies everything is OK with a new >> snapshot before updating the hosted snapshot on nightlies (although, I >> don’t actually know how to do that, it just seems like something that could >> be done). >> >> Cassandra >> On May 6, 2021, 10:13 AM -0500, David Smiley <[email protected]>, wrote: >> >> I asked on the Lucene dev list about possible use of the Nightlies >> server. We don't want to pollute nightlies on a regular basis (I think); >> this would be an on-demand thing. As such, I'm not sure a CI server is the >> best way to approach this vs a manual script publishing to an ASF personal >> Home space. >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Mon, May 3, 2021 at 10:55 AM Cassandra Targett <[email protected]> >> wrote: >> >>> I’m sort of surprised no one has mentioned the >>> https://nightlies.apache.org/ server, which could be used for this >>> purpose. Jenkins can push to it, and nothing gets deleted until we decide >>> to delete it (or overwrite it). Solr Operator builds go there as do drafts >>> of the Ref Guide pre-publication (in different directories now, but we’ll >>> fix that eventually). >>> >>> Cassandra >>> On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, wrote: >>> >>> RE "an external system like Maven" -- we're merely talking about adding >>> another JAR repo to the list of repos we already have. Heck, for this >>> limited purpose, we could even use http://home.apache.org/~dsmiley/ >>> Note that the Lucene project already uses home dirs of some users for >>> benchmark data. If we were talking about adding a Maven *build* then I >>> would totally appreciate your concern. >>> >>> I volunteer to use my space for this purpose. >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya < >>> [email protected]> wrote: >>> >>>> > Let’s not complicate things. >>>> By bringing in external systems like Maven, I think we're complicating >>>> things even though a straight forward way (git submodules) exists. >>>> >>>> On Sat, May 1, 2021 at 5:00 PM Jan Høydahl <[email protected]> >>>> wrote: >>>> >>>>> Sub modules is for organizing internal repos, not for pulling in >>>>> external deps. Let’s not complicate things. And once we switch main to >>>>> 10.x >>>>> we’d need to use pure jar dependencies in branch_9x to depend upon >>>>> actually >>>>> released and voted Lucene binaries. >>>>> >>>>> We need some tooling to smoothly work with bleeding edge Lucene >>>>> including local snapshot builds with not yet pushed changes to Lucene. I’d >>>>> rather put in some work to establish such tooling or procedures. >>>>> >>>>> Jan Høydahl >>>>> >>>>> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya < >>>>> [email protected]>: >>>>> >>>>> >>>>> > What submodules don't solve is releases - if you're >>>>> > on a >>>>> particular unreleased Lucene version then releasing >>>>> > Solr would still mean you need some kind of >>>>> > "public" pinned Lucene release for the >>>>> > Mavenworld. >>>>> >>>>> >>>>> Can we then update the submodule to point to the release tag or sha >>>>> that Lucene got released from? >>>>> >>>>> On Sat, 1 May, 2021, 11:45 am Dawid Weiss, <[email protected]> >>>>> wrote: >>>>> >>>>>> > Other than literally adding the git submodule, would we do anything >>>>>> else to modify the gradle build so that or do we (and Jenkinsfile) have >>>>>> to >>>>>> manually do a step to install Lucene before proceeding? >>>>>> >>>>>> Technically you add a submodule and then make a composite build from >>>>>> Solr side. I can provide a PR that does it if you wish... This is >>>>>> simple. What submodules don't solve is releases - if you're on a >>>>>> particular unreleased Lucene version then releasing Solr would still >>>>>> mean you need some kind of "public" pinned Lucene release for the >>>>>> Maven world. If you distribute the entire thing as binaries you can >>>>>> just publish a snapshot of Lucene code, of course. >>>>>> >>>>>> > I think adding git submodule means that we have to add back in all >>>>>> the build code for it. >>>>>> >>>>>> No. The submodule is just a reference to a particular repository (and >>>>>> commit) - the build code and remains nearly identical as it was with >>>>>> Lucene. Composite builds in gradle are quite nice in that they're >>>>>> (almost) transparent compared to dependency references. >>>>>> >>>>>> > At which point, I'd rather just copy and commit the >>>>>> > code they have so we don't have to learn another git system. >>>>>> >>>>>> It's not really a different git system - it's a mechanism of >>>>>> interacting with multiple repositories at once. Yes, it does add some >>>>>> additional complexity but I think sooner or later you'll have to >>>>>> interact with it anyway - many people favor monorepos these days and >>>>>> this is a common way to assemble them from fragmented >>>>>> sub-repositories. >>>>>> >>>>>> > I've >>>>>> > heard submodules don't play nice with Jenkins, but don't have any >>>>>> > direct experience with them. >>>>>> >>>>>> Some tools may require an additional flag to initialize and clone >>>>>> sub-modules on the initial pull. Or manually. >>>>>> >>>>>> git submodule init >>>>>> git submodule update --recursive. >>>>>> >>>>>> To summarize - I am not pushing for submodules, I do think they're a >>>>>> viable alternative but pinned Maven references are simpler. The >>>>>> problem with maven references is that we'd need a long(er) term maven >>>>>> repository where such pinned references would live (without being >>>>>> wiped out). Perhaps infra can help here and it'd solve the problem. >>>>>> >>>>>> Dawid >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> > -- > Uwe Schindler > Achterdiek 19, 28357 Bremen > https://www.thetaphi.de >
