It could be a manual process if that’s what you think is easiest and it could still be on nightlies. It’s just a curl command to push something up: https://nightlies.apache.org/authoring.html. If it is a manual process, I think it would be better for it to be hosted in a place any of us could access, in case you’re busy/something happens to you and we need to pin a different snapshot.
I’m not sure I see a reason why Solr couldn’t have our own Jenkins job that built Lucene once a week (or on demand, or whatever cadence that works) and pushes the snapshot to nightlies - we don’t have to use one of Lucene’s jobs, do we? I feel like we could also cascade a new Lucene snapshot build into a “Lucene validation” Solr build that verifies everything is OK with a new snapshot before updating the hosted snapshot on nightlies (although, I don’t actually know how to do that, it just seems like something that could be done). Cassandra On May 6, 2021, 10:13 AM -0500, David Smiley <[email protected]>, wrote: > I asked on the Lucene dev list about possible use of the Nightlies server. > We don't want to pollute nightlies on a regular basis (I think); this would > be an on-demand thing. As such, I'm not sure a CI server is the best way to > approach this vs a manual script publishing to an ASF personal Home space. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > > On Mon, May 3, 2021 at 10:55 AM Cassandra Targett <[email protected]> > > wrote: > > > I’m sort of surprised no one has mentioned the > > > https://nightlies.apache.org/ server, which could be used for this > > > purpose. Jenkins can push to it, and nothing gets deleted until we decide > > > to delete it (or overwrite it). Solr Operator builds go there as do > > > drafts of the Ref Guide pre-publication (in different directories now, > > > but we’ll fix that eventually). > > > > > > Cassandra > > > On May 3, 2021, 9:15 AM -0500, David Smiley <[email protected]>, wrote: > > > > RE "an external system like Maven" -- we're merely talking about adding > > > > another JAR repo to the list of repos we already have. Heck, for this > > > > limited purpose, we could even use http://home.apache.org/~dsmiley/ > > > > Note that the Lucene project already uses home dirs of some users for > > > > benchmark data. If we were talking about adding a Maven *build* then I > > > > would totally appreciate your concern. > > > > > > > > I volunteer to use my space for this purpose. > > > > > > > > ~ David Smiley > > > > Apache Lucene/Solr Search Developer > > > > http://www.linkedin.com/in/davidwsmiley > > > > > > > > > > > > > On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya > > > > > <[email protected]> wrote: > > > > > > > Let’s not complicate things. > > > > > > By bringing in external systems like Maven, I think we're > > > > > > complicating things even though a straight forward way (git > > > > > > submodules) exists. > > > > > > > > > > > > > On Sat, May 1, 2021 at 5:00 PM Jan Høydahl > > > > > > > <[email protected]> wrote: > > > > > > > > Sub modules is for organizing internal repos, not for pulling > > > > > > > > in external deps. Let’s not complicate things. And once we > > > > > > > > switch main to 10.x we’d need to use pure jar dependencies in > > > > > > > > branch_9x to depend upon actually released and voted Lucene > > > > > > > > binaries. > > > > > > > > > > > > > > > > We need some tooling to smoothly work with bleeding edge Lucene > > > > > > > > including local snapshot builds with not yet pushed changes to > > > > > > > > Lucene. I’d rather put in some work to establish such tooling > > > > > > > > or procedures. > > > > > > > > > > > > > > > > Jan Høydahl > > > > > > > > > > > > > > > > > 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya > > > > > > > > > <[email protected]>: > > > > > > > > > > > > > > > > > > > What submodules don't solve is releases - if you're > > > > > > > > > > on a > > > > > > > > > particular unreleased Lucene version then releasing > > > > > > > > > > Solr would still mean you need some kind of > > > > > > > > > > "public" pinned Lucene release for the > > > > > > > > > > Mavenworld. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can we then update the submodule to point to the release > > > > > > > > > > tag or sha that Lucene got released from? > > > > > > > > > > > > > > > > > > > > On Sat, 1 May, 2021, 11:45 am Dawid Weiss, > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > Other than literally adding the git submodule, would we > > > > > > > > > > > > do anything else to modify the gradle build so that or > > > > > > > > > > > > do we (and Jenkinsfile) have to manually do a step to > > > > > > > > > > > > install Lucene before proceeding? > > > > > > > > > > > > > > > > > > > > > > Technically you add a submodule and then make a composite > > > > > > > > > > > build from > > > > > > > > > > > Solr side. I can provide a PR that does it if you wish... > > > > > > > > > > > This is > > > > > > > > > > > simple. What submodules don't solve is releases - if > > > > > > > > > > > you're on a > > > > > > > > > > > particular unreleased Lucene version then releasing Solr > > > > > > > > > > > would still > > > > > > > > > > > mean you need some kind of "public" pinned Lucene release > > > > > > > > > > > for the > > > > > > > > > > > Maven world. If you distribute the entire thing as > > > > > > > > > > > binaries you can > > > > > > > > > > > just publish a snapshot of Lucene code, of course. > > > > > > > > > > > > > > > > > > > > > > > I think adding git submodule means that we have to add > > > > > > > > > > > > back in all the build code for it. > > > > > > > > > > > > > > > > > > > > > > No. The submodule is just a reference to a particular > > > > > > > > > > > repository (and > > > > > > > > > > > commit) - the build code and remains nearly identical as > > > > > > > > > > > it was with > > > > > > > > > > > Lucene. Composite builds in gradle are quite nice in that > > > > > > > > > > > they're > > > > > > > > > > > (almost) transparent compared to dependency references. > > > > > > > > > > > > > > > > > > > > > > > At which point, I'd rather just copy and commit the > > > > > > > > > > > > code they have so we don't have to learn another git > > > > > > > > > > > > system. > > > > > > > > > > > > > > > > > > > > > > It's not really a different git system - it's a mechanism > > > > > > > > > > > of > > > > > > > > > > > interacting with multiple repositories at once. Yes, it > > > > > > > > > > > does add some > > > > > > > > > > > additional complexity but I think sooner or later you'll > > > > > > > > > > > have to > > > > > > > > > > > interact with it anyway - many people favor monorepos > > > > > > > > > > > these days and > > > > > > > > > > > this is a common way to assemble them from fragmented > > > > > > > > > > > sub-repositories. > > > > > > > > > > > > > > > > > > > > > > > I've > > > > > > > > > > > > heard submodules don't play nice with Jenkins, but > > > > > > > > > > > > don't have any > > > > > > > > > > > > direct experience with them. > > > > > > > > > > > > > > > > > > > > > > Some tools may require an additional flag to initialize > > > > > > > > > > > and clone > > > > > > > > > > > sub-modules on the initial pull. Or manually. > > > > > > > > > > > > > > > > > > > > > > git submodule init > > > > > > > > > > > git submodule update --recursive. > > > > > > > > > > > > > > > > > > > > > > To summarize - I am not pushing for submodules, I do > > > > > > > > > > > think they're a > > > > > > > > > > > viable alternative but pinned Maven references are > > > > > > > > > > > simpler. The > > > > > > > > > > > problem with maven references is that we'd need a > > > > > > > > > > > long(er) term maven > > > > > > > > > > > repository where such pinned references would live > > > > > > > > > > > (without being > > > > > > > > > > > wiped out). Perhaps infra can help here and it'd solve > > > > > > > > > > > the problem. > > > > > > > > > > > > > > > > > > > > > > Dawid > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > >
