Plus, isn’t this a reason for folks in the Solr side to continue to be involved in the Lucene project? It’s the inverse of the days when folks wanted to cut releases of Lucene, and were waiting for Solr to be ready!
> On Feb 26, 2021, at 1:26 PM, Houston Putman <houstonput...@gmail.com> wrote: > > I don't think Jan's workflow blocks Solr releases on Lucene releases. It just > blocks this one feature branch merge on a Lucene release. Multiple Solr > releases can happen between step 6 and step 7. > > I completely agree with that being the workflow going forward with separate > repos, Jan. It will unfortunately be a pain to integrate changes that affect > both Lucene and Solr, but I think that's just a consequence of splitting the > projects. > > Neither option gives us everything we want, so here are the pros and cons in > my opinion. > > Using a snapshot lucene version > - Easier to make changes to lucene and solr concurrently > - Solr releases are blocked until the snapshot version being depended on is > released. > - Builds may break at any time, and possibly for different sets of users > depending on dependency caches. > > Using a released lucene version > - Harder to update lucene and solr concurrently > - Solr can make releases independent of Lucene's release schedule > - Builds are stable and consistent. > > Personally I think stability and the ability to own our own release schedule > outweigh the benefits of being able to iterate on both projects concurrently. > But it's definitely something that we should decide on as a community. > > On Fri, Feb 26, 2021 at 12:43 PM Mike Drob <md...@mdrob.com > <mailto:md...@mdrob.com>> wrote: > The part of this process that I really don't like is that Solr then still > becomes beholden to Lucene's release schedule. We don't know how long step 7 > takes, and will be effectively blocked from making our own releases until > that happens. > > On Fri, Feb 26, 2021 at 8:51 AM Jan Høydahl <jan....@cominvent.com > <mailto:jan....@cominvent.com>> wrote: > The developer workflow for adding something to both Lucene and Solr would be > as any other dependency, right? > So say we are on Luene 9.0. This is the process in my head: > Adapt Lucene as needed, and "mvn install" lucene-9.1-SNAPSHOT to your local > laptop (whatever command that is in gradle) > Make your Solr feature branch depend on lucene-9.1-SNAPSHOT instead of > lucene-9.0.0 -hopefully Gradle will pick the local version over Apache Nexus > version > Iterate 1-2 until happy > Make a Lucene PR and eventually commit the Lucene change > After next Jenkins build the feature is in Apache Nexus snapshot as > lucene-9.1-SNAPSHOT > Now the Solr Pull Request will compile and can be tested by others > Wait until Lucene 9.1 release > Upgrade Solr's lucene dependency on 'main' > Merge Solr PR > Backporting will be a similar process, i.e. first backport and release in > Lucene, then backport in Sol > Hmm, as I wrote this list I can understand why so many features were added > only to Solr and not to Lucene in the early days :) > > Jan > >> 26. feb. 2021 kl. 14:22 skrev Gus Heck <gus.h...@gmail.com >> <mailto:gus.h...@gmail.com>>: >> >> Except I just finished helping a contributor with a feature that touches >> both and I know for a fact that it was developed for his customer who was >> using solr (payload inequalities)... and have another in the works (the >> AQP).... Not being able to enhance lucene to support a feature in solr is an >> issue IMHO. >> >> On Thu, Feb 25, 2021 at 6:05 PM Mike Drob <md...@mdrob.com >> <mailto:md...@mdrob.com>> wrote: >> It is possible to publish snapshots into the Apache Nexus repository. That >> said, I think it is a bad idea for Solr to depend on Lucene snapshots >> because that constrains the ability to do releases. Either you have to wait >> for a Lucene release and then you can cut over, or you have to figure out >> what changes you need to roll back. >> >> Features today rarely touch both fronts anyway, they usually land in Lucene >> first and then percolate into Solr. For an easy example, we can see how WAND >> was developed recently. >> >> On Thu, Feb 25, 2021 at 5:02 PM Houston Putman <houstonput...@gmail.com >> <mailto:houstonput...@gmail.com>> wrote: >> Once the projects are on separate release cadences there wont be an ability >> to “add on both fronts” anymore. You will have to add to lucene, wait for a >> release, then add to Solr once Solr upgrades its lucene dependency to that >> new version. I dont imagine that we are going to keep Solr master/main, or >> even 8x, 9x, etc, depending on Lucene snaphsots in perpetuity. After it >> becomes possible (when lucene 9.0 is released) we should only be using >> released lucene versions as dependencies for every version branch in Solr. >> >> On Thu, Feb 25, 2021 at 5:49 PM Gus Heck <gus.h...@gmail.com >> <mailto:gus.h...@gmail.com>> wrote: >> Until the first feature that wants to add something on both fronts... Is it >> possible for Lucene to publish nightly snapshots? I know there is some level >> of support for snapshots in central, though I don't know what their usage >> policies are. If that's too restricted is there an artifact repo controlled >> by the ASF that could be used? (An implementation of Apache Archiva?) This >> would have the added benefit of allowing solr to detect when Lucene breaks >> something before its released. >> >> On Thu, Feb 25, 2021 at 4:50 PM Houston Putman <houstonput...@gmail.com >> <mailto:houstonput...@gmail.com>> wrote: >> Hey everyone, >> >> Currently there is discussion going on, in SOLR-14762 >> <https://issues.apache.org/jira/browse/SOLR-14762>, regarding the split of >> the lucene-solr repo into individual repos for Solr and Lucene. There seems >> to be agreement that we shouldn't wait for a Lucene release to do the split, >> and instead split now and release whenever that happens. >> >> The biggest issue that arises there is that Solr's master branch is >> obviously based on Lucene's master branch, since they are currently the >> same. So when the split happens, Solr master will have to depend on Lucene >> 9.0-SNAPSHOT. We can have solr merely depend on the lucene snapshot, but >> that will result in inconsistent builds, depending on whatever cached >> dependencies each dev has locally. Personally, I think that will cause a >> bunch of build errors and headaches for everyone trying to maintain Solr. >> >> There is another option though. We could instead do an alpha "release" of >> lucene-solr 9.0 right before the repo is split. Therefore Solr can reliably >> depend on a stable version of lucene until 9.0 is truly released. (And >> lucene can use a stable version of Solr, if it sees a need for that). There >> would be no guarantees for using this alpha release, and we don't have to >> advertise it at all. >> >> It's not perfect, but I think it would be preferable to depending on an >> ever-changing SNAPSHOT lucene. >> >> - Houston >> >> >> -- >> http://www.needhamsoftware.com <http://www.needhamsoftware.com/> (work) >> http://www.the111shift.com <http://www.the111shift.com/> (play) >> >> >> -- >> http://www.needhamsoftware.com <http://www.needhamsoftware.com/> (work) >> http://www.the111shift.com <http://www.the111shift.com/> (play) > _______________________ Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
