I would prefer Lucene as a git submodule inside Solr. That way, I can work with solr and Lucene together easily. The commit to which Lucene is pegged at could be from a release branch or a tag.
On Sat, 27 Feb, 2021, 3:03 am Adrien Grand, <[email protected]> wrote: > FYI Elasticsearch has been regularly depending on builds of specific > commits of Lucene for this case of features that need changes both in > Lucene and Elasticsearch. > > The workflow usually looks like this: > - Do work in Lucene. > - When it becomes clear that the next release of Lucene should happen > before the next feature freeze of Elasticsearch, we do a new build of > Lucene and upgrade Elasticsearch to it. > - Do work in Elasticsearch. > - When a new Lucene release is out, upgrade Elasticsearch to this Lucene > release. > > We have done this dozens of times and it has worked well for us. We do the > same when a vote for a new Lucene release is about to start in order to > check whether it breaks anything in Elasticsearch. > > The rest of the time (which is most of the time) Elasticsearch depends on > an actual Lucene release. > > Le ven. 26 févr. 2021 à 19:29, Eric Pugh <[email protected]> > a écrit : > >> Plus, isn’t this a reason for folks in the Solr side to continue to be >> involved in the Lucene project? It’s the inverse of the days when folks >> wanted to cut releases of Lucene, and were waiting for Solr to be ready! >> >> >> On Feb 26, 2021, at 1:26 PM, Houston Putman <[email protected]> >> wrote: >> >> I don't think Jan's workflow blocks Solr releases on Lucene releases. It >> just blocks this one feature branch merge on a Lucene release. Multiple >> Solr releases can happen between step 6 and step 7. >> >> I completely agree with that being the workflow going forward with >> separate repos, Jan. It will unfortunately be a pain to integrate changes >> that affect both Lucene and Solr, but I think that's just a consequence of >> splitting the projects. >> >> Neither option gives us everything we want, so here are the pros and cons >> in my opinion. >> >> Using a snapshot lucene version >> - Easier to make changes to lucene and solr concurrently >> - Solr releases are blocked until the snapshot version being depended on >> is released. >> - Builds may break at any time, and possibly for different sets of users >> depending on dependency caches. >> >> Using a released lucene version >> - Harder to update lucene and solr concurrently >> - Solr can make releases independent of Lucene's release schedule >> - Builds are stable and consistent. >> >> Personally I think stability and the ability to own our own release >> schedule outweigh the benefits of being able to iterate on both projects >> concurrently. But it's definitely something that we should decide on as a >> community. >> >> On Fri, Feb 26, 2021 at 12:43 PM Mike Drob <[email protected]> wrote: >> >>> The part of this process that I really don't like is that Solr then >>> still becomes beholden to Lucene's release schedule. We don't know how long >>> step 7 takes, and will be effectively blocked from making our own releases >>> until that happens. >>> >>> On Fri, Feb 26, 2021 at 8:51 AM Jan Høydahl <[email protected]> >>> wrote: >>> >>>> The developer workflow for adding something to both Lucene and Solr >>>> would be as any other dependency, right? >>>> So say we are on Luene 9.0. This is the process in my head: >>>> >>>> 1. Adapt Lucene as needed, and "mvn install" lucene-9.1-SNAPSHOT to >>>> your local laptop (whatever command that is in gradle) >>>> 2. Make your Solr feature branch depend on lucene-9.1-SNAPSHOT >>>> instead of lucene-9.0.0 -hopefully Gradle will pick the local version >>>> over >>>> Apache Nexus version >>>> 3. Iterate 1-2 until happy >>>> 4. Make a Lucene PR and eventually commit the Lucene change >>>> 5. After next Jenkins build the feature is in Apache Nexus snapshot >>>> as lucene-9.1-SNAPSHOT >>>> 6. Now the Solr Pull Request will compile and can be tested by >>>> others >>>> 7. Wait until Lucene 9.1 release >>>> 8. Upgrade Solr's lucene dependency on 'main' >>>> 9. Merge Solr PR >>>> 10. Backporting will be a similar process, i.e. first backport and >>>> release in Lucene, then backport in Sol >>>> >>>> Hmm, as I wrote this list I can understand why so many features were >>>> added only to Solr and not to Lucene in the early days :) >>>> >>>> Jan >>>> >>>> 26. feb. 2021 kl. 14:22 skrev Gus Heck <[email protected]>: >>>> >>>> Except I just finished helping a contributor with a feature that >>>> touches both and I know for a fact that it was developed for his customer >>>> who was using solr (payload inequalities)... and have another in the works >>>> (the AQP).... Not being able to enhance lucene to support a feature in solr >>>> is an issue IMHO. >>>> >>>> On Thu, Feb 25, 2021 at 6:05 PM Mike Drob <[email protected]> wrote: >>>> >>>>> It is possible to publish snapshots into the Apache Nexus repository. >>>>> That said, I think it is a bad idea for Solr to depend on Lucene snapshots >>>>> because that constrains the ability to do releases. Either you have to >>>>> wait >>>>> for a Lucene release and then you can cut over, or you have to figure out >>>>> what changes you need to roll back. >>>>> >>>>> Features today rarely touch both fronts anyway, they usually land in >>>>> Lucene first and then percolate into Solr. For an easy example, we can see >>>>> how WAND was developed recently. >>>>> >>>>> On Thu, Feb 25, 2021 at 5:02 PM Houston Putman < >>>>> [email protected]> wrote: >>>>> >>>>>> Once the projects are on separate release cadences there wont be an >>>>>> ability to “add on both fronts” anymore. You will have to add to lucene, >>>>>> wait for a release, then add to Solr once Solr upgrades its lucene >>>>>> dependency to that new version. I dont imagine that we are going to keep >>>>>> Solr master/main, or even 8x, 9x, etc, depending on Lucene snaphsots in >>>>>> perpetuity. After it becomes possible (when lucene 9.0 is released) we >>>>>> should only be using released lucene versions as dependencies for every >>>>>> version branch in Solr. >>>>>> >>>>>> On Thu, Feb 25, 2021 at 5:49 PM Gus Heck <[email protected]> wrote: >>>>>> >>>>>>> Until the first feature that wants to add something on both >>>>>>> fronts... Is it possible for Lucene to publish nightly snapshots? I know >>>>>>> there is some level of support for snapshots in central, though I don't >>>>>>> know what their usage policies are. If that's too restricted is there an >>>>>>> artifact repo controlled by the ASF that could be used? (An >>>>>>> implementation >>>>>>> of Apache Archiva?) This would have the added benefit of allowing solr >>>>>>> to >>>>>>> detect when Lucene breaks something before its released. >>>>>>> >>>>>>> On Thu, Feb 25, 2021 at 4:50 PM Houston Putman < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hey everyone, >>>>>>>> >>>>>>>> Currently there is discussion going on, in SOLR-14762 >>>>>>>> <https://issues.apache.org/jira/browse/SOLR-14762>, regarding the >>>>>>>> split of the lucene-solr repo into individual repos for Solr and >>>>>>>> Lucene. >>>>>>>> There seems to be agreement that we shouldn't wait for a Lucene >>>>>>>> release to >>>>>>>> do the split, and instead split now and release whenever that happens. >>>>>>>> >>>>>>>> The biggest issue that arises there is that Solr's master branch is >>>>>>>> obviously based on Lucene's master branch, since they are currently the >>>>>>>> same. So when the split happens, Solr master will have to depend on >>>>>>>> Lucene >>>>>>>> 9.0-SNAPSHOT. We can have solr merely depend on the lucene snapshot, >>>>>>>> but >>>>>>>> that will result in inconsistent builds, depending on whatever cached >>>>>>>> dependencies each dev has locally. Personally, I think that will cause >>>>>>>> a >>>>>>>> bunch of build errors and headaches for everyone trying to maintain >>>>>>>> Solr. >>>>>>>> >>>>>>>> There is another option though. We could instead do an *alpha* >>>>>>>> "release" of lucene-solr 9.0 right before the repo is split. Therefore >>>>>>>> Solr >>>>>>>> can reliably depend on a stable version of lucene until 9.0 is truly >>>>>>>> released. (And lucene can use a stable version of Solr, if it sees a >>>>>>>> need >>>>>>>> for that). There would be no guarantees for using this alpha release, >>>>>>>> and >>>>>>>> we don't have to advertise it at all. >>>>>>>> >>>>>>>> It's not perfect, but I think it would be preferable to depending >>>>>>>> on an ever-changing SNAPSHOT lucene. >>>>>>>> >>>>>>>> - Houston >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> http://www.needhamsoftware.com (work) >>>>>>> http://www.the111shift.com (play) >>>>>>> >>>>>> >>>> >>>> -- >>>> http://www.needhamsoftware.com (work) >>>> http://www.the111shift.com (play) >>>> >>>> >>>> >> _______________________ >> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >> | http://www.opensourceconnections.com | My Free/Busy >> <http://tinyurl.com/eric-cal> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >> This e-mail and all contents, including attachments, is considered to be >> Company Confidential unless explicitly stated otherwise, regardless >> of whether attachments are marked as such. >> >>
