Please, no sub modules! It is honestly a mess. And besides, for 95% of Solr development work there are no Lucene changes, so having to compile Lucene every time is not logical. I suppose you would be able to do some surgery on your local setup though, removing lucene dep from gradle and instead adding your Lucene checkout as a module dependency in your IDE?
Jan > 27. feb. 2021 kl. 04:28 skrev Ishan Chattopadhyaya > <[email protected]>: > > I would prefer Lucene as a git submodule inside Solr. That way, I can work > with solr and Lucene together easily. The commit to which Lucene is pegged at > could be from a release branch or a tag. > > On Sat, 27 Feb, 2021, 3:03 am Adrien Grand, <[email protected] > <mailto:[email protected]>> wrote: > FYI Elasticsearch has been regularly depending on builds of specific commits > of Lucene for this case of features that need changes both in Lucene and > Elasticsearch. > > The workflow usually looks like this: > - Do work in Lucene. > - When it becomes clear that the next release of Lucene should happen before > the next feature freeze of Elasticsearch, we do a new build of Lucene and > upgrade Elasticsearch to it. > - Do work in Elasticsearch. > - When a new Lucene release is out, upgrade Elasticsearch to this Lucene > release. > > We have done this dozens of times and it has worked well for us. We do the > same when a vote for a new Lucene release is about to start in order to check > whether it breaks anything in Elasticsearch. > > The rest of the time (which is most of the time) Elasticsearch depends on an > actual Lucene release. > > Le ven. 26 févr. 2021 à 19:29, Eric Pugh <[email protected] > <mailto:[email protected]>> a écrit : > Plus, isn’t this a reason for folks in the Solr side to continue to be > involved in the Lucene project? It’s the inverse of the days when folks > wanted to cut releases of Lucene, and were waiting for Solr to be ready! > > >> On Feb 26, 2021, at 1:26 PM, Houston Putman <[email protected] >> <mailto:[email protected]>> wrote: >> >> I don't think Jan's workflow blocks Solr releases on Lucene releases. It >> just blocks this one feature branch merge on a Lucene release. Multiple Solr >> releases can happen between step 6 and step 7. >> >> I completely agree with that being the workflow going forward with separate >> repos, Jan. It will unfortunately be a pain to integrate changes that affect >> both Lucene and Solr, but I think that's just a consequence of splitting the >> projects. >> >> Neither option gives us everything we want, so here are the pros and cons in >> my opinion. >> >> Using a snapshot lucene version >> - Easier to make changes to lucene and solr concurrently >> - Solr releases are blocked until the snapshot version being depended on is >> released. >> - Builds may break at any time, and possibly for different sets of users >> depending on dependency caches. >> >> Using a released lucene version >> - Harder to update lucene and solr concurrently >> - Solr can make releases independent of Lucene's release schedule >> - Builds are stable and consistent. >> >> Personally I think stability and the ability to own our own release schedule >> outweigh the benefits of being able to iterate on both projects >> concurrently. But it's definitely something that we should decide on as a >> community. >> >> On Fri, Feb 26, 2021 at 12:43 PM Mike Drob <[email protected] >> <mailto:[email protected]>> wrote: >> The part of this process that I really don't like is that Solr then still >> becomes beholden to Lucene's release schedule. We don't know how long step 7 >> takes, and will be effectively blocked from making our own releases until >> that happens. >> >> On Fri, Feb 26, 2021 at 8:51 AM Jan Høydahl <[email protected] >> <mailto:[email protected]>> wrote: >> The developer workflow for adding something to both Lucene and Solr would be >> as any other dependency, right? >> So say we are on Luene 9.0. This is the process in my head: >> Adapt Lucene as needed, and "mvn install" lucene-9.1-SNAPSHOT to your local >> laptop (whatever command that is in gradle) >> Make your Solr feature branch depend on lucene-9.1-SNAPSHOT instead of >> lucene-9.0.0 -hopefully Gradle will pick the local version over Apache Nexus >> version >> Iterate 1-2 until happy >> Make a Lucene PR and eventually commit the Lucene change >> After next Jenkins build the feature is in Apache Nexus snapshot as >> lucene-9.1-SNAPSHOT >> Now the Solr Pull Request will compile and can be tested by others >> Wait until Lucene 9.1 release >> Upgrade Solr's lucene dependency on 'main' >> Merge Solr PR >> Backporting will be a similar process, i.e. first backport and release in >> Lucene, then backport in Sol >> Hmm, as I wrote this list I can understand why so many features were added >> only to Solr and not to Lucene in the early days :) >> >> Jan >> >>> 26. feb. 2021 kl. 14:22 skrev Gus Heck <[email protected] >>> <mailto:[email protected]>>: >>> >>> Except I just finished helping a contributor with a feature that touches >>> both and I know for a fact that it was developed for his customer who was >>> using solr (payload inequalities)... and have another in the works (the >>> AQP).... Not being able to enhance lucene to support a feature in solr is >>> an issue IMHO. >>> >>> On Thu, Feb 25, 2021 at 6:05 PM Mike Drob <[email protected] >>> <mailto:[email protected]>> wrote: >>> It is possible to publish snapshots into the Apache Nexus repository. That >>> said, I think it is a bad idea for Solr to depend on Lucene snapshots >>> because that constrains the ability to do releases. Either you have to wait >>> for a Lucene release and then you can cut over, or you have to figure out >>> what changes you need to roll back. >>> >>> Features today rarely touch both fronts anyway, they usually land in Lucene >>> first and then percolate into Solr. For an easy example, we can see how >>> WAND was developed recently. >>> >>> On Thu, Feb 25, 2021 at 5:02 PM Houston Putman <[email protected] >>> <mailto:[email protected]>> wrote: >>> Once the projects are on separate release cadences there wont be an ability >>> to “add on both fronts” anymore. You will have to add to lucene, wait for a >>> release, then add to Solr once Solr upgrades its lucene dependency to that >>> new version. I dont imagine that we are going to keep Solr master/main, or >>> even 8x, 9x, etc, depending on Lucene snaphsots in perpetuity. After it >>> becomes possible (when lucene 9.0 is released) we should only be using >>> released lucene versions as dependencies for every version branch in Solr. >>> >>> On Thu, Feb 25, 2021 at 5:49 PM Gus Heck <[email protected] >>> <mailto:[email protected]>> wrote: >>> Until the first feature that wants to add something on both fronts... Is it >>> possible for Lucene to publish nightly snapshots? I know there is some >>> level of support for snapshots in central, though I don't know what their >>> usage policies are. If that's too restricted is there an artifact repo >>> controlled by the ASF that could be used? (An implementation of Apache >>> Archiva?) This would have the added benefit of allowing solr to detect when >>> Lucene breaks something before its released. >>> >>> On Thu, Feb 25, 2021 at 4:50 PM Houston Putman <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hey everyone, >>> >>> Currently there is discussion going on, in SOLR-14762 >>> <https://issues.apache.org/jira/browse/SOLR-14762>, regarding the split of >>> the lucene-solr repo into individual repos for Solr and Lucene. There seems >>> to be agreement that we shouldn't wait for a Lucene release to do the >>> split, and instead split now and release whenever that happens. >>> >>> The biggest issue that arises there is that Solr's master branch is >>> obviously based on Lucene's master branch, since they are currently the >>> same. So when the split happens, Solr master will have to depend on Lucene >>> 9.0-SNAPSHOT. We can have solr merely depend on the lucene snapshot, but >>> that will result in inconsistent builds, depending on whatever cached >>> dependencies each dev has locally. Personally, I think that will cause a >>> bunch of build errors and headaches for everyone trying to maintain Solr. >>> >>> There is another option though. We could instead do an alpha "release" of >>> lucene-solr 9.0 right before the repo is split. Therefore Solr can reliably >>> depend on a stable version of lucene until 9.0 is truly released. (And >>> lucene can use a stable version of Solr, if it sees a need for that). There >>> would be no guarantees for using this alpha release, and we don't have to >>> advertise it at all. >>> >>> It's not perfect, but I think it would be preferable to depending on an >>> ever-changing SNAPSHOT lucene. >>> >>> - Houston >>> >>> >>> -- >>> http://www.needhamsoftware.com <http://www.needhamsoftware.com/> (work) >>> http://www.the111shift.com <http://www.the111shift.com/> (play) >>> >>> >>> -- >>> http://www.needhamsoftware.com <http://www.needhamsoftware.com/> (work) >>> http://www.the111shift.com <http://www.the111shift.com/> (play) >> > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com <http://www.opensourceconnections.com/> > | My Free/Busy <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. >
