Thanks for sending this email, Mike and thanks for the follow up, David. The idea of having multiple repos under the project seems like the reasonable way to go for our project. This allows us to support more features/tooling/etc. without having to link them to Solr or Lucene releases.
An important thing here is to understand that if it comes from under the same umbrella, it should be treated with the same care and respect - at least we should attempt to. Q: Is it "okay" to release new Solr versions that break any of these > external contribs? Knowingly or unknowingly -- does it matter? I think it's really important to understand that breaking compat here should be a well thought off thing, especially as that's the differentiating factor for code that resides under the project vs. external repos. It doesn't mean that compat breaks can't happen, it's just that there would be more responsibility to providing a smooth upgrade path for users in case of compat breaks. >From my perspective, the code in the external repos here would be just like the code in the core repo, just with a different release cadence. Q: Would contribs be treated as first class citizens in the Solr Reference > Guide (they are still in the ASF after all), or would they be banished like > the DIH was? The repos are supposed to grow, and with that, adding more to the current ref guide would be just bad user experience. In addition, the different release cadence would make it difficult to support documentation for the code in these repos via the ref guide that would be released with the core. We should certainly aim for the same quality of documentation, but not make it to be a part of the ref guide. On Sat, Nov 14, 2020 at 8:54 PM David Smiley <[email protected]> wrote: > Thanks for shining a spotlight on this Mike. > I have some questions to consider. I'll call these additional repos, > "external contribs", or just contribs for short here; perhaps our internal > contribs would migrate. > > Q: Would each contrib be released at its own cadence unrelated to Solr? I > suppose so. > Q: Would each contrib have it's own release vote? I suppose so, as it has > its own artifact. I think the ASF requires this. > Q: Is it "okay" to release new Solr versions that break any of these > external contribs? Knowingly or unknowingly -- does it matter? > Q: What technical work is needed to extricate an internal contrib to an > external? > * source control history. (note: i've done this git history in a single > folder extraction before, with a popular Stackoverflow answer) > * mandatory ASF files, e.g. license, notice > * more files that we may want: CHANGES.txt > * More build files; copying the rules/setup/standards of the Solr > mothership and will become divergent over time no doubt. Or just KISS > principle; no sharing; simple Maven projects. > Q: Could & should many contribs live in one repo (no more internal > contribs), yet each still have its own release cycle? This could make > sharing build infrastructure easier, and detecting Solr compatibility with > them easier. Although it would mean sharing GitHub project area, thus > sharing issues/PRs. > Q: Should we create a separate JIRA for these contribs... or ditch JIRA > entirely for them, relying on GitHub alone? > Q: Would contribs be treated as first class citizens in the Solr Reference > Guide (they are still in the ASF after all), or would they be banished like > the DIH was? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Thu, Nov 12, 2020 at 6:40 PM Mike Drob <[email protected]> wrote: > >> Solr Devs, >> >> We've slowly been moving into a multi-repository model, and I wanted to >> bring some more attention to it and have a more focused discussion. We've >> recently embarked upon the acceptance of solr-operator as a distinct >> repo[1] under the care of the Lucene (soon to be Solr) PMC. I expect that >> there will be more cases of this as we transition additional contribs out >> of core, or as more plugins, packages, and integrations mature. Some will >> make sense as externally maintained code bases, but I believe other >> contributions may benefit our community more as part of the Apache >> Foundation. >> >> I think there was a very insightful comment[2] made by GP regarding >> adopting a similar model to Apache Commons governance, bringing attention >> to it here because I fear it may have gotten lost deep in the thread. Based >> on observations of Commons and a few other Apache projects with multi-repo >> setups, there thankfully does not appear to be a limit on how many >> repositories a PMC can maintain. The size and scope of each individual >> repository can vary greatly. I see potential ideas for anything that could >> be standalone and not tied to a release cycle (Admin UI, DIH, etc...), or >> anything that bridges integrations between Solr and other systems (k8s, >> HDFS, etc...). >> >> The risks that new repos face are similar to the risks they would have >> encountered as contrib modules, but I don't think they should dissuade us. >> Each project would need to start with a champion or sponsor and a >> discussion on the mailing list. From there, we can vote to accept the code, >> or just the idea if there is no code yet, as a community and create the >> repo. As part of a natural lifecycle, if there's not enough momentum or >> adoption over time, then we can update the README and docs and "retire" >> certain projects. The exact mechanisms can be undetermined for now; maybe >> it's a repo rename, maybe it's marking the repo read-only, maybe it's >> something else. >> >> The Commons model is that everyone is a committer on everything. There >> are other governance models, like Hadoop, with "area committers" who are >> limited to the specific repositories they have contributed frequently to. >> I'm not sure which model ultimately suits us better, but I think that >> leveraging area committers would allow us to recognize and empower >> contributors sooner and more frequently. Releases would still need to be >> voted on and approved by the singular PMC. >> >> There's no real action items here, it's more of a discussion prompt. If >> it looks like we have general consensus to this approach, then I'll start >> putting together individual proposals for a few repos to exercise the >> process and get more contributions going. I'll probably put the proposals >> together even if there's no replies here, but I'd much rather have some >> acknowledgement from the community that I'm headed in a sustainable >> direction! >> >> Mike >> >> [1]: >> https://lists.apache.org/thread.html/rb90f530155dc6edc6f1ccd5f056db1618142fdfcbd32d83f539d984b%40%3Cdev.lucene.apache.org%3E >> [2]: >> https://lists.apache.org/thread.html/r9965cb693369d927a942f805c134bfeb45c5e80f447ad0fe2f663fae%40%3Cdev.lucene.apache.org%3E >> >
