Thanks, Richard, for starting this discussion. When deciding if something belongs in Addons (or Sandbox) or Core, is expected usage a factor? You mentioned later down that if addons sees real adoption, it can be moved to Core (which makes sense). But does that imply that if we think a new feature might not have wide-spread adoption early on it should go into the Addons?
I strongly agree that Addons should not mean "graveyard". In that spirit, we could probably do a better job promoting the add-ons on the website and within OpenNLP itself. I think I remember Lucene renaming its sandbox to "contrib" and making it a part of the main repository. I don't know if the rename is applicable ehere, but combining them into a single repository could help determine where code should go and also improve interoperability. But this is a discussion for later on. In my 17-some years around OpenNLP, this is a new problem for the project, but it's one of those good problems we just need to work through. Thanks, Jeff On Tue, Aug 25, 2026 at 3:11 AM Richard Zowalla <[email protected]> wrote: > > Hi all, > > we currently have ~20 open PRs, most of them contributing substantial new > functionality: stemmers, subword tokenization, embeddings and vector > indexes, a gazetteer/geocoder, a lexical knowledge base, a dependency > parser, and more. First of all: this is a great problem to have. > > The breadth of this work would move OpenNLP much closer to what people > expect from the Python NLP ecosystem, and I'd like us to find a home > for as much of it as possible. > > That said, I don't think all of it belongs in core, and we've never > written down where the line is. Each PR currently triggers the same > case-by-case debate, which is slow for us and frustrating for > contributors. I'd like us to agree on a simple test and make it a > documented rule. > > My proposal: > > A contribution belongs in *core* if it: > - fits our existing abstractions / component model, > - introduces no new heavy runtime dependencies, and > - bundles no data files (models, dictionaries, datasets). > > A contribution belongs in *addons* (or sandbox) if it: > - brings its own dependencies, > - bundles or downloads data, or > - introduces a whole new component family that stands on its own. > > Applying that test to the current queue, as a starting point for > discussion: > > Core (fits existing abstractions, no new deps/data): > - OPENNLP-1903 BeamSearch optimisation (#1205) > - OPENNLP-1886 UniNE stemmer tiers (#1166) > - OPENNLP-1893 Hunspell affix stemming (#1190) > - smaller fixes like OPENNLP-1916 (#1235) > > Addons (own data and/or new component families): > - OPENNLP-1879 Gazetteer/geocoder with bundled Natural Earth data > (#1154) > - the embedding + vector index stack: OPENNLP-1877, -1895, -1910, > -1911, -1897 (#1152, #1213, #1214, #1215, #1212) > - the lexical knowledge base / WordNet work: OPENNLP-1880, -1887 > (#1155, #1167) > > Needs discussion (borderline under the test above): > - Document annotation layers: OPENNLP-1888, -1919, -1920 (#1182, > #1237, #1238) (my POV is: would love to have it) > - Dependency parser: OPENNLP-547 (#1236) > - SentencePiece inference: OPENNLP-1885 (#1165) > - CJK dictionary tokenization: OPENNLP-1894 (#1191) > - Verified resource installer: OPENNLP-1909 (#1211) > > Two important points that go with this: > > 1. "Addons" must not mean "graveyard". Redirecting work there is only > good if we actually release addons on a predictable cadence. So > part of this proposal is that we commit to regular addons releases > (and probably clean up the addons repo a bit first). I'd rather we > agree on that commitment here than quietly park stuff. > > 2. The line must be permeable IMHO. If an addons module sees > real adoption (downloads, traffic, user questions), that's exactly > the signal to promote it into core later. Starting in addons is not > a rejection, it's an incubation path with a much lower review bar > and a faster way to get code into users' hands. > > To be clear: this is not about declining contributions. It's about > matching our review capacity (which is honestly the bottleneck right > now - a handful of us cannot review 200k added lines in core in a > reasonable time frame) to a structure where good work lands somewhere > visible and released, instead of sitting in the queue for months. > > If we get rough consensus on the test itself, I'd follow up with: > - a short section in CONTRIBUTING.md documenting the rule, > - triaging the open PRs accordingly together with the authors, > - a plan (separate thread) for the addons release cadence. > > Looking forward to your thoughts - including, explicitly, from the > authors of the PRs above. > > Gruß > Richard
