[
https://issues.apache.org/jira/browse/OPENNLP-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kristian Rickert updated OPENNLP-1889:
--------------------------------------
Description:
A proof-of-concept modernization of OpenNLP around an immutable,
offset-anchored {{Document}} model, plus a set of pure-JVM analysis
capabilities built on top of it. It was developed as one integrated branch and
is being split into small,
independently reviewable pieces. The goal is a single-jar, no-Python,
span-faithful NLP stack that we can expose over gRPC as a serious alternative
to the popular Python packages.
h2. Rollout policy (applies to every ticket in this epic)
- Each feature is developed on its own branch and is *not* pushed to apache
until it
is ready for collaboration and review. These drafts exist so we can cut
branches in
our own environment first. I'd love to share, so just reply and I can make a
branch for collaboration and open to any changes along the way. I'd love the
help.
- *Only the document shape (OPENNLP-1888) is considered necessary for 3.0.*
Every other ticket here can land after 3.0 without blocking the release.
- As each feature lands, we add support for it to the gRPC service, so the
capability is exposed over the wire as it becomes available.
- Every ticket and feature will have Javadoc, unit tests, eval tests, manual
entries, and samples that are runnable. No exceptions.
- Any test results will me runnable and repeatable. We will focus most on
industry standard tests. Java is fast; I have full confidence we will have
even more impressive results than what you see below.
- All caveats will be documented (i.e. Java version of Model2Vec does lower
accuracy - at times too much - for 20x speedup.)
h2. PoC results (measured this cycle)
||Area||Result||Reference / target||
|Static embeddings (model2vec / potion-base-8M), pure-JVM inference|12.9x
faster single-thread, ~7x peak throughput multi-thread, at 0.22x memory|vs the
Python reference; vector parity verified *before* optimizing|
|Dependency parser (pure-JVM feedforward neural, trained in-process)|86.78 UAS
/ 84.61 LAS with gold tags (h400, beam 4)|UD English EWT test|
|Full all-neural pipeline (our tagger -> our parser)|83.73 UAS / 79.88 LAS at
448 tokens/s|Stanza end-to-end 88.90 / 86.77|
|Feedforward POS tagger|94.68%|classical maxent ceiling 93.75%|
|CJK segmentation (mecab-format Viterbi lattice, char-trie)|5.1M chars/s, ~146k
sentences/s; loads 392k IPADIC entries in ~0.7s|real IPADIC; one engine covers
JA + KO|
|UD lemmatizer (via the installer)|87.76% lemma accuracy (~22s train, default
params)|UD English EWT|
|Normalization / tokenization hot paths (OPENNLP-1878, #1161)|~2x on the
measured hot paths, non-breaking|already in review|
h2. Training and data provenance (legality)
- *We do our own training.* The feedforward parser and tagger are trained
in-process
in the JVM from open treebanks; the embedding tables are verified for parity
in-JVM. *No third-party model binaries are shipped.* We would need to find a
home for the newly trained models.
- *Nothing third-party is bundled.* A bring-your-own-data installer takes a
user-supplied URL, verifies SHA-256 before unpacking, and rejects path escapes.
This is how mecab dictionaries, UD treebanks, Hunspell dictionaries,
gazetteers, and Who's On First tables are acquired: the user points at the
source and thereby accepts its license. (Cleared via LEGAL-732)
- *Geographic data:* the only bundled gazetteer is Natural Earth populated
places
(public domain). GeoNames (CC-BY) and Overture are download-only. Who's On
First is BYOD-only (a large multi-source license patchwork). Census/IRS-derived
place profiles are gated behind legal review and never bundled: the engine
ships, the data does not.
- *Benchmark data:* UD English EWT is CC-BY-SA and used for measurement only;
we do not ship models derived from it.
I'm cleaning up the code now - but I'd be glad to put them as branches but it's
still in PoC and the stacked dependency makes it unrealistic and very
confusing. Once OPENNLP-1888 is committed, the complexity of the merge order
goes down and most tickets can land off main.
h3. The coolest part -
I try not to hype or get too opinionated, especially in a Jira ticket, but I'm
thrilled to work with anyone welcome on this. To say the least - Java is a
great language and initial numbers suggest that we'll gain a lot of customers
should we land these features. This opportunity is huge.
was:
A proof-of-concept modernization of OpenNLP around an immutable, offset-anchored
{{Document}} model, plus a set of pure-JVM analysis capabilities built on top of
it. It was developed as one integrated branch and is being split into small,
independently reviewable pieces. The goal is a single-jar, no-Python,
span-faithful
NLP stack that we can expose over gRPC as a serious alternative to the popular
Python packages.
h2. Rollout policy (applies to every ticket in this epic)
- Each feature is developed on its own branch and is *not* pushed to apache
until it
is ready for collaboration and review. These drafts exist so we can cut
branches in
our own environment first. I'd love to share, so just reply and I can make a
branch for collaboration and open to any changes along the way. I'd love the
help.
- *Only the document shape (OPENNLP-1888) is considered necessary for 3.0.*
Every other ticket here can land after 3.0 without blocking the release.
- As each feature lands, we add support for it to the gRPC service, so the
capability is exposed over the wire as it becomes available.
- Every ticket and feature will have Javadoc, unit tests, eval tests, manual
entries, and samples that are runnable. No exceptions.
- Any test results will me runnable and repeatable. We will focus most on
industry standard tests. Java is fast; I have full confidence we will have
even more impressive results than what you see below.
- All caveats will be documented (i.e. Java version of Model2Vec does lower
accuracy - at times too much - for 20x speedup.)
h2. PoC results (measured this cycle)
||Area||Result||Reference / target||
|Static embeddings (model2vec / potion-base-8M), pure-JVM inference|12.9x
faster single-thread, ~7x peak throughput multi-thread, at 0.22x memory|vs the
Python reference; vector parity verified *before* optimizing|
|Dependency parser (pure-JVM feedforward neural, trained in-process)|86.78 UAS
/ 84.61 LAS with gold tags (h400, beam 4)|UD English EWT test|
|Full all-neural pipeline (our tagger -> our parser)|83.73 UAS / 79.88 LAS at
448 tokens/s|Stanza end-to-end 88.90 / 86.77|
|Feedforward POS tagger|94.68%|classical maxent ceiling 93.75%|
|CJK segmentation (mecab-format Viterbi lattice, char-trie)|5.1M chars/s, ~146k
sentences/s; loads 392k IPADIC entries in ~0.7s|real IPADIC; one engine covers
JA + KO|
|UD lemmatizer (via the installer)|87.76% lemma accuracy (~22s train, default
params)|UD English EWT|
|Normalization / tokenization hot paths (OPENNLP-1878, #1161)|~2x on the
measured hot paths, non-breaking|already in review|
h2. Training and data provenance (legality)
- *We do our own training.* The feedforward parser and tagger are trained
in-process
in the JVM from open treebanks; the embedding tables are verified for parity
in-JVM. *No third-party model binaries are shipped.* We would need to find a
home for the newly trained models.
- *Nothing third-party is bundled.* A bring-your-own-data installer takes a
user-supplied URL, verifies SHA-256 before unpacking, and rejects path escapes.
This is how mecab dictionaries, UD treebanks, Hunspell dictionaries,
gazetteers, and Who's On First tables are acquired: the user points at the
source and thereby accepts its license. (Cleared via LEGAL-732)
- *Geographic data:* the only bundled gazetteer is Natural Earth populated
places
(public domain). GeoNames (CC-BY) and Overture are download-only. Who's On
First is BYOD-only (a large multi-source license patchwork). Census/IRS-derived
place profiles are gated behind legal review and never bundled: the engine
ships, the data does not.
- *Benchmark data:* UD English EWT is CC-BY-SA and used for measurement only;
we do not ship models derived from it.
I'm cleaning up the code now - but I'd be glad to put them as branches but it's
still in PoC and the stacked dependency makes it unrealistic and very
confusing. Once OPENNLP-1888 is committed, the complexity of the merge order
goes down and most tickets can land off main.
h3. The coolest part -
I try not to hype or get too opinionated, especially in a Jira ticket, but I'm
thrilled to work with anyone welcome on this. To say the least - Java is a
great language and initial numbers suggest that we'll gain a lot of customers
should we land these features. This opportunity is huge.
> OpenNLP modernization: document-shape pipeline and pure-JVM PoC feature set
> ---------------------------------------------------------------------------
>
> Key: OPENNLP-1889
> URL: https://issues.apache.org/jira/browse/OPENNLP-1889
> Project: OpenNLP
> Issue Type: Epic
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 3.0.0
>
>
> A proof-of-concept modernization of OpenNLP around an immutable,
> offset-anchored {{Document}} model, plus a set of pure-JVM analysis
> capabilities built on top of it. It was developed as one integrated branch
> and is being split into small,
> independently reviewable pieces. The goal is a single-jar, no-Python,
> span-faithful NLP stack that we can expose over gRPC as a serious alternative
> to the popular Python packages.
> h2. Rollout policy (applies to every ticket in this epic)
> - Each feature is developed on its own branch and is *not* pushed to apache
> until it
> is ready for collaboration and review. These drafts exist so we can cut
> branches in
> our own environment first. I'd love to share, so just reply and I can make a
> branch for collaboration and open to any changes along the way. I'd love the
> help.
> - *Only the document shape (OPENNLP-1888) is considered necessary for 3.0.*
> Every other ticket here can land after 3.0 without blocking the release.
> - As each feature lands, we add support for it to the gRPC service, so the
> capability is exposed over the wire as it becomes available.
> - Every ticket and feature will have Javadoc, unit tests, eval tests, manual
> entries, and samples that are runnable. No exceptions.
> - Any test results will me runnable and repeatable. We will focus most on
> industry standard tests. Java is fast; I have full confidence we will have
> even more impressive results than what you see below.
> - All caveats will be documented (i.e. Java version of Model2Vec does lower
> accuracy - at times too much - for 20x speedup.)
> h2. PoC results (measured this cycle)
> ||Area||Result||Reference / target||
> |Static embeddings (model2vec / potion-base-8M), pure-JVM inference|12.9x
> faster single-thread, ~7x peak throughput multi-thread, at 0.22x memory|vs
> the Python reference; vector parity verified *before* optimizing|
> |Dependency parser (pure-JVM feedforward neural, trained in-process)|86.78
> UAS / 84.61 LAS with gold tags (h400, beam 4)|UD English EWT test|
> |Full all-neural pipeline (our tagger -> our parser)|83.73 UAS / 79.88 LAS at
> 448 tokens/s|Stanza end-to-end 88.90 / 86.77|
> |Feedforward POS tagger|94.68%|classical maxent ceiling 93.75%|
> |CJK segmentation (mecab-format Viterbi lattice, char-trie)|5.1M chars/s,
> ~146k sentences/s; loads 392k IPADIC entries in ~0.7s|real IPADIC; one engine
> covers JA + KO|
> |UD lemmatizer (via the installer)|87.76% lemma accuracy (~22s train, default
> params)|UD English EWT|
> |Normalization / tokenization hot paths (OPENNLP-1878, #1161)|~2x on the
> measured hot paths, non-breaking|already in review|
> h2. Training and data provenance (legality)
> - *We do our own training.* The feedforward parser and tagger are trained
> in-process
> in the JVM from open treebanks; the embedding tables are verified for parity
> in-JVM. *No third-party model binaries are shipped.* We would need to find a
> home for the newly trained models.
> - *Nothing third-party is bundled.* A bring-your-own-data installer takes a
> user-supplied URL, verifies SHA-256 before unpacking, and rejects path
> escapes. This is how mecab dictionaries, UD treebanks, Hunspell dictionaries,
> gazetteers, and Who's On First tables are acquired: the user points at the
> source and thereby accepts its license. (Cleared via LEGAL-732)
> - *Geographic data:* the only bundled gazetteer is Natural Earth populated
> places
> (public domain). GeoNames (CC-BY) and Overture are download-only. Who's On
> First is BYOD-only (a large multi-source license patchwork).
> Census/IRS-derived place profiles are gated behind legal review and never
> bundled: the engine ships, the data does not.
> - *Benchmark data:* UD English EWT is CC-BY-SA and used for measurement
> only; we do not ship models derived from it.
>
> I'm cleaning up the code now - but I'd be glad to put them as branches but
> it's still in PoC and the stacked dependency makes it unrealistic and very
> confusing. Once OPENNLP-1888 is committed, the complexity of the merge order
> goes down and most tickets can land off main.
> h3. The coolest part -
> I try not to hype or get too opinionated, especially in a Jira ticket, but
> I'm thrilled to work with anyone welcome on this. To say the least - Java is
> a great language and initial numbers suggest that we'll gain a lot of
> customers should we land these features. This opportunity is huge.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
