henriquejsza opened a new pull request, #1520: URL: https://github.com/apache/opennlp/pull/1520
Thank you for contributing to Apache OpenNLP. ### For all changes: - [x] Is there a GitHub issue associated with this PR? Is it referenced in the PR description (e.g. "Fixes #1234")? - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0? (no new dependencies) - [x] If applicable, have you updated the LICENSE file, including the main LICENSE file in opennlp folder? (not applicable) - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found in opennlp folder? (not applicable) Fixes #1342 `DependencyContextGenerator` joined the word-tag and position values of a combined feature with `/` and `|` without escaping them. A value holding one of these characters could make a combined feature of one configuration read the same as that of another, for example `s1t=X|Y, s0t=Z` and `s1t=X, s0t=Y|Z` both produced `s1t,s0t=X|Y|Z`. This escapes the escape character `\`, `/`, and `|` in every value that is joined into a combined feature, so each combined feature is read back unambiguously. Single-position features carry one value, so a separator there cannot split the feature and the value is left as is. Values without any of these characters are returned unchanged. The dependency parser is new in 3.0 and no models have shipped yet, so there is no released model to keep compatible. Universal POS tags never contain `/` or `|`, so the trained-model feature strings only change for tokens that contain one of these characters; the pinned `UniversalDependencyParserEval` scores should be re-run on a newly trained model. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
