eddie-scio commented on PR #23870: URL: https://github.com/apache/beam/pull/23870#issuecomment-1402451178
Really appreciate the transparency! Also x-linking the issue: https://github.com/apache/beam/issues/22893 Here's our dependency chain: 1. We want to upgrade to Spacy V3 to leverage their new v3 models. 2. We also use `allennlp==2.9.3`, which has `spacy<3.3,>=2.1.0`. 4. There's [a bug](https://github.com/explosion/spaCy/issues/9858) leading to large latency issues in Spacy V3. The fix was only released in `spacy==3.3.0`. 5. `allennlp==2.10.0` has `spacy>=2.1.0,<3.4` which gets around (4), but it also has `dill>=0.3.4`, which leads us here, since Beam still has `dill<0.3.2`. Since `allennlp` is now in maintenance mode, we're unlikely to see any changes there, so the only other avenue I see is to remove our `allennlp` dependency fully, which is quite high lift on our end. And since we saw this `dill` version cap from Beam was pretty old, we figured you guys were going to work on this anyways soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
