TheNeuralBit commented on code in PR #29802:
URL: https://github.com/apache/beam/pull/29802#discussion_r1430527632
##########
sdks/python/setup.py:
##########
@@ -418,8 +418,13 @@ def get_portability_package_data():
],
'dataframe': dataframe_dependency,
'dask': [
- 'dask >= 2022.6',
- 'distributed >= 2022.6',
+ # FIXME(cisaacstern): The git+ link below is where
https://github.com/dask/distributed/pull/8400
+ # was merged into `distributed`. This PR is a fix for
https://github.com/apache/beam/issues/29365.
+ # Installing from here to move forward with development. Before
merge, this should be replaced with
+ # a lower bound release of `distributed`, once a release that
includes the linked PR is available.
+ # 'dask >= 2023.XX',
+ # 'distributed >= 2023.XX',
+ 'distributed @
git+https://github.com/dask/distributed.git@8c3eb6f0bf47d124c887c543599d80ff09c3f5ed',
Review Comment:
When you do update this we should consider adding an upper bound to protect
against breaking changes (e.g. I was seeing failures because
wait_for_workers(n_workers)` became a required parameter).
Unfortunately dask's use of CalVer isn't particularly conducive to this, as
the third number is just a unique id for a release in that month (e.g.
`2023.12.1` might be a bugfix release on `2023.12.0` or it could be the second
major release in December). I think we should just restrict on the month, even
though this might technically let a breaking change in. So something like
`>=2023.12,<2024.1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
