yuvipanda commented on issue #22349: URL: https://github.com/apache/beam/issues/22349#issuecomment-1190927047
Happy to get on a video call too :) What TZ are you in? The point isn't to use *beam* in an interactive context, but to use the libraries that are going to be used in the python code that is being run by beam in an interactive context. For example, https://github.com/pangeo-forge/staged-recipes/blob/master/recipes/cmip6/recipe.py will eventually be run on beam, and it uses https://github.com/pangeo-forge/pangeo-forge-recipes - which in turn uses libraries like xarray, zarr etc. So the goal is to test *that*. And these aren't being used in local Jupyter environments, but *cloud hosted environments* that are running on kubernetes (via z2jh.jupyter.org, or tools like mybinder.org). See the 'pangeo forge sandbox' here - https://pangeo-forge.readthedocs.io/en/latest/introduction_tutorial/index.html. It's running inside a ephemeral docker container in mybinder.org, which is based on JupyterHub. > Why not just install beam with pip / conda, and let users use the local runner? There is ongoing work on that too! The goal is to support three different contexts: 1. Users' local machines (via conda metapackages or similar that provide pinned versions - not implemented yet) 2. On cloud hosted interactive Jupyter environment, inside Docker / k8s 3. When called by Beam in dataflow. I hope this helps! @cisaacstern probably has more details than I do! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
