[
https://issues.apache.org/jira/browse/BEAM-8402?focusedWorklogId=376714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376714
]
ASF GitHub Bot logged work on BEAM-8402:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jan/20 05:50
Start Date: 24/Jan/20 05:50
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #9811: [BEAM-8402] Create a
class hierarchy to represent environments
URL: https://github.com/apache/beam/pull/9811#issuecomment-578000835
Beam 2.18 broke some users. Reverting back to 2.17 resolves the issues. I
believe this PR is the root cause. I am trying to get more information to file
a JIRA.
A stack trace I got looks like this:
```
File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line
339, in <module>
main()
File "/tfx-src/tfx/orchestration/kubeflow/container_entrypoint.py", line
332, in main
execution_info = launcher.launch()
File "/tfx-src/tfx/orchestration/launcher/base_component_launcher.py",
line 205, in launch
execution_decision.exec_properties)
File
"/tfx-src/tfx/orchestration/launcher/in_process_component_launcher.py", line
67, in _run_executor
executor.Do(input_dict, output_dict, exec_properties)
File "/tfx-src/tfx/components/example_gen/base_example_gen_executor.py",
line 230, in Do
artifact_utils.get_split_uri(output_dict['examples'], split_name)))
File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line
436, in __exit__
self.run().wait_until_finish()
File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line
416, in run
self._options).run(False)
File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line
429, in run
return self.runner.run_pipeline(self, self._options)
File
"/opt/venv/lib/python3.6/site-packages/apache_beam/runners/portability/fn_api_runner.py",
line 389, in run_pipeline
default_environment=self._default_environment))
File "/opt/venv/lib/python3.6/site-packages/apache_beam/pipeline.py", line
637, in to_runner_api
default_environment=default_environment)
File
"/opt/venv/lib/python3.6/site-packages/apache_beam/runners/pipeline_context.py",
line 134, in __init__
default_environment, label='default_environment')
File
"/opt/venv/lib/python3.6/site-packages/apache_beam/runners/pipeline_context.py",
line 66, in get_id
if obj not in self._obj_to_id:
TypeError: unhashable type: 'Environment'
```
I am suspecting this is related to the changes in the Environment class.
@violalyu @chadrik or other reviewers, do you know what this might be about?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 376714)
Time Spent: 5h 10m (was: 5h)
> Create a class hierarchy to represent environments
> --------------------------------------------------
>
> Key: BEAM-8402
> URL: https://issues.apache.org/jira/browse/BEAM-8402
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Chad Dombrova
> Assignee: Chad Dombrova
> Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> As a first step towards making it possible to assign different environments
> to sections of a pipeline, we first need to expose environment classes to the
> pipeline API. Unlike PTransforms, PCollections, Coders, and Windowings,
> environments exists solely in the portability framework as protobuf objects.
> By creating a hierarchy of "native" classes that represent the various
> environment types -- external, docker, process, etc -- users will be able to
> instantiate these and assign them to parts of the pipeline. The assignment
> portion will be covered in a follow-up issue/PR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
