[
https://issues.apache.org/jira/browse/BEAM-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-11826:
---------------------------------
Priority: P3 (was: P2)
> AttributeError: module 'apache_beam.coders.coders' has no attribute
> 'VarIntCoder'
> ---------------------------------------------------------------------------------
>
> Key: BEAM-11826
> URL: https://issues.apache.org/jira/browse/BEAM-11826
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.27.0
> Environment: OS: Ubuntu 18.04.5 LTS
> Python: 3.6.10
> GCC: 7.5.0
> Reporter: Lautaro Quiroz
> Priority: P3
> Labels: stale-P2
>
> Hi, I'm getting the following error when using apache-beam 2.27.0 in a
> Python3 virtualenv:
> {code:bash}
> airflow@airflow-worker-7fb797d459-nf8gh:~$
> /tmp/dataflow-venvtflya9ij/bin/python
> /home/airflow/gcs/dags/advisor/create_dataset/beam_pipelines/dummy.py
> Traceback (most recent call last):
> File
> "/home/airflow/gcs/dags/advisor/create_dataset/beam_pipelines/dummy.py", line
> 1, in <module>
> import apache_beam
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/__init__.py",
> line 95, in <module>
> from apache_beam import coders
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/coders/__init__.py",
> line 19, in <module>
> from apache_beam.coders.coders import *
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/coders/coders.py",
> line 43, in <module>
> from future.moves import pickle
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/future/moves/__init__.py",
> line 8, in <module>
> import_top_level_modules()
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/future/standard_library/__init__.py",
> line 810, in import_top_level_modules
> with exclude_local_folder_imports(*TOP_LEVEL_MODULES):
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/future/standard_library/__init__.py",
> line 781, in __enter__
> module = __import__(m, level=0)
> File
> "/home/airflow/gcs/dags/advisor/create_dataset/beam_pipelines/test.py", line
> 3, in <module>
> from apache_beam.options.pipeline_options import PipelineOptions
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/options/pipeline_options.py",
> line 41, in <module>
> from apache_beam.transforms.display import HasDisplayData
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/transforms/__init__.py",
> line 23, in <module>
> from apache_beam.transforms import combiners
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/transforms/combiners.py",
> line 45, in <module>
> from apache_beam.transforms import core
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/transforms/core.py",
> line 40, in <module>
> from apache_beam.coders import typecoders
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/coders/typecoders.py",
> line 198, in <module>
> registry = CoderRegistry()
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/coders/typecoders.py",
> line 91, in __init__
> self.register_standard_coders(fallback_coder)
> File
> "/tmp/dataflow-venvtflya9ij/lib/python3.6/site-packages/apache_beam/coders/typecoders.py",
> line 95, in register_standard_coders
> self._register_coder_internal(int, coders.VarIntCoder)
> AttributeError: module 'apache_beam.coders.coders' has no attribute
> 'VarIntCoder'
> {code}
> My `dummy.py` file consists of only:
> {code:python}
> import apache_beam
> if __name__ == '__main__': print('MAIN')
> {code}
> Strangely to me, I do not get the error when running the venv python
> interactively and executing the `import apache_beam` statement inside:
> {code:bash}
> airflow@airflow-worker-7fb797d459-nf8gh:~$
> /tmp/dataflow-venvtflya9ij/bin/python
> Python 3.6.10 (default, Feb 1 2021, 12:07:35)
> [GCC 7.5.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import apache_beam
> >>>
> {code}
> Note: In both cases the `sys.path` packages dirs are exactly the same.
> Equally strangely to me, the script runs in both scenarios (script &
> interactive) when I use the previous version `apache-beam==0.26.0`.
> ---
> A bit of context.
> I got to this error cause Im using a Google Cloud Composer (Airflow) operator
> ({{airflow.providers.google.cloud.operators.dataflow.DataflowCreatePythonJobOperator}})
> that launches the creation of a virtualenv & installs apache-beam, and
> executes the beam script in order the trigger run the Beam pipeline on Google
> Cloud DataFlow.
> I'm being able to reproduce these issue by connecting to the Kubernetes pod
> running this operator and manually executing the steps.
> —
> In order to reproduce this issue, you can:
> 1. create a python3 virtualenv: `virtualenv /tmp/venv --python=python3`.
> 2. create the dummy.py file with the following code inside:
> {code}
> import apache_beam
> {code}
> 3. install apache-beam 2.27.0: `/tmp/venv/bin/pip install
> apache-beam==2.27.0`.
> 4. run the script: `/tmp/venv/bin/python dummy.py`.
> 5. check it does not happen with `apache-beam==2.26.0`.
> —
> Also reported in:
> [https://stackoverflow.com/questions/66243327/inconsistent-behaviour-when-importing-a-package-interactively-vs-running-as-scri]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
