[
https://issues.apache.org/jira/browse/BEAM-7737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh Kumar updated BEAM-7737:
-------------------------------
Description:
microbenchmar scripts do not work consistently, at least run into problem for
me:
{code}
(beam) ➜ python git:(master) python -m
apache_beam.tools.distribution_counter_microbenchmark
Traceback (most recent call last):
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 72, in _run_code
exec code in run_globals
File
"/Users/rkumar/personal/opensource/beam/sdks/python/apache_beam/tools/distribution_counter_microbenchmark.py",
line 69, in <module>
'apache_beam.transforms.cy_dataflow_distribution_counter')
File "apache_beam/tools/utils.py", line 41, in check_compiled
"Profiling uncompiled code.\n"
RuntimeError: Profiling uncompiled code.
To compile beam, run 'pip install Cython; python setup.py build_ext --inplace'
{code}
Tihs is because [this
line|https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/sdks/python/apache_beam/tools/utils.py#L37]
of check_compiled method
doesn't work consistently. It every time just get the root module instead of
the submodule and fails to identify the the given moudle is compiled with
cython.
Also it is not recommended to use {{__import__}} method because it is python
internal and it is meant to be used by the Python interpreter.
{{importlib.import_module}} is the recommended way of finding the module.
was:
microbenchmar scripts do not work consistently, at least run into problem for
me:
{code}
(beam) ➜ python git:(master) python -m
apache_beam.tools.distribution_counter_microbenchmark
Traceback (most recent call last):
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 72, in _run_code
exec code in run_globals
File
"/Users/rkumar/personal/opensource/beam/sdks/python/apache_beam/tools/distribution_counter_microbenchmark.py",
line 69, in <module>
'apache_beam.transforms.cy_dataflow_distribution_counter')
File "apache_beam/tools/utils.py", line 41, in check_compiled
"Profiling uncompiled code.\n"
RuntimeError: Profiling uncompiled code.
To compile beam, run 'pip install Cython; python setup.py build_ext --inplace'
{code}
Tihs is because [this
line|https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/sdks/python/apache_beam/tools/utils.py#L37]
of check_compiled method
doesn't work consistently. It every time just get the root module instead of
the submodule and fails to identify the the given moudle is compiled with
cython.
Also it is not recommended to use {{__import__}} method because it is python
internal and it can change anytime. `importlib.import_module` is the
recommended way of finding the module.
> Microbenchmark script do not work consistently
> -----------------------------------------------
>
> Key: BEAM-7737
> URL: https://issues.apache.org/jira/browse/BEAM-7737
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Rakesh Kumar
> Assignee: Rakesh Kumar
> Priority: Major
>
> microbenchmar scripts do not work consistently, at least run into problem for
> me:
> {code}
> (beam) ➜ python git:(master) python -m
> apache_beam.tools.distribution_counter_microbenchmark
> Traceback (most recent call last):
> File
> "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
> line 174, in _run_module_as_main
> "__main__", fname, loader, pkg_name)
> File
> "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
> line 72, in _run_code
> exec code in run_globals
> File
> "/Users/rkumar/personal/opensource/beam/sdks/python/apache_beam/tools/distribution_counter_microbenchmark.py",
> line 69, in <module>
> 'apache_beam.transforms.cy_dataflow_distribution_counter')
> File "apache_beam/tools/utils.py", line 41, in check_compiled
> "Profiling uncompiled code.\n"
> RuntimeError: Profiling uncompiled code.
> To compile beam, run 'pip install Cython; python setup.py build_ext --inplace'
> {code}
> Tihs is because [this
> line|https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/sdks/python/apache_beam/tools/utils.py#L37]
> of check_compiled method
> doesn't work consistently. It every time just get the root module instead of
> the submodule and fails to identify the the given moudle is compiled with
> cython.
> Also it is not recommended to use {{__import__}} method because it is python
> internal and it is meant to be used by the Python interpreter.
> {{importlib.import_module}} is the recommended way of finding the module.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
