Rakesh Kumar created BEAM-7737:
----------------------------------
Summary: Microbenchmark script do not work consistently
Key: BEAM-7737
URL: https://issues.apache.org/jira/browse/BEAM-7737
Project: Beam
Issue Type: Improvement
Components: sdk-py-harness
Reporter: Rakesh Kumar
Assignee: Rakesh Kumar
microbenchmar scripts do not work consistently, at least run into problem for
me:
{code}
(beam) ➜ python git:(master) python -m
apache_beam.tools.distribution_counter_microbenchmark
Traceback (most recent call last):
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File
"/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 72, in _run_code
exec code in run_globals
File
"/Users/rkumar/personal/opensource/beam/sdks/python/apache_beam/tools/distribution_counter_microbenchmark.py",
line 69, in <module>
'apache_beam.transforms.cy_dataflow_distribution_counter')
File "apache_beam/tools/utils.py", line 41, in check_compiled
"Profiling uncompiled code.\n"
RuntimeError: Profiling uncompiled code.
To compile beam, run 'pip install Cython; python setup.py build_ext --inplace'
{code}
Tihs is because [this
line|https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/sdks/python/apache_beam/tools/utils.py#L37]
of check_compiled method
doesn't work consistently. It every time just get the root module instead of
the submodule and fails to identify the the given moudle is compiled with
cython.
Also it is not recommended to use {{__import__}} method because it is python
internal and it can change anytime. `importlib.import_module` is the
recommended way of finding the module.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
