[
https://issues.apache.org/jira/browse/FLINK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036942#comment-17036942
]
Dian Fu commented on FLINK-16026:
---------------------------------
The management of Python dependencies is quite different from the Java
dependencies. Regarding to the Java dependencies, they are bundled in the
distribution and so it's fine to use any version which could work. However, for
the Python dependencies, they are not bundled in the distribution and we rely
on the Python dependencies provided in the execution environment for Python UDF
execution. The execution environment may be shared between many applications.
This means that it may not be a good idea to limit the version to a very short
range if possible. For example, if we limit avro-python3 to version <= 1.9.1,
what will happen if users want to share the Python environment between multiple
applications and another application requires a high version, e.g. 1.9.2? This
introduces unnecessary dependency problems for users as we limit the version
just for test stability instead because that version is not supported actually.
Regarding to the avro-python3 problem, it's not caused by API breaking in the
new version. It was unexpected and an error package was released just by
mistaken and avro community has already provided a quick fix to address this
problem.
> Travis failed due to python setup
> ---------------------------------
>
> Key: FLINK-16026
> URL: https://issues.apache.org/jira/browse/FLINK-16026
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Jingsong Lee
> Assignee: Huang Xingbo
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.10.1, 1.11.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> [https://api.travis-ci.com/v3/job/286671652/log.txt]
> [https://api.travis-ci.org/v3/job/649754603/log.txt]
> [https://api.travis-ci.com/v3/job/286409130/log.txt]
> Collecting avro-python3<2.0.0,>=1.8.1; python_version >= "3.0" (from
> apache-beam==2.19.0->apache-flink==1.11.dev0) Using cached
> https://files.pythonhosted.org/packages/31/21/d98e2515e5ca0337d7e747e8065227ee77faf5c817bbb74391899613178a/avro-python3-1.9.2.tar.gz
> Complete output from command python setup.py egg_info: Traceback (most
> recent call last): File "<string>", line 1, in <module> File
> "/tmp/pip-install-d6uvsl_b/avro-python3/setup.py", line 41, in <module>
> import pycodestyle ModuleNotFoundError: No module named 'pycodestyle'
> ---------------------------------------- Command "python setup.py egg_info"
> failed with error code 1 in /tmp/pip-install-d6uvsl_b/avro-python3/ You are
> using pip version 10.0.1, however version 20.0.2 is available. You should
> consider upgrading via the 'pip install --upgrade pip' command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
