[
https://issues.apache.org/jira/browse/DATALAB-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vira Vitanska updated DATALAB-2462:
-----------------------------------
Description:
*Preconditions:*
# Dataproc is created on Zeppelin
*Steps to reproduce:*
# Run playbook for Dataproc
{code:java}
Interpreter gcp not found{code}
*Actual result:*
# Paybook running fails:
*Expected result:*
# Playbook running is successful
----
2. Get rid of PY2 Intrepeter for Dataproc/Apache Standalone cluster
----
3. Every first creation Dataproc on Zeppelin fails
{code:java}
Traceback (most recent call last):
File "/usr/lib/python3.8/datalab/actions_lib.py", line 1311, in install_python
subprocess.run('{0} && sudo -i {1} install -U
pip==9.0.3'.format(venv_command, pip_command), shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'source
/opt/python/python3.8.1/bin/activate && sudo -i
/opt/python/python3.8.1/bin/pip3.8 install -U pip==9.0.3' returned non-zero
exit status 127.
Traceback (most recent call last):
File "/root/scripts/zeppelin_install_dataengine-service_kernels.py", line 79,
in <module>
conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export
gcp_project_id="{0}"; export conf_resource="{1}"; /usr/bin/python3
/usr/local/bin/create_configs.py --bucket {2} --cluster_name {3}
--dataproc_version {4} --spark_version {5} --hadoop_version {6} --region {7}
--user_name {8} --os_user {9} --pip_mirror {10} --application {11}
--livy_version {12} --multiple_clusters {13} --r_enabled {14}' '''
File "<decorator-gen-4>", line 2, in sudo
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 30,
in opens
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 737,
in sudo
return self._sudo(self._remote_runner(), command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213, in
_sudo
return runner.run(cmd_str, watchers=watchers, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 363, in
run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 422, in
_run_body
return self.make_promise() if self._asynchronous else self._finish()
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 489, in
_finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit co
{code}
4. It is impossible to switch between interpreters via Zeppelin WEB UI
was:
*Preconditions:*
# Dataproc is created on Zeppelin
*Steps to reproduce:*
# Run playbook for Dataproc
{code:java}
Interpreter gcp not found{code}
*Actual result:*
# Paybook running fails:
*Expected result:*
# Playbook running is successful
----
2. Get rid of PY2 Intrepeter for Dataproc/Apache Standalone cluster
----
3. Every first creation Dataproc on Zeppelin fails
{code:java}
Traceback (most recent call last):
File "/usr/lib/python3.8/datalab/actions_lib.py", line 1311, in install_python
subprocess.run('{0} && sudo -i {1} install -U
pip==9.0.3'.format(venv_command, pip_command), shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'source
/opt/python/python3.8.1/bin/activate && sudo -i
/opt/python/python3.8.1/bin/pip3.8 install -U pip==9.0.3' returned non-zero
exit status 127.
Traceback (most recent call last):
File "/root/scripts/zeppelin_install_dataengine-service_kernels.py", line 79,
in <module>
conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export
gcp_project_id="{0}"; export conf_resource="{1}"; /usr/bin/python3
/usr/local/bin/create_configs.py --bucket {2} --cluster_name {3}
--dataproc_version {4} --spark_version {5} --hadoop_version {6} --region {7}
--user_name {8} --os_user {9} --pip_mirror {10} --application {11}
--livy_version {12} --multiple_clusters {13} --r_enabled {14}' '''
File "<decorator-gen-4>", line 2, in sudo
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 30,
in opens
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 737,
in sudo
return self._sudo(self._remote_runner(), command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213, in
_sudo
return runner.run(cmd_str, watchers=watchers, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 363, in
run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 422, in
_run_body
return self.make_promise() if self._asynchronous else self._finish()
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 489, in
_finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit co
{code}
> [Zeppelin] Interpreter for Dataproc/Apache Standalone cluster is not found
> during playbook running
> --------------------------------------------------------------------------------------------------
>
> Key: DATALAB-2462
> URL: https://issues.apache.org/jira/browse/DATALAB-2462
> Project: Apache DataLab
> Issue Type: Bug
> Security Level: Public(Regular Issues)
> Components: DataLab Main
> Reporter: Vira Vitanska
> Assignee: Mykola Bodnar
> Priority: Major
> Labels: Debian, DevOps, GCP
> Fix For: v.2.5
>
> Attachments: Dataproc playbook running.png, Intrepeter.png,
> Zeppelin.png
>
>
> *Preconditions:*
> # Dataproc is created on Zeppelin
> *Steps to reproduce:*
> # Run playbook for Dataproc
> {code:java}
> Interpreter gcp not found{code}
> *Actual result:*
> # Paybook running fails:
> *Expected result:*
> # Playbook running is successful
> ----
> 2. Get rid of PY2 Intrepeter for Dataproc/Apache Standalone cluster
> ----
> 3. Every first creation Dataproc on Zeppelin fails
> {code:java}
> Traceback (most recent call last):
> File "/usr/lib/python3.8/datalab/actions_lib.py", line 1311, in
> install_python
> subprocess.run('{0} && sudo -i {1} install -U
> pip==9.0.3'.format(venv_command, pip_command), shell=True, check=True)
> File "/usr/lib/python3.8/subprocess.py", line 512, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command 'source
> /opt/python/python3.8.1/bin/activate && sudo -i
> /opt/python/python3.8.1/bin/pip3.8 install -U pip==9.0.3' returned non-zero
> exit status 127.
> Traceback (most recent call last):
> File "/root/scripts/zeppelin_install_dataengine-service_kernels.py", line
> 79, in <module>
> conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export
> gcp_project_id="{0}"; export conf_resource="{1}"; /usr/bin/python3
> /usr/local/bin/create_configs.py --bucket {2} --cluster_name {3}
> --dataproc_version {4} --spark_version {5} --hadoop_version {6} --region {7}
> --user_name {8} --os_user {9} --pip_mirror {10} --application {11}
> --livy_version {12} --multiple_clusters {13} --r_enabled {14}' '''
> File "<decorator-gen-4>", line 2, in sudo
> File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line
> 30, in opens
> return method(self, *args, **kwargs)
> File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line
> 737, in sudo
> return self._sudo(self._remote_runner(), command, **kwargs)
> File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213,
> in _sudo
> return runner.run(cmd_str, watchers=watchers, **kwargs)
> File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 363,
> in run
> return self._run_body(command, **kwargs)
> File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 422,
> in _run_body
> return self.make_promise() if self._asynchronous else self._finish()
> File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 489,
> in _finish
> raise UnexpectedExit(result)
> invoke.exceptions.UnexpectedExit: Encountered a bad command exit co
> {code}
> 4. It is impossible to switch between interpreters via Zeppelin WEB UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
