[
https://issues.apache.org/jira/browse/DATALAB-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448067#comment-17448067
]
Leonid Frolov commented on DATALAB-2607:
----------------------------------------
fixed request_id {color:#172b4d}error, but got new error
{color}
usage: create_configs.py [-h] [--bucket BUCKET] [--cluster_name CLUSTER_NAME]
[--dry_run DRY_RUN]
[--dataproc_version DATAPROC_VERSION]
[--spark_version SPARK_VERSION]
[--hadoop_version HADOOP_VERSION] [--region REGION]
[--user_name USER_NAME] [--os_user OS_USER]
[--pip_mirror PIP_MIRROR] [--application APPLICATION]
create_configs.py: error: argument --pip_mirror: expected one argument
sudo -S -p '[sudo] password: ' \cp /tmp/create_configs.py
/usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' chmod 755 /usr/local/bin/create_configs.py
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib/python3.8/datalab/
mkdir -p /home/datalab-user/datalab_libs/
scp -i /root/keys/DLab-Dev-key.pem /usr/lib/python3.8/datalab/*.py
[email protected]:/home/datalab-user/datalab_libs/
chmod a+x /home/datalab-user/datalab_libs/*
sudo -S -p '[sudo] password: ' mv /home/datalab-user/datalab_libs/*
/usr/lib/python3.8/datalab/
sudo -S -p '[sudo] password: ' rm -rf /home/datalab-user/datalab_libs/
sudo -S -p '[sudo] password: ' mkdir -p /usr/lib64/python3.8
sudo -S -p '[sudo] password: ' ln -fs /usr/lib/python3.8/datalab
/usr/lib64/python3.8/datalab
sudo -S -p '[sudo] password: ' bash -l -c 'echo "[global]" > /etc/pip.conf;
echo "proxy = $(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=)" >>
/etc/pip.conf'
sudo -S -p '[sudo] password: ' bash -l -c 'echo "use_proxy=yes" > ~/.wgetrc;
proxy=$(cat /etc/profile | grep proxy | head -n1 | cut -f2 -d=); echo
"http_proxy=$proxy" >> ~/.wgetrc; echo "https_proxy=$proxy" >> ~/.wgetrc'
sudo -S -p '[sudo] password: ' bash -l -c 'unset http_proxy https_proxy; export
gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export
conf_resource="dataengine-service"; /usr/bin/python3
/usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket
--cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version
2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region
us-west1 --user_name proj1 --os_user datalab-user --pip_mirror --application
rstudio'
Traceback (most recent call last):
File "/root/scripts/rstudio_install_dataengine-service_kernels.py", line 75,
in <module>
conn.sudo('''bash -l -c 'unset http_proxy https_proxy; export
gcp_project_id="\{0}"; export conf_resource="\{1}"; /usr/bin/python3
/usr/local/bin/create_configs.py --bucket \{2} --cluster_name \{3}
--dataproc_version \{4} --spark_version \{5} --hadoop_version \{6} --region
\{7} --user_name \{8} --os_user \{9} --pip_mirror \{10} --application \{11}' '''
File "<decorator-gen-4>", line 2, in sudo
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 30,
in opens
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/fabric/connection.py", line 737,
in sudo
return self._sudo(self._remote_runner(), command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/context.py", line 213, in
_sudo
return runner.run(cmd_str, watchers=watchers, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 380, in
run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 442, in
_run_body
return self.make_promise() if self._asynchronous else self._finish()
File "/usr/local/lib/python3.8/dist-packages/invoke/runners.py", line 509, in
_finish
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
Command: 'sudo -S -p \'[sudo] password: \' bash -l -c \'unset http_proxy
https_proxy; export gcp_project_id="or2-msq-epmc-dlab-t1iylu"; export
conf_resource="dataengine-service"; /usr/bin/python3
/usr/local/bin/create_configs.py --bucket lfr-gc-23-11-proj1-local-bucket
--cluster_name lfr-gc-23-11-proj1-local-des-dp1 --dataproc_version
2.0.0-RC22-ubuntu18 --spark_version 3.0.1 --hadoop_version 3.2.1 --region
us-west1 --user_name proj1 --os_user datalab-user --pip_mirror --application
rstudio\' '
Exit code: 2
> [GCP] Data Engine Service (Dataproc) creation fails on RStudio/Zeppelin
> -----------------------------------------------------------------------
>
> Key: DATALAB-2607
> URL: https://issues.apache.org/jira/browse/DATALAB-2607
> Project: Apache DataLab
> Issue Type: Bug
> Security Level: Public(Regular Issues)
> Components: DataLab Main
> Affects Versions: v.2.5.1
> Reporter: Vira Vitanska
> Assignee: Leonid Frolov
> Priority: Major
> Labels: Debian, DevOps, GCP, pull-request-available
> Attachments: GCP Dataproc on RStudio Zeppelin .png
>
>
> *Preconditions:*
> # RStudio/Zeppelin is in running status
> *Steps to reproduce:*
> # Create Data Engine Service (Dataproc) on RStudio/Zeppelin
> *Actual result:*
> # Data Engine Service creation fails
> {code:java}
> [chan 1] Opened sftp connection (server version 3)
> Warning: Permanently added '172.31.16.4' (ECDSA) to the list of known hosts.
> Traceback (most recent call last):
> File "/usr/local/bin/create_configs.py", line 26, in <module>
> from datalab.actions_lib import *
> File "/usr/lib/python3.8/datalab/actions_lib.py", line 24, in <module>
> import datalab.common_lib
> File "/usr/lib/python3.8/datalab/common_lib.py", line 27, in <module>
> from datalab.logger import logging
> File "/usr/lib/python3.8/datalab/logger.py", line 32, in <module>
> local_log_filename = "{}_{}.log".format(os.environ['conf_resource'],
> os.environ['request_id'])
> File "/usr/lib/python3.8/os.py", line 675, in __getitem__
> raise KeyError(key) from None
> KeyError: 'request_id'
> {code}
> *Expected result:*
> # Data Engine Service creation is successful
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
