[
https://issues.apache.org/jira/browse/DATALAB-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vira Vitanska updated DATALAB-1365:
-----------------------------------
Description:
# -Retrieve available lib list group -error -ModuleNotFoundError: No module
named 'xmlrpclib'- -{color:#00875a} fixed 04/02/2021.{color}
# -Get rid of Python2 during SSN deployment, Notebook creation.-
*SSN:*
{code:java}
Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y
install python3-pip python-dev groff vim less git wget nano libssl-dev
libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/*
/var/tmp/*
---> Running in e349ec65c115 fixed 26/03/2021{code}
(i) {color:#00875a}Fixed{color} in '-tep 4/28 : RUN apt-get update &&'-
(03/02/2021).
{code:java}
sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build
--build-arg OS=debian --build-arg SRC_PATH= --file
general/files/gcp/base_Dockerfile -t docker.datalab-base:latest fixed 26/03/2021
{code}
(i) {color:#00875a}Fixed{color} in -'sudo: cd /opt/datalab/sources/'-
(03/02/2021).
{code:java}
sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi fixed 26/03/2021{code}
(i) {color:#00875a}Fixed{color} -in 'apt-get -y install supervisor-'
(03/02/2021).
(-) {color:#57d9a3}Done{color} -Script for Keycloak client creation should use
Python3.-
*{color:#57d9a3}Done{color} -Jupyter-:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep
-w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021{code}
*{color:#57d9a3}Done{color} -RStudio-:*
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
(i) -Py2 is still available in 'apt-get -y install nodejs'- -
{color:#00875a}fixed{color} (26/032021).
*{color:#57d9a3}Done{color} -Zeppelin-:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep
-w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
(i) -Py2 Is still available in 'apt-get -y install nodejs-'
{color:#00875a}fixed{color} (26/032021).
*3.* -Performance for SSN deployment. It has to take up to 40-50 minutes-. But
now it takes 2 hours. *{color:#00875a}Fixed{color}, now ~40m*
*4.1.* -It's impossible to retrieve available lib list for any group. Docker
runs with '1'- (26/03/2021): {color:#00875a}F{color}*{color:#00875a}ixed{color}
30.03.2021*
{code:java}
No idea what 'list_libs' is!
Traceback (most recent call last):
File "/root/entrypoint.py", line 154, in <module>
subprocess.run("/bin/list_libs.py", shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/bin/list_libs.py' returned non-zero
exit status 1.
{code}
*4.2.* Available lib list retrieving:
+RStudio:+
||Apt/Yum|Still in process: is too long, > 50 min.|
||Py3|Passed|
||R packages|Docker runs with 1, Error: package or namespace load failed for
‘SparkR’:
package ‘SparkR’ was installed before R 4.0.0: please re-install it|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Still in process: is too long, > 50 min. |
||Py3| Passed|
||R packages| Passed|
||Others| Passed|
*5.* Lib installation:
+RStudio:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
installed|
||Py3|Passed|
||R packages|Failed by lib list retrieving|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
installed|
||Py3|Passed|
||R packages|Passed|
||Java dependency|Passed|
||Others|Passed|
*6.* Get rid of kernel python2 for notebooks.
*7.* Docker of image creation runs with '1': *fixed 30.03.2021*
{code:java}
No idea what 'create_image' is!
Traceback (most recent call last):
File "/root/entrypoint.py", line 160, in <module>
subprocess.run("/bin/create_image.py", shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/bin/create_image.py' returned non-zero
exit status 1.
{code}
was:
# -Retrieve available lib list group -error -ModuleNotFoundError: No module
named 'xmlrpclib'- -{color:#00875a} fixed 04/02/2021.{color}
# Get rid of Python2 during SSN deployment, Notebook creation.
*SSN:*
{code:java}
Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y
install python3-pip python-dev groff vim less git wget nano libssl-dev
libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/*
/var/tmp/*
---> Running in e349ec65c115 fixed 26/03/2021{code}
(i) {color:#00875a}Fixed{color} in '-tep 4/28 : RUN apt-get update &&'-
(03/02/2021).
{code:java}
sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build
--build-arg OS=debian --build-arg SRC_PATH= --file
general/files/gcp/base_Dockerfile -t docker.datalab-base:latest fixed 26/03/2021
{code}
(i) {color:#00875a}Fixed{color} in -'sudo: cd /opt/datalab/sources/'-
(03/02/2021).
{code:java}
sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi fixed 26/03/2021{code}
(i) {color:#00875a}Fixed{color} -in 'apt-get -y install supervisor-'
(03/02/2021).
(-) {color:#57d9a3}Done{color} -Script for Keycloak client creation should use
Python3.-
*{color:#57d9a3}Done{color} -Jupyter-:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep
-w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021{code}
*{color:#57d9a3}Done{color} -RStudio-:*
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
(i) -Py2 is still available in 'apt-get -y install nodejs'- -
{color:#00875a}fixed{color} (26/032021).
*{color:#57d9a3}Done{color} -Zeppelin-:*
{code:java}
sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if ! grep
-w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
{code:java}
sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
"(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
/tmp/apt.log;fi - fixed 26/03/2021
{code}
(i) -Py2 Is still available in 'apt-get -y install nodejs-'
{color:#00875a}fixed{color} (26/032021).
*3.* -Performance for SSN deployment. It has to take up to 40-50 minutes-. But
now it takes 2 hours. *{color:#00875a}Fixed{color}, now ~40m*
*4.1.* -It's impossible to retrieve available lib list for any group. Docker
runs with '1'- (26/03/2021): *fixed 30.03.2021*
{code:java}
No idea what 'list_libs' is!
Traceback (most recent call last):
File "/root/entrypoint.py", line 154, in <module>
subprocess.run("/bin/list_libs.py", shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/bin/list_libs.py' returned non-zero
exit status 1.
{code}
*4.2.* Available lib list retrieving:
+RStudio:+
||Apt/Yum|Still in process: is too long, > 50 min.|
||Py3|Passed|
||R packages|Docker runs with 1, Error: package or namespace load failed for
‘SparkR’:
package ‘SparkR’ was installed before R 4.0.0: please re-install it|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Still in process: is too long, > 50 min. |
||Py3| Passed|
||R packages| Passed|
||Others| Passed|
*5.* Lib installation:
+RStudio:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
installed|
||Py3|Passed|
||R packages|Failed by lib list retrieving|
||Others|Passed|
+Zeppelin:+
||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
installed|
||Py3|Passed|
||R packages|Passed|
||Java dependency|Passed|
||Others|Passed|
*6.* Get rid of kernel python2 for notebooks.
*7.* Docker of image creation runs with '1': *fixed 30.03.2021*
{code:java}
No idea what 'create_image' is!
Traceback (most recent call last):
File "/root/entrypoint.py", line 160, in <module>
subprocess.run("/bin/create_image.py", shell=True, check=True)
File "/usr/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/bin/create_image.py' returned non-zero
exit status 1.
{code}
> Migrate to Python3
> ------------------
>
> Key: DATALAB-1365
> URL: https://issues.apache.org/jira/browse/DATALAB-1365
> Project: Apache DataLab
> Issue Type: Task
> Components: DataLab Main
> Reporter: Vira Vitanska
> Assignee: Leonid Frolov
> Priority: Major
> Labels: AWS, AZURE, Debian, DevOps, GCP, RedHat
> Fix For: v.2.5
>
> Attachments: Jupyter creation1.png, Jupyter creation2.png,
> P4-Rstudio-r-package-lib-list.png, RStudio creation.png, Zeppelin
> creation1.png, Zeppelin creation2.png
>
>
> # -Retrieve available lib list group -error -ModuleNotFoundError: No module
> named 'xmlrpclib'- -{color:#00875a} fixed 04/02/2021.{color}
> # -Get rid of Python2 during SSN deployment, Notebook creation.-
> *SSN:*
> {code:java}
> Step 4/28 : RUN apt-get update && apt-get -y upgrade && apt-get -y
> install python3-pip python-dev groff vim less git wget nano libssl-dev
> libffi-dev libffi6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/*
> /var/tmp/*
> ---> Running in e349ec65c115 fixed 26/03/2021{code}
> (i) {color:#00875a}Fixed{color} in '-tep 4/28 : RUN apt-get update &&'-
> (03/02/2021).
> {code:java}
> sudo: cd /opt/datalab/sources/infrastructure-provisioning/src/; docker build
> --build-arg OS=debian --build-arg SRC_PATH= --file
> general/files/gcp/base_Dockerfile -t docker.datalab-base:latest fixed
> 26/03/2021
> {code}
> (i) {color:#00875a}Fixed{color} in -'sudo: cd /opt/datalab/sources/'-
> (03/02/2021).
> {code:java}
> sudo: apt-get -y install supervisor 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
> "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
> /tmp/apt.log;fi fixed 26/03/2021{code}
> (i) {color:#00875a}Fixed{color} -in 'apt-get -y install supervisor-'
> (03/02/2021).
> (-) {color:#57d9a3}Done{color} -Script for Keycloak client creation should
> use Python3.-
> *{color:#57d9a3}Done{color} -Jupyter-:*
> {code:java}
> sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if !
> grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then
> echo "" > /tmp/apt.log;fi - fixed 26/03/2021
> {code}
> {code:java}
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
> "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
> /tmp/apt.log;fi - fixed 26/03/2021{code}
> *{color:#57d9a3}Done{color} -RStudio-:*
> {code:java}
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
> "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
> /tmp/apt.log;fi - fixed 26/03/2021
> {code}
> (i) -Py2 is still available in 'apt-get -y install nodejs'- -
> {color:#00875a}fixed{color} (26/032021).
> *{color:#57d9a3}Done{color} -Zeppelin-:*
> {code:java}
> sudo: apt-get -y build-dep python-matplotlib 2>&1 | tee /tmp/tee.tmp; if !
> grep -w -E "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then
> echo "" > /tmp/apt.log;fi - fixed 26/03/2021
> {code}
> {code:java}
>
> sudo: apt-get -y install nodejs 2>&1 | tee /tmp/tee.tmp; if ! grep -w -E
> "(frontend is locked|locked)" /tmp/tee.tmp > /tmp/apt.log; then echo "" >
> /tmp/apt.log;fi - fixed 26/03/2021
> {code}
> (i) -Py2 Is still available in 'apt-get -y install nodejs-'
> {color:#00875a}fixed{color} (26/032021).
> *3.* -Performance for SSN deployment. It has to take up to 40-50 minutes-.
> But now it takes 2 hours. *{color:#00875a}Fixed{color}, now ~40m*
> *4.1.* -It's impossible to retrieve available lib list for any group. Docker
> runs with '1'- (26/03/2021):
> {color:#00875a}F{color}*{color:#00875a}ixed{color} 30.03.2021*
> {code:java}
> No idea what 'list_libs' is!
> Traceback (most recent call last):
> File "/root/entrypoint.py", line 154, in <module>
> subprocess.run("/bin/list_libs.py", shell=True, check=True)
> File "/usr/lib/python3.8/subprocess.py", line 512, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command '/bin/list_libs.py' returned non-zero
> exit status 1.
> {code}
> *4.2.* Available lib list retrieving:
> +RStudio:+
> ||Apt/Yum|Still in process: is too long, > 50 min.|
> ||Py3|Passed|
> ||R packages|Docker runs with 1, Error: package or namespace load failed for
> ‘SparkR’:
> package ‘SparkR’ was installed before R 4.0.0: please re-install it|
> ||Others|Passed|
> +Zeppelin:+
> ||Apt/Yum|Still in process: is too long, > 50 min. |
> ||Py3| Passed|
> ||R packages| Passed|
> ||Others| Passed|
> *5.* Lib installation:
> +RStudio:+
> ||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
> installed|
> ||Py3|Passed|
> ||R packages|Failed by lib list retrieving|
> ||Others|Passed|
> +Zeppelin:+
> ||Apt/Yum|Blocked by lib list retrieving, but if open new window it can be
> installed|
> ||Py3|Passed|
> ||R packages|Passed|
> ||Java dependency|Passed|
> ||Others|Passed|
> *6.* Get rid of kernel python2 for notebooks.
> *7.* Docker of image creation runs with '1': *fixed 30.03.2021*
> {code:java}
> No idea what 'create_image' is!
> Traceback (most recent call last):
> File "/root/entrypoint.py", line 160, in <module>
> subprocess.run("/bin/create_image.py", shell=True, check=True)
> File "/usr/lib/python3.8/subprocess.py", line 512, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command '/bin/create_image.py' returned
> non-zero exit status 1.
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
