This is an automated email from the ASF dual-hosted git repository. chenliang613 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/carbondata.git
The following commit(s) were added to refs/heads/master by this push: new b9419832e1 Support using Apache CarbonData in notebook (#4317) b9419832e1 is described below commit b9419832e1f252c200eca1a691f15afe88432ac3 Author: Bo Xu <601450...@qq.com> AuthorDate: Fri Apr 14 04:13:02 2023 +0800 Support using Apache CarbonData in notebook (#4317) --- docs/images/using-carbondata-in-notebook.png | Bin 0 -> 148136 bytes docs/images/using-carbondata-in-notebook2.png | Bin 0 -> 350850 bytes docs/quick-start-guide.md | 2 + docs/using-carbondata-in-notebook.md | 76 ++++++++++++++++++++++++++ 4 files changed, 78 insertions(+) diff --git a/docs/images/using-carbondata-in-notebook.png b/docs/images/using-carbondata-in-notebook.png new file mode 100644 index 0000000000..2993953e22 Binary files /dev/null and b/docs/images/using-carbondata-in-notebook.png differ diff --git a/docs/images/using-carbondata-in-notebook2.png b/docs/images/using-carbondata-in-notebook2.png new file mode 100644 index 0000000000..47085ef16e Binary files /dev/null and b/docs/images/using-carbondata-in-notebook2.png differ diff --git a/docs/quick-start-guide.md b/docs/quick-start-guide.md index 0d9cee1bdf..b3a8b79cef 100644 --- a/docs/quick-start-guide.md +++ b/docs/quick-start-guide.md @@ -49,6 +49,8 @@ CarbonData can be integrated with Spark, Presto, Flink and Hive execution engine [Installing and Configuring CarbonData Thrift Server for Query Execution](#query-execution-using-the-thrift-server) +### Notebook +[Using CarbonData in notebook](#using-carbondata-in-notebook.md) #### Presto [Installing and Configuring CarbonData on Presto](#installing-and-configuring-carbondata-on-presto) diff --git a/docs/using-carbondata-in-notebook.md b/docs/using-carbondata-in-notebook.md new file mode 100644 index 0000000000..2bf239fd41 --- /dev/null +++ b/docs/using-carbondata-in-notebook.md @@ -0,0 +1,76 @@ +## Preparing +It should already installed docker environment before using carbondata in notebook. + +## Downloading docker images of carbondata notebook + +Downloading latest version of docker images of carbondata notebook + +```shell +docker pull xubo245/all-carbondata-notebook:latest +``` + +Downloading specify version of docker images of carbondata notebook + +```shell +docker pull xubo245/all-carbondata-notebook:carbondata-2.3.0-spark3.1.1-hadoop2.7.2-v1 +``` + +Refer to https://hub.docker.com/repository/docker/xubo245/all-carbondata-notebook/tags?page=1&ordering=last_updated +## Running the docker images of carbondata notebook + +``` +docker run -d -p 8888:8888 --restart always xubo245/all-carbondata-notebook:latest +``` + +## Opening the notebook +Command: +``` +docker ps |grep carbondata +docker logs <container_id> +``` +You can get the notebook address in the last line. + +For example: +```shell +localhost:carbondata xubo$ docker ps |grep carbondata +8a57e6fed80e xubo245/all-carbondata-notebook:latest "tini -g -- start-no…" 11 seconds ago Up 8 seconds 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp affectionate_bohr +localhost:carbondata xubo$ docker logs 8a57e6fed80e +WARN: Jupyter Notebook deprecation notice https://github.com/jupyter/docker-stacks#jupyter-notebook-deprecation-notice. +/usr/local/bin/start-notebook.sh: running hooks in /usr/local/bin/before-notebook.d +/usr/local/bin/start-notebook.sh: running /usr/local/bin/before-notebook.d/spark-config.sh +/usr/local/bin/start-notebook.sh: done running hooks in /usr/local/bin/before-notebook.d +Executing the command: jupyter notebook +[I 17:24:50.960 NotebookApp] Loading IPython parallel extension +[W 2023-04-12 17:24:51.833 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. +[W 2023-04-12 17:24:51.833 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. +[W 2023-04-12 17:24:51.833 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. +[W 2023-04-12 17:24:51.833 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. +[I 2023-04-12 17:24:51.842 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.9/site-packages/jupyterlab +[I 2023-04-12 17:24:51.842 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab +[I 17:24:51.849 NotebookApp] Serving notebooks from local directory: /home/jovyan +[I 17:24:51.849 NotebookApp] Jupyter Notebook 6.4.0 is running at: +[I 17:24:51.849 NotebookApp] http://8a57e6fed80e:8888/?token=f2f24cd38ddb1d2e11d8dd09ab27a2062dca66efbc50c75c +[I 17:24:51.849 NotebookApp] or http://127.0.0.1:8888/?token=f2f24cd38ddb1d2e11d8dd09ab27a2062dca66efbc50c75c +[I 17:24:51.849 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). +[C 17:24:51.853 NotebookApp] + + To access the notebook, open this file in a browser: + file:///home/jovyan/.local/share/jupyter/runtime/nbserver-8-open.html + Or copy and paste one of these URLs: + http://8a57e6fed80e:8888/?token=f2f24cd38ddb1d2e11d8dd09ab27a2062dca66efbc50c75c + or http://127.0.0.1:8888/?token=f2f24cd38ddb1d2e11d8dd09ab27a2062dca66efbc50c75c +``` + +Then you can open the notebook by put the notebook address to browser: +``` +http://127.0.0.1:8888/?token=f2f24cd38ddb1d2e11d8dd09ab27a2062dca66efbc50c75c +``` + +## Using carbondata in notebook: +Opening the carbondata_notebook.ipynb + + + +Running carbondata example in notebook file: + +