[
https://issues.apache.org/jira/browse/HDDS-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YiSheng Lien updated HDDS-2443:
-------------------------------
Description:
Original ideas:
Ozone Client(Python) for Data Science Notebook such as Jupyter.
# Size: Large
# PyArrow: [https://pypi.org/project/pyarrow/]
# Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API Impala
uses libhdfs
Path to try:
1. s3 interface: Ozone s3 gateway(already supported) + AWS python client (boto3)
2. python native RPC
3. pyarrow + libhdfs, which use the Java client under the hood.
was:
Original ideas:
Ozone Client(Python) for Data Science Notebook such as Jupyter.
# Size: Large
# PyArrow: [https://pypi.org/project/pyarrow/]
# Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API Impala
uses libhdfs
# How Jupyter iPython work:
[https://jupyter.readthedocs.io/en/latest/architecture/how_jupyter_ipython_work.html]
# Eco,
Architecture:[https://ipython-books.github.io/chapter-3-mastering-the-jupyter-notebook/]
Path to try:
1. s3 interface: Ozone s3 gateway(already supported) + AWS python client (boto3)
2. python native RPC
3. pyarrow + libhdfs, which use the Java client under the hood.
> Python client/interface for Ozone
> ---------------------------------
>
> Key: HDDS-2443
> URL: https://issues.apache.org/jira/browse/HDDS-2443
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Components: Ozone Client
> Reporter: Li Cheng
> Priority: Major
>
> Original ideas:
> Ozone Client(Python) for Data Science Notebook such as Jupyter.
> # Size: Large
> # PyArrow: [https://pypi.org/project/pyarrow/]
> # Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API
> Impala uses libhdfs
>
> Path to try:
> 1. s3 interface: Ozone s3 gateway(already supported) + AWS python client
> (boto3)
> 2. python native RPC
> 3. pyarrow + libhdfs, which use the Java client under the hood.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
