[ https://issues.apache.org/jira/browse/HDDS-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
YiSheng Lien updated HDDS-2443: ------------------------------- Description: Original ideas: Ozone Client(Python) for Data Science Notebook such as Jupyter. # Size: Large # PyArrow: [https://pypi.org/project/pyarrow/] # Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API Impala uses libhdfs Path to try: 1. s3 interface: Ozone s3 gateway(already supported) + AWS python client (boto3) 2. python native RPC 3. pyarrow + libhdfs, which use the Java client under the hood. was: Original ideas: Ozone Client(Python) for Data Science Notebook such as Jupyter. # Size: Large # PyArrow: [https://pypi.org/project/pyarrow/] # Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API Impala uses libhdfs # How Jupyter iPython work: [https://jupyter.readthedocs.io/en/latest/architecture/how_jupyter_ipython_work.html] # Eco, Architecture:[https://ipython-books.github.io/chapter-3-mastering-the-jupyter-notebook/] Path to try: 1. s3 interface: Ozone s3 gateway(already supported) + AWS python client (boto3) 2. python native RPC 3. pyarrow + libhdfs, which use the Java client under the hood. > Python client/interface for Ozone > --------------------------------- > > Key: HDDS-2443 > URL: https://issues.apache.org/jira/browse/HDDS-2443 > Project: Hadoop Distributed Data Store > Issue Type: New Feature > Components: Ozone Client > Reporter: Li Cheng > Priority: Major > > Original ideas: > Ozone Client(Python) for Data Science Notebook such as Jupyter. > # Size: Large > # PyArrow: [https://pypi.org/project/pyarrow/] > # Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API > Impala uses libhdfs > > Path to try: > 1. s3 interface: Ozone s3 gateway(already supported) + AWS python client > (boto3) > 2. python native RPC > 3. pyarrow + libhdfs, which use the Java client under the hood. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org