[
https://issues.apache.org/jira/browse/HDDS-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16978310#comment-16978310
]
Marton Elek commented on HDDS-2443:
-----------------------------------
[~cxorm], [~lichenga2404] : I tried to find my earlier experiments but they
were not up-to-date. I cleaned up the golang version and uploaded it to here:
[https://github.com/elek/ozone-go]
It's in very early phase (write path is not implemented and I didn't implement
iterations on the read path, works only for small files), but the remaining
parts just implementation details, can be improved easily. It shows how can it
work: it contains working hadoop-rpc and grpc calls to the ozone/ozone+ratis.
[~cxorm]: If you are interested to improve it, we can move it to the apache
repository, (to a separated feature branch, I guess) and we can improve it
gradually (Even if it won't be the chosen option to use python + ozone).
> Python client/interface for Ozone
> ---------------------------------
>
> Key: HDDS-2443
> URL: https://issues.apache.org/jira/browse/HDDS-2443
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Components: Ozone Client
> Reporter: Li Cheng
> Priority: Major
> Attachments: OzoneS3.py
>
>
> Original ideas: item#25 in
> [https://cwiki.apache.org/confluence/display/HADOOP/Ozone+project+ideas+for+new+contributors]
> Ozone Client(Python) for Data Science Notebook such as Jupyter.
> # Size: Large
> # PyArrow: [https://pypi.org/project/pyarrow/]
> # Python -> libhdfs HDFS JNI library (HDFS, S3,...) -> Java client API
> Impala uses libhdfs
>
> Path to try:
> # s3 interface: Ozone s3 gateway(already supported) + AWS python client
> (boto3)
> # python native RPC
> # pyarrow + libhdfs, which use the Java client under the hood.
> # python + C interface of go / rust ozone library. I created POC go / rust
> clients earlier which can be improved if the libhdfs interface is not good
> enough. [By [~elek]]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
