[
https://issues.apache.org/jira/browse/YARN-9230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755444#comment-16755444
]
Eric Yang commented on YARN-9230:
---------------------------------
Thanks to [[email protected]], I was able to rebase his [hdfs docker
driver|https://github.com/sakserv/distribution] for Docker Registry to latest
Docker registry code, and run the same benchmark as YARN-9229. The performance
is quite good:
| | Remote | Local |
| Pull | 18.673s | 13.109s |
| Push | 26.113s | 25.917s |
The numbers are measured on the same hardware that was used to measure NFS
Gateway flavor of Docker Registry.
The only drawback of this approach is Go HDFS client library does not fully
support Kerberos enabled environment. User will encounter ["unexpected
sequence number"|https://github.com/colinmarc/hdfs/issues/144] issue. Some
GSSAPI encryption related problems that are not fully resolved in Colin's Go
HDFS library.
> Write a go hdfs driver for Docker Registry
> ------------------------------------------
>
> Key: YARN-9230
> URL: https://issues.apache.org/jira/browse/YARN-9230
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Priority: Major
>
> A number of people have developed go client library for HDFS. [Example
> 1|https://medium.com/sqooba/making-hdfs-a-hundred-times-faster-ac75b8b5e0b4]
> [Example 2|https://github.com/colinmarc/hdfs].
> This can be enhanced into a real storage driver for Docker registry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
