[ https://issues.apache.org/jira/browse/ARROW-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890500#comment-16890500 ]
Neal Richardson commented on ARROW-5995: ---------------------------------------- It sounds like you already know more about HDFS checksums, but [https://wesmckinney.com/blog/python-hdfs-interfaces/] may also be interesting reading for you. You may need to look into what libhdfs(3) provide. > [Python] pyarrow: hdfs: support file checksum > --------------------------------------------- > > Key: ARROW-5995 > URL: https://issues.apache.org/jira/browse/ARROW-5995 > Project: Apache Arrow > Issue Type: Improvement > Components: Python > Reporter: Ruslan Kuprieiev > Priority: Minor > > I was not able to find how to retrieve checksum (`getFileChecksum` or `hadoop > fs/dfs -checksum`) for a file on hdfs. Judging by how it is implemented in > hadoop CLI [1], looks like we will also need to implement it manually in > pyarrow. Please correct me if I'm missing something. Is this feature > desirable? Or was there a good reason why it wasn't implemented already? > [1] > [https://github.com/hanborq/hadoop/blob/hadoop-hdh3u2.1/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java#L719] -- This message was sent by Atlassian JIRA (v7.6.14#76016)