[
https://issues.apache.org/jira/browse/HDFS-14325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780677#comment-16780677
]
Sahil Takiar edited comment on HDFS-14325 at 2/28/19 4:05 PM:
--------------------------------------------------------------
[~bobhansen], [~anatoli.shein], [~James Clampffer] wondering if you have any
comments on this since it involves libhdfs++
The TL;DR is that libhdfs and libhdfs++ have inconsistent handling of setting
{{errno}} on success, which causes issues for some of the shim-based libhdfs++
tests.
was (Author: stakiar):
[~bobhansen], [~anatoli.shein], [~James Clampffer] wondering if you have any
comments on this since it involves libhdfs++. The TL;DR is that libhdfs and
libhdfs++ have inconsistent handling of setting {{errno}} on success, which
causes issues for some of the shim-based libhdfs++ tests.
> Revise usage of errno in libhdfs
> --------------------------------
>
> Key: HDFS-14325
> URL: https://issues.apache.org/jira/browse/HDFS-14325
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, libhdfs, native
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> The usage of [errno|http://man7.org/linux/man-pages/man3/errno.3.html] in
> libhdfs has gone through several changes in the past: HDFS-3675, HDFS-4997,
> HDFS-3579, HDFS-8407, etc.
> As a result of these changes, some libhdfs functions set {{errno}} to 0 on
> success ({{hadoopReadZero}}, {{hdfsListDirectory}}), while several set
> {{errno}} to a meaningful value only on error.
> libhdfs++ on the other hand sets {{errno}} to 0 for all (successful)
> libhdfs++ operations. See
> [this|https://issues.apache.org/jira/browse/HDFS-10511?focusedCommentId=15322696&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15322696]
> comment in HDFS-10511 for why that was done.
> The inconsistent behavior between libhdfs and libhdfs++ causes issues for
> tests such as {{test_hdfs_ext_hdfspp_test_shim_static}} which uses a shim
> layer ({{tests/hdfs_shim.c}}) that delegates to both {{hdfs.c}} and
> {{hdfs.cc}} for various operations (e.g. opening / closing files uses both
> APIs, {{hdfsWrite}} delegates to libhdfs since libhdfs++ does not support
> writes yet). The tests expect {{errno}} to be set to 0 after successful
> operations against the shim layer. Since libhdfs is not guaranteed to set
> {{errno}} to 0 on success, tests can start failing.
> One example of the inconsistency causing issues is HDFS-14111, the patch for
> HDFS-14111 happens to change the {{errno}} from 0 to 2 for {{hdfsCloseFile}}.
> However, from libhdfs's perspective this seems to be by design. Quoting from
> the {{errno}} C docs:
> {quote}The value in errno is significant only when the return value of the
> call indicated an error (i.e., -1 from most system calls; -1 or NULL from
> most library functions); a function that succeeds is allowed to change errno.
> {quote}
> I was not able to pin down why the patch for HDFS-14111 changed the {{errno}}
> value, but I isolated the change to the {{FileSystem#close}} call. Most
> likely, some C function invoked as a result of calling {{#close}} failed and
> changed the {{errno}} value, but the {{#close}} was still able to succeed
> (this is most likely expected behavior, see
> [this|https://issues.apache.org/jira/browse/HDFS-8407?focusedCommentId=14551225&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14551225]
> comment in HDFS-8407 for further validation).
> Going forward we could (1) set {{errno}} to 0 for all successful libhdfs
> functions, which would make the libhdfs behavior consistent with the
> libhdfs++ behavior, or (2) we could live with the discrepancy, which would
> require modifying from the libhdfs++ shim tests (which assert that {{errno}}
> is 0 after certain operations) and just document the difference in behavior.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
