[
https://issues.apache.org/jira/browse/HDFS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496622#comment-13496622
]
Andy Isaacson commented on HDFS-4178:
-------------------------------------
bq. The data corruption occurs when the process opens a new file descriptor for
its data channel and gets file descriptor 2
Yep, exactly. If an app does {{fopen("datafile", "w+")}} (or the Java
equivalent) and is run under {{someapp 2>&-}} the file {{datafile}} will be
opened for write as filedescriptor 2. If the app then triggers glibc's "bad
free() detected" codepath (among many others), the error message will
potentially be written to the current file position in {{datafile}}.
(glibc has a workaround for the most common case, it opens {{/dev/tty}} for the
error message in interactive sessions, but it still does corrupt output files
when run in a noninteractive session with {{cron}}, {{at}}, {{batch}}, or at
startup time from {{/etc/init.d}}.)
> shell scripts should not close stderr
> -------------------------------------
>
> Key: HDFS-4178
> URL: https://issues.apache.org/jira/browse/HDFS-4178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: scripts
> Affects Versions: 2.0.2-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Attachments: hdfs4178.txt
>
>
> The {{start-dfs.sh}} and {{stop-dfs.sh}} scripts close stderr for some
> subprocesses using the construct
> bq. {{2>&-}}
> This is dangerous because child processes started up under this scenario will
> re-use filedescriptor 2 for opened files. Since libc and many other
> codepaths assume that filedescriptor 2 can be written to in error conditions,
> this can potentially result in data corruption.
> Much better to redirect stderr using the construct {{2>/dev/null}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
