have option to limit stderr output from user scripts in hadoop streaming
------------------------------------------------------------------------
Key: HADOOP-3984
URL: https://issues.apache.org/jira/browse/HADOOP-3984
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.17.2
Reporter: Joydeep Sen Sarma
People often echo to stderr to debug their programs. We have had cases where
someone inadvertently dumped large amounts of data to stderr. this created
large userlog directories and caused quite a few mapred slaves to become
dysfunctional (logs partition was full).
Hence looking for a defensive measure against such errant scripts. We are
currently putting in a limit on the number of bytes from the script's stderr
that are forwarded to System.err.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
