Suma Shivaprasad updated YARN-6550:
    Attachment: YARN-6550.003.patch

Attached patch with review comments addressed. 

These are the changes in the patch
1. Added command groups and redirection happen with command groups if stdout 
and stderr are set.
2. tee to the redirect files and  stdout and stderr are needed since 
DefaultContainerExecutor error propagation fails without this.
3. Redirection is happening to separate files for stdout and stderr instead of 
a combined stdout+stderr  - This is better since redirecting to same file from 
multiple streams is causing them to either be garbled or ordering is lost(since 
stderr is unbuffered and stdout is block buffered). Could simulate this easily 
with  a small script 
        echo "test"
        mkdir /inva;
} 2>"${STDERR}" | tee -a "${STDERR}" > "${STDOUT}"
The way to guarantee ordering would be to redirect stderr to out  2>&1 . 
However this would mean that the original stderr is lost and would break 
compatibility with current DefaultContainerExecutor behaviour. Hence havent 
addressed that
4. Also ran the script through shellcheck and no new errors were detected.

> Capture launch_container.sh logs
> --------------------------------
>                 Key: YARN-6550
>                 URL: https://issues.apache.org/jira/browse/YARN-6550
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-beta1
>            Reporter: Wangda Tan
>            Assignee: Suma Shivaprasad
>             Fix For: 3.0.0-beta1
>         Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch
> launch_container.sh which generated by NM will do a bunch of things (like 
> create link, etc.) while launch a process. No logs captured until {{exec}} is 
> called. We need capture all failures of launch_container.sh for easier 
> troubleshooting.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to