[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294490#comment-15294490
]
Vinod Kumar Vavilapalli commented on YARN-1134:
---
bq. It looks like you've implemented a feature to create a gzip'd tarball on
the client machine. The request to compress the log files that are served by
the NM logs web service.
Agreed. It was my mistake to close YARN-4919 as a duplicate of this, it isn't.
> Add support for zipping/unzipping logs while in transit for the NM logs
> web-service
> ---
>
> Key: YARN-1134
> URL: https://issues.apache.org/jira/browse/YARN-1134
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Xuan Gong
> Attachments: YARN-1134.1.patch
>
>
> As [~zjshen] pointed out at
> [YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
> {quote}
> For the long running applications, they may have a big log file, such that it
> will take a long time to download the log file via the RESTful API.
> Consequently, HTTP connection may timeout before downloading before
> downloading a complete log file. Maybe it is good to zip the log file before
> sending it, and unzip it after receiving it.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288927#comment-15288927
]
Varun Vasudev commented on YARN-1134:
-
[~xgong] - I think there's some confusion about the feature here. It looks like
you've implemented a feature to create a gzip'd tarball on the client machine.
The request to compress the log files that are served by the NM logs web
service. Like [~cnauroth] mentioned - you should check if the Jetty version we
run supports compression out of the box by sending the Accept-Encoding header.
> Add support for zipping/unzipping logs while in transit for the NM logs
> web-service
> ---
>
> Key: YARN-1134
> URL: https://issues.apache.org/jira/browse/YARN-1134
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Xuan Gong
> Attachments: YARN-1134.1.patch
>
>
> As [~zjshen] pointed out at
> [YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
> {quote}
> For the long running applications, they may have a big log file, such that it
> will take a long time to download the log file via the RESTful API.
> Consequently, HTTP connection may timeout before downloading before
> downloading a complete log file. Maybe it is good to zip the log file before
> sending it, and unzip it after receiving it.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286072#comment-15286072
]
Xuan Gong commented on YARN-1134:
-
Create a patch based on YARN-4913
> Add support for zipping/unzipping logs while in transit for the NM logs
> web-service
> ---
>
> Key: YARN-1134
> URL: https://issues.apache.org/jira/browse/YARN-1134
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Xuan Gong
> Attachments: YARN-1134.1.patch
>
>
> As [~zjshen] pointed out at
> [YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
> {quote}
> For the long running applications, they may have a big log file, such that it
> will take a long time to download the log file via the RESTful API.
> Consequently, HTTP connection may timeout before downloading before
> downloading a complete log file. Maybe it is good to zip the log file before
> sending it, and unzip it after receiving it.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256774#comment-15256774
]
Vinod Kumar Vavilapalli commented on YARN-1134:
---
We should also incorporate YARN-4919 in this - to have corresponding options in
"yarn logs" CLI.
> Add support for zipping/unzipping logs while in transit for the NM logs
> web-service
> ---
>
> Key: YARN-1134
> URL: https://issues.apache.org/jira/browse/YARN-1134
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Xuan Gong
>
> As [~zjshen] pointed out at
> [YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
> {quote}
> For the long running applications, they may have a big log file, such that it
> will take a long time to download the log file via the RESTful API.
> Consequently, HTTP connection may timeout before downloading before
> downloading a complete log file. Maybe it is good to zip the log file before
> sending it, and unzip it after receiving it.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758494#comment-13758494
]
Chris Nauroth commented on YARN-1134:
-
Is the intent to serve actual compressed files (i.e. it has a .gz extension),
or is the intent to layer compression over the HTTP transfer (i.e. the
Transfer-Encoding: gzip HTTP header). The original comment about how it will
take a long time to download makes me think the latter is appropriate.
Add support for zipping/unzipping logs while in transit for the NM logs
web-service
---
Key: YARN-1134
URL: https://issues.apache.org/jira/browse/YARN-1134
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
As [~zjshen] pointed out at
[YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
{quote}
For the long running applications, they may have a big log file, such that it
will take a long time to download the log file via the RESTful API.
Consequently, HTTP connection may timeout before downloading before
downloading a complete log file. Maybe it is good to zip the log file before
sending it, and unzip it after receiving it.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1134) Add support for zipping/unzipping logs while in transit for the NM logs web-service
[
https://issues.apache.org/jira/browse/YARN-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13758497#comment-13758497
]
Chris Nauroth commented on YARN-1134:
-
Also, if the latter is appropriate, then you may want to test the existing code
by sending an HTTP request with the header Accept-Encoding: gzip to see if
the response comes back compressed. Many web servers support this out of the
box, though I'm not sure about Jetty specifically.
Add support for zipping/unzipping logs while in transit for the NM logs
web-service
---
Key: YARN-1134
URL: https://issues.apache.org/jira/browse/YARN-1134
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
As [~zjshen] pointed out at
[YARN-649|https://issues.apache.org/jira/browse/YARN-649?focusedCommentId=13698415page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13698415],
{quote}
For the long running applications, they may have a big log file, such that it
will take a long time to download the log file via the RESTful API.
Consequently, HTTP connection may timeout before downloading before
downloading a complete log file. Maybe it is good to zip the log file before
sending it, and unzip it after receiving it.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
