[jira] [Comment Edited] (YARN-2838) Issues with TimeLineServer (Application History)

2014-11-13 Thread Zhijie Shen (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14210199#comment-14210199
 ] 

Zhijie Shen edited comment on YARN-2838 at 11/13/14 7:34 PM:
-

bq. 1. Whatever the CLI command user executes is historyserver or 
timelineserver it looks like ApplicationHistoryServer only run. So can we 
modify the name of the class  ApplicationHistoryServer to TimelineHistoryServer 
(or any other suitable name as it seems like any command user runs 
ApplicationHistoryServer is started)

Yes, not just the the main entry point class, but the whole sub-module needs to 
be refactor somehow to reflect the generalized conception (YARN-2043).

bq. 2. Instead of the "Starting the History Server anyway..." deprecated msg, 
can we have "Starting the Timeline History Server anyway...".

bq. 3. Based on start or stop, deprecated message should get modified to 
"Starting the Timeline History Server anyway..." or "Stopping the Timeline 
History Server anyway..."

See the comment before.

bq. But any way we need to fix this issue also right ? so already any jira is 
raised or shall i work as part of this jira ?

See YARN-2522. We can work this issue there.

bq. And also please inform if this issue needs to be split into mulitple jiras 
(apart from documentation which you have already raised) would like to split 
and work on them.

If you agree, we can close this Jira, and work on separate Jiras that focus on 
each individual issues.

bq. As already i have started looking into these issues, was also planning to 
work on documentation. If you don't mind can you assign the issue (YARN-2854) 
to me ?

No problem, assigned it to you.






was (Author: zjshen):
bq. 1. Whatever the CLI command user executes is historyserver or 
timelineserver it looks like ApplicationHistoryServer only run. So can we 
modify the name of the class  ApplicationHistoryServer to TimelineHistoryServer 
(or any other suitable name as it seems like any command user runs 
ApplicationHistoryServer is started)

Yes, not just the the main entry point class, but the whole sub-module needs to 
be refactor somehow to reflect the generalized conception (YARN-2043).

bq. 2. Instead of the "Starting the History Server anyway..." deprecated msg, 
can we have
"Starting the Timeline History Server anyway...".

bq. 3. Based on start or stop, deprecated message should get modified to 
"Starting the
Timeline History Server anyway..." or "Stopping the Timeline History Server 
anyway..."

See the comment before.

bq. But any way we need to fix this issue also right ? so already any jira is 
raised or shall i work as part of this jira ?

See YARN-2522. We can work this issue there.

bq. And also please inform if this issue needs to be split into mulitple jiras 
(apart from documentation which you have already raised) would like to split 
and work on them.

If you agree, we can close this Jira, and work on separate Jiras that focus on 
each individual issues.

bq. As already i have started looking into these issues, was also planning to 
work on documentation. If you don't mind can you assign the issue (YARN-2854) 
to me ?

No problem, assigned it to you.





> Issues with TimeLineServer (Application History)
> 
>
> Key: YARN-2838
> URL: https://issues.apache.org/jira/browse/YARN-2838
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: timelineserver
>Affects Versions: 2.6.0, 2.5.1
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: IssuesInTimelineServer.pdf
>
>
> Few issues in usage of Timeline server for generic application history access



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (YARN-2838) Issues with TimeLineServer (Application History)

2014-11-11 Thread Zhijie Shen (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206671#comment-14206671
 ] 

Zhijie Shen edited comment on YARN-2838 at 11/12/14 12:44 AM:
--

[~Naganarasimha], sorry for not responding you immediately as being busy on 
finalizing 2.6. A quick scan through your issue document. Here's my 
clarification:

1. While the entry point of the this sub-module is still called 
ApplicationHistoryServer, it is actually generalized to be TimelineServer right 
now (definitely we need to refactor the code at some point). The baseline 
service provided the the timeline server is to allow the cluster and its apps 
to store their history information, metrics and so on by complying with the 
defined timeline data model. Later on, users and admins can query this 
information to do the analysis.

2. Application history (or we prefer to call it generic history service) is now 
a built-in service in the timeline server to record the generic history 
information of YARN apps. It was on a separate store (on FS), but after 
YARN-2033, it has been moved to the timeline store too, as a payload. We 
replace the old storage layer, but keep the existing interfaces (web UI, 
services, CLI) not changed to be the analog of what RM provides for running 
apps. We still didn't integrate TimelineClient and AHSClient, the latter of 
which is RPC interface of getting generic history information via RPC 
interface. APPLICATION_HISTORY_ENABLED is the only remaining old config to 
control whether we also want to pull the app info from the generic history 
service inside the timeline server. You may want to take a look at YARN-2033 to 
get more context about the change. Moreover, as a number of limitation of the 
old history store, we're no longer going to support it.

3. The document is definitely staled. I'll file separate document Jira, 
however, it's too late for 2.6. Let's target 2.7 for an up-to-date document 
about timeline service and its built-in generic history service (YARN-2854). 
Does it sound good?


was (Author: zjshen):
[~Naganarasimha], sorry for not responding you immediately as being busy on 
finalizing 2.6. A quick scan through your issue document. Here's my 
clarification:

1. While the entry point of the this sub-module is still called 
ApplicationHistoryServer, it is actually generalized to be TimelineServer right 
now (definitely we need to refactor the code at some point). The baseline 
service provided the the timeline server is to allow the cluster and its apps 
to store their history information, metrics and so on by complying with the 
defined timeline data model. Later on, users and admins can query this 
information to do the analysis.

2. Application history (or we prefer to call it generic history service) is now 
a built-in service in the timeline server to record the generic history 
information of YARN apps. It was on a separate store (on FS), but after 
YARN-2033, it has been moved to the timeline store too, as a payload. We 
replace the old storage layer, but keep the existing interfaces (web UI, 
services, CLI) not changed to be the analog of what RM provides for running 
apps. We still didn't integrate TimelineClient and AHSClient, the latter of 
which is RPC interface of getting generic history information via RPC 
interface. APPLICATION_HISTORY_ENABLED is the only remaining old config to 
control whether we also want to pull the app info from the generic history 
service inside the timeline server. You may want to take a look at YARN-2033 to 
get more context about the change. Moreover, as a number of limitation of the 
old history store, we're no longer going to support it.

3. The document is definitely staled. I'll file separate document Jira, 
however, it's too late for 2.6. Let's target 2.7 for an up-to-date document 
about timeline service and its built-in generic history service. Does it sound 
good?

> Issues with TimeLineServer (Application History)
> 
>
> Key: YARN-2838
> URL: https://issues.apache.org/jira/browse/YARN-2838
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: timelineserver
>Affects Versions: 2.5.1
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: IssuesInTimelineServer.pdf
>
>
> Few issues in usage of Timeline server for generic application history access



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)