[ 
https://issues.apache.org/jira/browse/YARN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736140#comment-14736140
 ] 

Sangjin Lee commented on YARN-3901:
-----------------------------------

Somehow the jenkins info didn't make it to the JIRA: 
https://builds.apache.org/job/PreCommit-YARN-Build/9044/

-1 overall

| Vote |           Subsystem |  Runtime   | Comment
|  -1  |          pre-patch  |  15m 55s   | Findbugs (version ) appears to be 
|      |                     |            | broken on YARN-2928.
|  +1  |            @author  |  0m 1s     | The patch does not contain any 
|      |                     |            | @author tags.
|  +1  |     tests included  |  0m 0s     | The patch appears to include 2 new 
|      |                     |            | or modified test files.
|  +1  |              javac  |  8m 6s     | There were no new javac warning 
|      |                     |            | messages.
|  +1  |            javadoc  |  10m 12s   | There were no new javadoc warning 
|      |                     |            | messages.
|  +1  |      release audit  |  0m 23s    | The applied patch does not increase 
|      |                     |            | the total number of release audit
|      |                     |            | warnings.
|  +1  |         checkstyle  |  0m 16s    | There were no new checkstyle 
|      |                     |            | issues.
|  -1  |         whitespace  |  0m 31s    | The patch has 7 line(s) that end in 
|      |                     |            | whitespace. Use git apply
|      |                     |            | --whitespace=fix.
|  +1  |            install  |  1m 34s    | mvn install still works. 
|  +1  |    eclipse:eclipse  |  0m 40s    | The patch built with 
|      |                     |            | eclipse:eclipse.
|  -1  |           findbugs  |  0m 56s    | The patch appears to introduce 7 
|      |                     |            | new Findbugs (version 3.0.0)
|      |                     |            | warnings.
|  +1  |         yarn tests  |  1m 54s    | Tests passed in 
|      |                     |            | hadoop-yarn-server-timelineservice.
|      |                     |  40m 33s   | 


    Reason | Tests
 FindBugs  |  module:hadoop-yarn-server-timelineservice 


|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12754731/YARN-3901-YARN-2928.5.patch
 |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / e6afe26 |
| whitespace | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/whitespace.txt
 |
| Findbugs warnings | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-timelineservice.html
 |
| hadoop-yarn-server-timelineservice test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/9044/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |



> Populate flow run data in the flow_run & flow activity tables
> -------------------------------------------------------------
>
>                 Key: YARN-3901
>                 URL: https://issues.apache.org/jira/browse/YARN-3901
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>         Attachments: YARN-3901-YARN-2928.1.patch, 
> YARN-3901-YARN-2928.2.patch, YARN-3901-YARN-2928.3.patch, 
> YARN-3901-YARN-2928.4.patch, YARN-3901-YARN-2928.5.patch
>
>
> As per the schema proposed in YARN-3815 in 
> https://issues.apache.org/jira/secure/attachment/12743391/hbase-schema-proposal-for-aggregation.pdf
> filing jira to track creation and population of data in the flow run table. 
> Some points that are being  considered:
> - Stores per flow run information aggregated across applications, flow version
> RM’s collector writes to on app creation and app completion
> - Per App collector writes to it for metric updates at a slower frequency 
> than the metric updates to application table
> primary key: cluster ! user ! flow ! flow run id
> - Only the latest version of flow-level aggregated metrics will be kept, even 
> if the entity and application level keep a timeseries.
> - The running_apps column will be incremented on app creation, and 
> decremented on app completion.
> - For min_start_time the RM writer will simply write a value with the tag for 
> the applicationId. A coprocessor will return the min value of all written 
> values. - 
> - Upon flush and compactions, the min value between all the cells of this 
> column will be written to the cell without any tag (empty tag) and all the 
> other cells will be discarded.
> - Ditto for the max_end_time, but then the max will be kept.
> - Tags are represented as #type:value. The type can be not set (0), or can 
> indicate running (1) or complete (2). In those cases (for metrics) only 
> complete app metrics are collapsed on compaction.
> - The m! values are aggregated (summed) upon read. Only when applications are 
> completed (indicated by tag type 2) can the values be collapsed.
> - The application ids that have completed and been aggregated into the flow 
> numbers are retained in a separate column for historical tracking: we don’t 
> want to re-aggregate for those upon replay
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to