[
https://issues.apache.org/jira/browse/YARN-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184269#comment-15184269
]
Sangjin Lee commented on YARN-4062:
-----------------------------------
Sorry [~vrushalic] it took a while to get to this. It looks good for the most
part. I do have some questions and minor comments.
(YarnConfiguration.java)
- please define it in yarn-default.xml as well
(FlowScannerOperation.java)
- l.38, 43: typo: "Comapction" -> "compaction"
(FlowRunColumnPrefix.java)
- l.43: if we're removing the aggregation operation from the only enum entry,
should we remove the aggregation operation ({{aggOp}}) completely from this
class?
(FlowRunCoprocessor.java)
- l.268: Logging of {{request.isMajor()}} seems a bit cryptic. It would print
strings like "Compactionrequest= ... true for ObserverContext ...". Should we
do something like {{request.isMajor() ? "major compaction" : "minor
compaction"}} instead? And did you mean it to be an INFO logging statement?
(TimelineStorageUtils.java)
- l.524: nit: we probably don't need that beginning space
(FlowScanner.java)
- l.79: we should not set the action by default, as it is always passed in
through the constructor
- l.178: Shouldn't this be {{cellLimit}} still? {{limit}} is not an argument of
this method.
- l.220: could you elaborate on why this change is needed? I'm generally not
too clear on the difference between {{cellLimit}} and {{limit}}. How are these
values different and under what conditions?
- l.233: missing the row key value?
- l.381-382: should we throw an exception as this is not possible (currently)?
- l.484: Although this would be true for the most part, {{sum.longValue() >
0L}} is not equivalent to summations being performed on compaction here,
especially if there were negative individual values for example. Should we
introduce a boolean flag to denote the case where summations were performed
instead?
(TestHBaseStorageFlowRunCompaction.java)
- there are 3 unused imports
> Add the flush and compaction functionality via coprocessors and scanners for
> flow run table
> -------------------------------------------------------------------------------------------
>
> Key: YARN-4062
> URL: https://issues.apache.org/jira/browse/YARN-4062
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Vrushali C
> Assignee: Vrushali C
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-4062-YARN-2928.04.patch,
> YARN-4062-YARN-2928.05.patch, YARN-4062-YARN-2928.06.patch,
> YARN-4062-YARN-2928.07.patch, YARN-4062-YARN-2928.1.patch,
> YARN-4062-feature-YARN-2928.01.patch, YARN-4062-feature-YARN-2928.02.patch,
> YARN-4062-feature-YARN-2928.03.patch
>
>
> As part of YARN-3901, coprocessor and scanner is being added for storing into
> the flow_run table. It also needs a flush & compaction processing in the
> coprocessor and perhaps a new scanner to deal with the data during flushing
> and compaction stages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
