[
https://issues.apache.org/jira/browse/DRILL-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382940#comment-16382940
]
ASF GitHub Bot commented on DRILL-6197:
---------------------------------------
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1141#discussion_r171740245
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentStats.java ---
@@ -79,4 +71,21 @@ public void addOperatorStats(OperatorStats stats) {
operators.add(stats);
}
+ //DRILL-6197
+ public OperatorStats addOrReplaceOperatorStats(OperatorStats stats) {
+ //Remove existing stat
+ OperatorStats replacedStat = null;
+ int index = 0;
+ for (OperatorStats opStat : operators) {
--- End diff --
I see your point. Also, digging into the code shows I can substitute with a
LinkedHashMap, since the list is only referenced here for consumption of its
contents:
https://github.com/kkhatua/drill/blob/65efe3ea0c5777490488d3d56cbdb0cb011b9f33/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentStats.java#L45
I can't use a Set, because I need the Stats object hashed on the operator
ID & Type, and not the rest of the contents. I'll refactor and try to confirm
nothing else breaks.
> Duplicate entries in inputProfiles of minor fragments for specific operators
> ----------------------------------------------------------------------------
>
> Key: DRILL-6197
> URL: https://issues.apache.org/jira/browse/DRILL-6197
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Monitoring
> Affects Versions: 1.12.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
> Priority: Minor
> Fix For: 1.13.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Minor fragments for the following operators show duplicate entries of the
> inputProfile ({{org.apache.drill.exec.ops.OperatorStats}} instance) when
> viewed in the Profile UI.
> e.g
> {code:json}
> {
> ...
> "query": "select * from sys.version",
> ...
> [ ...
> {
> "inputProfile": [{
> "records": 0,
> "batches": 0,
> "schemas": 0
> }],
> "operatorId": 0,
> "operatorType": 13,
> "setupNanos": 0,
> "processNanos": 0,
> "peakLocalMemoryAllocated": 27131904,
> "waitNanos": 0
> },
> {
> "inputProfile": [{
> "records": 1,
> "batches": 1,
> "schemas": 1
> }],
> "operatorId": 0,
> "operatorType": 13,
> "setupNanos": 0,
> "processNanos": 752448,
> "peakLocalMemoryAllocated": 27131904,
> "metric": [{
> "metricId": 0,
> "longValue": 178
> }],
> "waitNanos": 889492
> }]
> ...
> }
> {code}
> {{operatorType: 13}} is the screen operator, for which there can be only one
> inputProfile.
> It turns out that by default, all minor fragments' operators are provide a
> list of inputProfiles by
> {{org.apache.drill.exec.ops.FragmentStats.newOperatorStats(OpProfileDef,
> BufferAllocator)}}. However, for the following 4 operators, the
> {{org.apache.drill.exec.physical.impl.BaseRootExec}} constructors also inject
> {{OperatorStats}}.
> {code:java}
> org.apache.drill.exec.proto.beans.CoreOperatorType.SCREEN
> org.apache.drill.exec.proto.beans.CoreOperatorType.SINGLE_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.BROADCAST_SENDER
> org.apache.drill.exec.proto.beans.CoreOperatorType.HASH_PARTITION_SENDER
> {code}
> All updates to the inputProfiles are done by the latter, while the former
> only reports zero values.
> The workaround is to have {{org.apache.drill.exec.ops.FragmentStats}} skip
> injecting the {{org.apache.drill.exec.ops.OperatorStats}} instance for these
> operators
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
