Kunal Khatua created DRILL-6197:
-----------------------------------
Summary: Duplicate entries in inputProfiles of minor fragments for
specific operators
Key: DRILL-6197
URL: https://issues.apache.org/jira/browse/DRILL-6197
Project: Apache Drill
Issue Type: Bug
Components: Execution - Monitoring
Affects Versions: 1.12.0
Reporter: Kunal Khatua
Assignee: Kunal Khatua
Fix For: 1.13.0
Minor fragments for the following operators show duplicate entries of the
inputProfile ({{org.apache.drill.exec.ops.OperatorStats}} instance) when viewed
in the Profile UI.
e.g
{code:json}
{
...
"query": "select * from sys.version",
...
[ ...
{
"inputProfile": [{
"records": 0,
"batches": 0,
"schemas": 0
}],
"operatorId": 0,
"operatorType": 13,
"setupNanos": 0,
"processNanos": 0,
"peakLocalMemoryAllocated": 27131904,
"waitNanos": 0
},
{
"inputProfile": [{
"records": 1,
"batches": 1,
"schemas": 1
}],
"operatorId": 0,
"operatorType": 13,
"setupNanos": 0,
"processNanos": 752448,
"peakLocalMemoryAllocated": 27131904,
"metric": [{
"metricId": 0,
"longValue": 178
}],
"waitNanos": 889492
}]
...
}
{code}
{{operatorType: 13}} is the screen operator, for which there can be only one
inputProfile.
It turns out that by default, all minor fragments' operators are provide a list
of inputProfiles by
{{org.apache.drill.exec.ops.FragmentStats.newOperatorStats(OpProfileDef,
BufferAllocator)}}. However, for the following 4 operators, the
{{org.apache.drill.exec.physical.impl.BaseRootExec}} constructors also inject
{{OperatorStats}}.
{code:java}
org.apache.drill.exec.proto.beans.CoreOperatorType.SCREEN
org.apache.drill.exec.proto.beans.CoreOperatorType.SINGLE_SENDER
org.apache.drill.exec.proto.beans.CoreOperatorType.BROADCAST_SENDER
org.apache.drill.exec.proto.beans.CoreOperatorType.HASH_PARTITION_SENDER
{code}
All updates to the inputProfiles are done by the latter, while the former only
reports zero values.
The workaround is to have {{org.apache.drill.exec.ops.FragmentStats}} skip
injecting the {{org.apache.drill.exec.ops.OperatorStats}} instance for these
operators
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
