RalfJL commented on issue #12949:
URL: https://github.com/apache/pinot/issues/12949#issuecomment-2063088775
Here the two completet query plans:
First with filter a.acctuniquesessionid = 'da0f86138ec36b2364889f048bf2ac82'
```
Execution Plan
LogicalSort(sort0=[$1], dir0=[DESC])
PinotLogicalSortExchange(distribution=[hash], collation=[[1 DESC]],
isSortOnSender=[false], isSortOnReceiver=[true])
LogicalProject(acctuniquesessionid=[$14], Zeit=[/(-($17, $36), 1000)],
acctstatustype=[$12], acctstatustype0=[$31], eventtime=[$17], eventtime0=[$36])
LogicalFilter(condition=[IS NULL($48)])
LogicalJoin(condition=[AND(=($14, $42), =($33, $43), =($38, $44),
=($39, $45), =($40, $46), =($41, $47))], joinType=[left])
PinotLogicalExchange(distribution=[hash[14, 33, 38, 39, 40, 41]])
LogicalProject($docId=[$0], $hostName=[$1], $segmentName=[$2],
USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5], _3gppsgsnmccmnc=[$6],
acctinputoctets=[$7], acctinputpackets=[$8], acctoutputoctets=[$9],
acctoutputpackets=[$10], acctsessiontime=[$11], acctstatustype=[$12],
acctterminatecause=[$13], acctuniquesessionid=[$14], calledstationid=[$15],
callingstationid=[$16], eventtime=[$17], eventtimestamp=[$18], $docId0=[$21],
$hostName0=[$22], $segmentName0=[$23], USERNAME0=[$24], _3gppimsi0=[$25],
_3gppimsimccmnc0=[$26], _3gppsgsnmccmnc0=[$27], acctinputoctets0=[$28],
acctinputpackets0=[$29], acctoutputoctets0=[$30], acctoutputpackets0=[$31],
acctsessiontime0=[$32], acctstatustype0=[$33], acctterminatecause0=[$34],
acctuniquesessionid0=[$35], calledstationid0=[$36], callingstationid0=[$37],
eventtime0=[$38], eventtimestamp0=[$39], $f40=[$19], $f41=[$40], $f42=[$41],
$f43=[$20])
LogicalJoin(condition=[AND(=($35, $14), >($17, $38))],
joinType=[inner])
PinotLogicalExchange(distribution=[hash[14]])
LogicalProject($docId=[$0], $hostName=[$1],
$segmentName=[$2], USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5],
_3gppsgsnmccmnc=[$6], acctinputoctets=[$7], acctinputpackets=[$8],
acctoutputoctets=[$9], acctoutputpackets=[$10], acctsessiontime=[$11],
acctstatustype=[$12], acctterminatecause=[$13], acctuniquesessionid=[$14],
calledstationid=[$15], callingstationid=[$16], eventtime=[$17],
eventtimestamp=[$18], $f40=[>($17, 0)], $f43=[-($17, 1)])
**LogicalFilter(condition=[AND(=($14,
_UTF-8'da0f86138ec36b2364889f048bf2ac82'), =($12, _UTF-8'Stop'))])**
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[14]])
LogicalProject($docId=[$0], $hostName=[$1],
$segmentName=[$2], USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5],
_3gppsgsnmccmnc=[$6], acctinputoctets=[$7], acctinputpackets=[$8],
acctoutputoctets=[$9], acctoutputpackets=[$10], acctsessiontime=[$11],
acctstatustype=[$12], acctterminatecause=[$13], acctuniquesessionid=[$14],
calledstationid=[$15], callingstationid=[$16], eventtime=[$17],
eventtimestamp=[$18], $f41=[>($17, 0)], $f42=[+($17, 1)])
LogicalFilter(condition=[=($12, _UTF-8'Start')])
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5]])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}], agg#0=[MIN($6)])
PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5]])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}], agg#0=[MIN($6)])
LogicalProject(acctuniquesessionid0=[$2],
acctuniquesessionid00=[$3], $f40=[$4], $f41=[$5], $f42=[$6], $f43=[$7],
$f0=[true])
LogicalJoin(condition=[AND(=($3, $0), =($2, $0), >=($1,
$6), <=($1, $7))], joinType=[inner])
PinotLogicalExchange(distribution=[hash[0, 0]])
LogicalProject(acctuniquesessionid=[$14],
eventtime=[$17])
LogicalFilter(condition=[OR(=($12, _UTF-8'Start'),
=($12, _UTF-8'Stop'))])
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0, 1]])
LogicalProject(acctuniquesessionid=[$0],
acctuniquesessionid0=[$3], $f40=[$1], $f41=[$4], $f42=[$5], $f43=[$2])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
PinotLogicalExchange(distribution=[hash[0, 1, 2,
3, 4, 5]])
LogicalAggregate(group=[{0, 2, 3, 4, 6, 7}])
LogicalJoin(condition=[AND(=($4, $0), >($1,
$5))], joinType=[inner])
PinotLogicalExchange(distribution=[hash[0]])
LogicalProject(acctuniquesessionid=[$14], eventtime=[$17], $f40=[>($17, 0)],
$f43=[-($17, 1)])
**LogicalFilter(condition=[AND(=($14,
_UTF-8'da0f86138ec36b2364889f048bf2ac82'), =($12, _UTF-8'Stop'), >($17, 0))])**
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0]])
LogicalProject(acctuniquesessionid=[$14], eventtime=[$17], $f41=[>($17, 0)],
$f42=[+($17, 1)])
LogicalFilter(condition=[AND(=($12,
_UTF-8'Start'), >($17, 0))])
LogicalTableScan(table=[[radius_json]])
```
Without the filter:
```
Execution Plan
LogicalSort(sort0=[$1], dir0=[DESC])
PinotLogicalSortExchange(distribution=[hash], collation=[[1 DESC]],
isSortOnSender=[false], isSortOnReceiver=[true])
LogicalProject(acctuniquesessionid=[$14], Zeit=[/(-($17, $36), 1000)],
acctstatustype=[$12], acctstatustype0=[$31], eventtime=[$17], eventtime0=[$36])
LogicalFilter(condition=[IS NULL($48)])
LogicalJoin(condition=[AND(=($14, $42), =($33, $43), =($38, $44),
=($39, $45), =($40, $46), =($41, $47))], joinType=[left])
PinotLogicalExchange(distribution=[hash[14, 33, 38, 39, 40, 41]])
LogicalProject($docId=[$0], $hostName=[$1], $segmentName=[$2],
USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5], _3gppsgsnmccmnc=[$6],
acctinputoctets=[$7], acctinputpackets=[$8], acctoutputoctets=[$9],
acctoutputpackets=[$10], acctsessiontime=[$11], acctstatustype=[$12],
acctterminatecause=[$13], acctuniquesessionid=[$14], calledstationid=[$15],
callingstationid=[$16], eventtime=[$17], eventtimestamp=[$18], $docId0=[$21],
$hostName0=[$22], $segmentName0=[$23], USERNAME0=[$24], _3gppimsi0=[$25],
_3gppimsimccmnc0=[$26], _3gppsgsnmccmnc0=[$27], acctinputoctets0=[$28],
acctinputpackets0=[$29], acctoutputoctets0=[$30], acctoutputpackets0=[$31],
acctsessiontime0=[$32], acctstatustype0=[$33], acctterminatecause0=[$34],
acctuniquesessionid0=[$35], calledstationid0=[$36], callingstationid0=[$37],
eventtime0=[$38], eventtimestamp0=[$39], $f40=[$19], $f41=[$40], $f42=[$41],
$f43=[$20])
LogicalJoin(condition=[AND(=($35, $14), >($17, $38))],
joinType=[inner])
PinotLogicalExchange(distribution=[hash[14]])
LogicalProject($docId=[$0], $hostName=[$1],
$segmentName=[$2], USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5],
_3gppsgsnmccmnc=[$6], acctinputoctets=[$7], acctinputpackets=[$8],
acctoutputoctets=[$9], acctoutputpackets=[$10], acctsessiontime=[$11],
acctstatustype=[$12], acctterminatecause=[$13], acctuniquesessionid=[$14],
calledstationid=[$15], callingstationid=[$16], eventtime=[$17],
eventtimestamp=[$18], $f40=[>($17, 0)], $f43=[-($17, 1)])
**LogicalFilter(condition=[=($12, _UTF-8'Stop')])**
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[14]])
LogicalProject($docId=[$0], $hostName=[$1],
$segmentName=[$2], USERNAME=[$3], _3gppimsi=[$4], _3gppimsimccmnc=[$5],
_3gppsgsnmccmnc=[$6], acctinputoctets=[$7], acctinputpackets=[$8],
acctoutputoctets=[$9], acctoutputpackets=[$10], acctsessiontime=[$11],
acctstatustype=[$12], acctterminatecause=[$13], acctuniquesessionid=[$14],
calledstationid=[$15], callingstationid=[$16], eventtime=[$17],
eventtimestamp=[$18], $f41=[>($17, 0)], $f42=[+($17, 1)])
LogicalFilter(condition=[=($12, _UTF-8'Start')])
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5]])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}], agg#0=[MIN($6)])
PinotLogicalExchange(distribution=[hash[0, 1, 2, 3, 4, 5]])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}], agg#0=[MIN($6)])
LogicalProject(acctuniquesessionid0=[$2],
acctuniquesessionid00=[$3], $f40=[$4], $f41=[$5], $f42=[$6], $f43=[$7],
$f0=[true])
LogicalJoin(condition=[AND(=($3, $0), =($2, $0), >=($1,
$6), <=($1, $7))], joinType=[inner])
PinotLogicalExchange(distribution=[hash[0, 0]])
LogicalProject(acctuniquesessionid=[$14],
eventtime=[$17])
LogicalFilter(condition=[OR(=($12, _UTF-8'Start'),
=($12, _UTF-8'Stop'))])
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0, 1]])
LogicalProject(acctuniquesessionid=[$0],
acctuniquesessionid0=[$3], $f40=[$1], $f41=[$4], $f42=[$5], $f43=[$2])
LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
PinotLogicalExchange(distribution=[hash[0, 1, 2,
3, 4, 5]])
LogicalAggregate(group=[{0, 2, 3, 4, 6, 7}])
LogicalJoin(condition=[AND(=($4, $0), >($1,
$5))], joinType=[inner])
PinotLogicalExchange(distribution=[hash[0]])
LogicalProject(acctuniquesessionid=[$14], eventtime=[$17], $f40=[>($17, 0)],
$f43=[-($17, 1)])
**LogicalFilter(condition=[AND(=($12,
_UTF-8'Stop'), >($17, 0))])**
LogicalTableScan(table=[[radius_json]])
PinotLogicalExchange(distribution=[hash[0]])
LogicalProject(acctuniquesessionid=[$14], eventtime=[$17], $f41=[>($17, 0)],
$f42=[+($17, 1)])
LogicalFilter(condition=[AND(=($12,
_UTF-8'Start'), >($17, 0))])
LogicalTableScan(table=[[radius_json]])
```
Enclosed in "\*\*" the 2 lines that differs between the 2 plans. Please
note: the "\*\*" is not part of the plan, it should be bold which does not work
in code.
How can I check if any resource limit is hit?
Logs do not show an error.
The server log shows:
```
2024/04/18 06:19:45.695 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2875] Processed
requestId=5100306610870553857,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/215/148/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=192,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=264651,scanInFilter=0,scanPostFilter=529302,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:45.714 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2864] Processed
requestId=5100306610870557697,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/215/148/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=212,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=264651,scanInFilter=0,scanPostFilter=529302,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:46.555 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2850] Processed
requestId=5100306610870555905,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/215/148/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=1052,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=264651,scanInFilter=0,scanPostFilter=5028369,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:46.556 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2874] Processed
requestId=5100306610870552065,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/215/148/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=1053,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=264651,scanInFilter=0,scanPostFilter=5028369,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:46.690 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2857] Processed
requestId=5100306610870557441,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/158/158/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=1183,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=202133,scanInFilter=0,scanPostFilter=404266,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:46.729 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2837] Processed
requestId=5100306610870553601,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/158/158/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=1226,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=202133,scanInFilter=0,scanPostFilter=404266,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:47.461 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2832] Processed
requestId=5100306610870556673,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/257/257/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=1955,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=466784,scanInFilter=0,scanPostFilter=933568,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:47.545 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2855] Processed
requestId=5100306610870552833,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/257/257/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=2039,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=466784,scanInFilter=0,scanPostFilter=933568,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:47.851 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2841] Processed
requestId=5100306610870555649,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/158/158/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=2346,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=202133,scanInFilter=0,scanPostFilter=3840527,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
2024/04/18 06:19:48.745 INFO [ServerQueryLogger]
[query-runner-on-33095-1-thread-2849] Processed
requestId=5100306610870554625,table=radius_json_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=880/257/257/10/0/0/0/0/0,schedulerWaitMs=-1,reqDeserMs=-1,totalExecMs=3241,resSerMs=-1,totalTimeMs=-1,minConsumingFreshnessMs=1713358280165,broker=unknown,numDocsScanned=466784,scanInFilter=0,scanPostFilter=8868896,sched=MultistageEngine,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
```
Thanks
Ralf
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
