The aclSubmitApps can specify users or groups, so if the user name is also a group name, that can lead to confusion, e.g. https://issues.apache.org/jira/browse/IMPALA-2578 I'm not saying that is necessarily the problem, but something to check. If we still can't find anything configured incorrectly I'd recommend that you file a JIRA.
On Thu, Feb 8, 2018 at 9:05 AM, Fawze Abujaber <[email protected]> wrote: > Hi Matt, > > If this is the situation, how we can explain that when i submit the query it > runs on my secondary group pool and when nira do the same it runs under her > secondary group pool. > > More than this, the issue not related to LDAP group as i'm using the > submission restriction to a user and not a group. > > On Thu, Feb 8, 2018 at 7:00 PM, Matthew Jacobs <[email protected]> wrote: >> >> As far as I can tell from this information, your configuration looks >> fine. The only other potential issue I can think of is that the >> secondary groups placement rule isn't picking up your LDAP groups >> properly. Does the ACL behavior work if you remove the secondary >> groups rule? >> >> On Wed, Feb 7, 2018 at 10:09 PM, Fawze Abujaber <[email protected]> wrote: >> > Hi Matt, >> > >> > v2.11.0-cdh5.14.0 >> > >> > Query Runtime Profile: >> > Query (id=c843dae48ce9b289:6ad3ffc000000000): >> > Summary: >> > Session ID: ca420ac477772b56:51ae460ce9e44999 >> > Session Type: BEESWAX >> > Start Time: 2018-02-08 01:02:01.023558000 >> > End Time: 2018-02-08 01:02:01.501506000 >> > Query Type: QUERY >> > Query State: FINISHED >> > Query Status: OK >> > Impala Version: impalad version 2.11.0-cdh5.14.0 RELEASE (build >> > d68206561bce6b26762d62c01a78e6cd27aa7690) >> > User: fawzea >> > Connected User: fawzea >> > Delegated User: >> > Network Address: ::ffff:10.16.144.131:50078 >> > Default Db: default >> > Sql Statement: select count(*) from dv_entersiteevent_prq_remote >> > where >> > year=2017 and month=10 >> > Coordinator: aopr-dhc001.lpdomain.com:22000 >> > Query Options (set by configuration): REQUEST_POOL=impreports >> > Query Options (set by configuration and planner): >> > REQUEST_POOL=impreports,MT_DOP=0 >> > Plan: >> > ---------------- >> > Max Per-Host Resource Reservation: Memory=0B >> > Per-Host Resource Estimates: Memory=21.00MB >> > WARNING: The following tables are missing relevant table and/or column >> > statistics. >> > default.dv_entersiteevent_prq_remote >> > >> > F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 >> > | Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B >> > PLAN-ROOT SINK >> > | mem-estimate=0B mem-reservation=0B >> > | >> > 03:AGGREGATE [FINALIZE] >> > | output: count:merge(*) >> > | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB >> > | tuple-ids=1 row-size=8B cardinality=1 >> > | >> > 02:EXCHANGE [UNPARTITIONED] >> > | mem-estimate=0B mem-reservation=0B >> > | tuple-ids=1 row-size=8B cardinality=1 >> > | >> > F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5 >> > Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B >> > 01:AGGREGATE >> > | output: >> > sum_init_zero(default.dv_entersiteevent_prq_remote.parquet-stats: >> > num_rows) >> > | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB >> > | tuple-ids=1 row-size=8B cardinality=1 >> > | >> > 00:SCAN HDFS [default.dv_entersiteevent_prq_remote, RANDOM] >> > partitions=31/441 files=2292 size=1.39GB >> > stats-rows=unavailable extrapolated-rows=disabled >> > table stats: rows=unavailable size=unavailable >> > column stats: all >> > mem-estimate=1.00MB mem-reservation=0B >> > tuple-ids=0 row-size=8B cardinality=unavailable >> > ---------------- >> > Estimated Per-Host Mem: 22020096 >> > Tables Missing Stats: default.dv_entersiteevent_prq_remote >> > Per Host Min Reservation: aopr-dhc001.lpdomain.com:22000(0) >> > aopr-dhc002.lpdomain.com:22000(0) aopr-dhc003.lpdomain.com:22000(0) >> > aopr-dhc004.lpdomain.com:22000(0) aopr-dhc005.lpdomain.com:22000(0) >> > Request Pool: root.impreports >> > Admission result: Admitted immediately >> > ExecSummary: >> > Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem >> > Est. Peak Mem Detail >> > >> > ------------------------------------------------------------------------------------------------------------------------------ >> > 03:AGGREGATE 1 222.958us 222.958us 1 1 36.00 KB >> > 10.00 MB FINALIZE >> > 02:EXCHANGE 1 357.508ms 357.508ms 5 1 0 >> > 0 UNPARTITIONED >> > 01:AGGREGATE 5 1.027ms 1.541ms 5 1 20.00 KB >> > 10.00 MB >> > 00:SCAN HDFS 5 282.797ms 333.715ms 2.29K -1 2.62 MB >> > 1.00 MB default.dv_entersiteevent_prq_remote >> > Errors: >> > Planner Timeline: 13.508ms >> > - Analysis finished: 3.303ms (3.303ms) >> > - Value transfer graph computed: 3.465ms (162.667us) >> > - Single node plan created: 10.352ms (6.886ms) >> > - Runtime filters computed: 10.677ms (325.364us) >> > - Distributed plan created: 10.733ms (55.854us) >> > - Lineage info computed: 10.898ms (164.937us) >> > - Planning finished: 13.508ms (2.610ms) >> > Query Timeline: 481.950ms >> > - Query submitted: 79.971us (79.971us) >> > - Planning finished: 33.947ms (33.867ms) >> > - Submit for admission: 40.067ms (6.120ms) >> > - Completed admission: 40.197ms (130.146us) >> > - Ready to start on 5 backends: 40.905ms (707.852us) >> > - All 5 execution backends (6 fragment instances) started: >> > 46.417ms >> > (5.512ms) >> > - Rows available: 424.548ms (378.131ms) >> > - First row fetched: 473.488ms (48.939ms) >> > - Unregister query: 477.951ms (4.462ms) >> > - ComputeScanRangeAssignmentTimer: 3.980ms >> > ImpalaServer: >> > - ClientFetchWaitTimer: 51.659ms >> > - RowMaterializationTimer: 1.726ms >> > Execution Profile c843dae48ce9b289:6ad3ffc000000000:(Total: 385.029ms, >> > non-child: 0.000ns, % non-child: 0.00%) >> > Number of filters: 0 >> > Filter routing table: >> > ID Src. Node Tgt. Node(s) Target type Partition filter Pending >> > (Expected) First arrived Completed Enabled >> > >> > ------------------------------------------------------------------------------------------------------------------- >> > >> > Backend startup latencies: Count: 5, min / max: 3ms / 4ms, 25th >> > %-ile: >> > 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile: 4ms, >> > 99.9th %-ile: 4ms >> > Per Node Peak Memory Usage: aopr-dhc001.lpdomain.com:22000(2.68 MB) >> > aopr-dhc002.lpdomain.com:22000(2.54 MB) >> > aopr-dhc003.lpdomain.com:22000(2.54 >> > MB) aopr-dhc004.lpdomain.com:22000(2.41 MB) >> > aopr-dhc005.lpdomain.com:22000(2.56 MB) >> > - FiltersReceived: 0 (0) >> > - FinalizationTimer: 0.000ns >> > - NumBackends: 5 (5) >> > - NumFragmentInstances: 6 (6) >> > - NumFragments: 2 (2) >> > Averaged Fragment F01:(Total: 427.530ms, non-child: 49.428ms, % >> > non-child: 11.56%) >> > split sizes: min: 0, max: 0, avg: 0, stddev: 0 >> > completion times: min:428.693ms max:428.693ms mean: 428.693ms >> > stddev:0.000ns >> > execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec >> > stddev:0.00 /sec >> > num instances: 1 >> > - AverageThreadTokens: 0.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 44.02 KB (45072) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.68 MB (2806113) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 357.481ms >> > - TotalNetworkSendTime: 0.000ns >> > - TotalStorageWaitTime: 0.000ns >> > - TotalThreadsInvoluntaryContextSwitches: 0 (0) >> > - TotalThreadsTotalWallClockTime: 406.708ms >> > - TotalThreadsSysTime: 0.000ns >> > - TotalThreadsUserTime: 0.000ns >> > - TotalThreadsVoluntaryContextSwitches: 7 (7) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 49.055ms >> > - ExecTreeExecTime: 4.251us >> > - OpenTime: 357.663ms >> > - ExecTreeOpenTime: 357.649ms >> > - PrepareTime: 20.764ms >> > - ExecTreePrepareTime: 81.759us >> > PLAN_ROOT_SINK: >> > - PeakMemoryUsage: 0 >> > CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child: >> > 100.00%) >> > - CodegenTime: 0.000ns >> > - CompileTime: 0.000ns >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 0 (0) >> > - NumInstructions: 0 (0) >> > - OptimizationTime: 0.000ns >> > - PeakMemoryUsage: 0 >> > - PrepareTime: 19.685ms >> > AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child: 222.958us, % >> > non-child: 0.06%) >> > - BuildTime: 7.368us >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 36.00 KB (36864) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 2.00 /sec >> > - SpilledPartitions: 0 (0) >> > EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child: 357.508ms, % >> > non-child: 100.00%) >> > - ConvertRowBatchTime: 8.881us >> > - PeakMemoryUsage: 0 >> > - RowsReturned: 5 (5) >> > - RowsReturnedRate: 13.00 /sec >> > DataStreamReceiver: >> > - BytesReceived: 80.00 B (80) >> > - DeserializeRowBatchTimer: 24.165us >> > - FirstBatchArrivalWaitTime: 277.194ms >> > - PeakMemoryUsage: 4.02 KB (4112) >> > - SendersBlockedTimer: 0.000ns >> > - SendersBlockedTotalTimer(*): 0.000ns >> > Coordinator Fragment F01: >> > Instance c843dae48ce9b289:6ad3ffc000000000 >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 427.530ms, non-child: >> > 49.428ms, % non-child: 11.56%) >> > - AverageThreadTokens: 0.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 44.02 KB (45072) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.68 MB (2806113) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 357.481ms >> > - TotalNetworkSendTime: 0.000ns >> > - TotalStorageWaitTime: 0.000ns >> > - TotalThreadsInvoluntaryContextSwitches: 0 (0) >> > - TotalThreadsTotalWallClockTime: 406.708ms >> > - TotalThreadsSysTime: 0.000ns >> > - TotalThreadsUserTime: 0.000ns >> > - TotalThreadsVoluntaryContextSwitches: 7 (7) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 49.055ms >> > - ExecTreeExecTime: 4.251us >> > - OpenTime: 357.663ms >> > - ExecTreeOpenTime: 357.649ms >> > - PrepareTime: 20.764ms >> > - ExecTreePrepareTime: 81.759us >> > PLAN_ROOT_SINK: >> > - PeakMemoryUsage: 0 >> > CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child: >> > 100.00%) >> > - CodegenTime: 0.000ns >> > - CompileTime: 0.000ns >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 0 (0) >> > - NumInstructions: 0 (0) >> > - OptimizationTime: 0.000ns >> > - PeakMemoryUsage: 0 >> > - PrepareTime: 19.685ms >> > AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child: 222.958us, >> > % >> > non-child: 0.06%) >> > - BuildTime: 7.368us >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 36.00 KB (36864) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 2.00 /sec >> > - SpilledPartitions: 0 (0) >> > EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child: 357.508ms, % >> > non-child: 100.00%) >> > - ConvertRowBatchTime: 8.881us >> > - PeakMemoryUsage: 0 >> > - RowsReturned: 5 (5) >> > - RowsReturnedRate: 13.00 /sec >> > DataStreamReceiver: >> > - BytesReceived: 80.00 B (80) >> > - DeserializeRowBatchTimer: 24.165us >> > - FirstBatchArrivalWaitTime: 277.194ms >> > - PeakMemoryUsage: 4.02 KB (4112) >> > - SendersBlockedTimer: 0.000ns >> > - SendersBlockedTotalTimer(*): 0.000ns >> > Averaged Fragment F00:(Total: 327.360ms, non-child: 1.578ms, % >> > non-child: 0.48%) >> > split sizes: min: 283.31 MB, max: 284.33 MB, avg: 284.04 MB, >> > stddev: >> > 382.00 KB >> > completion times: min:299.457ms max:378.950ms mean: 328.881ms >> > stddev:31.507ms >> > execution rates: min:750.18 MB/sec max:949.01 MB/sec mean:871.27 >> > MB/sec stddev:79.74 MB/sec >> > num instances: 5 >> > - AverageThreadTokens: 15.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.54 MB (2666852) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.55 MB (2669309) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 303.541us >> > - TotalStorageWaitTime: 4s821ms >> > - TotalThreadsInvoluntaryContextSwitches: 104 (104) >> > - TotalThreadsTotalWallClockTime: 6s835ms >> > - TotalThreadsSysTime: 31.784ms >> > - TotalThreadsUserTime: 682.084ms >> > - TotalThreadsVoluntaryContextSwitches: 2.12K (2121) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 500.816us >> > - ExecTreeExecTime: 8.200us >> > - OpenTime: 302.848ms >> > - ExecTreeOpenTime: 281.309ms >> > - PrepareTime: 23.946ms >> > - ExecTreePrepareTime: 2.513ms >> > DataStreamSender (dst_id=2):(Total: 269.902us, non-child: >> > 269.902us, % >> > non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 373.13 KB/sec >> > - OverallThroughput: 63.53 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 14.525us >> > - TransmitDataRPCTime: 43.122us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 41.687ms, non-child: 41.687ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.194ms >> > - CompileTime: 7.360ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 13.529ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 20.084ms >> > AGGREGATION_NODE (id=1):(Total: 283.825ms, non-child: 1.027ms, % >> > non-child: 0.36%) >> > - BuildTime: 1.259ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 2.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 282.797ms, non-child: 282.797ms, % >> > non-child: 100.00%) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 14.40 >> > - BytesRead: 44.52 MB (46686028) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 44.52 MB (46686028) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 44.52 MB (46686028) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 458 (458) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 458 (458) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.50 MB (2625539) >> > - PerReadThreadRawHdfsThroughput: 1.29 GB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 269.787ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 458 (458) >> > - RowsReturnedRate: 1.64 K/sec >> > - ScanRangesComplete: 458 (458) >> > - ScannerThreadsInvoluntaryContextSwitches: 99 (99) >> > - ScannerThreadsTotalWallClockTime: 6s532ms >> > - MaterializeTupleTime(*): 70.664us >> > - ScannerThreadsSysTime: 26.184ms >> > - ScannerThreadsUserTime: 650.089ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.67K (1674) >> > - TotalRawHdfsReadTime(*): 34.416ms >> > - TotalReadThroughput: 36.86 MB/sec >> > Fragment F00: >> > Instance c843dae48ce9b289:6ad3ffc000000004 >> > (host=aopr-dhc004.lpdomain.com:22000):(Total: 377.384ms, non-child: >> > 1.619ms, >> > % non-child: 0.43%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:36/21.95 MB 11:33/19.87 MB 9:27/15.16 MB 5:44/28.07 MB 6:56/33.32 MB >> > 1:38/22.24 MB 10:44/27.90 MB 3:34/20.26 MB 2:45/28.21 MB 7:31/19.33 MB >> > 4:42/26.73 MB 8:34/21.25 MB >> > MemoryUsage(500.000ms): 1.91 MB >> > ThreadUsage(500.000ms): 25 >> > - AverageThreadTokens: 25.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.41 MB (2531689) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.41 MB (2531689) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 382.629us >> > - TotalStorageWaitTime: 5s790ms >> > - TotalThreadsInvoluntaryContextSwitches: 115 (115) >> > - TotalThreadsTotalWallClockTime: 8s122ms >> > - TotalThreadsSysTime: 33.984ms >> > - TotalThreadsUserTime: 702.879ms >> > - TotalThreadsVoluntaryContextSwitches: 2.04K (2037) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 604.005us >> > - ExecTreeExecTime: 8.338us >> > - OpenTime: 352.927ms >> > - ExecTreeOpenTime: 331.921ms >> > - PrepareTime: 23.783ms >> > - ExecTreePrepareTime: 2.559ms >> > DataStreamSender (dst_id=2):(Total: 331.818us, non-child: >> > 331.818us, >> > % non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 289.00 KB/sec >> > - OverallThroughput: 47.09 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 14.466us >> > - TransmitDataRPCTime: 54.066us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 40.950ms, non-child: 40.950ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.122ms >> > - CompileTime: 7.318ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 13.106ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 19.773ms >> > AGGREGATION_NODE (id=1):(Total: 334.482ms, non-child: 766.527us, >> > % >> > non-child: 0.23%) >> > ExecOption: Codegen Enabled >> > - BuildTime: 1.192ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 2.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 333.715ms, non-child: 333.715ms, % >> > non-child: 100.00%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:36/21.95 MB 11:33/19.87 MB 9:27/15.16 MB 5:44/28.07 MB 6:56/33.32 MB >> > 1:38/22.24 MB 10:44/27.90 MB 3:34/20.26 MB 2:45/28.21 MB 7:31/19.33 MB >> > 4:42/26.73 MB 8:34/21.25 MB >> > ExecOption: PARQUET Codegen Enabled, Codegen enabled: 464 out >> > of >> > 464 >> > Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% >> > 4:0% >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% >> > File Formats: PARQUET/Unknown(Skipped):464 >> > BytesRead(500.000ms): 12.27 MB >> > - FooterProcessingTime: (Avg: 13.555ms ; Min: 5.433ms ; Max: >> > 62.957ms ; Number of samples: 464) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 24.00 >> > - BytesRead: 45.01 MB (47193988) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 45.01 MB (47193988) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 45.01 MB (47193988) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 464 (464) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 464 (464) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.38 MB (2490376) >> > - PerReadThreadRawHdfsThroughput: 1016.44 MB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 320.860ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 464 (464) >> > - RowsReturnedRate: 1.39 K/sec >> > - ScanRangesComplete: 464 (464) >> > - ScannerThreadsInvoluntaryContextSwitches: 108 (108) >> > - ScannerThreadsTotalWallClockTime: 7s769ms >> > - MaterializeTupleTime(*): 75.514us >> > - ScannerThreadsSysTime: 29.984ms >> > - ScannerThreadsUserTime: 669.884ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.60K (1597) >> > - TotalRawHdfsReadTime(*): 44.279ms >> > - TotalReadThroughput: 24.53 MB/sec >> > Instance c843dae48ce9b289:6ad3ffc000000002 >> > (host=aopr-dhc003.lpdomain.com:22000):(Total: 351.014ms, non-child: >> > 1.641ms, >> > % non-child: 0.47%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:35/20.89 MB 7:44/26.64 MB 11:27/17.21 MB 9:32/18.61 MB 5:25/14.56 MB >> > 2:38/24.32 MB 6:31/19.85 MB 8:50/30.76 MB 10:41/25.92 MB 3:31/19.78 MB >> > 4:60/37.59 MB 1:44/27.19 MB >> > MemoryUsage(500.000ms): 1.16 MB >> > ThreadUsage(500.000ms): 25 >> > - AverageThreadTokens: 25.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.54 MB (2662753) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.54 MB (2662753) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 342.746us >> > - TotalStorageWaitTime: 4s796ms >> > - TotalThreadsInvoluntaryContextSwitches: 93 (93) >> > - TotalThreadsTotalWallClockTime: 7s286ms >> > - TotalThreadsSysTime: 27.988ms >> > - TotalThreadsUserTime: 671.887ms >> > - TotalThreadsVoluntaryContextSwitches: 2.08K (2079) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 524.006us >> > - ExecTreeExecTime: 7.920us >> > - OpenTime: 327.802ms >> > - ExecTreeOpenTime: 307.154ms >> > - PrepareTime: 22.617ms >> > - ExecTreePrepareTime: 2.650ms >> > DataStreamSender (dst_id=2):(Total: 348.990us, non-child: >> > 348.990us, >> > % non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 354.47 KB/sec >> > - OverallThroughput: 44.77 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 15.602us >> > - TransmitDataRPCTime: 44.080us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 39.215ms, non-child: 39.215ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.183ms >> > - CompileTime: 7.168ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 12.929ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 18.453ms >> > AGGREGATION_NODE (id=1):(Total: 309.807ms, non-child: 948.677us, >> > % >> > non-child: 0.31%) >> > ExecOption: Codegen Enabled >> > - BuildTime: 1.552ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 3.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 308.859ms, non-child: 308.859ms, % >> > non-child: 100.00%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:35/20.89 MB 7:44/26.64 MB 11:27/17.21 MB 9:32/18.61 MB 5:25/14.56 MB >> > 2:38/24.32 MB 6:31/19.85 MB 8:50/30.76 MB 10:41/25.92 MB 3:31/19.78 MB >> > 4:60/37.59 MB 1:44/27.19 MB >> > ExecOption: PARQUET Codegen Enabled, Codegen enabled: 458 out >> > of >> > 458 >> > Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% >> > 4:0% >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% >> > File Formats: PARQUET/Unknown(Skipped):458 >> > BytesRead(500.000ms): 36.74 MB >> > - FooterProcessingTime: (Avg: 11.502ms ; Min: 3.668ms ; Max: >> > 23.564ms ; Number of samples: 458) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 24.00 >> > - BytesRead: 44.51 MB (46668432) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 44.51 MB (46668432) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 44.51 MB (46668432) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 458 (458) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 458 (458) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.50 MB (2621440) >> > - PerReadThreadRawHdfsThroughput: 1.26 GB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 295.595ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 458 (458) >> > - RowsReturnedRate: 1.48 K/sec >> > - ScanRangesComplete: 458 (458) >> > - ScannerThreadsInvoluntaryContextSwitches: 91 (91) >> > - ScannerThreadsTotalWallClockTime: 6s958ms >> > - MaterializeTupleTime(*): 71.630us >> > - ScannerThreadsSysTime: 16.989ms >> > - ScannerThreadsUserTime: 645.891ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.63K (1630) >> > - TotalRawHdfsReadTime(*): 34.377ms >> > - TotalReadThroughput: 73.49 MB/sec >> > Instance c843dae48ce9b289:6ad3ffc000000001 >> > (host=aopr-dhc005.lpdomain.com:22000):(Total: 308.195ms, non-child: >> > 1.784ms, >> > % non-child: 0.58%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 2:46/28.97 MB 8:39/24.34 MB 10:33/21.03 MB 3:43/26.50 MB 5:32/20.27 MB >> > 9:32/19.86 MB 6:25/16.01 MB 7:41/24.77 MB 1:42/25.01 MB 0:55/32.51 MB >> > 4:40/24.29 MB 11:33/20.77 MB >> > - AverageThreadTokens: 0.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.56 MB (2683241) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.56 MB (2683241) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 292.765us >> > - TotalStorageWaitTime: 4s482ms >> > - TotalThreadsInvoluntaryContextSwitches: 113 (113) >> > - TotalThreadsTotalWallClockTime: 6s310ms >> > - TotalThreadsSysTime: 35.983ms >> > - TotalThreadsUserTime: 682.884ms >> > - TotalThreadsVoluntaryContextSwitches: 2.14K (2143) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 532.572us >> > - ExecTreeExecTime: 8.377us >> > - OpenTime: 280.241ms >> > - ExecTreeOpenTime: 258.652ms >> > - PrepareTime: 27.342ms >> > - ExecTreePrepareTime: 2.863ms >> > DataStreamSender (dst_id=2):(Total: 309.794us, non-child: >> > 309.794us, >> > % non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 356.91 KB/sec >> > - OverallThroughput: 50.44 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 14.262us >> > - TransmitDataRPCTime: 43.778us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 44.584ms, non-child: 44.584ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.382ms >> > - CompileTime: 7.346ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 13.462ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 23.006ms >> > AGGREGATION_NODE (id=1):(Total: 261.516ms, non-child: 836.656us, >> > % >> > non-child: 0.32%) >> > ExecOption: Codegen Enabled >> > - BuildTime: 1.280ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 3.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 260.680ms, non-child: 260.680ms, % >> > non-child: 100.00%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 2:46/28.97 MB 8:39/24.34 MB 10:33/21.03 MB 3:43/26.50 MB 5:32/20.27 MB >> > 9:32/19.86 MB 6:25/16.01 MB 7:41/24.77 MB 1:42/25.01 MB 0:55/32.51 MB >> > 4:40/24.29 MB 11:33/20.77 MB >> > ExecOption: PARQUET Codegen Enabled, Codegen enabled: 461 out >> > of >> > 461 >> > Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% >> > 5:0% >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% >> > File Formats: PARQUET/Unknown(Skipped):461 >> > - FooterProcessingTime: (Avg: 10.793ms ; Min: 3.087ms ; Max: >> > 20.253ms ; Number of samples: 461) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 0.00 >> > - BytesRead: 44.73 MB (46906407) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 44.73 MB (46906407) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 44.73 MB (46906407) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 461 (461) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 461 (461) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.52 MB (2641928) >> > - PerReadThreadRawHdfsThroughput: 1.31 GB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 246.691ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 461 (461) >> > - RowsReturnedRate: 1.77 K/sec >> > - ScanRangesComplete: 461 (461) >> > - ScannerThreadsInvoluntaryContextSwitches: 109 (109) >> > - ScannerThreadsTotalWallClockTime: 6s030ms >> > - MaterializeTupleTime(*): 68.735us >> > - ScannerThreadsSysTime: 33.983ms >> > - ScannerThreadsUserTime: 647.889ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.70K (1696) >> > - TotalRawHdfsReadTime(*): 33.405ms >> > - TotalReadThroughput: 0.00 /sec >> > Instance c843dae48ce9b289:6ad3ffc000000003 >> > (host=aopr-dhc002.lpdomain.com:22000):(Total: 302.116ms, non-child: >> > 1.604ms, >> > % non-child: 0.53%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:27/16.21 MB 11:33/21.23 MB 6:36/23.15 MB 10:41/25.09 MB 1:41/24.13 MB >> > 9:32/18.79 MB 5:41/25.31 MB 2:42/26.49 MB 7:36/22.87 MB 3:51/31.00 MB >> > 4:41/25.99 MB 8:38/23.79 MB >> > MemoryUsage(500.000ms): 700.35 KB >> > ThreadUsage(500.000ms): 25 >> > - AverageThreadTokens: 25.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.54 MB (2662753) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.54 MB (2662753) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 300.675us >> > - TotalStorageWaitTime: 4s661ms >> > - TotalThreadsInvoluntaryContextSwitches: 90 (90) >> > - TotalThreadsTotalWallClockTime: 6s256ms >> > - TotalThreadsSysTime: 30.983ms >> > - TotalThreadsUserTime: 709.880ms >> > - TotalThreadsVoluntaryContextSwitches: 2.17K (2175) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 492.627us >> > - ExecTreeExecTime: 8.191us >> > - OpenTime: 277.922ms >> > - ExecTreeOpenTime: 255.395ms >> > - PrepareTime: 23.647ms >> > - ExecTreePrepareTime: 2.243ms >> > DataStreamSender (dst_id=2):(Total: 194.517us, non-child: >> > 194.517us, >> > % non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 371.86 KB/sec >> > - OverallThroughput: 80.33 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 14.035us >> > - TransmitDataRPCTime: 42.019us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 42.676ms, non-child: 42.676ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.236ms >> > - CompileTime: 7.717ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 14.130ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 20.109ms >> > AGGREGATION_NODE (id=1):(Total: 257.640ms, non-child: 1.541ms, % >> > non-child: 0.60%) >> > ExecOption: Codegen Enabled >> > - BuildTime: 1.252ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 3.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 256.099ms, non-child: 256.099ms, % >> > non-child: 100.00%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 0:27/16.21 MB 11:33/21.23 MB 6:36/23.15 MB 10:41/25.09 MB 1:41/24.13 MB >> > 9:32/18.79 MB 5:41/25.31 MB 2:42/26.49 MB 7:36/22.87 MB 3:51/31.00 MB >> > 4:41/25.99 MB 8:38/23.79 MB >> > ExecOption: PARQUET Codegen Enabled, Codegen enabled: 459 out >> > of >> > 459 >> > Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% >> > 4:0% >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% >> > File Formats: PARQUET/Unknown(Skipped):459 >> > BytesRead(500.000ms): 43.15 MB >> > - FooterProcessingTime: (Avg: 11.257ms ; Min: 3.989ms ; Max: >> > 23.004ms ; Number of samples: 459) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 24.00 >> > - BytesRead: 44.52 MB (46678031) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 44.52 MB (46678031) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 44.52 MB (46678031) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 459 (459) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 459 (459) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.50 MB (2621440) >> > - PerReadThreadRawHdfsThroughput: 1.33 GB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 243.426ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 459 (459) >> > - RowsReturnedRate: 1.79 K/sec >> > - ScanRangesComplete: 459 (459) >> > - ScannerThreadsInvoluntaryContextSwitches: 85 (85) >> > - ScannerThreadsTotalWallClockTime: 5s978ms >> > - MaterializeTupleTime(*): 67.997us >> > - ScannerThreadsSysTime: 22.984ms >> > - ScannerThreadsUserTime: 678.884ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.72K (1724) >> > - TotalRawHdfsReadTime(*): 32.626ms >> > - TotalReadThroughput: 86.30 MB/sec >> > Instance c843dae48ce9b289:6ad3ffc000000005 >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 298.093ms, non-child: >> > 1.242ms, >> > % non-child: 0.42%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 9:55/35.38 MB 0:37/24.23 MB 5:37/23.31 MB 11:22/13.79 MB 8:30/18.53 MB >> > 4:33/20.45 MB 1:38/24.06 MB 10:29/18.40 MB 3:39/24.35 MB 2:41/25.45 MB >> > 6:42/27.03 MB 7:47/29.22 MB >> > - AverageThreadTokens: 0.00 >> > - BloomFilterBytes: 0 >> > - PeakMemoryUsage: 2.66 MB (2793825) >> > - PeakReservation: 0 >> > - PeakUsedReservation: 0 >> > - PerHostPeakMemUsage: 2.68 MB (2806113) >> > - RowsProduced: 1 (1) >> > - TotalNetworkReceiveTime: 0.000ns >> > - TotalNetworkSendTime: 198.894us >> > - TotalStorageWaitTime: 4s374ms >> > - TotalThreadsInvoluntaryContextSwitches: 110 (110) >> > - TotalThreadsTotalWallClockTime: 6s201ms >> > - TotalThreadsSysTime: 29.983ms >> > - TotalThreadsUserTime: 642.892ms >> > - TotalThreadsVoluntaryContextSwitches: 2.17K (2174) >> > Fragment Instance Lifecycle Timings: >> > - ExecTime: 350.873us >> > - ExecTreeExecTime: 8.173us >> > - OpenTime: 275.347ms >> > - ExecTreeOpenTime: 253.425ms >> > - PrepareTime: 22.341ms >> > - ExecTreePrepareTime: 2.251ms >> > DataStreamSender (dst_id=2):(Total: 164.394us, non-child: >> > 164.394us, >> > % non-child: 100.00%) >> > - BytesSent: 16.00 B (16) >> > - NetworkThroughput(*): 493.42 KB/sec >> > - OverallThroughput: 95.05 KB/sec >> > - PeakMemoryUsage: 16.00 KB (16384) >> > - RowsReturned: 1 (1) >> > - SerializeBatchTime: 14.262us >> > - TransmitDataRPCTime: 31.667us >> > - UncompressedRowBatchSize: 16.00 B (16) >> > CodeGen:(Total: 41.008ms, non-child: 41.008ms, % non-child: >> > 100.00%) >> > - CodegenTime: 1.049ms >> > - CompileTime: 7.248ms >> > - LoadTime: 0.000ns >> > - ModuleBitcodeSize: 1.86 MB (1950880) >> > - NumFunctions: 15 (15) >> > - NumInstructions: 214 (214) >> > - OptimizationTime: 14.018ms >> > - PeakMemoryUsage: 107.00 KB (109568) >> > - PrepareTime: 19.080ms >> > AGGREGATION_NODE (id=1):(Total: 255.678ms, non-child: 1.046ms, % >> > non-child: 0.41%) >> > ExecOption: Codegen Enabled >> > - BuildTime: 1.015ms >> > - GetResultsTime: 0.000ns >> > - HTResizeTime: 0.000ns >> > - HashBuckets: 0 (0) >> > - LargestPartitionPercent: 0 (0) >> > - MaxPartitionLevel: 0 (0) >> > - NumRepartitions: 0 (0) >> > - PartitionsCreated: 0 (0) >> > - PeakMemoryUsage: 20.00 KB (20480) >> > - RowsRepartitioned: 0 (0) >> > - RowsReturned: 1 (1) >> > - RowsReturnedRate: 3.00 /sec >> > - SpilledPartitions: 0 (0) >> > HDFS_SCAN_NODE (id=0):(Total: 254.631ms, non-child: 254.631ms, % >> > non-child: 100.00%) >> > Hdfs split stats (<volume id>:<# splits>/<split lengths>): >> > 9:55/35.38 MB 0:37/24.23 MB 5:37/23.31 MB 11:22/13.79 MB 8:30/18.53 MB >> > 4:33/20.45 MB 1:38/24.06 MB 10:29/18.40 MB 3:39/24.35 MB 2:41/25.45 MB >> > 6:42/27.03 MB 7:47/29.22 MB >> > ExecOption: PARQUET Codegen Enabled, Codegen enabled: 450 out >> > of >> > 450 >> > Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% >> > 5:0% >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% >> > File Formats: PARQUET/Unknown(Skipped):450 >> > - FooterProcessingTime: (Avg: 10.727ms ; Min: 3.813ms ; Max: >> > 23.755ms ; Number of samples: 450) >> > - AverageHdfsReadThreadConcurrency: 0.00 >> > - AverageScannerThreadConcurrency: 0.00 >> > - BytesRead: 43.85 MB (45983284) >> > - BytesReadDataNodeCache: 0 >> > - BytesReadLocal: 43.85 MB (45983284) >> > - BytesReadRemoteUnexpected: 0 >> > - BytesReadShortCircuit: 43.85 MB (45983284) >> > - CachedFileHandlesHitCount: 0 (0) >> > - CachedFileHandlesMissCount: 450 (450) >> > - CollectionItemsRead: 0 (0) >> > - DecompressionTime: 0.000ns >> > - MaxCompressedTextFileLength: 0 >> > - NumColumns: 0 (0) >> > - NumDictFilteredRowGroups: 0 (0) >> > - NumDisksAccessed: 12 (12) >> > - NumRowGroups: 450 (450) >> > - NumScannerThreadsStarted: 24 (24) >> > - NumScannersWithNoReads: 0 (0) >> > - NumStatsFilteredRowGroups: 0 (0) >> > - PeakMemoryUsage: 2.62 MB (2752512) >> > - PerReadThreadRawHdfsThroughput: 1.56 GB/sec >> > - RemoteScanRanges: 0 (0) >> > - RowBatchQueueGetWaitTime: 242.362ms >> > - RowBatchQueuePutWaitTime: 0.000ns >> > - RowsRead: 0 (0) >> > - RowsReturned: 450 (450) >> > - RowsReturnedRate: 1.77 K/sec >> > - ScanRangesComplete: 450 (450) >> > - ScannerThreadsInvoluntaryContextSwitches: 106 (106) >> > - ScannerThreadsTotalWallClockTime: 5s926ms >> > - MaterializeTupleTime(*): 69.447us >> > - ScannerThreadsSysTime: 26.984ms >> > - ScannerThreadsUserTime: 607.897ms >> > - ScannerThreadsVoluntaryContextSwitches: 1.72K (1723) >> > - TotalRawHdfsReadTime(*): 27.393ms >> > - TotalReadThroughput: 0.00 /sec >> > =================================================== >> > >> > Coordinator log: >> > >> > 01: query (string) = "select count(*) from dv_entersiteevent_prq_remote >> > where year=2017 and month=10", >> > 03: configuration (list) = list<string>[1] { >> > [0] = "REQUEST_POOL=impreports", >> > }, >> > 04: hadoop_user (string) = "fawzea", >> > } >> > I0208 01:07:06.321281 7154 impala-beeswax-server.cc:459] >> > TClientRequest.queryOptions: TQueryOptions { >> > 01: abort_on_error (bool) = false, >> > 02: max_errors (i32) = 100, >> > 03: disable_codegen (bool) = false, >> > 04: batch_size (i32) = 0, >> > 05: num_nodes (i32) = 0, >> > 06: max_scan_range_length (i64) = 0, >> > 07: num_scanner_threads (i32) = 0, >> > 08: max_io_buffers (i32) = 0, >> > 09: allow_unsupported_formats (bool) = false, >> > 10: default_order_by_limit (i64) = -1, >> > 11: debug_action (string) = "", >> > 12: mem_limit (i64) = 0, >> > 13: abort_on_default_limit_exceeded (bool) = false, >> > 15: hbase_caching (i32) = 0, >> > 16: hbase_cache_blocks (bool) = false, >> > 17: parquet_file_size (i64) = 0, >> > 18: explain_level (i32) = 1, >> > 19: sync_ddl (bool) = false, >> > 20: request_pool (string) = "impreports", >> > 23: disable_cached_reads (bool) = false, >> > 24: disable_outermost_topn (bool) = false, >> > 25: rm_initial_mem (i64) = 0, >> > 26: query_timeout_s (i32) = 0, >> > 28: appx_count_distinct (bool) = false, >> > 29: disable_unsafe_spills (bool) = false, >> > 31: exec_single_node_rows_threshold (i32) = 100, >> > 32: optimize_partition_key_scans (bool) = false, >> > 33: replica_preference (i32) = 0, >> > 34: schedule_random_replica (bool) = false, >> > 35: scan_node_codegen_threshold (i64) = 1800000, >> > 36: disable_streaming_preaggregations (bool) = false, >> > 37: runtime_filter_mode (i32) = 2, >> > 38: runtime_bloom_filter_size (i32) = 1048576, >> > 39: runtime_filter_wait_time_ms (i32) = 0, >> > 40: disable_row_runtime_filtering (bool) = false, >> > 41: max_num_runtime_filters (i32) = 10, >> > 42: parquet_annotate_strings_utf8 (bool) = false, >> > 43: parquet_fallback_schema_resolution (i32) = 0, >> > 45: s3_skip_insert_staging (bool) = true, >> > 46: runtime_filter_min_size (i32) = 1048576, >> > 47: runtime_filter_max_size (i32) = 16777216, >> > 48: prefetch_mode (i32) = 1, >> > 49: strict_mode (bool) = false, >> > 50: scratch_limit (i64) = -1, >> > 51: enable_expr_rewrites (bool) = true, >> > 52: decimal_v2 (bool) = false, >> > 53: parquet_dictionary_filtering (bool) = true, >> > 54: parquet_array_resolution (i32) = 2, >> > 55: parquet_read_statistics (bool) = true, >> > 56: default_join_distribution_mode (i32) = 0, >> > 57: disable_codegen_rows_threshold (i32) = 50000, >> > 58: default_spillable_buffer_size (i64) = 2097152, >> > 59: min_spillable_buffer_size (i64) = 65536, >> > 60: max_row_size (i64) = 524288, >> > } >> > I0208 01:07:06.324410 7154 Frontend.java:909] Compiling query: select >> > count(*) from dv_entersiteevent_prq_remote where year=2017 and month=10 >> > I0208 01:07:06.327203 7154 Frontend.java:948] Compiled query. >> > I0208 01:07:06.359748 7154 admission-controller.cc:510] Schedule for >> > id=5f49e927591a9042:faf8f28600000000 in pool_name=root.impreports >> > cluster_mem_needed=105.00 MB PoolConfig: max_requests=-1 max_queued=200 >> > max_mem=-1.00 B >> > I0208 01:07:06.359776 7154 admission-controller.cc:515] Stats: >> > agg_num_running=0, agg_num_queued=0, agg_mem_reserved=0, >> > local_host(local_mem_admitted=0, num_admitted_running=0, num_queued=0, >> > backend_mem_reserved=0) >> > I0208 01:07:06.359784 7154 admission-controller.cc:531] Admitted query >> > id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.359807 7154 coordinator.cc:99] Exec() >> > query_id=5f49e927591a9042:faf8f28600000000 stmt=select count(*) from >> > dv_entersiteevent_prq_remote where year=2017 and month=10 >> > I0208 01:07:06.360694 7154 coordinator.cc:357] starting execution on 5 >> > backends for query 5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.363255 63583 impala-internal-service.cc:44] >> > ExecQueryFInstances(): query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.363282 63583 query-exec-mgr.cc:46] StartQueryFInstances() >> > query_id=5f49e927591a9042:faf8f28600000000 >> > coord=aopr-dhc001.lpdomain.com:22000 >> > I0208 01:07:06.363296 63583 query-state.cc:173] Buffer pool limit for >> > 5f49e927591a9042:faf8f28600000000: 10802010521 >> > I0208 01:07:06.363364 63583 initial-reservations.cc:60] Successfully >> > claimed >> > initial reservations (0) for query 5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.363544 7155 query-state.cc:286] StartFInstances(): >> > query_id=5f49e927591a9042:faf8f28600000000 #instances=2 >> > I0208 01:07:06.363844 7155 query-state.cc:299] descriptor table for >> > query=5f49e927591a9042:faf8f28600000000 >> > tuples: >> > Tuple(id=2 size=0 slots=[] tuple_path=[]) >> > Tuple(id=1 size=8 slots=[Slot(id=2 type=BIGINT col_path=[] offset=0 >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[]) >> > Tuple(id=0 size=8 slots=[Slot(id=3 type=BIGINT col_path=[] offset=0 >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[]) >> > I0208 01:07:06.364013 7156 query-state.cc:377] Executing instance. >> > instance_id=5f49e927591a9042:faf8f28600000000 fragment_idx=0 >> > per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=1 >> > I0208 01:07:06.364135 7157 query-state.cc:377] Executing instance. >> > instance_id=5f49e927591a9042:faf8f28600000005 fragment_idx=1 >> > per_fragment_instance_idx=4 coord_state_idx=0 #in-flight=2 >> > I0208 01:07:06.364241 7157 hdfs-scan-node.cc:160] Max row batch queue >> > size >> > for scan node '0' in fragment instance >> > '5f49e927591a9042:faf8f28600000005': >> > 160 >> > I0208 01:07:06.366775 7154 coordinator.cc:370] started execution on 5 >> > backends for query 5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.385625 7155 query-exec-mgr.cc:149] ReleaseQueryState(): >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=4 >> > I0208 01:07:06.671193 7559 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.3:55598 closed, closing 1 associated session(s) >> > I0208 01:07:06.676138 544 data-stream-mgr.cc:228] Reduced stream ID >> > cache >> > from 2 items, to 0, eviction took: 0 >> > I0208 01:07:06.677202 544 coordinator.cc:955] Backend completed: >> > host=aopr-dhc002.lpdomain.com:22000 remaining=4 >> > I0208 01:07:06.677214 544 coordinator-backend-state.cc:229] >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: >> > aopr-dhc001.lpdomain.com:22000 >> > I0208 01:07:06.684221 7157 query-state.cc:384] Instance completed. >> > instance_id=5f49e927591a9042:faf8f28600000005 #in-flight=1 status=OK >> > I0208 01:07:06.684237 7157 query-exec-mgr.cc:149] ReleaseQueryState(): >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=3 >> > I0208 01:07:06.700275 520 coordinator.cc:955] Backend completed: >> > host=aopr-dhc005.lpdomain.com:22000 remaining=3 >> > I0208 01:07:06.700289 520 coordinator-backend-state.cc:229] >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: >> > aopr-dhc001.lpdomain.com:22000 >> > I0208 01:07:06.704620 65258 coordinator.cc:955] Backend completed: >> > host=aopr-dhc003.lpdomain.com:22000 remaining=2 >> > I0208 01:07:06.704666 65258 coordinator-backend-state.cc:229] >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: >> > aopr-dhc001.lpdomain.com:22000 >> > I0208 01:07:06.706041 7156 data-stream-mgr.cc:238] DeregisterRecvr(): >> > fragment_instance_id=5f49e927591a9042:faf8f28600000000, node=2 >> > I0208 01:07:06.706070 7156 data-stream-recvr.cc:235] cancelled stream: >> > fragment_instance_id_=5f49e927591a9042:faf8f28600000000 node_id=2 >> > I0208 01:07:06.706809 64916 coordinator.cc:955] Backend completed: >> > host=aopr-dhc004.lpdomain.com:22000 remaining=1 >> > I0208 01:07:06.706821 64916 coordinator-backend-state.cc:229] >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress backend: >> > aopr-dhc001.lpdomain.com:22000 >> > I0208 01:07:06.789568 7154 impala-beeswax-server.cc:193] >> > get_results_metadata(): query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.790439 7154 coordinator.cc:789] Coordinator waiting for >> > backends to finish, 1 remaining >> > I0208 01:07:06.791052 7154 coordinator.cc:794] All backends finished >> > successfully. >> > I0208 01:07:06.791086 7154 coordinator.cc:1090] Release admssion >> > control >> > resources for query 5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.791363 7156 query-state.cc:384] Instance completed. >> > instance_id=5f49e927591a9042:faf8f28600000000 #in-flight=0 status=OK >> > I0208 01:07:06.791414 7156 query-exec-mgr.cc:149] ReleaseQueryState(): >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=2 >> > I0208 01:07:06.793586 7154 impala-beeswax-server.cc:235] close(): >> > query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.793602 7154 impala-server.cc:992] UnregisterQuery(): >> > query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.793614 7154 impala-server.cc:1075] Cancel(): >> > query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.793624 7154 coordinator.cc:895] Cancel() >> > query_id=5f49e927591a9042:faf8f28600000000 >> > I0208 01:07:06.793637 7154 coordinator.cc:905] CancelBackends() >> > query_id=5f49e927591a9042:faf8f28600000000, tried to cancel 0 backends >> > I0208 01:07:06.802776 7154 query-exec-mgr.cc:149] ReleaseQueryState(): >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=1 >> > I0208 01:07:06.814214 7154 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.144.131:51012 closed, closing 1 associated session(s) >> > I0208 01:07:07.258891 7567 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.2:56375 closed, closing 1 associated session(s) >> > I0208 01:07:11.600131 7819 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.3:27460 closed, closing 1 associated session(s) >> > I0208 01:07:12.262739 7821 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.2:52672 closed, closing 1 associated session(s) >> > I0208 01:07:16.665451 7822 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.3:10693 closed, closing 1 associated session(s) >> > I0208 01:07:17.235321 7825 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.2:34752 closed, closing 1 associated session(s) >> > I0208 01:07:21.410611 5858 impala-beeswax-server.cc:193] >> > get_results_metadata(): query_id=1d4fbf0f2571f1e9:ea346f7900000000 >> > I0208 01:07:21.414175 5858 impala-beeswax-server.cc:235] close(): >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000 >> > I0208 01:07:21.414185 5858 impala-server.cc:992] UnregisterQuery(): >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000 >> > I0208 01:07:21.414191 5858 impala-server.cc:1075] Cancel(): >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000 >> > I0208 01:07:21.421505 5858 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.26.180.75:60072 closed, closing 1 associated session(s) >> > I0208 01:07:21.605690 7830 impala-server.cc:1796] Connection from >> > client >> > ::ffff:10.16.136.3:46660 closed, closing 1 associated session(s) >> > >> > >> > On Thu, Feb 8, 2018 at 6:20 AM, Matthew Jacobs <[email protected]> >> > wrote: >> >> >> >> Thanks! I think that configuration looks OK. >> >> >> >> 1) What version of Impala is this? >> >> >> >> 2) Can you send the query profile and the Impala coordinator log for >> >> the query where it's not behaving as expected? >> >> >> >> > When i'm running the following with fawzea user: >> >> > set request_pool=impreports; >> >> > select count(*) from dv_entersiteevent_prq_remote where year=2017 and >> >> > month=10; >> >> > I can see that the query is running with impreports pool. >> >> >> >> On Wed, Feb 7, 2018 at 7:59 PM, Fawze Abujaber <[email protected]> >> >> wrote: >> >> > Hi Matt, >> >> > >> >> > [root@aopr-dhc001 ~]# cat >> >> > >> >> > >> >> > /var/run/cloudera-scm-agent/process/4558-impala-IMPALAD/impala-conf/fair-scheduler.xml >> >> > <?xml version="1.0" encoding="UTF-8" standalone="yes"?> >> >> > <allocations> >> >> > <queue name="root"> >> >> > <weight>1.0</weight> >> >> > <schedulingPolicy>fair</schedulingPolicy> >> >> > <queue name="impiaas"> >> >> > <weight>1.0</weight> >> >> > <schedulingPolicy>fair</schedulingPolicy> >> >> > <aclSubmitApps>fawzea </aclSubmitApps> >> >> > </queue> >> >> > <queue name="impreports"> >> >> > <weight>1.0</weight> >> >> > <schedulingPolicy>fair</schedulingPolicy> >> >> > <aclSubmitApps>nira </aclSubmitApps> >> >> > </queue> >> >> > </queue> >> >> > <defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPolicy> >> >> > <queuePlacementPolicy> >> >> > <rule name="specified" create="false"/> >> >> > <rule name="secondaryGroupExistingQueue"/> >> >> > <rule name="default"/> >> >> > </queuePlacementPolicy> >> >> > </allocations> >> >> > >> >> > >> >> > On Thu, Feb 8, 2018 at 4:47 AM, Matthew Jacobs <[email protected]> >> >> > wrote: >> >> >> >> >> >> Yes, I saw that, but I'd like to see the actual configuration that >> >> >> is >> >> >> passed to Impala because it may not be specified in the right way to >> >> >> accomplish what you're trying to do in "b". >> >> >> >> >> >> On Wed, Feb 7, 2018 at 4:32 PM, Fawze Abujaber <[email protected]> >> >> >> wrote: >> >> >> > Hi Matt, >> >> >> > >> >> >> > I created the pools manually and it’s completely nfigurations as >> >> >> > shown >> >> >> > in b. >> >> >> > >> >> >> > On Wed, 7 Feb 2018 at 23:17 Matthew Jacobs <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> How, more specifically, are you configuring these pools? Can you >> >> >> >> share >> >> >> >> the configuration files? >> >> >> >> >> >> >> >> On Wed, Feb 7, 2018 at 11:42 AM, Fawze Abujaber >> >> >> >> <[email protected]> >> >> >> >> wrote: >> >> >> >> > Hello All, >> >> >> >> > >> >> >> >> > Will appreciate any insight of what i'm missing in my >> >> >> >> > configuration >> >> >> >> > to >> >> >> >> > get >> >> >> >> > the impala submission control working as expected .... >> >> >> >> > >> >> >> >> > a- Here is my placement Placement Rule: >> >> >> >> > 1- Use the pool Specified at run time., only if the pool >> >> >> >> > exists. >> >> >> >> > 2- Use the pool root.[secondary group]. >> >> >> >> > 3- Use the pool root.default. >> >> >> >> > >> >> >> >> > b- Here is the impala resource pools >> >> >> >> > >> >> >> >> > Name Max Memory Max Running Queries Max >> >> >> >> > Queued >> >> >> >> > Queries >> >> >> >> > Queue Timeout Default Query Memory Limit Access Control >> >> >> >> > root.impiaas Unlimited Unlimited >> >> >> >> > 200 1 minute(s) No Default >> >> >> >> > Custom >> >> >> >> > root.impreports Unlimited Unlimited >> >> >> >> > 200 1 minute(s) No Default >> >> >> >> > Custom >> >> >> >> > >> >> >> >> > >> >> >> >> > C- Here is my Access control for root.impiaas pool: >> >> >> >> > >> >> >> >> > Allow these users and groups to submit to this pool >> >> >> >> > >> >> >> >> > Users: fawzea >> >> >> >> > >> >> >> >> > The same for impreports the allowed user and groups to submit >> >> >> >> > to >> >> >> >> > pool >> >> >> >> > is >> >> >> >> > only nira. >> >> >> >> > >> >> >> >> > >> >> >> >> > LDAP groups for fawzea user: >> >> >> >> > >> >> >> >> > Member in groups: >> >> >> >> > >> >> >> >> > - dba >> >> >> >> > - impiaas >> >> >> >> > - mysql >> >> >> >> > >> >> >> >> > LDAP group for nira user >> >> >> >> > Member in groups: >> >> >> >> > - dba >> >> >> >> > - impreports >> >> >> >> > - mysql >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > Impala Queries UI shows: >> >> >> >> > >> >> >> >> > 1- select count(*) from dv_entersiteevent_prq_remote where >> >> >> >> > year=2017 >> >> >> >> > >> >> >> >> > User: fawzea >> >> >> >> > Pool: root.impiaas >> >> >> >> > >> >> >> >> > 2- select count(*) from dv_entersiteevent_prq_remote where >> >> >> >> > year=2017 >> >> >> >> > and >> >> >> >> > month=10 >> >> >> >> > User: nira >> >> >> >> > Pool: root.impreports >> >> >> >> > >> >> >> >> > As shown above that the queries submitted based on the >> >> >> >> > secondary >> >> >> >> > group >> >> >> >> > rule. >> >> >> >> > >> >> >> >> > When i'm running the following with fawzea user: >> >> >> >> > >> >> >> >> > set request_pool=impreports; >> >> >> >> > select count(*) from dv_entersiteevent_prq_remote where >> >> >> >> > year=2017 >> >> >> >> > and >> >> >> >> > month=10; >> >> >> >> > >> >> >> >> > I can see that the query is running with impreports pool. >> >> >> >> > >> >> >> >> > >> >> >> >> > What i'm missing here in my configuration to get this works as >> >> >> >> > expected. >> >> >> >> > >> >> > >> >> > >> > >> > > >
