It is recommended to set a per query memory limit as part of admission and
not rely on estimates as they are sometimes inaccurate.
Can you please include the full query profile?


On Thu, Feb 22, 2018 at 12:13 PM, Fawze Abujaber <fawz...@gmail.com> wrote:

> Hi Mostafa,
>
> It's not a specific query, almost all the query has such differene between
> the 2 values.
>
> I can see even queries showing the estimate per node is 8.2 Tib
>
>
>    - User: psanalytics
>    - Database: default
>    - Query Type: QUERY
>    - Coordinator: slpr-dhc014.lpdomain.com
>    <http://slvr-cmc101:7180/cmf/hardware/hosts/22/status>
>    - Duration: 6.48s
>    - Rows Produced: 708
>    - Estimated per Node Peak Memory: 8.2 TiB
>    - Per Node Peak Memory Usage: 1.1 GiB
>    - Pool: root.impanalytics
>    - Threads: CPU Time: 20.1m
>
>
>
> How you can explain this behavior, and for sure i don't have 8.2 Tib
> memory per node to give neither you.
>
> Can you please explain me how i should treat Estimated per Node Peak
> Memory and if it used by impala for the resource pool and admission control
> and what is the relation of this value to the default memory per node that
> i setup for each resource pool?
>
> Below is part of one of the queries profile which the estimate per node
> was ~ @GB and the used was 200MB per node.
>
>
>  Instance 744de1b6228736fa:b54bfaa70000000f 
> (host=slpr-dhc004.lpdomain.com:22000):(Total:
> 1s455ms, non-child: 1s292ms, % non-child: 88.82%)
>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 6:1/1.20 MB 2:1/199.83 KB 1:1/1.20 MB 10:2/1.42 MB 8:1/225.57 KB 9:1/191.64
> KB 5:2/289.57 KB 3:2/1012.83 KB
>         MemoryUsage(500.000ms): 6.09 MB, 6.09 MB, 3.00 MB
>         ThreadUsage(500.000ms): 1, 1, 1
>          - AverageThreadTokens: 1.00
>          - BloomFilterBytes: 0
>          - PeakMemoryUsage: 7.17 MB (7521751)
>          - PeakReservation: 0
>          - PeakUsedReservation: 0
>          - PerHostPeakMemUsage: 106.53 MB (111709581)
>          - RowsProduced: 32.83K (32826)
>          - TotalNetworkReceiveTime: 0.000ns
>          - TotalNetworkSendTime: 1s297ms
>          - TotalStorageWaitTime: 234.356ms
>          - TotalThreadsInvoluntaryContextSwitches: 66 (66)
>          - TotalThreadsTotalWallClockTime: 1s715ms
>            - TotalThreadsSysTime: 5.998ms
>            - TotalThreadsUserTime: 124.975ms
>          - TotalThreadsVoluntaryContextSwitches: 303 (303)
>         Fragment Instance Lifecycle Timings:
>            - ExecTime: 1s394ms
>              - ExecTreeExecTime: 67.115ms
>            - OpenTime: 32.795ms
>              - ExecTreeOpenTime: 73.243us
>            - PrepareTime: 27.602ms
>              - ExecTreePrepareTime: 243.141us
>         DataStreamSender (dst_id=11):(Total: 38.747ms, non-child:
> 38.747ms, % non-child: 100.00%)
>            - BytesSent: 39.71 MB (41643000)
>            - NetworkThroughput(*): 1.97 GB/sec
>            - OverallThroughput: 1.00 GB/sec
>            - PeakMemoryUsage: 59.38 KB (60800)
>            - RowsReturned: 32.83K (32826)
>            - SerializeBatchTime: 16.860ms
>            - TransmitDataRPCTime: 19.698ms
>            - UncompressedRowBatchSize: 77.58 MB (81350840)
>         CodeGen:(Total: 56.573ms, non-child: 56.573ms, % non-child:
> 100.00%)
>            - CodegenTime: 1.299ms
>            - CompileTime: 10.672ms
>            - LoadTime: 0.000ns
>            - ModuleBitcodeSize: 1.96 MB (2050180)
>            - NumFunctions: 16 (16)
>            - NumInstructions: 250 (250)
>            - OptimizationTime: 21.023ms
>            - PeakMemoryUsage: 125.00 KB (128000)
>            - PrepareTime: 24.116ms
>         SUBPLAN_NODE (id=6):(Total: 67.311ms, non-child: 12.013ms, %
> non-child: 17.85%)
>            - PeakMemoryUsage: 627.94 KB (643015)
>            - RowsReturned: 32.77K (32768)
>            - RowsReturnedRate: 486.81 K/sec
>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 33.999ms, non-child:
> 25.197ms, % non-child: 74.11%)
>              - BuildRows: 0 (0)
>              - BuildTime: 0.000ns
>              - PeakMemoryUsage: 24.00 KB (24576)
>              - ProbeRows: 32.83K (32826)
>              - ProbeTime: 0.000ns
>              - RowsReturned: 16.80M (16795311)
>              - RowsReturnedRate: 493.99 M/sec
>             Nested Loop Join Builder:
>                - PeakMemoryUsage: 8.00 KB (8192)
>             SINGULAR_ROW_SRC_NODE (id=7):
>                - PeakMemoryUsage: 0
>                - RowsReturned: 0 (0)
>                - RowsReturnedRate: 0
>           UNNEST_NODE (id=8):(Total: 8.801ms, non-child: 8.801ms, %
> non-child: 100.00%)
>              - AvgCollectionSize: 1.00
>              - MaxCollectionSize: 1 (1)
>              - MinCollectionSize: 1 (1)
>              - NumCollections: 32.83K (32826)
>              - PeakMemoryUsage: 0
>              - RowsReturned: 1 (1)
>              - RowsReturnedRate: 113.00 /sec
>         HDFS_SCAN_NODE (id=5):(Total: 21.299ms, non-child: 21.299ms, %
> non-child: 100.00%)
>           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 6:1/1.20 MB 2:1/199.83 KB 1:1/1.20 MB 10:2/1.42 MB 8:1/225.57 KB 9:1/191.64
> KB 5:2/289.57 KB 3:2/1012.83 KB
>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11 out of
> 11
>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0%
> 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>           File Formats: PARQUET/SNAPPY:66
>           BytesRead(500.000ms): 2.26 MB, 2.26 MB, 2.26 MB
>            - FooterProcessingTime: (Avg: 12.082ms ; Min: 9.079ms ; Max:
> 29.275ms ; Number of samples: 11)
>            - AverageHdfsReadThreadConcurrency: 0.00
>            - AverageScannerThreadConcurrency: 0.00
>            - BytesRead: 2.26 MB (2373738)
>            - BytesReadDataNodeCache: 0
>            - BytesReadLocal: 2.26 MB (2373738)
>            - BytesReadRemoteUnexpected: 0
>            - BytesReadShortCircuit: 2.26 MB (2373738)
>            - CachedFileHandlesHitCount: 0 (0)
>            - CachedFileHandlesMissCount: 77 (77)
>            - DecompressionTime: 1.543ms
>            - MaxCompressedTextFileLength: 0
>            - NumColumns: 6 (6)
>            - NumDictFilteredRowGroups: 0 (0)
>            - NumDisksAccessed: 8 (8)
>            - NumRowGroups: 11 (11)
>            - NumScannerThreadsStarted: 11 (11)
>            - NumScannersWithNoReads: 0 (0)
>            - NumStatsFilteredRowGroups: 0 (0)
>            - PeakMemoryUsage: 6.51 MB (6821203)
>            - PerReadThreadRawHdfsThroughput: 764.51 MB/sec
>            - RemoteScanRanges: 0 (0)
>            - RowBatchQueueGetWaitTime: 18.825ms
>            - RowBatchQueuePutWaitTime: 0.000ns
>            - RowsRead: 73.46K (73464)
>            - RowsReturned: 32.83K (32826)
>            - RowsReturnedRate: 1.54 M/sec
>            - ScanRangesComplete: 11 (11)
>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>            - ScannerThreadsTotalWallClockTime: 287.702ms
>              - MaterializeTupleTime(*): 11.921ms
>              - ScannerThreadsSysTime: 999.000us
>              - ScannerThreadsUserTime: 26.990ms
>            - ScannerThreadsVoluntaryContextSwitches: 232 (232)
>            - TotalRawHdfsReadTime(*): 2.961ms
>            - TotalReadThroughput: 1.51 MB/sec
>       Instance 744de1b6228736fa:b54bfaa70000000d
> (host=slpr-dhc002.lpdomain.com:22000):(Total: 1s453ms, non-child:
> 1s254ms, % non-child: 86.28%)
>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 0:1/1.32 MB 4:1/1.18 MB 5:1/1.19 MB 10:2/1.06 MB
>         MemoryUsage(500.000ms): 5.32 MB, 5.32 MB, 1.27 MB
>         ThreadUsage(500.000ms): 1, 1, 1
>          - AverageThreadTokens: 1.00
>          - BloomFilterBytes: 0
>          - PeakMemoryUsage: 5.86 MB (6145964)
>          - PeakReservation: 0
>          - PeakUsedReservation: 0
>          - PerHostPeakMemUsage: 106.66 MB (111842766)
>          - RowsProduced: 30.34K (30340)
>          - TotalNetworkReceiveTime: 0.000ns
>          - TotalNetworkSendTime: 1s256ms
>          - TotalStorageWaitTime: 106.530ms
>          - TotalThreadsInvoluntaryContextSwitches: 177 (177)
>          - TotalThreadsTotalWallClockTime: 1s557ms
>            - TotalThreadsSysTime: 6.998ms
>            - TotalThreadsUserTime: 143.976ms
>          - TotalThreadsVoluntaryContextSwitches: 136 (136)
>         Fragment Instance Lifecycle Timings:
>            - ExecTime: 1s390ms
>              - ExecTreeExecTime: 91.500ms
>            - OpenTime: 34.085ms
>              - ExecTreeOpenTime: 81.436us
>            - PrepareTime: 28.524ms
>              - ExecTreePrepareTime: 217.949us
>         DataStreamSender (dst_id=11):(Total: 48.674ms, non-child:
> 48.674ms, % non-child: 100.00%)
>            - BytesSent: 36.61 MB (38388760)
>            - NetworkThroughput(*): 1.51 GB/sec
>            - OverallThroughput: 752.15 MB/sec
>            - PeakMemoryUsage: 59.38 KB (60800)
>            - RowsReturned: 30.34K (30340)
>            - SerializeBatchTime: 23.132ms
>            - TransmitDataRPCTime: 23.609ms
>            - UncompressedRowBatchSize: 71.70 MB (75187520)
>         CodeGen:(Total: 59.024ms, non-child: 59.024ms, % non-child:
> 100.00%)
>            - CodegenTime: 1.382ms
>            - CompileTime: 10.435ms
>            - LoadTime: 0.000ns
>            - ModuleBitcodeSize: 1.96 MB (2050180)
>            - NumFunctions: 16 (16)
>            - NumInstructions: 250 (250)
>            - OptimizationTime: 22.325ms
>            - PeakMemoryUsage: 125.00 KB (128000)
>            - PrepareTime: 25.459ms
>         SUBPLAN_NODE (id=6):(Total: 91.711ms, non-child: 20.997ms, %
> non-child: 22.90%)
>            - PeakMemoryUsage: 699.55 KB (716335)
>            - RowsReturned: 29.70K (29696)
>            - RowsReturnedRate: 323.80 K/sec
>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 53.723ms, non-child:
> 40.263ms, % non-child: 74.95%)
>              - BuildRows: 0 (0)
>              - BuildTime: 0.000ns
>              - PeakMemoryUsage: 24.00 KB (24576)
>              - ProbeRows: 30.34K (30340)
>              - ProbeTime: 0.000ns
>              - RowsReturned: 15.43M (15426890)
>              - RowsReturnedRate: 287.16 M/sec
>             Nested Loop Join Builder:
>                - PeakMemoryUsage: 8.00 KB (8192)
>             SINGULAR_ROW_SRC_NODE (id=7):
>                - PeakMemoryUsage: 0
>                - RowsReturned: 0 (0)
>                - RowsReturnedRate: 0
>           UNNEST_NODE (id=8):(Total: 13.460ms, non-child: 13.460ms, %
> non-child: 100.00%)
>              - AvgCollectionSize: 1.00
>              - MaxCollectionSize: 1 (1)
>              - MinCollectionSize: 1 (1)
>              - NumCollections: 30.34K (30340)
>              - PeakMemoryUsage: 0
>              - RowsReturned: 1 (1)
>              - RowsReturnedRate: 74.00 /sec
>         HDFS_SCAN_NODE (id=5):(Total: 16.990ms, non-child: 16.990ms, %
> non-child: 100.00%)
>           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 0:1/1.32 MB 4:1/1.18 MB 5:1/1.19 MB 10:2/1.06 MB
>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 5 out of 5
>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0%
> 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>           File Formats: PARQUET/SNAPPY:30
>           BytesRead(500.000ms): 1.56 MB, 1.56 MB, 1.56 MB
>            - FooterProcessingTime: (Avg: 12.351ms ; Min: 7.816ms ; Max:
> 28.687ms ; Number of samples: 5)
>            - AverageHdfsReadThreadConcurrency: 0.00
>            - AverageScannerThreadConcurrency: 0.00
>            - BytesRead: 1.56 MB (1630546)
>            - BytesReadDataNodeCache: 0
>            - BytesReadLocal: 1.56 MB (1630546)
>            - BytesReadRemoteUnexpected: 0
>            - BytesReadShortCircuit: 1.56 MB (1630546)
>            - CachedFileHandlesHitCount: 0 (0)
>            - CachedFileHandlesMissCount: 35 (35)
>            - DecompressionTime: 1.267ms
>            - MaxCompressedTextFileLength: 0
>            - NumColumns: 6 (6)
>            - NumDictFilteredRowGroups: 0 (0)
>            - NumDisksAccessed: 4 (4)
>            - NumRowGroups: 5 (5)
>            - NumScannerThreadsStarted: 5 (5)
>            - NumScannersWithNoReads: 0 (0)
>            - NumStatsFilteredRowGroups: 0 (0)
>            - PeakMemoryUsage: 5.59 MB (5859290)
>            - PerReadThreadRawHdfsThroughput: 920.32 MB/sec
>            - RemoteScanRanges: 0 (0)
>            - RowBatchQueueGetWaitTime: 15.335ms
>            - RowBatchQueuePutWaitTime: 0.000ns
>            - RowsRead: 64.69K (64692)
>            - RowsReturned: 30.34K (30340)
>            - RowsReturnedRate: 1.79 M/sec
>            - ScanRangesComplete: 5 (5)
>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>            - ScannerThreadsTotalWallClockTime: 132.548ms
>              - MaterializeTupleTime(*): 10.585ms
>              - ScannerThreadsSysTime: 999.000us
>              - ScannerThreadsUserTime: 18.995ms
>            - ScannerThreadsVoluntaryContextSwitches: 79 (79)
>            - TotalRawHdfsReadTime(*): 1.689ms
>            - TotalReadThroughput: 1.04 MB/sec
>       Instance 744de1b6228736fa:b54bfaa700000002
> (host=slpr-dhc003.lpdomain.com:22000):(Total: 1s452ms, non-child:
> 1s255ms, % non-child: 86.41%)
>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 0:1/875.15 KB 10:1/927.27 KB 6:1/1.07 MB 9:1/215.54 KB 8:3/1.84 MB
>         MemoryUsage(500.000ms): 5.58 MB, 5.58 MB, 3.94 MB
>         ThreadUsage(500.000ms): 1, 1, 1
>          - AverageThreadTokens: 1.00
>          - BloomFilterBytes: 0
>          - PeakMemoryUsage: 6.17 MB (6473729)
>          - PeakReservation: 0
>          - PeakUsedReservation: 0
>          - PerHostPeakMemUsage: 107.32 MB (112537640)
>          - RowsProduced: 29.97K (29969)
>          - TotalNetworkReceiveTime: 0.000ns
>          - TotalNetworkSendTime: 1s260ms
>          - TotalStorageWaitTime: 130.815ms
>          - TotalThreadsInvoluntaryContextSwitches: 15 (15)
>          - TotalThreadsTotalWallClockTime: 1s586ms
>            - TotalThreadsSysTime: 11.995ms
>            - TotalThreadsUserTime: 146.975ms
>          - TotalThreadsVoluntaryContextSwitches: 197 (197)
>         Fragment Instance Lifecycle Timings:
>            - ExecTime: 1s354ms
>              - ExecTreeExecTime: 67.791ms
>            - OpenTime: 52.012ms
>              - ExecTreeOpenTime: 101.232us
>            - PrepareTime: 45.946ms
>              - ExecTreePrepareTime: 309.644us
>         DataStreamSender (dst_id=11):(Total: 38.995ms, non-child:
> 38.995ms, % non-child: 100.00%)
>            - BytesSent: 35.95 MB (37697260)
>            - NetworkThroughput(*): 2.14 GB/sec
>            - OverallThroughput: 921.92 MB/sec
>            - PeakMemoryUsage: 59.38 KB (60800)
>            - RowsReturned: 29.97K (29969)
>            - SerializeBatchTime: 18.055ms
>            - TransmitDataRPCTime: 16.428ms
>            - UncompressedRowBatchSize: 70.83 MB (74269800)
>         CodeGen:(Total: 90.385ms, non-child: 90.385ms, % non-child:
> 100.00%)
>            - CodegenTime: 2.224ms
>            - CompileTime: 15.517ms
>            - LoadTime: 0.000ns
>            - ModuleBitcodeSize: 1.96 MB (2050180)
>            - NumFunctions: 16 (16)
>            - NumInstructions: 250 (250)
>            - OptimizationTime: 35.099ms
>            - PeakMemoryUsage: 125.00 KB (128000)
>            - PrepareTime: 38.847ms
>         SUBPLAN_NODE (id=6):(Total: 68.126ms, non-child: 12.625ms, %
> non-child: 18.53%)
>            - PeakMemoryUsage: 636.17 KB (651439)
>            - RowsReturned: 29.70K (29696)
>            - RowsReturnedRate: 435.89 K/sec
>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 35.347ms, non-child:
> 27.325ms, % non-child: 77.31%)
>              - BuildRows: 0 (0)
>              - BuildTime: 0.000ns
>              - PeakMemoryUsage: 24.00 KB (24576)
>              - ProbeRows: 29.97K (29969)
>              - ProbeTime: 0.000ns
>              - RowsReturned: 15.26M (15256601)
>              - RowsReturnedRate: 431.62 M/sec
>             Nested Loop Join Builder:
>                - PeakMemoryUsage: 8.00 KB (8192)
>             SINGULAR_ROW_SRC_NODE (id=7):
>                - PeakMemoryUsage: 0
>                - RowsReturned: 0 (0)
>                - RowsReturnedRate: 0
>           UNNEST_NODE (id=8):(Total: 8.021ms, non-child: 8.021ms, %
> non-child: 100.00%)
>              - AvgCollectionSize: 1.00
>              - MaxCollectionSize: 1 (1)
>              - MinCollectionSize: 1 (1)
>              - NumCollections: 29.97K (29969)
>              - PeakMemoryUsage: 0
>              - RowsReturned: 1 (1)
>              - RowsReturnedRate: 124.00 /sec
>         HDFS_SCAN_NODE (id=5):(Total: 20.153ms, non-child: 20.153ms, %
> non-child: 100.00%)
>           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 0:1/875.15 KB 10:1/927.27 KB 6:1/1.07 MB 9:1/215.54 KB 8:3/1.84 MB
>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 7 out of 7
>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0%
> 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>           File Formats: PARQUET/SNAPPY:42
>           BytesRead(500.000ms): 1.75 MB, 1.75 MB, 1.75 MB
>            - FooterProcessingTime: (Avg: 8.683ms ; Min: 3.937ms ; Max:
> 10.464ms ; Number of samples: 7)
>            - AverageHdfsReadThreadConcurrency: 0.00
>            - AverageScannerThreadConcurrency: 0.00
>            - BytesRead: 1.75 MB (1833560)
>            - BytesReadDataNodeCache: 0
>            - BytesReadLocal: 1.75 MB (1833560)
>            - BytesReadRemoteUnexpected: 0
>            - BytesReadShortCircuit: 1.75 MB (1833560)
>            - CachedFileHandlesHitCount: 0 (0)
>            - CachedFileHandlesMissCount: 49 (49)
>            - DecompressionTime: 1.758ms
>            - MaxCompressedTextFileLength: 0
>            - NumColumns: 6 (6)
>            - NumDictFilteredRowGroups: 0 (0)
>            - NumDisksAccessed: 5 (5)
>            - NumRowGroups: 7 (7)
>            - NumScannerThreadsStarted: 7 (7)
>            - NumScannersWithNoReads: 0 (0)
>            - NumStatsFilteredRowGroups: 0 (0)
>            - PeakMemoryUsage: 5.82 MB (6098991)
>            - PerReadThreadRawHdfsThroughput: 621.09 MB/sec
>            - RemoteScanRanges: 0 (0)
>            - RowBatchQueueGetWaitTime: 17.938ms
>            - RowBatchQueuePutWaitTime: 0.000ns
>            - RowsRead: 65.17K (65166)
>            - RowsReturned: 29.97K (29969)
>            - RowsReturnedRate: 1.49 M/sec
>            - ScanRangesComplete: 7 (7)
>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>            - ScannerThreadsTotalWallClockTime: 179.698ms
>              - MaterializeTupleTime(*): 13.922ms
>              - ScannerThreadsSysTime: 2.997ms
>              - ScannerThreadsUserTime: 29.993ms
>            - ScannerThreadsVoluntaryContextSwitches: 130 (130)
>            - TotalRawHdfsReadTime(*): 2.815ms
>            - TotalReadThroughput: 1.17 MB/sec
>       Instance 744de1b6228736fa:b54bfaa700000008
> (host=slpr-dhc009.lpdomain.com:22000):(Total: 1s450ms, non-child:
> 1s305ms, % non-child: 90.01%)
>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 7:1/1.08 MB 6:2/2.52 MB 3:2/2.19 MB 8:1/810.47 KB
>         MemoryUsage(500.000ms): 5.97 MB, 5.88 MB, 2.24 MB
>         ThreadUsage(500.000ms): 1, 1, 1
>          - AverageThreadTokens: 1.00
>          - BloomFilterBytes: 0
>          - PeakMemoryUsage: 6.69 MB (7017157)
>          - PeakReservation: 0
>          - PeakUsedReservation: 0
>          - PerHostPeakMemUsage: 106.39 MB (111556990)
>          - RowsProduced: 42.68K (42681)
>          - TotalNetworkReceiveTime: 0.000ns
>          - TotalNetworkSendTime: 1s315ms
>          - TotalStorageWaitTime: 201.411ms
>          - TotalThreadsInvoluntaryContextSwitches: 43 (43)
>          - TotalThreadsTotalWallClockTime: 1s658ms
>            - TotalThreadsSysTime: 6.997ms
>            - TotalThreadsUserTime: 108.980ms
>          - TotalThreadsVoluntaryContextSwitches: 179 (179)
>         Fragment Instance Lifecycle Timings:
>            - ExecTime: 1s411ms
>              - ExecTreeExecTime: 68.017ms
>            - OpenTime: 19.935ms
>              - ExecTreeOpenTime: 48.883us
>            - PrepareTime: 19.160ms
>              - ExecTreePrepareTime: 134.576us
>         DataStreamSender (dst_id=11):(Total: 40.671ms, non-child:
> 40.671ms, % non-child: 100.00%)
>            - BytesSent: 51.36 MB (53856680)
>            - NetworkThroughput(*): 2.44 GB/sec
>            - OverallThroughput: 1.23 GB/sec
>            - PeakMemoryUsage: 59.38 KB (60800)
>            - RowsReturned: 42.68K (42681)
>            - SerializeBatchTime: 18.299ms
>            - TransmitDataRPCTime: 20.591ms
>            - UncompressedRowBatchSize: 100.88 MB (105782860)
>         CodeGen:(Total: 36.171ms, non-child: 36.171ms, % non-child:
> 100.00%)
>            - CodegenTime: 828.865us
>            - CompileTime: 6.170ms
>            - LoadTime: 0.000ns
>            - ModuleBitcodeSize: 1.96 MB (2050180)
>            - NumFunctions: 16 (16)
>            - NumInstructions: 250 (250)
>            - OptimizationTime: 12.988ms
>            - PeakMemoryUsage: 125.00 KB (128000)
>            - PrepareTime: 16.608ms
>         SUBPLAN_NODE (id=6):(Total: 68.046ms, non-child: 12.434ms, %
> non-child: 18.27%)
>            - PeakMemoryUsage: 689.95 KB (706507)
>            - RowsReturned: 41.98K (41984)
>            - RowsReturnedRate: 616.99 K/sec
>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 39.837ms, non-child:
> 29.422ms, % non-child: 73.86%)
>              - BuildRows: 0 (0)
>              - BuildTime: 0.000ns
>              - PeakMemoryUsage: 24.00 KB (24576)
>              - ProbeRows: 42.68K (42681)
>              - ProbeTime: 0.000ns
>              - RowsReturned: 21.76M (21760053)
>              - RowsReturnedRate: 546.22 M/sec
>             Nested Loop Join Builder:
>                - PeakMemoryUsage: 8.00 KB (8192)
>             SINGULAR_ROW_SRC_NODE (id=7):
>                - PeakMemoryUsage: 0
>                - RowsReturned: 0 (0)
>                - RowsReturnedRate: 0
>           UNNEST_NODE (id=8):(Total: 10.415ms, non-child: 10.415ms, %
> non-child: 100.00%)
>              - AvgCollectionSize: 1.00
>              - MaxCollectionSize: 1 (1)
>              - MinCollectionSize: 1 (1)
>              - NumCollections: 42.68K (42681)
>              - PeakMemoryUsage: 0
>              - RowsReturned: 1 (1)
>              - RowsReturnedRate: 96.00 /sec
>         HDFS_SCAN_NODE (id=5):(Total: 15.774ms, non-child: 15.774ms, %
> non-child: 100.00%)
>           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 7:1/1.08 MB 6:2/2.52 MB 3:2/2.19 MB 8:1/810.47 KB
>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 6 out of 6
>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0%
> 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0% 17:0%
>           File Formats: PARQUET/SNAPPY:36
>           BytesRead(500.000ms): 2.08 MB, 2.08 MB, 2.08 MB
>            - FooterProcessingTime: (Avg: 22.052ms ; Min: 7.783ms ; Max:
> 42.732ms ; Number of samples: 6)
>            - AverageHdfsReadThreadConcurrency: 0.00
>            - AverageScannerThreadConcurrency: 0.00
>            - BytesRead: 2.08 MB (2186159)
>            - BytesReadDataNodeCache: 0
>            - BytesReadLocal: 2.08 MB (2186159)
>            - BytesReadRemoteUnexpected: 0
>            - BytesReadShortCircuit: 2.08 MB (2186159)
>            - CachedFileHandlesHitCount: 0 (0)
>            - CachedFileHandlesMissCount: 42 (42)
>            - DecompressionTime: 1.480ms
>            - MaxCompressedTextFileLength: 0
>            - NumColumns: 6 (6)
>            - NumDictFilteredRowGroups: 0 (0)
>            - NumDisksAccessed: 4 (4)
>            - NumRowGroups: 6 (6)
>            - NumScannerThreadsStarted: 6 (6)
>            - NumScannersWithNoReads: 0 (0)
>            - NumStatsFilteredRowGroups: 0 (0)
>            - PeakMemoryUsage: 6.42 MB (6730467)
>            - PerReadThreadRawHdfsThroughput: 77.02 MB/sec
>            - RemoteScanRanges: 0 (0)
>            - RowBatchQueueGetWaitTime: 13.979ms
>            - RowBatchQueuePutWaitTime: 0.000ns
>            - RowsRead: 89.91K (89910)
>            - RowsReturned: 42.68K (42681)
>            - RowsReturnedRate: 2.71 M/sec
>            - ScanRangesComplete: 6 (6)
>            - ScannerThreadsInvoluntaryContextSwitches: 3 (3)
>            - ScannerThreadsTotalWallClockTime: 227.825ms
>              - MaterializeTupleTime(*): 9.931ms
>              - ScannerThreadsSysTime: 1.998ms
>              - ScannerThreadsUserTime: 18.994ms
>            - ScannerThreadsVoluntaryContextSwitches: 85 (85)
>            - TotalRawHdfsReadTime(*): 27.069ms
>            - TotalReadThroughput: 1.39 MB/sec
>       Instance 744de1b6228736fa:b54bfaa700000005
> (host=slpr-dhc001.lpdomain.com:22000):(Total: 1s447ms, non-child:
> 1s301ms, % non-child: 89.88%)
>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 6:1/1.20 MB 1:2/2.50 MB 3:1/1.32 MB
>         MemoryUsage(500.000ms): 5.58 MB, 5.58 MB, 1.25 MB
>         ThreadUsage(500.000ms): 1, 1, 1
> ...
>
> [Message clipped]

Reply via email to