Mostafa Mokhtar created IMPALA-6657:
---------------------------------------
Summary: Investigate why memory allocation in Exchange receiver
node takes a long time
Key: IMPALA-6657
URL: https://issues.apache.org/jira/browse/IMPALA-6657
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 2.11.0
Reporter: Mostafa Mokhtar
Assignee: Mostafa Mokhtar
Attachments: Impala query profile.txt
It was observed while inserting large amounts of data into a Kudu table
Exchange operator was running slow, query profile showed a big portion of the
time was spent in memory allocation in the buffer pool
{code}
EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s853ms, % non-child:
0.23%)
- ConvertRowBatchTime: 20s289ms
- PeakMemoryUsage: 19.53 MB (20483562)
- RowsReturned: 575.30M (575298780)
- RowsReturnedRate: 27.10 K/sec
Buffer pool:
- AllocTime: 2h53m
- CumulativeAllocationBytes: 261.26 GB (280526643200)
- CumulativeAllocations: 13.70M (13697590)
- PeakReservation: 18.06 MB (18939904)
- PeakUnpinnedBytes: 0
- PeakUsedReservation: 18.06 MB (18939904)
- ReadIoBytes: 0
- ReadIoOps: 0 (0)
- ReadIoWaitTime: 0.000ns
- WriteIoBytes: 0
- WriteIoOps: 0 (0)
- WriteIoWaitTime: 0.000ns
RecvrSide:
BytesReceived(8m32s): 20.91 GB, 37.03 GB, 45.62 GB, 53.22 GB, 60.17
GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.30 GB, 98.17
GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 GB, 121.30
GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85 GB, 139.32
GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 GB, 148.29
GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29
GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29
GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30
GB, 148.30 GB
- FirstBatchArrivalWaitTime: 1s071ms
- TotalBytesReceived: 148.30 GB (159234237617)
- TotalGetBatchTime: 5h53m
- DataArrivalTimer: 5h52m
SenderSide:
- DeserializeRowBatchTime: 3h4m
- NumBatchesArrived: 6.85M (6848795)
- NumBatchesDeferred: 99.67K (99667)
- NumBatchesEnqueued: 6.85M (6848795)
- NumBatchesReceived: 6.85M (6848795)
- NumEarlySenders: 0 (0)
- NumEosReceived: 0 (0)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
