Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2706#discussion_r217048144
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
---
@@ -570,23 +589,31 @@ public int
writeRawRowAsIntermediateSortTempRowToUnsafeMemory(Object[] row,
private void packNoSortFieldsToBytes(Object[] row, ByteBuffer rowBuffer)
{
// convert dict & no-sort
for (int idx = 0; idx < this.dictNoSortDimCnt; idx++) {
+ // cannot exceed default 2MB, hence no need to call ensureArraySize
rowBuffer.putInt((int) row[this.dictNoSortDimIdx[idx]]);
}
// convert no-dict & no-sort
for (int idx = 0; idx < this.noDictNoSortDimCnt; idx++) {
byte[] bytes = (byte[]) row[this.noDictNoSortDimIdx[idx]];
+ // cannot exceed default 2MB, hence no need to call ensureArraySize
--- End diff --
2 MB is not enough for varchar and complex complex columns.
---
