Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2056#discussion_r174005326
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/UnsafeSortDataRows.java
---
@@ -141,13 +141,18 @@ public void initialize() throws MemoryException {
semaphore = new Semaphore(parameters.getNumberOfCores());
}
- private UnsafeCarbonRowPage createUnsafeRowPage() throws MemoryException
{
+ private UnsafeCarbonRowPage createUnsafeRowPage()
+ throws MemoryException, CarbonSortKeyAndGroupByException {
MemoryBlock baseBlock =
UnsafeMemoryManager.allocateMemoryWithRetry(this.taskId,
inMemoryChunkSize);
boolean isMemoryAvailable =
UnsafeSortMemoryManager.INSTANCE.isMemoryAvailable(baseBlock.size());
if (isMemoryAvailable) {
UnsafeSortMemoryManager.INSTANCE.allocateDummyMemory(baseBlock.size());
+ } else {
+ LOGGER.info("trigger in-memory merge and spill for table " +
parameters.getTableName());
--- End diff --
FYI: The main entry point is here
---
