jiangjiangtian opened a new issue, #13062: URL: https://github.com/apache/gluten/issues/13062
### Backend VL (Velox) ### Bug description When testing, I find a task with following logs: ``` I20260920 14:24:00.305145 38755 LocalPartitionWriter.cc:475] Merge 183 spills to 4000 partitions. W20260920 14:24:43.041427 38755 Utils.cc:283] madvise willneed failed: Invalid argument W20260920 14:24:55.489629 38755 Utils.cc:283] madvise willneed failed: Invalid argument ... W20260920 14:26:52.805511 38755 Utils.cc:283] madvise willneed failed: Invalid argument W20260920 14:26:52.808156 38755 Utils.cc:283] madvise willneed failed: Invalid argument I20260920 14:26:52.812170 38755 LocalPartitionWriter.cc:507] Merge spills done in 172 s, detail: mergeTempFileTime:159 s payloadCacheWriteTime:0 s payloadMergerWriteTime:8 s totalTempFileSize:1433658590 bytes writeSize:1564238492 bytes ``` After some investigation, I find there is a bug in [this line of code](https://github.com/apache/gluten/blob/main/cpp/core/shuffle/Utils.cc#L100). We shouldn't calculate `fetchLen` using `pos_`, because `pos_` can lag behind `posFetch_`. If we use `pos_` to calculate `fetchLen`, we will have two problem: 1. **Unaligned madvise addresses, prefetch silently dropped.** When the result is not page-aligned, `posFetch_` lands on a non-page-aligned offset inside the file and stays there. Every subsequent `madvise(MADV_WILLNEED)` then passes an unaligned address, which the kernel rejects with `EINVAL`, so all further prefetching is lost. 2. **Advised range overruns the mapping.** When `pos_` lags behind `posFetch_`, the result length is too large by the gap, so the advised range extends past the end of the mapping. The overrunning part silently operates on whatever mapping happens to follow in the address space (e.g. other mmap'd files or allocator arenas). ### Gluten version _No response_ ### Spark version None ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
