LuciferYang opened a new pull request, #12583:
URL: https://github.com/apache/gluten/pull/12583
### What changes were proposed in this pull request?
Final follow-up in the Spark 3.2 cleanup series (#12525, #12541, #12550;
open #12543 / #12548 / #12574).
The fallback path in `ReadBufferBuilder.cpp` at
```cpp
if (!file_size.has_value())
{
// only for spark3.2 file partition not contained file size
// so first compute file size first
auto tmp_read_buffer = std::make_unique<DB::ReadBufferFromHDFS>(...);
file_size = tmp_read_buffer->getFileSize();
}
```
is still triggered on Spark 3.3+ whenever the Substrait `LocalFilesNode` was
built without a `fileSizes` list — e.g. by `IcebergLocalFilesBuilder` — so the
comment "only for spark3.2" is factually wrong on the currently supported
version matrix.
This PR rewords the comment to describe the actual condition. The fallback
code itself is unchanged.
### How was this patch tested?
- Comment-only change — no code path touched.
- Verified via `gluten-substrait/.../LocalFilesNode.java:194-203` that the
`fileProperties.setFileSize(...)` block only fires when the caller passes a
non-empty `fileSizes` list; producers such as `IcebergLocalFilesBuilder` do not
pass one, so the fallback is unavoidable on Spark 3.3+.
- Repo-wide `grep -rn 'spark3.2\|Spark 3.2' cpp-ch`: 0 remaining occurrences
after this change.
- clang-format was not run: the local toolchain is `clang-format 12.0.1` and
the project requires `clang-format-15`; running the mismatched version would
produce a diff CI rejects (see AGENTS.md).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-4-7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
