Copilot commented on code in PR #6470:
URL: https://github.com/apache/paimon/pull/6470#discussion_r2459568539
##########
paimon-filesystems/paimon-oss-impl/src/main/java/org/apache/paimon/oss/OssTwoPhaseOutputStream.java:
##########
@@ -39,8 +39,8 @@ public OssTwoPhaseOutputStream(
}
@Override
- public long partSizeThreshold() {
- return 10L << 20;
+ public int partSizeThreshold() {
Review Comment:
The threshold value changed from 10MB to 8MB without explanation. Consider
adding a comment explaining why 8MB was chosen or documenting this breaking
change.
```suggestion
public int partSizeThreshold() {
// Changed from 10MB to 8MB to optimize multipart upload performance
and reduce memory usage.
// This is a breaking change; ensure downstream systems are
compatible with 8MB part sizes.
```
##########
paimon-common/src/test/java/org/apache/paimon/fs/MultiPartUploadTwoPhaseOutputStreamTest.java:
##########
@@ -257,12 +293,12 @@ private TestCommitter(
String uploadId,
List<TestPart> parts,
String objectName,
- long byteLength) {
+ long position) {
this.store = store;
this.uploadId = uploadId;
this.parts = new ArrayList<>(parts);
this.objectName = objectName;
- this.byteLength = byteLength;
+ this.byteLength = (int) position;
Review Comment:
Downcasting position from long to int can cause data loss for files larger
than 2GB. This could result in incorrect byte length values being passed to
completeMultipartUpload.
##########
paimon-common/src/main/java/org/apache/paimon/fs/BaseMultiPartUploadCommitter.java:
##########
@@ -35,10 +35,10 @@ public abstract class BaseMultiPartUploadCommitter<T, C>
implements TwoPhaseOutp
private final String uploadId;
private final String objectName;
private final List<T> uploadedParts;
- private final long byteLength;
+ private final int byteLength;
Review Comment:
Changing byteLength from long to int limits maximum file size to ~2GB. This
could prevent uploading larger files that multipart upload is designed to
handle. Consider keeping this as long to support files larger than
Integer.MAX_VALUE bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
