stevenzwu commented on code in PR #17144:
URL: https://github.com/apache/iceberg/pull/17144#discussion_r3581691403
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -208,18 +225,37 @@ void testDVDeleteFileAdapterDelegation() {
.cardinality(10L)
.build();
+ TrackingStruct tracking =
+ new TrackingStruct(
+ EntryStatus.ADDED,
+ 42L,
+ DATA_SEQUENCE_NUMBER,
+ FILE_SEQUENCE_NUMBER,
+ 42L,
+ FIRST_ROW_ID,
+ null,
+ null);
+ tracking.setManifestLocation(MANIFEST_LOCATION);
+ tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
+
TrackedFile file =
- TrackedFileBuilder.data(42L)
- .formatVersion(FORMAT_VERSION_V4)
- .location(DATA_FILE_LOCATION)
- .fileFormat(FileFormat.PARQUET)
- .partition(PARTITION)
- .recordCount(100L)
- .fileSizeInBytes(1024L)
- .specId(PARTITIONED_SPEC_ID)
- .deletionVector(dv)
- .build();
- populateTrackingFields(file);
+ new TrackedFileStruct(
Review Comment:
nit: it would help readers if each argument is annotated with a comment for
the param name for null values and literals (like numbers).
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -387,18 +471,23 @@ private static PartitionData partition(String category) {
/** Minimal file with no tracking, used by the rejection and null-tracking
tests. */
private static TrackedFileStruct dummyTrackedFile(FileContent contentType) {
- TrackedFileStruct file = new TrackedFileStruct();
- file.set(CONTENT_TYPE_ORDINAL, contentType.id());
- return file;
- }
-
- private static void populateTrackingFields(TrackedFile file) {
- TrackingStruct tracking = (TrackingStruct) file.tracking();
- tracking.set(DATA_SEQUENCE_NUMBER_ORDINAL, DATA_SEQUENCE_NUMBER);
- tracking.set(FILE_SEQUENCE_NUMBER_ORDINAL, FILE_SEQUENCE_NUMBER);
- tracking.set(FIRST_ROW_ID_ORDINAL, FIRST_ROW_ID);
- tracking.setManifestLocation(MANIFEST_LOCATION);
- tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
+ return new TrackedFileStruct(
+ null,
Review Comment:
`recordCount=0L` and `fileSizeInBytes=0L` read as a valid empty file (0
bytes, 0 rows), not as "unset placeholders," which they're meant to be here —
same for `formatVersion=0` and `null` location. If a future adapter or write
path adds a `< 0` unset guard (`Preconditions.checkArgument(recordCount >= 0,
...)`), calls through this dummy silently pass instead of tripping the guard.
Real-ish placeholders like `FORMAT_VERSION_V4`, `"s3://bucket/file"`,
`FileFormat.PARQUET`, `1L`, `1L` (matching the pattern used elsewhere in this
file) read as "obviously a fixture" and avoid the ambiguity. This would also
match the behavior prior to [PR
16769](https://github.com/apache/iceberg/pull/16769/changes#diff-fc382f17c4ec991bab0102083d77fb7820e05f4ebb05cbaf7100cd4ed88c8dd4).
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -59,40 +59,41 @@ class TestTrackedFileAdapters {
.build();
private static final PartitionData PARTITION = partition("books");
- // Tracking field ordinals, looked up from the schema so the tests do not
hard-code offsets.
- private static final int DATA_SEQUENCE_NUMBER_ORDINAL =
- ordinalOf(Tracking.schema(), "sequence_number");
- private static final int FILE_SEQUENCE_NUMBER_ORDINAL =
- ordinalOf(Tracking.schema(), "file_sequence_number");
- private static final int FIRST_ROW_ID_ORDINAL = ordinalOf(Tracking.schema(),
"first_row_id");
- // manifestPos is appended after the tracking schema fields by the manifest
reader.
private static final int MANIFEST_POS_ORDINAL =
Tracking.schema().fields().size();
- // TrackedFile optional field ordinals, looked up from the schema.
- private static final Types.StructType TRACKED_FILE_SCHEMA =
- TrackedFile.schemaWithContentStats(Types.StructType.of(),
Types.StructType.of());
- private static final int CONTENT_TYPE_ORDINAL =
ordinalOf(TRACKED_FILE_SCHEMA, "content_type");
- private static final int SPEC_ID_ORDINAL = ordinalOf(TRACKED_FILE_SCHEMA,
"spec_id");
- private static final int DELETION_VECTOR_ORDINAL =
- ordinalOf(TRACKED_FILE_SCHEMA, "deletion_vector");
-
@Test
void testDataFileAdapterDelegation() {
+ TrackingStruct tracking =
+ new TrackingStruct(
+ EntryStatus.ADDED,
+ 42L,
+ DATA_SEQUENCE_NUMBER,
+ FILE_SEQUENCE_NUMBER,
+ null,
+ FIRST_ROW_ID,
+ null,
+ null);
+ tracking.setManifestLocation(MANIFEST_LOCATION);
+ tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
Review Comment:
I think we should add a `setManifestPos` setter to `TrackingStruct`, similar
to `setManifestLocation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
