gaborkaszab commented on code in PR #17144:
URL: https://github.com/apache/iceberg/pull/17144#discussion_r3586965849
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -59,40 +59,41 @@ class TestTrackedFileAdapters {
.build();
private static final PartitionData PARTITION = partition("books");
- // Tracking field ordinals, looked up from the schema so the tests do not
hard-code offsets.
- private static final int DATA_SEQUENCE_NUMBER_ORDINAL =
- ordinalOf(Tracking.schema(), "sequence_number");
- private static final int FILE_SEQUENCE_NUMBER_ORDINAL =
- ordinalOf(Tracking.schema(), "file_sequence_number");
- private static final int FIRST_ROW_ID_ORDINAL = ordinalOf(Tracking.schema(),
"first_row_id");
- // manifestPos is appended after the tracking schema fields by the manifest
reader.
private static final int MANIFEST_POS_ORDINAL =
Tracking.schema().fields().size();
- // TrackedFile optional field ordinals, looked up from the schema.
- private static final Types.StructType TRACKED_FILE_SCHEMA =
- TrackedFile.schemaWithContentStats(Types.StructType.of(),
Types.StructType.of());
- private static final int CONTENT_TYPE_ORDINAL =
ordinalOf(TRACKED_FILE_SCHEMA, "content_type");
- private static final int SPEC_ID_ORDINAL = ordinalOf(TRACKED_FILE_SCHEMA,
"spec_id");
- private static final int DELETION_VECTOR_ORDINAL =
- ordinalOf(TRACKED_FILE_SCHEMA, "deletion_vector");
-
@Test
void testDataFileAdapterDelegation() {
+ TrackingStruct tracking =
+ new TrackingStruct(
+ EntryStatus.ADDED,
+ 42L,
+ DATA_SEQUENCE_NUMBER,
+ FILE_SEQUENCE_NUMBER,
+ null,
+ FIRST_ROW_ID,
+ null,
+ null);
+ tracking.setManifestLocation(MANIFEST_LOCATION);
+ tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
Review Comment:
I don't think we should add such a setter for `manifestPos`. Currently,
readers populate this by using the `set(ordinal, value)` function. Any other
setter would be introduced only to be used in tests. Is there a usage where
such a setter would be useful in prod code?
`setManifestLocation` is somewhat different, because how @anoopj implemented
the reader, it won't be set by the file format reader under the hood, but
"manually" by the manifest reader. However, I also had comments to also remove
setting manifest location through this setter and follow the way manifest pos
is set. But kind of irrelevant for this comment.
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -208,18 +225,37 @@ void testDVDeleteFileAdapterDelegation() {
.cardinality(10L)
.build();
+ TrackingStruct tracking =
+ new TrackingStruct(
+ EntryStatus.ADDED,
+ 42L,
+ DATA_SEQUENCE_NUMBER,
+ FILE_SEQUENCE_NUMBER,
+ 42L,
+ FIRST_ROW_ID,
+ null,
+ null);
+ tracking.setManifestLocation(MANIFEST_LOCATION);
+ tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
+
TrackedFile file =
- TrackedFileBuilder.data(42L)
- .formatVersion(FORMAT_VERSION_V4)
- .location(DATA_FILE_LOCATION)
- .fileFormat(FileFormat.PARQUET)
- .partition(PARTITION)
- .recordCount(100L)
- .fileSizeInBytes(1024L)
- .specId(PARTITIONED_SPEC_ID)
- .deletionVector(dv)
- .build();
- populateTrackingFields(file);
+ new TrackedFileStruct(
Review Comment:
hmm, might be useful reading code on GitHub, but otherwise can't we rely on
the IDE show the param name anyway?
<img width="255" height="408" alt="Image"
src="https://github.com/user-attachments/assets/7e1596bd-9d5f-4aea-8b21-c76691118a1c";
/>
null params are the ones that are probably irrelevant for the test, does
adding such a comment give any value for them?
##########
core/src/test/java/org/apache/iceberg/TestTrackedFileAdapters.java:
##########
@@ -387,18 +471,23 @@ private static PartitionData partition(String category) {
/** Minimal file with no tracking, used by the rejection and null-tracking
tests. */
private static TrackedFileStruct dummyTrackedFile(FileContent contentType) {
- TrackedFileStruct file = new TrackedFileStruct();
- file.set(CONTENT_TYPE_ORDINAL, contentType.id());
- return file;
- }
-
- private static void populateTrackingFields(TrackedFile file) {
- TrackingStruct tracking = (TrackingStruct) file.tracking();
- tracking.set(DATA_SEQUENCE_NUMBER_ORDINAL, DATA_SEQUENCE_NUMBER);
- tracking.set(FILE_SEQUENCE_NUMBER_ORDINAL, FILE_SEQUENCE_NUMBER);
- tracking.set(FIRST_ROW_ID_ORDINAL, FIRST_ROW_ID);
- tracking.setManifestLocation(MANIFEST_LOCATION);
- tracking.set(MANIFEST_POS_ORDINAL, MANIFEST_POS);
+ return new TrackedFileStruct(
+ null,
Review Comment:
Reverted the values to the state before the PR you linked. Let me know if
this is what you meant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
