szehon-ho commented on pull request #4142:
URL: https://github.com/apache/iceberg/pull/4142#issuecomment-1049345572
Added new method as suggested @flyrain
Output for this method is now:
```
BaseSnapshot{id=8051839554011841790, timestamp_ms=1645660550950,
operation=append, summary={spark.app.id=local-1645660524540,
added-data-files=1, added-records=1, added-files-size=643,
changed-partition-count=1, total-records=5, total-files-size=1948,
total-data-files=3, total-delete-files=0, total-position-deletes=0,
total-equality-deletes=0},
manifest-list=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/snap-8051839554011841790-1-171220c0-d44c-47ff-9a28-92bbd2f32960.avro,
schema-id=0}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/171220c0-d44c-47ff-9a28-92bbd2f32960-m0.avro,
length=6561, partition_spec_id=0, added_snapshot_id=8051839554011841790,
added_data_files_count=1, added_rows_count=1, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=5, min_sequence_number=5}
+---GenericManifestEntry{status=ADDED, snapshot_id=8051839554011841790,
sequence_number=5, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-403-8ed90937-faae-4151-9a87-aaf82a4318ed-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=1,
file_size_in_bytes=643, column_sizes={1=46, 2=48}, value_counts={1=1, 2=1},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/8bdf97d4-7ad4-428f-8ef3-65a9454f449a-m1.avro,
length=6559, partition_spec_id=0, added_snapshot_id=2008302932940166540,
added_data_files_count=1, added_rows_count=1, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=4, min_sequence_number=4}
+---GenericManifestEntry{status=ADDED, snapshot_id=2008302932940166540,
sequence_number=4, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-402-c135e7a5-98ee-4336-9d3e-264b33aa6606-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=1,
file_size_in_bytes=643, column_sizes={1=46, 2=48}, value_counts={1=1, 2=1},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/45a03772-f196-49dd-8d9a-7395e4e7ea3f-m1.avro,
length=6563, partition_spec_id=0, added_snapshot_id=8202500298836421079,
added_data_files_count=1, added_rows_count=3, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=3, min_sequence_number=3}
+---GenericManifestEntry{status=ADDED, snapshot_id=8202500298836421079,
sequence_number=3, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-401-263e4856-3c17-4137-8146-ad6c9117a92e-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=3,
file_size_in_bytes=662, column_sizes={1=57, 2=57}, value_counts={1=3, 2=3},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f85,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f85,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
```
versus for the original method (printTable):
```
------------------------------------------------------------
Table spark_catalog.default.table
------------------------------------------------------------
BaseSnapshot{id=8051839554011841790, timestamp_ms=1645660550950,
operation=append, summary={spark.app.id=local-1645660524540,
added-data-files=1, added-records=1, added-files-size=643,
changed-partition-count=1, total-records=5, total-files-size=1948,
total-data-files=3, total-delete-files=0, total-position-deletes=0,
total-equality-deletes=0},
manifest-list=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/snap-8051839554011841790-1-171220c0-d44c-47ff-9a28-92bbd2f32960.avro,
schema-id=0}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/171220c0-d44c-47ff-9a28-92bbd2f32960-m0.avro,
length=6561, partition_spec_id=0, added_snapshot_id=8051839554011841790,
added_data_files_count=1, added_rows_count=1, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=5, min_sequence_number=5}
+---GenericManifestEntry{status=ADDED, snapshot_id=8051839554011841790,
sequence_number=5, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-403-8ed90937-faae-4151-9a87-aaf82a4318ed-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=1,
file_size_in_bytes=643, column_sizes={1=46, 2=48}, value_counts={1=1, 2=1},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/8bdf97d4-7ad4-428f-8ef3-65a9454f449a-m1.avro,
length=6559, partition_spec_id=0, added_snapshot_id=2008302932940166540,
added_data_files_count=1, added_rows_count=1, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=4, min_sequence_number=4}
+---GenericManifestEntry{status=ADDED, snapshot_id=2008302932940166540,
sequence_number=4, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-402-c135e7a5-98ee-4336-9d3e-264b33aa6606-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=1,
file_size_in_bytes=643, column_sizes={1=46, 2=48}, value_counts={1=1, 2=1},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f8d,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
\---GenericManifestFile{content=DATA,
path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/metadata/45a03772-f196-49dd-8d9a-7395e4e7ea3f-m1.avro,
length=6563, partition_spec_id=0, added_snapshot_id=8202500298836421079,
added_data_files_count=1, added_rows_count=3, existing_data_files_count=0,
existing_rows_count=0, deleted_data_files_count=0, deleted_rows_count=0,
partitions=[], key_metadata=null, sequence_number=3, min_sequence_number=3}
+---GenericManifestEntry{status=ADDED, snapshot_id=8202500298836421079,
sequence_number=3, file=GenericDataFile{content=data,
file_path=file:/var/folders/wy/5b87_qx57n974szn9_wrn6lw0000gn/T/hive15893599931471374639/table/data/00000-401-263e4856-3c17-4137-8146-ad6c9117a92e-00001.parquet,
file_format=PARQUET, spec_id=0, partition=PartitionData{}, record_count=3,
file_size_in_bytes=662, column_sizes={1=57, 2=57}, value_counts={1=3, 2=3},
null_value_counts={1=0, 2=0}, nan_value_counts={},
lower_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f85,
upper_bounds=org.apache.iceberg.SerializableByteBufferMap@94446f85,
key_metadata=null, split_offsets=[4], equality_ids=null, sort_order_id=0}}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
