romandenysov-akv commented on code in PR #17:
URL:
https://github.com/apache/iceberg-verification/pull/17#discussion_r4481931724
##########
table-spec/partition-transforms/bucket/cases.json:
##########
@@ -0,0 +1,148 @@
+{
+ "cases": [
+ {
+ "id": "bucket-int-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "int", "values":
[null, 34] },
+ "decoded": { "result_types": ["int"], "values": [null, 79] },
+ "clause": "Bucket Transform Details and Appendix B: int hash reference
vector",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-long-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "long", "values":
[null, "34"] },
+ "decoded": { "result_types": ["int"], "values": [null, 79] },
+ "clause": "Appendix B: integer and long hash results must be identical
for all integer values",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-decimal-9-2-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "decimal(9, 2)",
"values": [null, "14.20"] },
+ "decoded": { "result_types": ["int"], "values": [null, 59] },
+ "clause": "Appendix B: decimal 14.20 hashes from its minimum
two's-complement big-endian unscaled value; bucket discards the sign bit",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-decimal-10-3-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "decimal(10, 3)",
"values": [null, "1.420"] },
+ "decoded": { "result_types": ["int"], "values": [null, 59] },
+ "clause": "Appendix B: decimal hashes use the unscaled value and hash
results are not dependent on decimal scale",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-date-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "date", "values":
[null, "2017-11-16"] },
+ "decoded": { "result_types": ["int"], "values": [null, 26] },
+ "clause": "Appendix B: date hash reference vector; bucket discards the
sign bit",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-time-100",
+ "valid": true,
+ "input": { "transform": "bucket[100]", "source_type": "time", "values":
[null, "22:31:08"] },
+ "decoded": { "result_types": ["int"], "values": [null, 59] },
+ "clause": "Appendix B: time hash reference vector; bucket discards the
sign bit",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-timestamp-100",
+ "valid": true,
+ "input": {
+ "transform": "bucket[100]",
+ "source_type": "timestamp",
+ "values": [null, "2017-11-16T22:31:08", "2017-11-16T22:31:08.000001"]
+ },
+ "decoded": { "result_types": ["int"], "values": [null, 7, 38] },
+ "clause": "Appendix B: timestamp hash reference vectors at whole-second
and microsecond precision",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-timestamptz-100",
+ "valid": true,
+ "input": {
+ "transform": "bucket[100]",
+ "source_type": "timestamptz",
+ "values": [
+ null,
+ "2017-11-16T14:31:08-08:00",
+ "2017-11-16T22:31:08Z",
+ "2017-11-16T14:31:08.000001-08:00",
+ "2017-11-16T22:31:08.000001Z"
+ ]
+ },
+ "decoded": { "result_types": ["int"], "values": [null, 7, 7, 38, 38] },
+ "clause": "Appendix B: timestamptz hashes microseconds from the Unix
epoch, so equivalent instants hash identically",
+ "spec_ref": "format/spec.md#appendix-b-32-bit-hash-requirements"
+ },
+ {
+ "id": "bucket-timestamp-ns-100",
+ "valid": true,
+ "input": {
+ "transform": "bucket[100]",
+ "source_type": "timestamp_ns",
+ "values": [null, "2017-11-16T22:31:08",
"2017-11-16T22:31:08.000001001", "2017-11-16T22:31:08.000001999"]
+ },
+ "decoded": { "result_types": ["int"], "values": [null, 7, 38, 38] },
Review Comment:
@laskoviymishka Thanks for the example. It distinguishes floor division
(bucket 12) from truncation toward zero (bucket 76).
I checked Iceberg Java, Go, and Rust; all three use floor division, and Rust
explicitly changed its implementation to match Java.
Appendix B requires conversion to microseconds but does not state the
rounding rule for negative values between microseconds.
Should we clarify that rule in the spec before making bucket 12 a
conformance expectation, or do you think the current wording is sufficient to
include this case in the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
