rahulsmahadev commented on code in PR #4409:
URL: https://github.com/apache/iceberg-python/pull/4409#discussion_r4617208234


##########
pyiceberg/conversions.py:
##########
@@ -352,14 +356,28 @@ def _(_: PrimitiveType, b: bytes) -> int:
 
 
 @from_bytes.register(TimeType)
-@from_bytes.register(TimestampType)
 @from_bytes.register(TimestamptzType)
-@from_bytes.register(TimestampNanoType)
 @from_bytes.register(TimestamptzNanoType)
 def _(_: PrimitiveType, b: bytes) -> int:
     return _LONG_STRUCT.unpack(b)[0]
 
 
+@from_bytes.register(TimestampType)
+def _(_: TimestampType, b: bytes) -> int:
+    if len(b) < 8:
+        # If the length is 4 bytes, it is a date promoted to timestamp, stored 
as days from the epoch
+        return _INT_STRUCT.unpack(b)[0] * _MICROS_PER_DAY

Review Comment:
   Happy to look into it. One constraint: the spec says readers must infer the 
write-time type from the bound's length, because manifests do not store the 
bound type and promotion does not rewrite bounds ("Schema Evolution" in 
`format/spec.md`). The manifest's `schema` metadata is the schema when the 
manifest was written, which can be newer than the data file when manifests are 
rewritten, so it cannot replace the length check everywhere. I'll dig into 
whether there is a reliable source for the write-time type, and open a 
follow-up PR if there is.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to