rahulsmahadev opened a new pull request, #4343:
URL: https://github.com/apache/iceberg-python/pull/4343
# Rationale for this change
A table partitioned by `bucket[N]` on a `time` column can be created, but
every `append()` to it fails:
```python
schema = Schema(NestedField(1, "id", LongType()), NestedField(2,
"event_time", TimeType()))
spec = PartitionSpec(PartitionField(source_id=2, field_id=1000,
transform=BucketTransform(4), name="event_time_bucket"))
table = catalog.create_table("default.t", schema=schema, partition_spec=spec)
table.append(pa.table({"id": [1], "event_time": pa.array([time(22, 31, 8)],
pa.time64("us"))}))
# AttributeError: 'int' object has no attribute 'hour'
```
The spec lists `time` as a source type for `bucket[N]`, and Java supports
it. The bucket transform returns an `int`, which `_to_partition_representation`
already passes through for date, timestamp and uuid sources, but the `TimeType`
handler always called `time_to_micros` on it. This change makes the `TimeType`
handler accept an `int` the same way.
## Are these changes tested?
Yes. Both new tests fail without the fix:
- `test_to_partition_representation_time` in `tests/test_transforms.py`
covers `None`, a `time` value, and the `int` a bucket transform returns.
`test_to_partition_representation_unrecognized_type_raises` now also covers
`TimeType`.
- `test_append_to_table_bucketed_by_time` in `tests/io/test_pyarrow.py`
appends to a table bucketed on a time column and reads the rows back with and
without a filter.
## Are there any user-facing changes?
Bug fix only: appends to tables partitioned by bucket on a time column now
succeed. No API changes.
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
