kevinjqliu commented on issue #3758: URL: https://github.com/apache/iceberg-python/issues/3758#issuecomment-5331741698
I was able to validate that this issue is fixed by https://github.com/apache/iceberg-python/pull/3320/changes#diff-23e8153e0fd497a9212215bd2067068f3b56fa071770c7ef326db3d3d03cee9bR833 Tested on these: - 0.11.1 -> ✅ works as intended - commit this issue references (`154288fb7`) -> ❌ failed - current `main` (c86eb8e5c) -> ✅ works as intended Here's the script I used to repro: <details> ``` uv run python - <<'PY' import datetime as dt import tempfile import pyarrow as pa from pyiceberg.catalog.sql import SqlCatalog from pyiceberg.partitioning import PartitionField, PartitionSpec from pyiceberg.schema import Schema from pyiceberg.transforms import DayTransform from pyiceberg.types import IntegerType, NestedField, StringType, TimestampType warehouse = tempfile.mkdtemp() catalog = SqlCatalog("c", uri=f"sqlite:///{warehouse}/c.db", warehouse=f"file://{warehouse}") catalog.create_namespace("db") table = catalog.create_table( "db.t", schema=Schema( NestedField(1, "k", StringType(), required=False), NestedField(2, "v", IntegerType(), required=False), NestedField(3, "ts", TimestampType(), required=False), ), partition_spec=PartitionSpec(PartitionField(3, 1000, DayTransform(), "p")), properties={"format-version": "2"}, ) schema = pa.schema( [pa.field("k", pa.string()), pa.field("v", pa.int32()), pa.field("ts", pa.timestamp("us"))] ) when = dt.datetime(2026, 1, 6, 12) def batch(pairs): return pa.table( { "k": [k for k, _ in pairs], "v": pa.array([v for _, v in pairs], type=pa.int32()), "ts": [when] * len(pairs), }, schema=schema, ) table.append(batch([("a", 1), ("b", 1)])) # one manifest, two rows table.refresh() table.upsert(batch([("a", 2)]), join_cols=["k"]) # replace one of them table.refresh() result = table.scan().to_arrow() rows = sorted(zip(result["k"].to_pylist(), result["v"].to_pylist())) print(rows) assert rows == [("a", 2), ("b", 1)], rows PY ``` </details> I'll take a look at the related PRs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
