This is an automated email from the ASF dual-hosted git repository.
gabotechs pushed a change to branch gabotechs/port-iceberg-rust-history
in repository https://gitbox.apache.org/repos/asf/datafusion-iceberg.git
discard 9b1a2e9 deps: upgrade to DataFusion 55.0 and Arrow/Parquet 59.2
(#2958)
discard 96783fc refactor: parse table properties lazily (#3030)
discard 72b8fcb docs: fix references in public docs (#2937)
discard d1819cb refactor: Migrate TableProperties consumers to getters (#2989)
discard fbf6a21 refactor: Add manifest reader (#2884)
discard e140f9f feat(encryption) [15/N] write encrypted parquet data files
(#2701)
discard b7819c1 fix (datafusion) IcebergWriteExec emits incorrect schema
(#2975)
discard 051b19b feat(parquet): wire in additional parquet writer settings
(#2887)
discard 49098eb feat: add history metadata table (#2825)
discard 823b746 chore(lint) check unused_qualifications (#2839)
discard 2c9644d feat: Variant Support (#2188)
discard faba05f chore(deps): Bump DataFusion to 54.0.0 (#2648)
discard cdc31c1 chore: Update changelog for 0.10.0 (#2782)
discard e835b75 feat(writer): honor write.parquet.* via
ParquetWriterBuilder::from_table_properties (#2561)
discard 7334077 fix(datafusion): reject non-append insert operations (#2714)
discard 21bd3b6 fix(datafusion): return single row with count 0 for empty
inserts (#2712)
discard a030a1a chore: Update changelog and dependency lists for 0.10.0
(#2745)
discard 05bed56 chore: Bump version to 0.10.0, update changelog (#2700)
discard 45c3605 chore: Move dependency list generation from a release task to
a PR author responsibility (#2676)
discard 46dfe94 feat(datafusion): push down isnan over NaN-preserving numeric
expressions (#2592)
discard 999cc09 chore: Bump MSRV to 1.94 (#2652)
discard ac36ed0 refactor!(writer): borrow TableMetadata in
DefaultLocationGenerator::new (#2604)
discard c421c95 chore(refactor): add manifest list reader (#2541)
discard 2baad09 chore(ci): omit blanket/auto-trait/auto-derived impls from
public API (#2542)
discard 360f749 feat(ci): add cargo-public-api checks for public API
stability (#2525)
discard 29928a5 Use cargo publish to publish crates (#2489)
discard d9162fb feat: HuggingFace Hub storage backend and CDC table
properties (#2375)
discard a972016 feat!(runtime): Support custom Runtime in Catalog (#2308)
discard a3077ed fix(tests): remove unnecessary `.to_string()` calls for
metadata_location (#2407)
discard 09ead8c feat(datafusion): show pushed-down limit in IcebergTableScan
EXPLAIN output (#2360)
discard f0b7875 Make `convert_filters_to_predicate` public (#2118)
discard 33f4202 deps: upgrade DataFusion to 53.0, Arrow to 58 (#2206)
discard 3753ec0 chore: Bump version to 0.9.0, update dep list and changelog
(#2223)
discard c35ca67 chore: Update MSRV to 1.91.0 (#2204)
discard 0b9ab78 feat!(storage): Integration Storage trait with FileIO (#2116)
discard 2f2afcf feat(datafusion): support isnan predicate pushdown to Iceberg
(#2142)
discard 5c2d83c feat(datafusion): Add Timestamp scalar value conversion for
predicate pushdown (#2069)
discard 0afd6d0 feat(datafusion): Add Boolean predicate pushdown support
(#2082)
discard 46a64c7 feat(datafusion): Add LIKE predicate pushdown for StartsWith
patterns (#2014)
discard 27a2259 feat(datafusion): Add support for DROP TABLE (#2033)
discard 868ea8b feat(datafusion): Add Binary scalar value conversion for
predicate pushdown (#2048)
discard 851b18c feat(datafusion): Support `CREATE TABLE` for DataFusion
(#1972)
discard ab86392 feat(datafusion): Add schema validation for partition
projection (#2008)
discard 99ddb0a feat(spec): add table_properties() method to TableMetadata
(#2002)
discard a87d10f feat(datafusion): Add LIMIT pushdown support (#2006)
discard 5124214 feat(datafusion): Apply SortExec when writing in clustered
mode (#2005)
discard b90f91d feat: make FanoutWriter writer configurable (#1962)
discard 50dfba6 Bump to version 0.8.0 (#1938)
discard 043bf35 deps: bump DataFusion to 51, Arrow to 57, pyo to 0.26 (#1899)
discard 20f87db chore: bump MSRV to 1.88, fix warnings and clippy errors
(#1902)
discard f86209e feat(datafusion): Add `sort_by_partition` to sort the input
partitioned data (#1618)
discard 10b781d feat(datafusion): Split IcebergTableProvider into static and
non-static table provider (#1879)
discard 5bf2ff3 feat(datafusion): Support `INSERT INTO` partitioned tables
(#1827)
discard 7cdec71 feat(datafusion): Add TaskWriter for DataFusion (#1769)
discard 09ac6ae feat(datafusion): implement the partitioning node for
DataFusion to define the partitioning (#1620)
discard 33bd90a chore: inline format args (#1805)
discard ba6653c feat: Update Datafusion to v49 (#1704)
discard 07f0b67 refactor(arrow,datafusion): Reuse PartitionValueCalculator in
RecordBatchPartitionSplitter (#1781)
discard 26e25c6 feat(datafusion): implement the project node to add the
partition columns (#1602)
discard 0c11819 feat(writer): Add clustered and fanout writer (#1735)
discard bea2fe2 feat(spec): add `table_properties.rs` to spec (#1733)
discard ced4c0b Improve `IcebergCommitExec` to correctly populate
properties/schema (#1721)
discard 4d98e77 fix: ensure CoalescePartitionsExec is enabled for
IcebergCommitExec (#1723)
discard 8f12434 refactor(writer): Refactor writers for the future
partitioning writers (#1657)
discard 87ce715 chore: update changelog and dependencies for 0.7.0 release
(#1713)
discard 4b9dc43 chore: Bump toolchain version to rust 1.87 and update
dependencies list (#1688)
discard e026b43 feat: bump datafusion to 48 (#1501)
discard ce7188c feat(writer): Make LocationGenerator partition-aware (#1625)
discard 960ecfc fix: await get catalog in datafusion integration test (#1635)
discard 5feba97 feat(datafusion): Implement `insert_into` for
`IcebergTableProvider` (#1600)
discard 7d9ccb1 feat(catalog): Implement catalog loader for in memory (#1623)
discard 26c0bec Include license and notice files in published crates (#1601)
discard 7d2425f feat(arrow): Allow ArrowArrayAccessor to use field.name to
match fields (#1561)
discard c1e2bdd feat(datafusion): Implement IcebergWriteExec for DataFusion
write support (#1585)
discard 9da5fa8 feat(datafusion): Add IcebergCommitExec to commit the written
data files (#1588)
discard d7827d7 feat(iceberg-datafusion): make IcebergTableScan public (#1570)
discard 0ebed78 Preparing 0.6.0 release (update dep list) (#1525)
discard 8ccaf10 refactor: Move memory catalog into core (#1480)
discard 3a5c028 feat(datafusion): support metadata tables for Datafusion
(#879)
discard 285fcab Bump iceberg-rust version to 0.5.1 (#1380)
discard 60b46d0 Regenerate the licenses and update the allowed list (#1363)
discard 1a5b05d Bump iceberg-rust version to 0.5.0 (#1345)
discard 2dc29af Bump iceberg-rust version to 0.5.0 (Round 1) (#1342)
discard c5ea4c2 Expose datafusion table provider as python binding (#1324)
discard 9f5ad68 chore: bump up arrow/parquet/datafusion (#1294)
discard cbd9732 refactor: use the same MSRV for datafusion integration (#1197)
discard ae721fc ci: Use taplo to replace cargo-sort (#1186)
discard ec439df refactor: Bump MSRV to 1.84 for preparing next release (#1185)
discard 0d41351 feat: nan_value_counts support (#907)
discard c08b3ec fix: Do not extract expression from cast to date (#977)
discard d86825f Add pyspark DataFusion integration test (#850)
discard 8f8f58b Datafusion: Support cast operations (#821)
discard 5a8583f ci: check MSRV correctly (#849)
discard 1b8972a chore: datafusion 44 upgrade (#867)
discard 2ae61b2 Align schemas for DataFusion plan and stream (#829)
discard b847bcd chore: Generate Changelog Dependencies for 0.4.0 release
(#812)
discard acfb0a8 Derive Clone for IcebergTableProvider (#722)
discard 8ee8e7a Fix error running data fusion queries - Physical input schema
should be the same as the one converted from logical input schema (#664)
discard 0c88d1f datafusion: Create table provider for a snapshot. (#707)
discard 9c98c89 Upgrade to DataFusion 43 (#691)
discard 563d76c feat: implement IcebergTableProviderFactory for datafusion
(#600)
discard 430ed4d feat(datafusion): Support pushdown more datafusion exprs to
Iceberg (#649)
discard a86ecaa test (datafusion): add test for table provider creation (#651)
discard e95278d making table provider pub (#650)
discard 85cdbc4 feat (datafusion integration): convert datafusion expr
filters to Iceberg Predicate (#588)
discard 15a59b3 Migrate to arrow-* v53 (#626)
discard 9468aa7 feat: support projection pushdown for datafusion iceberg
(#594)
discard 4e5195b test: refactor datafusion test with memory catalog (#557)
discard db2301c Bump to version 0.3.0 (#549)
discard 17ca188 chore: Bump datafusion to 41 (#548)
discard 73f8992 feat: podman support (#489)
discard 4fd5a21 chore: Enable new rust code format settings (#483)
discard 820677c fix: Hms test on macos should use correct arch (#461)
discard 9cb8da4 chore: remove compose obsolete version (#452) (#454)
discard a2984bc reuse docker container to save compute resources (#428)
discard 500ae71 chore: Use nightly toolchain for check (#445)
discard 4e421f6 feat: Add storage features for iceberg (#400)
discard 93bc0c4 deps: Bump arrow related crates to 52 (#403)
discard 0452002 chore(deps): Update datafusion requirement from 37.0.0 to
38.0.0 (#369)
discard 19e6c87 feat: Extract FileRead and FileWrite trait (#364)
discard 0905609 Basic Integration with Datafusion (#324)
new 8877b4b Add ASF configuration
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9b1a2e9)
\
N -- N -- N refs/heads/gabotechs/port-iceberg-rust-history
(8877b4b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yml | 38 ++
Cargo.toml | 49 --
DEPENDENCIES.rust.tsv | 412 ------------
LICENSE | 201 ------
NOTICE | 5 -
README.md | 22 -
public-api.txt | 139 ----
src/catalog.rs | 93 ---
src/error.rs | 32 -
src/lib.rs | 30 -
src/physical_plan/commit.rs | 713 --------------------
src/physical_plan/expr_to_predicate.rs | 927 --------------------------
src/physical_plan/metadata_scan.rs | 98 ---
src/physical_plan/mod.rs | 31 -
src/physical_plan/project.rs | 590 -----------------
src/physical_plan/repartition.rs | 853 ------------------------
src/physical_plan/scan.rs | 252 -------
src/physical_plan/sort.rs | 240 -------
src/physical_plan/write.rs | 653 ------------------
src/schema.rs | 447 -------------
src/table/metadata_table.rs | 84 ---
src/table/mod.rs | 893 -------------------------
src/table/table_provider_factory.rs | 331 ----------
src/task_writer.rs | 517 ---------------
testdata/table_metadata/TableMetadataV2.json | 121 ----
tests/integration_datafusion_test.rs | 948 ---------------------------
tests/test_data/TableMetadataV2Valid.json | 122 ----
27 files changed, 38 insertions(+), 8803 deletions(-)
create mode 100644 .asf.yml
delete mode 100644 Cargo.toml
delete mode 100644 DEPENDENCIES.rust.tsv
delete mode 100644 LICENSE
delete mode 100644 NOTICE
delete mode 100644 README.md
delete mode 100644 public-api.txt
delete mode 100644 src/catalog.rs
delete mode 100644 src/error.rs
delete mode 100644 src/lib.rs
delete mode 100644 src/physical_plan/commit.rs
delete mode 100644 src/physical_plan/expr_to_predicate.rs
delete mode 100644 src/physical_plan/metadata_scan.rs
delete mode 100644 src/physical_plan/mod.rs
delete mode 100644 src/physical_plan/project.rs
delete mode 100644 src/physical_plan/repartition.rs
delete mode 100644 src/physical_plan/scan.rs
delete mode 100644 src/physical_plan/sort.rs
delete mode 100644 src/physical_plan/write.rs
delete mode 100644 src/schema.rs
delete mode 100644 src/table/metadata_table.rs
delete mode 100644 src/table/mod.rs
delete mode 100644 src/table/table_provider_factory.rs
delete mode 100644 src/task_writer.rs
delete mode 100644 testdata/table_metadata/TableMetadataV2.json
delete mode 100644 tests/integration_datafusion_test.rs
delete mode 100644 tests/test_data/TableMetadataV2Valid.json
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
