Pandas886 opened a new pull request, #511:
URL: https://github.com/apache/paimon-rust/pull/511
### Purpose
Linked issue: close #506
**Depends on:** the full incremental stack (Delta → Changelog → Diff). This
PR only adds the DataFusion SQL surface on top.
I wanted something you can actually type in SQL without dropping into the
Rust API:
```sql
SELECT * FROM paimon_incremental_query('t', 0, 2);
SELECT * FROM paimon_incremental_query('t', 0, 2, 'diff');
SELECT rowkind, id, value FROM paimon_incremental_query('t$audit_log', 0, 2,
'changelog');
```
### Brief change log
- Register TVF `paimon_incremental_query` in `SQLContext`
- 3-arg form defaults to `auto`; optional 4th arg: `auto` / `delta` /
`changelog` / `diff`
- Physical plan delegates to core `IncrementalScan` / `AuditLogTable` — **no
second planning implementation** in the DF crate
- Support `$audit_log` suffix for rowkind output
- Projection + residual filter (filter is not pushed into core incremental
yet; kept conservative)
- Update `docs/src/sql.md` with the function and a couple examples
I skipped the Java golden / lookup-compact E2E path on purpose: those need
compact/lookup generation which isn't in this contribution series. Coverage is
pure Rust / input-changelog based.
### Tests
```bash
cargo test -p paimon-datafusion --test incremental_query
cargo fmt --all -- --check
cargo clippy -p paimon -p paimon-datafusion --all-targets --features
fulltext,vortex -- -D warnings
```
### API and Format
- New SQL table function only; storage format unchanged
- Relies on the core incremental APIs from the parent PRs
### Documentation
Yes — `docs/src/sql.md` updated.
### Review tip
If reviewing against `main`, the DF-only commit is the tip: `feat: support
DataFusion paimon_incremental_query`. Happy to restack onto latest `main` once
the core PRs land.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
