Abacn opened a new pull request, #40695:
URL: https://github.com/apache/beam/pull/40695
- Add `BeamCrossJoinRel` and `BeamCrossJoinRule` to support
bounded-by-bounded pure cross joins in Beam Calcite SQL.
- Extend `BeamCoGBKJoinRule` to split mixed equi/non-equi inner join
predicates into an equi-join followed by a post-join `BeamCalcRel` filter.
- Support `LAG`, `LEAD`, `FIRST_VALUE`, `LAST_VALUE`, `NTH_VALUE`
(`RESPECT`/`IGNORE NULLS`), and `CUME_DIST` analytic window functions in
`BeamWindowRel` and `BeamBuiltinAnalyticFunctions`, and convert window inputs
to `BeamLogicalConvention` in `BeamWindowRule`.
- Fix `NULLS FIRST` and `NULLS LAST` comparator ordering in `BeamSortRel`
when combined with `ASC`/`DESC`.
- Normalize set operator (`UNION`, `INTERSECT`, `EXCEPT`) input schemas in
`BeamSetOperatorRelBase` when inputs differ only by field names or nullability.
- Return `NULL` instead of `0.0` from `VarianceFn` for empty or
single-element sample variance/stddev groups to match SQL standard semantics.
- Remove workarounds in TPC-DS table schemas used to cast `DECIMAL` to
`DOUBLE` for fractional columns
Having these improvements, 55 out of 103 queries now passing (up from 24).
Then for test suite,
- add `--queries=supported` and `10M`/`10MB` dataset size support to
`TpcdsParametersReader`.
- Add the Python TPC-DS benchmark and regression test suite
(`sdks/python/apache_beam/testing/benchmarks/tpcds`) running on `PrismRunner`
and cross-language `SqlTransform`.
- Update Java TPC-DS GitHub Actions workflows to use `--queries=supported`
and add the `beam_PostCommit_Python_Tpcds_Prism.yml` workflow.
<!--
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
- [ ] Replace these instructions with a description of your change and why
you have made it.
- [ ] Mention the appropriate issue in your description (for example:
`addresses #123` or `fixes #456`).
- [ ] Update `CHANGES.md` with noteworthy changes.
See [CONTRIBUTING.md]() for more details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
