rmannibucau commented on PR #9353: URL: https://github.com/apache/hop/pull/9353#issuecomment-6893446555
Rebased onto current main (`947ecbe21e`); updated head is `243b87bcd6`. Review blockers and follow-ups addressed: - Added classic database and native Spark integration workflows for partitioned Table Input; classic coverage uses four transform copies. - Preview/Debug now respects per-copy partitioning rather than duplicating the complete result in every copy. - Native Spark falls back when explicit output fields are configured, preserving classic mapping/conversion semantics. - Classic partitioned reads normalize trailing SQL semicolons before wrapping the query; regression coverage added. - Preserved upstream SQL preparation, specified-field handling and UTF-8 translations during rebase. Validation on Java 21 / Maven 3.9.16: - Focused verify with Checkstyle enabled: Table Input 80 unit tests + 12 integration tests; Spark 26 unit tests + 3 integration tests, zero failures/errors/skips. - Client assembly succeeded; `git diff --check` passed. - `database/main-0008-table-input.hwf` passed via the supported Docker/Hop runner: all 10 pipeline variants succeeded; four partitioned copies returned 12 rows total and matched golden data. - `spark-native/main-0011-table-input-partitioned.hwf` passed via repository `run-tests.sh` / `hop-run.sh`, using the freshly assembled PR client in its base image: Spark 4.1.3 local[2], H2 file database, four JDBC partitions, all 12 expected rows including null/out-of-bound keys matched golden data. - All GitHub checks are green; GitHub reports MERGEABLE / CLEAN. Environment notes: the Spark Docker wrapper initially failed before tests because the proxy hostname could not resolve during Docker Hub authorization; the supported Hop runner fallback above succeeded. An earlier broad local reactor run hit the unrelated existing SVG font-assumption test failure; current GitHub build passes. GUI Preview/Debug and distributed Spark-cluster execution were not exercised interactively. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
