mattcasters opened a new pull request, #8162: URL: https://github.com/apache/hop/pull/8162
fixes #8154 Hop Binary stream fields (hash keys, raw bytes, BLOBs) can now be bulk loaded on every transform whose engine accepts binary in its bulk protocol. There is no new dialog option: the stream type drives the encoding, the same way PostgreSQL already serializes booleans as `t`/`f`. The previous failure (`PGBulkLoader doesn't handle the type Binary`) is gone. Text-based loaders call `getBinary()` and write hex in the format that engine actually reads: | Loader | Encoding | |---|---| | PostgreSQL / Greenplum | `\xdeadbeef` (`bytea` hex) | | MySQL | lowercase hex + `SET col = UNHEX(@col)` | | Oracle | hex in the data file + `HEXTORAW` in the SQL*Loader control file (replaces the broken `startlob` / `byte[].toString()` path) | | Snowflake | hex + `BINARY_FORMAT = 'HEX'` on `COPY` | | Redshift | hex **without** `\x` (`VARBYTE`) | | MonetDB | hex for `BLOB` | | TeraFast | hex for FastLoad `BYTE`/`VARBYTE` | | CrateDB | hex string (no native binary type; lands in `STRING`) | | SQL Server | unchanged — already sent as `byte[]` | | Vertica | unchanged native bytes; `BINARY`/`VARBINARY` now use `getBinary()` | Out of scope: Doris (it stream-loads a payload field you already built) and the MySQL/SQL Server **actions** (they load an existing file, not Hop rows). ## Tests Unit tests cover the hex helpers and write paths for PostgreSQL, MySQL, Oracle, Snowflake, Redshift, MonetDB, TeraFast, CrateDB, and Vertica. Existing integration tests were run successfully for projects **database**, **mssql**, **vertica**, and **monetdb**. Project **cratedb** is disabled. None of those suites currently load Binary/`BYTEA`/`VARBINARY` columns. ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`. - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
