JWuCines opened a new pull request, #19698:
URL: https://github.com/apache/druid/pull/19698
### Description
The `export-metadata` tool currently only supports exporting from Derby
metadata stores. This PR adds support for exporting from PostgreSQL by
implementing a generic JDBC-based `exportTable` in `SQLMetadataConnector`,
which PostgreSQL (and any future connector) inherits automatically.
#### Added generic JDBC export in `SQLMetadataConnector`
Implemented `exportTableWithJdbc`, a `protected` method that exports any
table to CSV using standard JDBC. Binary/BLOB columns (including PostgreSQL
`BYTEA`) are hex-encoded, booleans are written as `true`/`false` strings, and
string values containing commas, quotes, `\n`, or `\r` are properly CSV-escaped
per RFC 4180. The base `exportTable` delegates to this method, while
`DerbyConnector` continues to override it with Derby's native
`SYSCS_EXPORT_TABLE`.
#### Updated `ExportMetadata` for PostgreSQL compatibility
- Table names are no longer unconditionally uppercased; uppercasing is now
applied only for Derby (detected via `jdbc:derby` URI prefix), since PostgreSQL
uses lowercase table names.
- Updated the `@Command` description to mention PostgreSQL support.
#### Added unit tests
Four new tests in `SQLMetadataConnectorTest` exercise the generic JDBC
export path via `TestDerbyConnector.exportTableGeneric()`:
- **`testExportTable`** — verifies hex-encoded BLOBs and `true`/`false`
boolean strings
- **`testExportTableWithSpecialCharacters`** — verifies CSV quoting/escaping
for commas, double quotes, and plain values
- **`testExportTableWithNullValues`** — verifies NULL columns produce empty
CSV fields
- **`testExportTablePreservesAllColumns`** — verifies all columns (including
nullable trailing columns like `used_status_last_updated`) are exported
#### Updated documentation
- **`export-metadata.md`** — removed the Derby-only limitation, added a
"PostgreSQL" section under "Running the tool" with the required
`-Ddruid.extensions.loadList` and `-Ddruid.metadata.storage.type` flags,
clarified `_raw.csv` description
- **`metadata-migration.md`** — updated intro and export tool reference to
include PostgreSQL
- **`deep-storage-migration.md`** — updated export tool reference, added
note about no running processes needed when migrating from PostgreSQL
#### Release note
The `export-metadata` tool now supports exporting from PostgreSQL metadata
stores in addition to Derby. When exporting from PostgreSQL, pass
`-Ddruid.extensions.loadList='["postgresql-metadata-storage"]'
-Ddruid.metadata.storage.type=postgresql` on the command line along with the
appropriate `--connectURI`.
<hr>
##### Key changed/added classes in this PR
* `SQLMetadataConnector` — added `exportTable` and `exportTableWithJdbc`
for generic JDBC CSV export
* `ExportMetadata` — added `isDerby()` helper, conditional table name casing
* `TestDerbyConnector` — added `exportTableGeneric()` to test the generic
JDBC path
* `SQLMetadataConnectorTest` — added 4 export tests
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] a release note entry in the PR description.
- [x] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for code coverage is met.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
