ramackri opened a new pull request, #691: URL: https://github.com/apache/atlas/pull/691
## Summary Implements the Trino extractor test suite that was previously only sketched as comments in `TrinoExtractorIT`. Tests exercise the **standalone extractor tarball** (`run-trino-extractor.sh`) against a live Atlas + Trino stack, matching production layout rather than the Maven test classpath. JIRA: https://issues.apache.org/jira/browse/ATLAS-5338 ## Changes ### `TrinoExtractorIT` Full live integration test suite: - Invalid CLI arguments and cron expression handling - Catalog, schema, table, and column metadata import - Unregistered catalog via command-line `-c` - Stale entity cleanup after delete/rename (opt-in via `TRINO_IT_DELETE_SYNC=1`) - Hive hook linkage to `trino_column` (opt-in via `TRINO_IT_HOOK_ENABLED=1`) - PII tag propagation via hook (opt-in via `TRINO_IT_TAG_PROPAGATION=1`) - Tarball Jersey classpath verification ### `TrinoExtractorITSupport` Shared test infrastructure: - Tarball workdir preparation and cleanup - Atlas REST entity existence/absence assertions - Stale entity seeding for delete-sync scenarios - Property overrides and extractor script execution helpers ### `TrinoExtractorTest` Unit tests (no live stack required): - Invalid/valid Quartz cron expression validation - `@DisallowConcurrentExecution` on `MetadataJob` ### `pom.xml` - New Maven profile **`trino-extractor-it`**: builds extractor tarball in `pre-integration-test`, runs Failsafe `*IT` tests - Test dependencies: `testng`, `commons-io` ### `src/test/assemblies/trino-extractor-tarball.xml` Assembly descriptor for IT tarball packaging (same `lib/` layout as distro). ### `README.md` Documents how to run unit and integration tests, required environment variables, optional flags, and coverage map. ## How to test **Unit tests (no Atlas/Trino required):** ```bash mvn -pl addons/trino-extractor test -Dtest=TrinoExtractorTest ``` **Live integration tests:** ```bash export ATLAS_REST_URL=http://localhost:21000 export TRINO_JDBC_URL=jdbc:trino://localhost:8080/ export ATLAS_USERNAME=admin export ATLAS_PASSWORD=atlasR0cks! export TRINO_EXTRACTOR_SCHEMA=hr export TRINO_EXTRACTOR_TABLE=trino_pii_lab export TRINO_EXTRACTOR_COLUMN=ssn mvn -pl addons/trino-extractor -Ptrino-extractor-it verify ``` **Optional flags:** | Variable | Purpose | |----------|---------| | `TRINO_IT_HOOK_ENABLED=1` | Verify `trino_column` links to `hive_column` | | `TRINO_IT_TAG_PROPAGATION=1` | Verify PII tag propagation | | `TRINO_IT_DELETE_SYNC=1` | Verify stale catalog/schema/table removal | ## Test plan - [x] `mvn -pl addons/trino-extractor test -Dtest=TrinoExtractorTest` passes - [ ] `mvn -pl addons/trino-extractor -Ptrino-extractor-it verify` against Atlas + Trino lab (manual; requires `ATLAS_REST_URL`) - [x] ITs skip gracefully when live stack is not configured Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
