SEPURI-SAI-KRISHNA commented on PR #13046:
URL: https://github.com/apache/seatunnel/pull/13046#issuecomment-6488674723
@DanielLeens thanks, agreed. The Druid cluster is started once for the class
and the template runs once per engine, so with a fixed `druid_null_field`
datasource a later engine could pass on the rows an earlier one indexed.
Updated as your Option A:
- `testDruidSinkWithNullFields` now builds the datasource name from the
engine, `druid_null_field_` plus `container.identifier()` lower cased (for
example `druid_null_field_seatunnel`, `druid_null_field_flink_1_13`), passes it
to the job as the `nullFieldDatasource` variable, and queries that same
datasource.
- `fakesource_to_druid_with_null.conf` reads `datasource =
"${nullFieldDatasource}"`.
- The response is parsed instead of matched as a substring. The test asserts
exactly 3 rows, then by id: row 1 has `name` `a` and `amount` 10, row 2 has
`amount` 20 (the value after the null name), row 3 has `name` `c`. The null
values themselves are still not asserted, because how Druid renders them
depends on its null handling.
Until a run's own indexing task is queryable, the query returns an error
object, the parse fails, and Awaitility retries. So each engine can only pass
on its own three rows, and the check no longer depends on JSON property order.
The production change and the unit tests are unchanged. The other DruidIT
tests use the same fixed datasource pattern; I left them as they are to keep
this PR to the null-field fix.
Checks: `DruidIT.java` compiles, and `spotless:check` on
`connector-druid-e2e` is clean. Docker is not available to me, so the new
head's E2E result will come from CI. On the previous head the fork
`updated-modules-integration-test-part-1` ran `DruidIT` with 14 tests, 0
failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
