kfaraz opened a new pull request, #18302: URL: https://github.com/apache/druid/pull/18302
### Description This contains the same changes as #18265. Since that PR had a very long history of tuning the workflows (sincere thanks to @Akshat-Jain for the assistance!), I have created a fresh PR instead to simplify reviews. The old PR had only a couple of comments: https://github.com/apache/druid/pull/18265#discussion_r2214466052 https://github.com/apache/druid/pull/18265#discussion_r2214471841 which have already been answered. ### Changes - Add extension `druid-testcontainers` that can be later published as an [individual module](https://testcontainers.com/modules/) for `Testcontainers` - Add `DruidContainer` - `Testcontainer` impl for running individual Druid services - Add `DruidContainerResource` to allow use of `DruidContainer` in embedded cluster tests - Make minor modifications to `EmbeddedDruidServer` and `EmbeddedDruidCluster` to support running both embedded servers and container-based services in the same cluster - Add job to run the `*DockerTest`s (2 right now, this number should remain small over time). Note: __Usage of `DruidContainer` will not be the norm but the exception to test out backward compatibility and/or Docker related changes only. All other use cases should continue to use `EmbeddedDruidServers`.__ ### Pending items - Populate all the test methods using the builders in #18207 once it is merged ### Advantages - Can be used to run embedded tests against the distribution Docker image as opposed to the current ITs which use a custom IT-only image - Takes us 1 step closer to completely removing the old IT frameworks - Leverages all the benefits of the embedded test framework while also testing out the Docker containers - Can be easily used for backward compatibility tests ### Next steps - Once we have phased out the old ITs, all integration tests will be based on `EmbeddedClusterTestBase` - Most of these tests will just use `EmbeddedDruidServer` (and required containers only, like `KafkaResource`) - These tests will use Indexer as worker as they are faster and lighter than MiddleManagers - There will be only a small number of tests that use `DruidContainer`, some of which will use MiddleManagers <hr> This PR has: - [ ] been self-reviewed. - [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.) - [ ] added documentation for new or modified features or behaviors. - [ ] a release note entry in the PR description. - [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links. - [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md) - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. - [ ] added integration tests. - [ ] been tested in a test Druid cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
