maytasm opened a new pull request #10186:
URL: https://github.com/apache/druid/pull/10186
Add integration tests for Appends
### Description
The native batch ingestion currently allows appending dynamically
partitioned segments regardless of the original partitioning scheme. The new
integration tests include the followings:
- Append dynamically-partitioned segments to a dynamically-partitioned
datasource
- Append dynamically-partitioned segments to a hash-partitioned datasource.
Compact hash-partitioned segments and dynamically-partitioned segments after
append.
- Append dynamically-partitioned segments to a range-partitioned datasource.
Compact range-partitioned segments and dynamically-partitioned segments after
append.
Some notes on the tests:
- Note that the intervals of the appended segment overlap with those of
original segments. This is done by using the same data for initial write and
append.
- Batch append itself is pretty well covered by unit tests (the unit tests
mimic the distributed ingestion using multi-threaded ingestion testing
framework (see AbstractParallelIndexSupervisorTaskTest). The integration tests
focus on missing parts such as querying segments of mixed partitioning schemes.
This PR has:
- [x] been self-reviewed.
- [ ] using the [concurrency
checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md)
(Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in
[licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml)
- [ ] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [x] added integration tests.
- [ ] been tested in a test Druid cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
