kfaraz opened a new pull request, #13310:
URL: https://github.com/apache/druid/pull/13310
The fetch of pending segments happens behind a lock and can cause others
threads to remain stuck while deserializing the payload fetched from the
metadata store.
__Changes:__
- Deserialize the payload only when needed
__Notes:__
The query to fetch the payload uses a `<=` and `>=` on the start and end
intervals.
```
SELECT payload FROM pending_segments
WHERE datasource = 'search-ds'
AND start <= 'search-end-time'
AND end >= 'search-start-time'
```
This might often end up fetching more segments than would actually have an
overlap.
We can try to improve the query by adding clearer conditions but that would
look something like:
```
SELECT payload FROM pending_segments
WHERE datasource = 'search-ds'
AND 'search-end-time' BETWEEN start and end
OR 'search-start-time' BETWEEN start and end
```
I am not sure if this would be able to leverage the existing indexes on the
table better than the existing query.
So leaving it untouched for now.
<hr>
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency
checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md)
(Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in
[licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
