yamass opened a new pull request, #19093:
URL: https://github.com/apache/iotdb/pull/19093
## Description
Introduce new Session methods: `insertTablets(List<Tablet>)` and
`insertAlignedTablets(List<Tablet>)`.
The existing methods `insertTablets(Map<String, Tablet>)` and
`insertAlignedTablets(Map<String, Tablet>)` allow only one tablet per device,
because the map is keyed by device id.
This PR adds `insertTablets(List<Tablet>)` and
`insertAlignedTablets(List<Tablet>)` to `ISession`/`Session` and
`ISessionPool`/`SessionPool`. They accept several tablets of the same device
and send them in one request. With redirection enabled, each tablet is routed
by its own device id, so tablets of devices with the same leader go to that
leader together. No behaviour change for existing methods.
### Why
The devices in our system sample their measurements independently, so the
timestamps of their measurements rarely match. Packing such measurements into
one tablet per device fills most value cells with nulls. So in our case, the
natural representation is one tablet per measurement, i.e. several tablets per
device. `TSInsertTabletsReq` already accepts the same device several times;
only the client API does not.
### Workarounds
With the current map-based API, there are two workarounds to send several
tablets of one device in one request:
1. **Key the tablets by something unique, e.g. the timeseries path.** This
works, but the map key is what selects the connection, and such keys are never
in the leader cache. So with redirection enabled, every request goes to the
default connection, and the receiving DataNode has to forward it to the leader.
2. **Pass a fake `Map` that contains the same device id several times**, one
entry per tablet. This routes correctly, but breaks the `Map` contract and
relies on `Session` only iterating the map. If `Session` ever copied it into a
real map, all but one tablet per device would be dropped without an error.
*This is our current workaround, guarded by tests that fail if an IoTDB upgrade
breaks it.*
### Changes
- `Session.insertTablets(List)` / `insertAlignedTablets(List)`: with
redirection enabled, they key each tablet by its device id and use the existing
leader-cache path. Without redirection, they send one request over the default
connection, like the map methods.
- `insertTabletsWithLeaderCache` now takes a list of
`KeyedTablet(leaderCacheKey, tablet)` instead of a map, so a key may occur
several times. The map methods use their map keys as leader cache keys, so
their routing stays the same.
- No `sorted` overloads: `sorted` is deprecated, and unsorted tablets are
sorted anyway.
### Tests
**`SessionInsertTabletListTest`** (new) uses a mocked connection that
records every request and, like a DataNode, answers with a redirect when a
request contains devices led by another endpoint.
| Test
| Checks
|
|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|
|
`test_InsertTablets_WhenLeadersAreUnknown_SendsAllTabletsToDefaultEndpoint`
| first request goes to the default endpoint; the redirect fills the leader
cache |
| `test_InsertTablets_WhenLeadersAreKnown_SendsOneRequestPerLeader`
| three tablets of one device go to its leader in one request, another
device's tablet to its own leader |
|
`test_InsertTablets_WhenRedirectionIsDisabled_SendsOneRequestToDefaultEndpoint`
| one non-aligned request over the default connection
|
|
`test_InsertAlignedTablets_WhenLeadersAreKnown_SendsAlignedRequestPerLeader`
| aligned flag and routing
|
|
`test_InsertAlignedTablets_WhenRedirectionIsDisabled_SendsOneAlignedRequest`
| aligned flag without redirection
|
**`SessionPoolTest`:**
| Test
| Checks |
|---------------------------------------------------------------------------------------|------------------------------------------|
|
`test_InsertTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBack`
| pool delegates and puts the session back |
|
`test_InsertAlignedTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBack`
| same, aligned |
| `test_InsertTabletList_WhenSessionRejectsIt_RethrowsAndPutsSessionBack`
| rejected insert |
| `test_InsertAlignedTabletList_WhenConnectionBreaks_ClosesSessionAndThrows`
| connection error |
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added Javadocs for most classes and all non-trivial methods.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold
for code coverage.
<!-- Check the items by putting "x" in the brackets for the done things. Not
all of these items
apply to every PR. Remove the items which are not done or not relevant to
the PR. None of the items
from the checklist above are strictly necessary, but it would be very
helpful if you at least
self-review the PR. -->
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
* `ISession`, `ISessionPool`, `Session`, `SessionPool` + tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]