yamass opened a new pull request, #19093:
URL: https://github.com/apache/iotdb/pull/19093

   ## Description
   
   Introduce new Session methods: `insertTablets(List<Tablet>)` and 
`insertAlignedTablets(List<Tablet>)`.
   
   The existing methods `insertTablets(Map<String, Tablet>)` and 
`insertAlignedTablets(Map<String, Tablet>)` allow only one tablet per device, 
because the map is keyed by device id. 
   
   This PR adds `insertTablets(List<Tablet>)` and 
`insertAlignedTablets(List<Tablet>)` to `ISession`/`Session` and 
`ISessionPool`/`SessionPool`. They accept several tablets of the same device 
and send them in one request. With redirection enabled, each tablet is routed 
by its own device id, so tablets of devices with the same leader go to that 
leader together. No behaviour change for existing methods.
   
   ### Why
   
   The devices in our system sample their measurements independently, so the 
timestamps of their measurements rarely match. Packing such measurements into 
one tablet per device fills most value cells with nulls. So in our case, the 
natural representation is one tablet per measurement, i.e. several tablets per 
device. `TSInsertTabletsReq` already accepts the same device several times; 
only the client API does not.
   
   ### Workarounds
   
   With the current map-based API, there are two workarounds to send several 
tablets of one device in one request:
   
   1. **Key the tablets by something unique, e.g. the timeseries path.** This 
works, but the map key is what selects the connection, and such keys are never 
in the leader cache. So with redirection enabled, every request goes to the 
default connection, and the receiving DataNode has to forward it to the leader.
   2. **Pass a fake `Map` that contains the same device id several times**, one 
entry per tablet. This routes correctly, but breaks the `Map` contract and 
relies on `Session` only iterating the map. If `Session` ever copied it into a 
real map, all but one tablet per device would be dropped without an error. 
*This is our current workaround, guarded by tests that fail if an IoTDB upgrade 
breaks it.*
   
   ### Changes
   
   - `Session.insertTablets(List)` / `insertAlignedTablets(List)`: with 
redirection enabled, they key each tablet by its device id and use the existing 
leader-cache path. Without redirection, they send one request over the default 
connection, like the map methods.
   - `insertTabletsWithLeaderCache` now takes a list of 
`KeyedTablet(leaderCacheKey, tablet)` instead of a map, so a key may occur 
several times. The map methods use their map keys as leader cache keys, so 
their routing stays the same.
   - No `sorted` overloads: `sorted` is deprecated, and unsorted tablets are 
sorted anyway.
                                       
   ### Tests
   
   **`SessionInsertTabletListTest`** (new) uses a mocked connection that 
records every request and, like a DataNode, answers with a redirect when a 
request contains devices led by another endpoint.
   
   | Test                                                                       
     | Checks                                                                   
                              |
   
|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|
   | 
`test_InsertTablets_WhenLeadersAreUnknown_SendsAllTabletsToDefaultEndpoint`     
| first request goes to the default endpoint; the redirect fills the leader 
cache                        |
   | `test_InsertTablets_WhenLeadersAreKnown_SendsOneRequestPerLeader`          
     | three tablets of one device go to its leader in one request, another 
device's tablet to its own leader |
   | 
`test_InsertTablets_WhenRedirectionIsDisabled_SendsOneRequestToDefaultEndpoint` 
| one non-aligned request over the default connection                           
                         |
   | 
`test_InsertAlignedTablets_WhenLeadersAreKnown_SendsAlignedRequestPerLeader`    
| aligned flag and routing                                                      
                         |
   | 
`test_InsertAlignedTablets_WhenRedirectionIsDisabled_SendsOneAlignedRequest`    
| aligned flag without redirection                                              
                         |
   
   **`SessionPoolTest`:**
   
   | Test                                                                       
           | Checks                                   |
   
|---------------------------------------------------------------------------------------|------------------------------------------|
   | 
`test_InsertTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBack`  
      | pool delegates and puts the session back |
   | 
`test_InsertAlignedTabletList_WhenSessionAcceptsIt_PassesTabletsOnAndPutsSessionBack`
 | same, aligned                            |
   | `test_InsertTabletList_WhenSessionRejectsIt_RethrowsAndPutsSessionBack`    
           | rejected insert                          |
   | `test_InsertAlignedTabletList_WhenConnectionBreaks_ClosesSessionAndThrows` 
           | connection error                         |
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added Javadocs for most classes and all non-trivial methods.
   - [x] added unit tests or modified existing tests to cover new code paths, 
ensuring the threshold
     for code coverage.
   
   <!-- Check the items by putting "x" in the brackets for the done things. Not 
all of these items 
   apply to every PR. Remove the items which are not done or not relevant to 
the PR. None of the items 
   from the checklist above are strictly necessary, but it would be very 
helpful if you at least 
   self-review the PR. -->
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) 
in this PR
   
   * `ISession`, `ISessionPool`, `Session`, `SessionPool` + tests
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to