THUMarkLau commented on code in PR #12113:
URL: https://github.com/apache/iotdb/pull/12113#discussion_r1510671751
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -2737,6 +2774,78 @@ private void updateTSInsertTabletsReq(
request.addToSizeList(tablet.rowSize);
}
+ public void invertToTabletsAndInsert(
+ List<String> deviceIds,
+ List<Long> times,
+ List<List<String>> measurementsList,
+ List<List<TSDataType>> typesList,
+ List<List<Object>> valuesList,
+ boolean isAligned)
+ throws IoTDBConnectionException, StatementExecutionException {
+ while (!deviceIds.isEmpty()) {
+ Map<String, Tablet> tablets = new HashMap<>();
+ Map<String, List<String>> deviceMeasurement = new HashMap<>();
+
+ List<String> remainDeviceIds = new ArrayList<>();
+ List<Long> remainTimes = new ArrayList<>();
+ List<List<String>> remainMeasurementsList = new ArrayList<>();
+ List<List<TSDataType>> remainTypesList = new ArrayList<>();
+ List<List<Object>> remainValuesList = new ArrayList<>();
+ // Set the max rows of the tablet as the count of the most frequently
occurring deviceId.
+ long maxRow =
+ deviceIds.stream()
+ .collect(Collectors.groupingBy(e -> e, Collectors.counting()))
+ .values()
+ .stream()
+ .max(Long::compare)
+ .orElse((long) deviceIds.size());
+ for (int i = 0; i < deviceIds.size(); i++) {
+ String deviceId = deviceIds.get(i);
+ if (tablets.containsKey(deviceId)) {
+ if (measurementsList.get(i) == deviceMeasurement.get(deviceId)) {
Review Comment:
Use `equals` instead of `==` here.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -1944,6 +1964,23 @@ public void insertAlignedRecords(
throw new IllegalArgumentException(
"prefixPaths, times, subMeasurementsList and valuesList's size
should be equal");
}
+ // judge if convert records to tablets.
+ // two devices are not same when:
+ // 1. they have different deviceId
+ // 2. they have same deviceId but different measurements.
+ List<List<String>> deviceMeasurementsList = new ArrayList<>();
Review Comment:
Same as above, add an experiment with both aligned and not aligned series.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -2737,6 +2774,78 @@ private void updateTSInsertTabletsReq(
request.addToSizeList(tablet.rowSize);
}
+ public void invertToTabletsAndInsert(
+ List<String> deviceIds,
+ List<Long> times,
+ List<List<String>> measurementsList,
+ List<List<TSDataType>> typesList,
+ List<List<Object>> valuesList,
+ boolean isAligned)
+ throws IoTDBConnectionException, StatementExecutionException {
+ while (!deviceIds.isEmpty()) {
Review Comment:
Even if all the records are about the same measurement under the same
device, if the order of measurements in each record is different, it could lead
to one set of records being split into multiple tablets for transmission. In
conditions of average network quality, the multiple RPC delays could result in
high latency and potentially poor performance. I lean towards placing all
records of a device into one tablet to complete the transmission in a single
step.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/Session.java:
##########
@@ -1901,6 +1903,24 @@ public void insertRecords(
throw new IllegalArgumentException(
"deviceIds, times, measurementsList and valuesList's size should be
equal");
}
+ // judge if convert records to tablets.
+ // two devices are not same when:
+ // 1. they have different deviceId
+ // 2. they have same deviceId but different measurements.
+ List<List<String>> deviceMeasurementsList = new ArrayList<>();
Review Comment:
Let's discuss this further. Suppose each row in the records represents
different measurements from the same device. Would converting these records
into tablets bring any benefits? If there are benefits, then we only need to
consider the differences in deviceID. We could add an experiment to see what
happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
