Copilot commented on code in PR #15320:
URL: https://github.com/apache/iotdb/pull/15320#discussion_r2038771240
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/utils/FileUtils.java:
##########
@@ -379,6 +379,35 @@ public static void moveFileWithMD5Check(final File
sourceFile, final File target
}
}
+ public static void copyFileWithMD5Check(final File sourceFile, final File
targetDir)
+ throws IOException {
+ final String sourceFileName = sourceFile.getName();
+ final File targetFile = new File(targetDir, sourceFileName);
+ if (targetFile.exists()) {
+ if (!haveSameMD5(sourceFile, targetFile)) {
+ final String renameFile = copyFileRenameWithMD5(sourceFile, targetDir);
+ LOGGER.info(
+ "Copy file {} to {} because it already exists in the target
directory: {}",
+ sourceFile.getName(),
+ renameFile,
+ targetDir.getAbsolutePath());
+ }
+ } else {
+ if (!(targetDir.exists() || targetDir.mkdirs())) {
+ final String log =
+ String.format("failed to create target directory: %s",
targetDir.getAbsolutePath());
+ LOGGER.warn(log);
+ throw new IOException(log);
Review Comment:
Consider enhancing error handling when target directory creation fails by
providing additional context or throwing a custom exception to clearly indicate
the failure reason.
```suggestion
final String errorMessage =
String.format("Failed to create target directory: %s. Please
check permissions or disk space.", targetDir.getAbsolutePath());
LOGGER.error(errorMessage);
throw new DirectoryCreationException(errorMessage);
```
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/analyze/LoadTsFileAnalyzer.java:
##########
@@ -228,6 +237,69 @@ public Analysis analyzeFileByFile(Analysis analysis) {
return analysis;
}
+ private boolean doAsyncLoad(final Analysis analysis) {
+ final String[] loadActiveListeningDirs =
+ IoTDBDescriptor.getInstance().getConfig().getLoadActiveListeningDirs();
+ String targetFilePath = null;
+ for (int i = 0, size = loadActiveListeningDirs == null ? 0 :
loadActiveListeningDirs.length;
+ i < size;
+ i++) {
+ if (loadActiveListeningDirs[i] != null) {
+ targetFilePath = loadActiveListeningDirs[i];
+ break;
+ }
+ }
+ if (targetFilePath == null) {
+ LOGGER.warn("Load active listening dir is not set. Will try sync load
instead.");
+ return false;
+ }
+
+ try {
+ loadTsFilesAsyncToTargetDir(new File(targetFilePath), tsFiles);
+ } catch (Exception e) {
+ LOGGER.warn(
+ "Failed to async load tsfiles {} to target dir {}. Will try sync
load instead.",
+ tsFiles,
+ targetFilePath,
+ e);
+ return false;
+ }
+
+ analysis.setFinishQueryAfterAnalyze(true);
+ analysis.setFailStatus(RpcUtils.getStatus(TSStatusCode.SUCCESS_STATUS));
+ analysis.setStatement(loadTsFileStatement);
+ return true;
+ }
+
+ private void loadTsFilesAsyncToTargetDir(final File targetDir, final
List<File> files)
+ throws IOException {
+ for (final File file : files) {
+ if (file == null) {
+ continue;
+ }
+
+ loadTsFileAsyncToTargetDir(targetDir, file);
+ loadTsFileAsyncToTargetDir(targetDir, new File(file.getAbsolutePath() +
".resource"));
+ loadTsFileAsyncToTargetDir(targetDir, new File(file.getAbsolutePath() +
".mods"));
+ }
+ }
+
+ private void loadTsFileAsyncToTargetDir(final File targetDir, final File
file)
+ throws IOException {
+ if (!file.exists()) {
+ return;
+ }
+ RetryUtils.retryOnException(
Review Comment:
[nitpick] Consider specifying explicit retry parameters (e.g. max attempts
or backoff interval) when using RetryUtils.retryOnException to improve control
over the async load failure handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
