sollhui commented on code in PR #67353:
URL: https://github.com/apache/doris/pull/67353#discussion_r4332659816
##########
fe/fe-core/src/main/java/org/apache/doris/load/loadv2/BulkLoadJob.java:
##########
@@ -141,6 +142,10 @@ public static BulkLoadJob fromLoadCommand(LoadCommand
command, StmtExecutor exec
default:
throw new DdlException("Unknown load job type.");
}
+ if (ctx.getSessionVariable().enableProfile()) {
+ bulkLoadJob.setMaskedProfileSql(
+
SqlStatementMasker.mask(executor.getOriginStmt().originStmt, command));
Review Comment:
[P2] Move SQL masking outside the LoadManager write lock
`LoadManager.createLoadJobFromCommand()` holds its global write lock while
calling `fromLoadCommand()`, so this new masking operation runs inside that
critical section. Passing the parsed `command` does not avoid reparsing:
`NeedAuditEncryption.geneEncryptionSQL()` calls
`NereidsParser.parseForEncryption()` and builds another parse tree for the
complete SQL.
With profiling enabled, a LOAD containing many file paths or DATA
descriptions therefore extends the global lock hold time and delays other load
submissions and LoadManager operations requiring that lock. Please compute the
masked SQL before acquiring the LoadManager write lock and pass the result into
job construction, keeping the setter in the existing initialization path. This
is based on the call chain; the performance impact has not been benchmarked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
