hi:
Don't split with semicolon [";"], it's a problem
For example, SQL contains comments with semicolons [";"]
In this way, there will be problems when the program is executed
--------------------------------------
BoYi ZhangE-mail : zhangboyi...@163.com
On 09/23/2020 15:06，lgcareer2...@outlook.com<lgcareer2...@outlook.com> wrote：
Hi,If execute executeFuncAndSql in a loop,it will close and open connection
frequently.
You can try use the `addBatch` and `executeBatch` method of PrepareStatement,
But it seems hive jdbc didn't support it.
BTW,Beacause the sql task can send the result by the email now,So If support
multiple queries,How to send the result.
Whether we only support mulitiple non-query statement and support only one
select statement?
Welcome discuss!Thanks!
Thanks,
Gang Li
DolphinScheduler(Incubator) PPMC
Gang Li 李岗
lgcar...@apache.org
From: zx liu
Date: 2020-09-21 17:09
To: dev
Subject: [DISCUSS] support multi-queries in single sql transformation task #3710
hi all,
The current sql task cannot support multiple queries, but my project is
very emergency about this feature. I chang this feature in my local.but
I'm not sure if my change is reasonable, so i need some help or suggestions。
hive and sql use the same code to execute sql，
executeFuncAndSql(mainSqlBinds, preStatementSqlBinds,
postStatementSqlBinds, createFuncs);
But the problem here is that hive JDBC does not support multiple queries,
mysql mssql supports multiple queries. If both mysql mssql and hive
supports multiple queries , so I consider the compatibility of the two
here, and modify it to split the sql statement according to the
conventional ";", but execute them one by one.
// support multiQueries 2020/9/16
String[] sqlsplit = mainSqlBinds.getSql().split(";");
if (sqlsplit.length == 0) {
logger.error("please check sqlscript can splited by ';'");
return;
}
Map<Integer, Property> sqlParamsMap = mainSqlBinds.getParamsMap();
for (String sqlscript : sqlsplit) {
executeFuncAndSql(new SqlBinds(sqlscript, sqlParamsMap),
preStatementSqlBinds, postStatementSqlBinds, createFuncs);
}
I don’t know if this change is reasonable, I hope you can give some
reasonable suggestions
https://github.com/apache/incubator-dolphinscheduler/issues/3710
thk!
当前的sql组件支持正常的多条语句同时执行，但是我们项目对这个功能比较着急，我在本地做了修改，但是我不确定我改的是否合理。
希望得到大家的建议或者帮助。
跟踪代码发现，hive 与mysql 使用了同样的执行语句的入口：
executeFuncAndSql(mainSqlBinds, preStatementSqlBinds,
postStatementSqlBinds, createFuncs);
有些数据源可能直接在jdbc 的连接参数加上allowmultiqueries=true就可以,但是hive jdbc是不支持 批量sql支持的这个功能，
我的考虑是在不能确定其它的数据源是否也支持多sql查询的功能，所以使用";"分割sql语句依然保持一条一条的语句执行，这样能
保证这两种情况兼容，同样能够实现基本的功能，代码的改动量很少。修改成这样：
// support multiQueries 2020/9/16
String[] sqlsplit = mainSqlBinds.getSql().split(";");
if (sqlsplit.length == 0) {
logger.error("please check sqlscript can splited by ';'");
return;
}
Map<Integer, Property> sqlParamsMap = mainSqlBinds.getParamsMap();
for (String sqlscript : sqlsplit) {
executeFuncAndSql(new SqlBinds(sqlscript, sqlParamsMap),
preStatementSqlBinds, postStatementSqlBinds, createFuncs);
}
这样修改是否合理，希望能够给予帮助或者建议。
谢谢！ https://github.com/apache/incubator-dolphinscheduler/issues/3710
