Cyanty commented on code in PR #9906:
URL: https://github.com/apache/seatunnel/pull/9906#discussion_r2417152077
##########
seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/executor/SqlUtils.java:
##########
@@ -17,24 +17,65 @@
package
org.apache.seatunnel.connectors.seatunnel.clickhouse.sink.client.executor;
+import org.apache.seatunnel.shade.com.google.common.collect.Streams;
+
+import org.apache.seatunnel.api.table.type.SeaTunnelDataType;
+import org.apache.seatunnel.api.table.type.SeaTunnelRowType;
+
+import com.clickhouse.client.ClickHouseDataType;
+
import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
import java.util.stream.Collectors;
-
-import static java.lang.String.format;
+import java.util.stream.IntStream;
public class SqlUtils {
+ private static final Map<String, String> TYPE_CASTS =
+ new HashMap<String, String>() {
+ {
+ String CAST_TEMPLATE = "CAST(? AS %s)";
+ put(ClickHouseDataType.JSON.name(),
String.format(CAST_TEMPLATE, "String"));
+ }
+ };
+
+ public static String getTypeCast(String clickHouseDataType) {
+ return TYPE_CASTS.getOrDefault(clickHouseDataType, "?");
+ }
+
public static String quoteIdentifier(String identifier) {
return "\"" + identifier + "\"";
}
- public static String getInsertIntoStatement(String tableName, String[]
fieldNames) {
+ public static String getInsertIntoStatement(
+ String tableName, SeaTunnelRowType rowType, Map<String, String>
tableSchema) {
+ String[] fieldNames = rowType.getFieldNames();
+ SeaTunnelDataType<?>[] fieldTypes = rowType.getFieldTypes();
String columns =
Arrays.stream(fieldNames)
.map(SqlUtils::quoteIdentifier)
.collect(Collectors.joining(", "));
+
+ String[] typeNames =
+ IntStream.range(0, fieldNames.length)
+ .mapToObj(
+ i ->
+ tableSchema.containsKey(fieldNames[i])
+ &&
ClickHouseDataType.JSON
+ .name()
+
.equalsIgnoreCase(
+
tableSchema.get(
+
fieldNames[i]))
+ ?
ClickHouseDataType.JSON.name()
+ :
fieldTypes[i].getSqlType().name())
Review Comment:
> The sqltype is come from SeaTunnel. Why it can used in clickhouse sql
directly? We should use `ClickhouseTypeConverter` to convert.
Hi @Hisoka-X , Thank you for your guidance. I'm facing a roadblock and would
appreciate your input.
To force `SqlBasedPreparedStatement` for writing JSON types, I'm adding a
CAST in the VALUES() clause. And that all other types are converted to a
generic `?`.
Since only the JSON type requires special handling, I'd like to skip
implementing `ClickhouseTypeConverter#convert` and instead directly use the
data types from `tableSchema` for the CAST.
e.g.,`String typeName = tableSchema.getOrDefault(fieldName, "")`, Do you
think this is feasible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
