yuzelin commented on code in PR #1621:
URL: https://github.com/apache/incubator-paimon/pull/1621#discussion_r1270557733
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlTableSchemaBuilder.java:
##########
@@ -53,50 +47,40 @@ public MySqlTableSchemaBuilder(Map<String, String>
tableConfig, boolean caseSens
}
@Override
- public Optional<Schema> build(MySqlCreateTableStatement statement) {
- List<SQLTableElement> columns = statement.getTableElementList();
- LinkedHashMap<String, Tuple2<DataType, String>> fields = new
LinkedHashMap<>();
-
- for (SQLTableElement element : columns) {
- if (element instanceof SQLColumnDefinition) {
- SQLColumnDefinition column = (SQLColumnDefinition) element;
- SQLName name = column.getName();
- SQLDataType dataType = column.getDataType();
- List<SQLExpr> arguments = dataType.getArguments();
- Integer precision = null;
- Integer scale = null;
- if (arguments.size() >= 1) {
- precision = (int) (((SQLIntegerExpr)
arguments.get(0)).getValue());
- }
-
- if (arguments.size() >= 2) {
- scale = (int) (((SQLIntegerExpr)
arguments.get(1)).getValue());
- }
-
- SQLCharExpr comment = (SQLCharExpr) column.getComment();
- fields.put(
- name.getSimpleName(),
- Tuple2.of(
- MySqlTypeUtils.toDataType(
- column.getDataType().getName(),
- precision,
- scale,
-
MYSQL_CONVERTER_TINYINT1_BOOL.defaultValue()),
- comment == null ? null :
String.valueOf(comment.getValue())));
- }
+ public Optional<Schema> build(JsonNode tableChange) {
+ JsonNode jsonTable = tableChange.get("table");
+ String tableName = tableChange.get("id").asText();
+ ArrayNode columns = (ArrayNode) jsonTable.get("columns");
+ LinkedHashMap<String, DataType> fields = new LinkedHashMap<>();
+
+ for (JsonNode element : columns) {
+ Integer precision = element.has("length") ?
element.get("length").asInt() : null;
+ Integer scale = element.has("scale") ?
element.get("scale").asInt() : null;
+ fields.put(
+ element.get("name").asText(),
+ MySqlTypeUtils.toDataType(
+ element.get("typeExpression").asText(),
+ precision,
+ scale,
+
MYSQL_CONVERTER_TINYINT1_BOOL.defaultValue())
+ .copy(element.get("optional").asBoolean()));
Review Comment:
I'm not familiar with debezium json, I want to know here can we find field
comments? If we can, we should also get the comments.
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlDebeziumJsonEventParser.java:
##########
@@ -231,20 +227,26 @@ public Optional<Schema> parseNewTable() {
try {
String historyRecordString = historyRecord.asText();
- String ddl =
objectMapper.readTree(historyRecordString).get("ddl").asText();
- if (Strings.isNullOrEmpty(ddl)) {
- return Optional.empty();
+ JsonNode tableChanges =
objectMapper.readTree(historyRecordString).get("tableChanges");
+ if (tableChanges.size() != 1) {
+ throw new IllegalArgumentException(
+ "Invalid historyRecord, because tableChanges should
contain exactly 1 item.\n"
+ + historyRecord.asText());
}
- SQLStatement statement = SQLUtils.parseSingleStatement(ddl,
JdbcConstants.MYSQL);
- if (!(statement instanceof MySqlCreateTableStatement)) {
+ if (!tableChanges
+ .get(0)
+ .get("type")
+ .asText()
+ .equals(TableChanges.TableChangeType.CREATE.name())) {
return Optional.empty();
}
- MySqlCreateTableStatement createTableStatement =
(MySqlCreateTableStatement) statement;
- List<String> primaryKeys =
createTableStatement.getPrimaryKeyNames();
- String tableName = createTableStatement.getTableName();
- if (primaryKeys.isEmpty()) {
+ JsonNode primaryKeyColumnNames =
+
tableChanges.get(0).get("table").get("primaryKeyColumnNames");
+ if (primaryKeyColumnNames.size() == 0) {
+ String id = tableChanges.get(0).get("id").asText();
+ String tableName = id.replaceAll("\"", "").split("\\.")[1];
Review Comment:
I think it's OK to use `id` in `LOG.debug`.
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/mysql/MySqlDebeziumJsonEventParser.java:
##########
@@ -231,20 +227,26 @@ public Optional<Schema> parseNewTable() {
try {
String historyRecordString = historyRecord.asText();
- String ddl =
objectMapper.readTree(historyRecordString).get("ddl").asText();
- if (Strings.isNullOrEmpty(ddl)) {
- return Optional.empty();
+ JsonNode tableChanges =
objectMapper.readTree(historyRecordString).get("tableChanges");
+ if (tableChanges.size() != 1) {
+ throw new IllegalArgumentException(
+ "Invalid historyRecord, because tableChanges should
contain exactly 1 item.\n"
+ + historyRecord.asText());
}
- SQLStatement statement = SQLUtils.parseSingleStatement(ddl,
JdbcConstants.MYSQL);
- if (!(statement instanceof MySqlCreateTableStatement)) {
+ if (!tableChanges
+ .get(0)
Review Comment:
Too many tableChanges.get(0).
You can use a variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
