hanjun996 removed a comment on issue #4698:
URL: https://github.com/apache/hudi/issues/4698#issuecomment-1031040459
Environment Description
Hudi version :0.10.0
flink version :1.13.0
Steps to reproduce the behavior:
mysql timestamp type, such as: 1970-01-01T00:00:01Z, the corresponding type
in flink is TIMESTAMP_WITH_LOCAL_TIME_ZONE, which is not supported by avro
during conversion in hudi version 0.10.0
codeļ¼
public static final DataType ROW_DATA_TYPE = DataTypes.ROW(
DataTypes.FIELD("id", DataTypes.STRING()),
DataTypes.FIELD("timestamp",
DataTypes.TIMESTAMP_WITH_LOCAL_TIME_ZONE()))
.notNull();
public static final RowType ROW_TYPE = (RowType)
ROW_DATA_TYPE.getLogicalType();
@Test
public void test() {
String inferredSchema =
AvroSchemaConverter.convertToSchema(ROW_TYPE).toString();
Configuration configuration = new Configuration();
configuration.setString(FlinkOptions.PATH, "test");
configuration.setString(FlinkOptions.TABLE_NAME, "test");
configuration.setString(FlinkOptions.TABLE_TYPE, "MERGE_ON_READ");
configuration.setString(FlinkOptions.RECORD_KEY_FIELD, "id");
configuration.setString(FlinkOptions.SOURCE_AVRO_SCHEMA,
inferredSchema);
Map<String, String> record1 = new HashMap<>();
record1.put("id", "1");
record1.put("timestamp", "1970-01-01T00:00:01Z");
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
DataStreamSource source = env.fromElements(record1);
Pipelines.bootstrap(configuration, ROW_TYPE, 1, source).print();
}
ERROR:
java.lang.UnsupportedOperationException: Unsupported to derive Schema for
type: TIMESTAMP_LTZ(6)
at
org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:292)
at
org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:263)
at
org.apache.hudi.util.AvroSchemaConverter.convertToSchema(AvroSchemaConverter.java:169)
Expected behavior:
support TIMESTAMP_WITH_LOCAL_TIME_ZONE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
