sollhui opened a new pull request, #39479:
URL: https://github.com/apache/doris/pull/39479
when using routine load to consume Kafka stream:
```
CREATE ROUTINE LOAD jobName ON tableName
COLUMNS TERMINATED BY ",",
COLUMNS(k1,k2,v1,v2,v3,tmp,v4=json_unquote(tmp))
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "${externalEnvIp}:${kafka_port}",
"kafka_topic" = "${kafkaCsvTpoics[0]}",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
There is no exception but data can not consume. The root cause is the json
function -` json unquote` parse data error will cancel load directly and do not
record url and rows statistic, making routine load data quality checker invalid
and always retry to consume Kafka data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
