mincwang commented on issue #6550: URL: https://github.com/apache/incubator-doris/issues/6550#issuecomment-919842453
> > @JNSimba thanks for your reply. here is my code, and some errors. > > doris ddl： > > ```sql > > CREATE TABLE `product` ( > > `id` int(11) NULL COMMENT "", > > ... > > `gmt_modified` datetime NULL COMMENT "", > > `gmt_create` datetime NULL COMMENT "" > > ) ENGINE=OLAP > > DUPLICATE KEY(`id`, `id_hash`) > > COMMENT "OLAP" > > DISTRIBUTED BY HASH(`id`) BUCKETS 10 > > PROPERTIES ( > > "replication_num" = "3", > > "in_memory" = "false", > > "storage_format" = "V2" > > ); > > ``` > > > > > > > > > > > > > > > > > > > > > > > > flink sql doris-sink ddl： > > ```sql > > CREATE TABLE product ( > > id INT, > > ... > > gmt_modified TIMESTAMP(6), > > gmt_create TIMESTAMP(6) ) > > WITH ( > > 'connector' = 'doris', > > 'fenodes' = 'n3:8031', > > 'table.identifier' = 'dev.product', > > 'username' = 'root', > > 'sink.batch.size' = '100', > > 'sink.properties.column_separator' = '\x01', > > 'sink.properties.line_delimiter' = '\x02', > > 'sink.batch.interval' = '2s' , > > 'password' = 'doris' > > ); > > ``` > > > > > > > > > > > > > > > > > > > > > > > > ```sql > > # flink insert sql > > insert into product > > select * from product_cdc_table ; > > ``` > > > > > > > > > > > > > > > > > > > > > > > > When we write data to dorsi by flink streaming, have following error occurs. > >  > > doris version: 0.14 > flink-doris-connector version : master > flink version ： 1.12.2 > > When we repalce `column_separator=%x01 ` and `line_delimiter=%x02`, have other error occurs : > > ```shell > Reason: actual column number is more than schema column number. actual number: 1900 sep: %, schema number: 19; . src line: [689%x018a7c7089faaf00c850b90154e74fb821%x01101787%x0111%x010%x010%x010%x01%x01%x01QARFEE > ``` > > We check barch-0.14 and master, look the commit message `[Load] Support multi bytes LineDelimiter and ColumnSeparator` (#5462) in `broker_scanner.cpp`, so we are going to verify it. @JNSimba Can this PR fix the above problems? please help to confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
