Carl-Zhou-CN commented on issue #5881:
URL: https://github.com/apache/seatunnel/issues/5881#issuecomment-1819091360
Thank you for your feedback. It seems to be missing the ability to
automatically detect the delimiter.
> > > > hi @codeDing18 Can you provide more information, such as ddl of hive
table
> > >
> > >
> > > thx for reply.I have solved the problem. Just need add
`delimiter="\\|"` to Hive.
> >
> >
> > Can you elaborate? Because we had a similar problem before
>
> the ddl of hive table is below:
>
> ```shell
> create table lineitem (
> l_orderkey int,
> l_partkey int,
> l_suppkey int,
> l_linenumber int,
> l_quantity double,
> l_extendedprice double,
> l_discount double,
> l_tax double,
> l_returnflag string,
> l_linestatus string,
> l_shipdate string,
> l_commitdate string,
> l_receiptdate string,
> l_shipinstruct string,
> l_shipmode string,
> l_comment string)
> row format delimited
> fields terminated by '|'
> stored as textfile;
> ```
>
> the delimiter of the table's data is '|'.So I need to specify the
delimiiter in the config. the seatunnel config is below:
>
> ```
> source {
>
> Hive {
> table_name = "tpch.linetem"
> metastore_uri = "thrift://xxx:9083"
> hdfs_site_path =
"/usr/local/apache-seatunnel-2.3.3/hadoop_config/hdfs-site.xml"
> hive_site_path =
"/usr/local/apache-seatunnel-2.3.3/hadoop_config/hive-site.xml"
> kerberos_principal = "hdfs/xx@TEST"
> kerberos_keytab_path = "/etc/security/keytabs/hdfs.keytab"
> delimiter="\\|"
> }
>
> sink {
>
> Doris {
> fenodes = "localhost:8031"
> username = root
> password = "xxx"
> table.identifier = "seatunnel.test2"
> sink.label-prefix = "hive-doris"
> sink.enable-delete = "true"
> doris.config {
> format="json"
> read_json_by_line="true"
> }
> }
> ```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
