wang-zhiang commented on issue #4776:
URL:
https://github.com/apache/incubator-seatunnel/issues/4776#issuecomment-1552858540
Thank you for your reply. I tried again according to your method, but the
result was still garbled. Is this related to the coding of my hive setting or
is it a problem in my cluster.Do you have any ideas? Thank you again
hive version:3.1.2
hadoop:3.1.3
spark:2.4.8
CREATE TABLE `characters_source`(
`id` string,
`name` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
TBLPROPERTIES (
'bucketing_version'='2',
'last_modified_by'='smartpath',
'last_modified_time'='1663656042',
'transient_lastDdlTime'='1663656042');
INSERT INTO TABLE characters_source
VALUES (1,'詹姆斯'),(2,'乔丹');
drop table characters_sink;
CREATE TABLE `characters_sink`(
`id` string,
`name` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
TBLPROPERTIES (
'bucketing_version'='2',
'last_modified_by'='smartpath',
'last_modified_time'='1663656042',
'transient_lastDdlTime'='1663656042');
Seatunnetl Hive conf:
env {
# You can set spark configuration here
execution.parallelism = 1
}
source {
Hive {
table_name = "default.characters_source"
metastore_uri = "thrift://hadoop104:9083"
}
}
sink {
Hive {
table_name = "default.characters_sink"
metastore_uri = "thrift://hadoop104:9083"
}
}
select * from characters_sink
------------------ 原始邮件 ------------------
发件人:
"apache/incubator-seatunnel"
***@***.***>;
发送时间: 2023年5月18日(星期四) 下午5:50
***@***.***>;
***@***.******@***.***>;
主题: Re: [apache/incubator-seatunnel] Synchronizing data to hive
contains garbled characters (Issue #4776)
drop table characters_source; CREATE TABLE `characters_source`( `id`
string, `name` string) ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION
'hdfs://localhost:9000/user/hive/warehouse/characters_source' TBLPROPERTIES (
'bucketing_version'='2', 'last_modified_by'='smartpath',
'last_modified_time'='1663656042', 'transient_lastDdlTime'='1663656042');
INSERT INTO TABLE characters_source VALUES (1,'詹姆斯'),(2,'乔丹'); drop table
characters_sink; CREATE TABLE `characters_sink`( `id` string, `name` string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' STORED AS
INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION
'hdfs://localhost:9000/user/hive/warehouse/characters_sink' TBLPROPERTIES (
'bucketing_versi
on'='2', 'last_modified_by'='smartpath', 'last_modified_time'='1663656042',
'transient_lastDdlTime'='1663656042');
Seatunnetl Hive conf:
env { # You can set spark configuration here execution.parallelism = 1
} source { Hive { table_name = "default.characters_source"
metastore_uri = "thrift://localhost:9083" } } sink { Hive { table_name =
"default.characters_sink" metastore_uri = "thrift://localhost:9083" } }
2023-05-18 17:47:15,088 INFO [de8e2b80-70b4-448a-9733-c49260f041d0 main]
exec.ListSinkOperator (Operator.java:logStats(1028)) -
RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_LIST_SINK_3:2, 1 詹姆斯 2
乔丹 Time taken: 0.331 seconds, Fetched: 2 row(s)
Seatunnel version: Current dev version
Hive:3.0.0
Hadoop:3.0.0
I have no garbled codes for the time being. Could you please recheck your
configuration
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
