[
https://issues.apache.org/jira/browse/FLINK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438106#comment-17438106
]
Fabian Paul commented on FLINK-24533:
-------------------------------------
Thanks for opening this ticket! We should really investigate what is causing
this problem. Unfortunately, I do not have much experience with CDC and SQL
although I can assist on the HBase side.
[~licp] Can you try to isolate the problem to the Hbase sink or does this only
happen when running the exact job you show?
[~jark] can you also have a quick look if something looks odd w.r.t the quer?
> Flink SQL Upsert To Hbase Appear data loss
> ---------------------------------------------
>
> Key: FLINK-24533
> URL: https://issues.apache.org/jira/browse/FLINK-24533
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / HBase
> Affects Versions: 1.12.0
> Environment: Flink 1.12.0 on Yarn
> HBase1.4
> Reporter: licp
> Priority: Major
> Attachments: Check_Result.png, Specify_some_key.png
>
>
> Data flow direction is described below：
> Source：Mysql
> Sink：HBase
> Parallelism：1
> -------------------Code Example One Using Left Join -------------------
> -- {color:#FF0000}Mysql Source ,Total Records:4829{color}
> create table user(
> user_id string,
> user_name string,
> primary key(user_id) not enforced
> )with(
> 'connector' = 'mysql-cdc',
> 'hostname' = 'localhost',
> 'port' = '3308',
> 'username' = 'user_name',
> 'password' = '******',
> 'database-name' = 'database_name',
> 'table-name' = 'table_name',
> 'debezium.event.processing.failure.handling.mode' = 'warn',
> 'debezium.snapshot.locking.mode' = 'none'
> );
> create table user_profile(
> user_id string,
> age int,
> primary key(user_id) not enforced
> )with(
> 'connector' = 'mysql-cdc',
> 'hostname' = 'localhost',
> 'port' = '3308',
> 'username' = 'user_name',
> 'password' = '******',
> 'database-name' = 'database_name',
> 'table-name' = 'table_name',
> 'debezium.event.processing.failure.handling.mode' = 'warn',
> 'debezium.snapshot.locking.mode' = 'none'
> );
> {color:#FF0000}-- HBase sink ;Total Record:4826{color}
> create table real_dwd_user_info_to_hbase(
> rowkey string,
> f ROW(user_name string,age int)
> )with(
> 'connector' = 'hbase-1.4',
> 'table-name' = 'table_name',
> 'zookeeper.quorum' = 'zk',
> 'zookeeper.znode.parent' = '/hbase'
> );
> insert into real_dwd_user_info_to_hbase
> select
> u.user_id,
> row(u.user_name,up.age) as
> from user u
> left join user_profile up
> on u.user_id = up.user_id
> where u.user_id<1000000
> ;
> ----------Code Example Two Using Left Join And Specify some key---------
> insert into real_dwd_user_info_to_hbase
> select
> u.user_id,
> row(u.user_name,up.age) as
> from user u
> left join user_profile up
> on u.user_id = up.user_id
> where u.user_id=0
> -------------------------------------------------------------------------------
> I printed the same code logic results, I can find that the specified key has
> three results of " + i -d + i", but in HBase is still not find the key
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
