退订

2024-03-31 文章 杨东树 
申请退订邮件通知，谢谢！

FlinkSQL connector jdbc 用户密码是否可以加密/隐藏

2024-03-10 文章 杨东树 
各位好，
   考虑到数据库用户、密码安全性问题，使用FlinkSQL connector 
jdbc时，请问如何对数据库的用户密码进行加密/隐藏呢。例如如下常用sql中的password:
CREATE TABLE wordcount_sink (
 word String,
 cnt BIGINT,
 primary key (word) not enforced
) WITH (
 'connector' = 'jdbc',
 'url' = 'jdbc:mysql://localhost:3306/flink',
 'username' = 'root',
 'password' = '123456',
 'table-name' = 'wordcount_sink'
);

flink1.14.5 sql-client无法查询hbase1.4.3数据

2023-08-05 文章 杨东树 
各位好，
   目前使用sql-client查询hbase数据时，无法查询成功，麻烦指导下，谢谢。
   复现方法：
1、hbase操作:
hbase(main):005:0> create 'flink_to_hbase','cf1'
0 row(s) in 2.2900 seconds
hbase(main):006:0> put 'flink_to_hbase', 'rk0001', 'cf1:username', 'zhangsan'
0 row(s) in 0.0510 seconds


2、flink操作:
./start-cluster.sh
./sql-client.sh
CREATE TABLE flink_to_hbase(
rowkey STRING,
cf1 ROW,
PRIMARY KEY (rowkey) NOT ENFORCED
)WITH(
'connector'='hbase-1.4',
'table-name'='flink_to_hbase',
'zookeeper.quorum'='192.168.21.128:2181',
'zookeeper.znode.parent'='/hbase'
);


3、flink 报错日志：
2023-08-05 21:00:35,081 INFO  org.apache.flink.table.client.cli.CliClient   
   [] - Command history file path: /root/.flink-sql-history
2023-08-05 21:00:52,011 INFO  org.apache.flink.configuration.Configuration  
   [] - Config uses fallback configuration key 'jobmanager.rpc.address' 
instead of key 'rest.address'
2023-08-05 21:00:52,026 INFO  
org.apache.flink.client.program.rest.RestClusterClient   [] - Submitting 
job 'collect' (0c147bc0da5a43a5a382f2ec20740b45).
2023-08-05 21:00:52,480 INFO  
org.apache.flink.client.program.rest.RestClusterClient   [] - Successfully 
submitted job 'collect' (0c147bc0da5a43a5a382f2ec20740b45) to 
'http://localhost:8081'.
2023-08-05 21:00:55,809 INFO  org.apache.flink.configuration.Configuration  
   [] - Config uses fallback configuration key 'jobmanager.rpc.address' 
instead of key 'rest.address'
2023-08-05 21:00:55,830 INFO  org.apache.flink.configuration.Configuration  
   [] - Config uses fallback configuration key 'jobmanager.rpc.address' 
instead of key 'rest.address'
2023-08-05 21:07:52,481 INFO  org.apache.flink.configuration.Configuration  
   [] - Config uses fallback configuration key 'jobmanager.rpc.address' 
instead of key 'rest.address'
2023-08-05 21:07:52,484 INFO  
org.apache.flink.client.program.rest.RestClusterClient   [] - Submitting 
job 'collect' (d29904103fa3c83e3089c09f093372c9).
2023-08-05 21:07:52,728 INFO  
org.apache.flink.client.program.rest.RestClusterClient   [] - Successfully 
submitted job 'collect' (d29904103fa3c83e3089c09f093372c9) to 
'http://localhost:8081'.
2023-08-05 21:07:55,972 WARN  org.apache.flink.table.client.cli.CliClient   
   [] - Could not execute SQL statement.
org.apache.flink.table.client.gateway.SqlExecutionException: Could not execute 
SQL statement.
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.executeOperation(LocalExecutor.java:211)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.executeQuery(LocalExecutor.java:231)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at org.apache.flink.table.client.cli.CliClient.callSelect(CliClient.java:532) 
~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.cli.CliClient.callOperation(CliClient.java:423) 
~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.cli.CliClient.lambda$executeStatement$1(CliClient.java:332)
 [flink-sql-client_2.11-1.14.5.jar:1.14.5]
at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_231]
at 
org.apache.flink.table.client.cli.CliClient.executeStatement(CliClient.java:325)
 [flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.cli.CliClient.executeInteractive(CliClient.java:297)
 [flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:221)
 [flink-sql-client_2.11-1.14.5.jar:1.14.5]
at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:151) 
[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at org.apache.flink.table.client.SqlClient.start(SqlClient.java:95) 
[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187) 
[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161) 
[flink-sql-client_2.11-1.14.5.jar:1.14.5]
Caused by: org.apache.flink.table.api.TableException: Failed to execute sql
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeQueryOperation(TableEnvironmentImpl.java:828)
 ~[flink-table_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:1274)
 ~[flink-table_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeOperation$3(LocalExecutor.java:209)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.context.ExecutionContext.wrapClassLoader(ExecutionContext.java:88)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.executeOperation(LocalExecutor.java:209)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
... 12 more
Caused by: org.apache.flink.util.FlinkException: Failed to execute job 
'collect'.
at

Re:Re: flink1.14.5 sql-client 运行在yarn-session模式提交任务报错

2023-07-15 文章 杨东树 
:1.14.5]
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:754)
 ~[flink-table_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeModifyOperations$4(LocalExecutor.java:222)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.context.ExecutionContext.wrapClassLoader(ExecutionContext.java:88)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]
at 
org.apache.flink.table.client.gateway.local.LocalExecutor.executeModifyOperations(LocalExecutor.java:222)
 ~[flink-sql-client_2.11-1.14.5.jar:1.14.5]


在 2023-07-10 10:53:33，"Shammon FY"  写道：
>Hi,
>
>邮件里的图片看不到
>
>Best,
>Shammon FY
>
>On Sun, Jul 9, 2023 at 7:30 PM 杨东树  wrote:
>
>> 各位好，
>>目前我在使用flink1.14.5版本的sql-client on
>> yarn-session模式时，发现无法正常执行sql任务，日志报如下错误，希望能得到指导，谢谢：
>>背景信息：
>>1、当flink配置execution.target:
>> yarn-per-job时，随后进入sql-client执行sql任务，可正常执行。
>>2、当flink配置execution.target: yarn-session，并启动flink
>> yarn-session集群，随后进入sql-client执行同样的sql任务，报上图中的错误。
>>

flink1.14.5 sql-client 运行在yarn-session模式提交任务报错

2023-07-09 文章 杨东树 
各位好，
   目前我在使用flink1.14.5版本的sql-client on 
yarn-session模式时，发现无法正常执行sql任务，日志报如下错误，希望能得到指导，谢谢：
   背景信息：
   1、当flink配置execution.target: yarn-per-job时，随后进入sql-client执行sql任务，可正常执行。
   2、当flink配置execution.target: yarn-session，并启动flink 
yarn-session集群，随后进入sql-client执行同样的sql任务，报上图中的错误。