Questions about UPDATE_BEFORE row kind in ElasticSearch sink

2021-06-27 Thread Kai Fu
Hi team, We notice the UPDATE_BEFORE row kind is not dropped and treated as DELETE as in code

Re: [Question] Why doesn't Flink use Calcite adapter?

2021-06-27 Thread JING ZHANG
Hi guangyuan, The question is an interesting and broad topic. I try to give my opinion based on my limited knowledge. Flink introduces dynamic sources to read from an external system[1]. Flink connector modules are completely decoupled with Calcite. There are two benefits: (1) If users need to

退订

2021-06-27 Thread 高耀军
退订

[Flink SQL] Lookup join hbase problem

2021-06-27 Thread 纳兰清风
Hi, When I was using hbase table as my lookup table, I got this error: Caused by: java.lang.IllegalArgumentException: Currently, HBase table can only be lookup by single row key. at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138) My SQL is insert into

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread zhisheng
看下你引入的 jar 包是咋引入的,scope 设置的是 provided 吧? Wei JI10 季伟 于2021年6月28日周一 下午12:19写道: > 您好, > 版本都是1.12.3的,有其他方式能够定位到这个问题出在哪么? > > 在 2021/6/28 上午11:59,“Jingsong Li” 写入: > > 注意:此封邮件来自于公司外部,请注意信息安全! > Attention: This email comes from outside of the company, please pay > attention to the

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Wei JI10 季伟
您好, 版本都是1.12.3的,有其他方式能够定位到这个问题出在哪么? 在 2021/6/28 上午11:59,“Jingsong Li” 写入: 注意:此封邮件来自于公司外部,请注意信息安全! Attention: This email comes from outside of the company, please pay attention to the information security! Hi, 你的版本check下?集群和flink-parquet是同一个版本吗? BEST, Jingsong

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Jingsong Li
Hi, 你的版本check下?集群和flink-parquet是同一个版本吗? BEST, Jingsong On Mon, Jun 28, 2021 at 11:22 AM Wei JI10 季伟 wrote: > 您好, > 不是sql client,是在yarn环境上跑的,运行的jar包中引入了上面的依赖,然后就看到有这个报错。 > > -- Best, Jingsong Lee

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Wei JI10 季伟
您好, 不是sql client,是在yarn环境上跑的,运行的jar包中引入了上面的依赖,然后就看到有这个报错。

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread zhisheng
使用的是 sql client 测试的 sql 吗?如果是的话,记得在 flink lib 目录下添加 flink-sql-parquet jar 包,然后重启集群和 sql client Wei JI10 季伟 于2021年6月28日周一 上午9:35写道: > 您好, > 添加的parquet 依赖如下,不知道全不全 > > org.apache.flink > flink-parquet_${scala.binary.version} > ${flink.version} > > > org.apache.parquet >

Re: 回复:flink 1.12如何实现window topN功能

2021-06-27 Thread zhisheng
可以将 1.13 的这个功能打在 flink 1.12 上面,然后引用你们新打的依赖 casel.chen 于2021年6月23日周三 下午12:08写道: > -- source > CREATE TABLE tmall_item( > itemID VARCHAR, > itemType VARCHAR, > onSellTime TIMESTAMP(3), > price DOUBLE, > proctime AS PROCTIME(), > WATERMARK FOR onSellTime AS onSellTime - INTERVAL '5' SECOND > )

回复:flinksql写入hive问题

2021-06-27 Thread 杨光跃
写入hive在读取,我试了下是可以的。。。 第一步: CREATE CATALOG myhive WITH ( 'type' = 'hive', 'default-database' = 'default', 'hive-conf-dir' = '/home/admin/hive/conf' ); 第二部 USE CATALOG myhive; 第三步 select * from hive_table; 猜测可能的问题,我们本地部署设置的slot都是1,你可能是在跑着写入任务,没有资源跑读取任务?

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Wei JI10 季伟
您好, 添加的parquet 依赖如下,不知道全不全 org.apache.flink flink-parquet_${scala.binary.version} ${flink.version} org.apache.parquet parquet-avro 1.10.1 

Re: flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Zhiwen Sun
parquet 相关依赖增加了吗? Zhiwen Sun On Sun, Jun 27, 2021 at 3:57 PM Wei JI10 季伟 wrote: > Hi: >在使用flink sql connector的filesytem时,指定format为parquet。抛出异常信息 > Caused by: org.apache.flink.table.api.ValidationException: Could not find > any format factory for identifier 'parquet' in the classpath.

Re: [Question] Why doesn't Flink use Calcite adapter?

2021-06-27 Thread Israel Ekpo
Maybe this question was better addressed to the DEV list. On Fri, Jun 25, 2021 at 11:57 PM guangyuan wang wrote: > > > > I have read the design doc of the Flink planner recently. I've found the > Flink only uses Calcite as an SQL optimizer. It

Re: Yarn Application Crashed?

2021-06-27 Thread Thomas Wang
Just found some additional info. It looks like one of the EC2 instances got terminated at the time the crash happened and this job had 7 Task Managers running on that EC2 instance. Now I suspect it's possible that when Yarn tried to migrate the Task Managers, there were no idle containers as this

Yarn Application Crashed?

2021-06-27 Thread Thomas Wang
Hi, I recently experienced a job crash due to the underlying Yarn application failing for some reason. Here is the only error message I saw. It seems I can no longer see any of the Flink job logs. Application application_1623861596410_0010 failed 1 times (global limit =2; local limit is =1) due

flink sql报错: Could not find any format factory for identifier 'parquet' in the classpath

2021-06-27 Thread Wei JI10 季伟
Hi: 在使用flink sql connector的filesytem时,指定format为parquet。抛出异常信息 Caused by: org.apache.flink.table.api.ValidationException: Could not find any format factory for identifier 'parquet' in the classpath. at