flink clickhouse connector
使用阿里的clickhouse connector 报找不到factory 大佬们遇到吗 现在flnk sql写入clickhouse 除了使用阿里的 还有别的方式吗？ org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'clickhouse' that implements 'org.apache.flink.table.factories.DynamicTableSourceFactory' in the classpath. | | 阿华田 | | a15733178...@163.com | 签名由网易邮箱大师定制
Re: Re: flink clickhouse connector
有，但是贴做附件时因为超长没法发出去 : ezmlm-reject: fatal: Sorry, I don't accept messages larger than 100 bytes (#5.2.3) 发你私人邮箱是否方便 guoliubi...@foxmail.com From: magichuang Date: 2020-12-17 20:18 To: user-zh Subject: Re: Re: flink clickhouse connector 您是用java写的还是pyflink 啊？ 我是用pyflink写的程序，所以需要一个jar包，您那里有嘛，我本地是新安装的maven，在打包 但是一直在下载依赖好多。。 > -- 原始邮件 -- > 发 件 人："guoliubi...@foxmail.com" > 发送时间：2020-12-17 19:36:55 > 收 件 人：user-zh > 抄 送： > 主 题：Re: flink clickhouse connector > > 我这也是往clickhouse写数据，用官方的或是其他第三方的JDBC驱动（我用的https://github.com/blynkkk/clickhouse4j），然后用JdbcSink就能写入了，不需要另外写connector。 > > > > guoliubi...@foxmail.com > > From: magichuang > Date: 2020-12-17 18:41 > To: user-zh > Subject: flink clickhouse connector > hi 想问一下有小伙伴使用flink > 往clickhouse里面写数据嘛？我是使用的https://help.aliyun.com/document_detail/185696.html?spm=a2c4g.11186623.6.606.6222693bubxXzX > 这个flink-connector，但是运行报错了: > > Caused by: java.io.IOException: unable to establish connection to ClickHouse > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:79) > > at > org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) > > at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) > > at > org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:73) > > at > org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479) > > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528) > > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.io.IOException: table `default`.`traffic` is not a > Distributed table > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.establishShardConnections(ClickHouseShardOutputFormat.java:96) > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:76) > > ... 12 more > > > > > 但 traffic 这个表我在clickhouse里面创建了，flink版本是1.11 > > > > > 有小伙伴成功对接的嘛，可否分享一下connector呀 > > > > > > > > > > > > > > > > > > > > Best， > > MagicHuang > > > > > -- Best， MagicHuang
Re: Re: flink clickhouse connector
您是用java写的还是pyflink 啊？ 我是用pyflink写的程序，所以需要一个jar包，您那里有嘛，我本地是新安装的maven，在打包 但是一直在下载依赖好多。。 > -- 原始邮件 -- > 发 件 人："guoliubi...@foxmail.com" > 发送时间：2020-12-17 19:36:55 > 收 件 人：user-zh > 抄 送： > 主 题：Re: flink clickhouse connector > > 我这也是往clickhouse写数据，用官方的或是其他第三方的JDBC驱动（我用的https://github.com/blynkkk/clickhouse4j），然后用JdbcSink就能写入了，不需要另外写connector。 > > > > guoliubi...@foxmail.com > > From: magichuang > Date: 2020-12-17 18:41 > To: user-zh > Subject: flink clickhouse connector > hi 想问一下有小伙伴使用flink > 往clickhouse里面写数据嘛？我是使用的https://help.aliyun.com/document_detail/185696.html?spm=a2c4g.11186623.6.606.6222693bubxXzX > 这个flink-connector，但是运行报错了: > > Caused by: java.io.IOException: unable to establish connection to ClickHouse > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:79) > > at > org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) > > at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) > > at > org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:73) > > at > org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479) > > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528) > > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.io.IOException: table `default`.`traffic` is not a > Distributed table > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.establishShardConnections(ClickHouseShardOutputFormat.java:96) > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:76) > > ... 12 more > > > > > 但 traffic 这个表我在clickhouse里面创建了，flink版本是1.11 > > > > > 有小伙伴成功对接的嘛，可否分享一下connector呀 > > > > > > > > > > > > > > > > > > > > Best， > > MagicHuang > > > > > -- Best， MagicHuang
Re: flink clickhouse connector
我使用的是第三方的驱动，clickhouse-native-jdbc，通过JDBC的方式。 > 2020年12月17日 18:41，magichuang 写道： > > hi想问一下有小伙伴使用flink > 往clickhouse里面写数据嘛？我是使用的https://help.aliyun.com/document_detail/185696.html?spm=a2c4g.11186623.6.606.6222693bubxXzX > 这个flink-connector，但是运行报错了: > > Caused by: java.io.IOException: unable to establish connection to ClickHouse > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:79) > > at > org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) > > at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) > > at > org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:73) > > at > org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479) > > at > org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528) > > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.io.IOException: table `default`.`traffic` is not a > Distributed table > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.establishShardConnections(ClickHouseShardOutputFormat.java:96) > > at > com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:76) > > ... 12 more > > > > > 但 traffic 这个表我在clickhouse里面创建了，flink版本是1.11 > > > > > 有小伙伴成功对接的嘛，可否分享一下connector呀 > > > > > > > > > > > > > > > > > > > > Best， > > MagicHuang > > > > >
Re: flink clickhouse connector
我这也是往clickhouse写数据，用官方的或是其他第三方的JDBC驱动（我用的https://github.com/blynkkk/clickhouse4j），然后用JdbcSink就能写入了，不需要另外写connector。 guoliubi...@foxmail.com From: magichuang Date: 2020-12-17 18:41 To: user-zh Subject: flink clickhouse connector hi想问一下有小伙伴使用flink 往clickhouse里面写数据嘛？我是使用的https://help.aliyun.com/document_detail/185696.html?spm=a2c4g.11186623.6.606.6222693bubxXzX 这个flink-connector，但是运行报错了: Caused by: java.io.IOException: unable to establish connection to ClickHouse at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:79) at org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:73) at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: table `default`.`traffic` is not a Distributed table at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.establishShardConnections(ClickHouseShardOutputFormat.java:96) at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:76) ... 12 more 但 traffic 这个表我在clickhouse里面创建了，flink版本是1.11 有小伙伴成功对接的嘛，可否分享一下connector呀 Best， MagicHuang
flink clickhouse connector
hi想问一下有小伙伴使用flink 往clickhouse里面写数据嘛？我是使用的https://help.aliyun.com/document_detail/185696.html?spm=a2c4g.11186623.6.606.6222693bubxXzX 这个flink-connector，但是运行报错了: Caused by: java.io.IOException: unable to establish connection to ClickHouse at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:79) at org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) at org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:73) at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: table `default`.`traffic` is not a Distributed table at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.establishShardConnections(ClickHouseShardOutputFormat.java:96) at com.aliyun.flink.connector.clickhouse.table.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:76) ... 12 more 但 traffic 这个表我在clickhouse里面创建了，flink版本是1.11 有小伙伴成功对接的嘛，可否分享一下connector呀 Best， MagicHuang
