Thank You Xintong, I will look for these updates in the near future.
Regards,
Vijay
On Wed, May 26, 2021 at 6:40 PM Xintong Song wrote:
> Hi Vijay,
>
> Currently, Flink only supports shipping files from the local machine where
> job is submitted.
>
> There are tickets [1][2][3] tracking the
Hi,
I forgot to mention that we are using Flink 1.12.0. This is a job that has
only minimum components. Reading from source and printing it.
Profiling was my next step to do. Regarding memory I didn't see any
bottlenecks.
I guess I will have to do some investigating in the metric part of Flink.
Hi,
I forgot to mention that we are running Flink 1.12.0.
This is the main function (some parts of codes are abbreviated and this is
the main part). As you can see the job was simplified to minimum. Just
reading from source and printing.
[image: Screenshot 2021-05-26 at 08.05.53.png]
And this
Hi,
Imagine I have one class having 4 fields: ID, A, B, C. There are three
data sources providing data in the form of (ID, A), (ID, B), (ID, C)
respectively. I want to join these three data sources to get final (ID, A,
B, C) without any window. For example, (ID, A) could come one month after
Hi，
集群重启，具体是指什么重启呢，这个能在描述详细一点吗？
Best,
LakeShen
datayangl 于2021年5月26日周三 上午9:43写道：
> FixedDelaStrategy 默认是从最近一个ck
> 恢复，其他的策略可以看官网。如果你是想问怎么实现的，不建议在邮件列表里问实现原理的问题。可以google找相关文章、相关flip 或者
> 直接debug源码。
>
>
>
> --
> Sent from: http://apache-flink.147419.n8.nabble.com/
看下你的 flink 命令对不对，然后去 Flink Web UI Checkpoint 界面，看下是否从 Savepoint 恢复（下面有个
restore path）.
之后再看下你的窗口时间类型用的是什么。
Best,
LakeShen
王春浩 于2021年5月27日周四 上午9:26写道：
> hi, 社区
>
> 版本flink 1.7
>
>
> 我正在尝试从保存点（或检查点）还原flink作业，该作业的工作是从kafka读取->执行30分钟的窗口聚合（只是AggregationFunction，就像一个计数器）->下沉到kafka。
>
>
Hi igyu:
You can submit job use these arguements like this
```
-m yarn-cluster \
-yqu root.realtime \
-ynm “test" \
-yjm 2g \
-ytm 2g \
-n \
-d \
-ys 1 \
-yD security.kerberos.login.principal=xxx...@x.com \
-yD security.kerberos.login.keytab=/tmp/xx.keytab \
...
```
> On May 27, 2021,
Hi Vijay,
Currently, Flink only supports shipping files from the local machine where
job is submitted.
There are tickets [1][2][3] tracking the efforts that shipping files from
remote paths, e.g., http, hdfs, etc. Once the efforts are done, adding s3
as an additional supported schema should be
关于兰州哪里有开机械设备发票〖⒈⒌⒍一徴一⒎⒎⒍⒈一電一⒍⒍⒊⒈〗罗生-100%-项目齐全【餐饮】〖住宿〗「建筑」｛手撕｝《定额》〈运输〉〔材料〕＜钢材＞〔机械〕『咨询』【广告】｛服务｝【租赁】《设计》【培训】『劳务本书是为教师写的“教育常识”。本书所讲的教育常识，首先是我自己眼中的教育常识，它绝非教育常识的全部。我期待通过对教育常识的追问，唤醒对常识的尊重和敬畏。
--
Sent from: http://apache-flink.147419.n8.nabble.com/
关于北京哪里有开化工原料发票〖⒈⒌⒍一徴一⒎⒎⒍⒈一電一⒍⒍⒊⒈〗罗生-100%-项目齐全【餐饮】〖住宿〗「建筑」｛手撕｝《定额》〈运输〉〔材料〕＜钢材＞〔机械〕『咨询』【广告】｛服务｝【租赁】《设计》【培训】『劳务本书是为教师写的“教育常识”。本书所讲的教育常识，首先是我自己眼中的教育常识，它绝非教育常识的全部。我期待通过对教育常识的追问，唤醒对常识的尊重和敬畏。
--
Sent from: http://apache-flink.147419.n8.nabble.com/
hi, 社区
版本flink 1.7
我正在尝试从保存点（或检查点）还原flink作业，该作业的工作是从kafka读取->执行30分钟的窗口聚合（只是AggregationFunction，就像一个计数器）->下沉到kafka。
我使用rocksdb和启用检查点。
现在我尝试手动触发一个保存点。 每个汇总的期望值是30（1个数据/每分钟）。 但是，当我从保存点还原时（flink运行-d -s
{savepoint的url}），聚合值不是30（小于30，取决于我取消flink作业并还原的时间）。 但是当作业正常运行时，它将达到30。
我不知道为什么有些数据似乎会丢失？
I use CDH 6.3.2
flink-1.12.3
I enabel kerberos
I want to use different user with different keytab,because I creat many queue
in YARN , different user use different queue.
igyu
Hi Yang,
Thanks for getting back to me.
By “restart master node”, I mean do “kubctl get nodes” to find the node’s role
as master and “ssh” into one of master nodes as ubuntu user. Then run “sudo
/sbin/reboot -f” to restart the master node.
It looks like The JobManager would cancel the running
Found https://issues.apache.org/jira/browse/FLINK-22766
From: Alexey Trenikhun
Sent: Tuesday, May 25, 2021 3:25 PM
To: Ardhani Narasimha ; 陳樺威
; Flink User Mail List
Subject: Re: KafkaSource metrics
Looks like when KafkaSource is used instead of
Hello,
We are facing an error restarting a job from a savepoint. We believe it is
because one of the common classes used across all of our jobs was changed
but there was no *serialVersionUID* assigned to the class. There error we
are facing is
java.lang.Exception: Exception while creating
Hi Pohl,
I tried to ship my property file. Example: *-yarn.ship-files
s3://applib/xx/xx/1.0-SNAPSHOT/application.properties \*
*Error:*
6:21:37.163 [main] ERROR org.apache.flink.client.cli.CliFrontend - Invalid
command line arguments.
org.apache.flink.client.cli.CliArgsException: Could not
On Wed, May 26, 2021 at 01:03:53PM +0200, Dawid Wysakowicz wrote:
> Hi,
>
> No there is no API in the operator to know which mode it works in. We
> aim to have separate operators for both modes if required. You can check
> e.g. how we do it in KeyedBroadcastStateTransformationTranslator[1].
Hi team,
I have two queries as mentioned below:
*Query1:*
I am using PrometheusReporter to expose metrics to Prometheus Server.
What should be the minimum recommended scrape interval to be defined on
Prometheus server?
Is there any interval in which Flink reports metrics?
*Query2:*
Is there any
张斌，你好：如需退订，请回复信息到 user-zh-unsubscr...@flink.apache.org
后，根据提示完成后续流程，即可退订。祝好。Best, flinker.
在 2021-05-26 17:05:59，"张斌" 写道：
>
>
>退订
>| |
>张斌
>|
>|
>herobin1...@163.com
>|
>签名由网易邮箱大师定制
>
Glad to hear it! Thanks for confirming that it works.
Piotrek
śr., 26 maj 2021 o 12:59 Barak Ben Nathan
napisał(a):
>
>
> Hi Piotrek,
>
>
>
> This is exactly what I was searching for. Thanks!
>
>
>
> Barak
>
>
>
> *From:* Piotr Nowojski
> *Sent:* Wednesday, May 26, 2021 9:59 AM
> *To:* Barak
Hi,
No there is no API in the operator to know which mode it works in. We
aim to have separate operators for both modes if required. You can check
e.g. how we do it in KeyedBroadcastStateTransformationTranslator[1].
Yes, it should be possible to register a timer for Long.MAX_WATERMARK if
you
/mnt/data is a local disk, so there shouldn’t be any additional latency. I’ll
provide more information when/if this happens again.
Peter
From: Roman Khachatryan
Date: Tuesday, May 25, 2021 at 6:54 PM
To: Peter Westermann
Cc: user@flink.apache.org
Subject: Re: Job recovery issues with state
Hi Dawid,
+1 (non-binding)
Thanks for driving this release. I checked the following things:
- downloaded and build source code
- verified checksums
- double-checked diff of pom files between 1.13.0 and 1.13.1-rc1
- did a visual check of the release blog post
- started cluster and ran jobs
退订
| |
张斌
|
|
herobin1...@163.com
|
签名由网易邮箱大师定制
退订
退订
退订
Hi，all：
我使用flink1.11.2进行作业开发，由于要兼容内部历史代码，需要把source手动注册成一张表，调用为：
> tableEnv.createTemporaryView(tableSource.getName, source, fields.toArray:
> _*)
>
其中，tableEnv为 StreamTableEnvironment类型，source是 DataStream[Row] 类型，代表source
connector生成的算子，fields是 由处理过的source table的 filed name 转换成的 Expression，将filed
Just FYI https://hub.docker.com/_/flink is updated now as well.
On Wed, May 26, 2021 at 9:57 AM Nikola Hrusov wrote:
> Hello Arvid,
>
> Thank you for your fast response
>
> Regards
> ,
> Nikola
>
>
> On Tue, May 25, 2021 at 7:11 PM Arvid Heise wrote:
>
>> Hi Nikola,
>>
>>
谢谢，好奇为什么要这么做，动态编译么？
Qishang 于2021年5月26日周三 下午1:57写道：
> Hi.
>
> 会生成 `${project.basedir}/target/generated-sources/`
>
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-datastream-allround-test/pom.xml#L97
>
> r pp 于2021年5月25日周二 上午9:58写道：
>
> > 各位好，请问下，
> >
> >
>
可以使用 State Processor [1]。
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/libs/state_processor_api/
Best,
Paul Lam
> 2021年5月26日 09:14，casel.chen 写道：
>
> 我有一个flink sql写的数据实时同步作业，从mysql binlog cdc消费发到mongodb，仅此而已，没有lookup，也没有join。
> 查看checkpoint页显示状态有17MB，checkpoint耗时要2s。
-- Forwarded message -
From: Priyanka Manickam
Date: Tue, 25 May 2021, 21:13
Subject: Fwd: Getting error in pod template
To: user , Yang Wang
-- Forwarded message -
From: Priyanka Manickam
Date: Tue, 25 May 2021, 21:11
Subject: Re: Getting error in pod
Hello Arvid,
Thank you for your fast response
Regards
,
Nikola
On Tue, May 25, 2021 at 7:11 PM Arvid Heise wrote:
> Hi Nikola,
>
> https://hub.docker.com/r/apache/flink now contains the images. It takes a
> few days until https://hub.docker.com/_/flink is updated though.
>
> Sorry for the
Hi,
Maybe before deleting the pods, you could look inside them and inspect your
job's jar? What classes does it have inside it? The job's jar should be in
a local directory. Or maybe even first inspect the jar before submitting it?
Best, Piotrek
wt., 25 maj 2021 o 17:40 Georgi Stoyanov
Hi Barak,
Before starting the JobManager I don't think there is any API running at
all. If you want to be able to submit/stop multiple jobs to the same
cluster session mode is indeed the way to go. But first you need to to
start the cluster ( start-cluster.sh ) [1]
Piotrek
[1]
Hi Vijay,
have you tried yarn-ship-files [1] or yarn-ship-archives [2]? Maybe, that's
what you're looking for...
Best,
Matthias
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/#yarn-ship-files
[2]
退订
wujing...@shantaijk.cn
mysql-cdc connector只能设置一个并行度，主要可能有这些原因：
1. mysql binlog本质上是一个文件，多个并行度消费需要避免重复
2. 多个并行度消费难以保证顺序
sink可以设为多个并行度，但是顺序不一定，如果需要主键相同的记录发到同一个sink线程可以先做一个keyby，并且保证keyby并行度与sink并行度相同，这样基本上能够保证数据forward传输，不过也不能100%保证有序。
如果需要保证有序还是建议sink并行度为1
--
Sent from: http://apache-flink.147419.n8.nabble.com/
39 matches
Mail list logo
