Another possibility is the JM is killed externally, e.g. K8s may kill JM/TM
if it exceeds the resource limit.
Thanks,
Zhu Zhu
Zhu Zhu 于2019年8月12日周一 下午1:45写道：
> Hi Cam,
>
> Flink master should not die when getting disconnected with task managers.
> It may exit for cases below:
> 1. when the job
可以分离，客户端提交的时候，初始化是在客户端上完成的，JobGraph 提交到
JobManager 之后不需要配置文件了
-邮件原件-
发件人: user-zh-return-797-wxchunjhyy=163@flink.apache.org
代表
jinxiaolong_al...@163.com
发送时间: Saturday, August 10, 2019 12:33 AM
收件人: user-zh
主题: Flink命令提交任务时是否支持配置文件与任务jar包分离
各位社区大佬：
并不是没条消息会触发watermark，而是有一定时间间隔的，默认是200ms触发一次watermark。
当你的数据来的比较集中的时候，经常会发生最新的消息的时间戳已经过了window end，但是window还没fire的情况。
-邮件原件-
发件人: Ever <439674...@qq.com>
发送时间: Sunday, July 14, 2019 5:00 PM
收件人: user-zh
主题: 回复： flink 1.8.1 时间窗口无法关闭以及消息丢失的问题
第四条数据来的时间戳是: 03:17:55, 水印时间这时候应该是03:17:50,
Hi Cam,
Flink master should not die when getting disconnected with task managers.
It may exit for cases below:
1. when the job terminated(FINISHED/FAILED/CANCELED). If you job is
configured with no restart retry, a TM failure can cause the job to be
FAILED.
2. JM lost HA leadership, e.g. lost
Hi Cam,
This case is expected due to slot sharing.
A slot can be shared by one instance of different tasks. So the used slot
is count of your max parallelism of a task.
You can specify the shared group with slotSharingGroup(String
slotSharingGroup) on operators.
Thanks,
Zhu Zhu
Abhishek Jain
What you'se seeing is likely operator chaining. This is the default
behaviour of grouping sub tasks to avoid transer overhead (from one slot to
another). You can disable chaining if you need to. Please refer task and
operator chains
Hi,
I had the same exception recently. I want to confirm that if it is due to
transaction timeout,
then I will lose those data. Am I right? Can I make it fall back to at
least once semantic in
this situation?
Best,
Tony Wei
Piotr Nowojski 於 2018年3月21日 週三 下午10:28寫道：
> Hi,
>
> But that’s
Hello Flink expert,
I have a cluster with 10 Task Managers, configured with 6 task slot each,
and a pipeline that has 13 tasks/operators with parallelism of 5. But when
running the pipeline I observer that only 5 slots are being used, the
other 55 slots are available/free. It should use all of
Hello Flink experts,
We are running Flink under Kubernetes and see that Job Manager
die/restarted whenever Task Manager die/restarted or couldn't get connected
each other. Is there any specific configurations/parameters that we need to
turn on to stop this? Or this is expected?
Thanks,
Cam
你好，谢谢，图片显示确实有问题，不过没关系，图片不是很重要。
pengcheng...@bonc.com.cn
发件人： Xintong Song
发送时间： 2019-08-09 20:09
收件人： user-zh
主题： Re: 有一些TaskManager的slot不可用，尽管没有任务正在运行
Hi，
邮件中的图片显示不出来。Flink邮件列表的图片附件是有点问题的，如果是截图最好上传到其他地方然后把链接贴出来。
Thank you~
Xintong Song
On Fri, Aug 9, 2019 at 10:06 AM
Flink 编译的默认 Scala 版本是 2.11.x，你可以试着把 Scala 版本切换成 2.11.x 再编译一下。
Best,
tison.
苟刚 于2019年8月10日周六 下午11:08写道：
>
>
>
> Hi,All:
>
>
> 我再尝试编译flink 1.7的源码时，遇到如下错误，本人对scala不是很了解，不知道是不是版本问题引起，另外可以去掉sacla模块编译吗：
> 本机scala版本：2.13.0
> JDK 版本： 1.8.0_91
> [ERROR] Failed to execute goal
>
异常原因如上所说是 akka ask timeout 的问题，这个问题前两天有人在部署 k8s 的时候也遇过[1]
他的情况是配置资源过少导致 JM 未能及时响应。除了调整上述参数外也可看看是不是这个问题。
Best,
tison.
[1]
https://lists.apache.org/thread.html/84db9dca2e990dd0ebc30aa35390ac75a0e9c7cbfcdbc2029986d4d7@%3Cuser-zh.flink.apache.org%3E
Biao Liu 于2019年8月8日周四 下午8:00写道：
> 你好，
>
>
Dear community,
happy to share this week's community update. The first real release
candidate of Apache Flink 1.9.0 has been published and the developer
community has moved more towards the 1.10 development cycle with more
discussion threads popping up again.
As always, please feel free to add
你好：
感谢你的回答，使用配置中心托管配置是可以解决这个问题。
其实我提问的初衷是想确认下flink自身在应用或部署层面上有没有相关的能力或使用技巧解决这个问题，或者是不是后续的演进已经考虑这方面的问题，
可能我还不知道，求社区大佬们指点。
我在用的版本是1.7.2
jinxiaolong_al...@163.com
发件人： huanqinghappy
发送时间： 2019-08-11 00:48
收件人： user-zh
主题： 回复：Flink命令提交任务时是否支持配置文件与任务jar包分离
你好：
是不是可以直接使用配置中心
Hello,
We are using Apache Flink 1.7.2 version. During our security scans following
issues are reported by our scan tool. Please let us know your comments on these
issues.
[1] 150085 Slow HTTP POST vulnerability
Severity Potential Vulnerability - Level 3
Group Information Disclosure
Threat
Hi Chad,
We have (Blink) jobs each running with over 10 thousands of TMs.
In our experience, the main regression caused by large scale TMs is the in
TM allocation stage in ResourceManager, that some times it fails to
allocate enough TMs before the allocation timeout.
It does not deteriorate much
Hi Chad,
In our cases, 1~2k TMs with up to ~10k TM slots are used in one job. In
general, the CPU/memory of Job Manager should be increased with more TMs.
Regards,
Qi
> On Aug 11, 2019, at 2:03 AM, Chad Dombrova wrote:
>
> Hi,
> I'm still on my task management investigation, and I'm curious
17 matches
Mail list logo
