Thanks for the updates, Rui.
It does seem challenging to ensure evenness in slot deployment unless
we introduce batch slot requests in SlotPool. However, one possibility
is to add a delay of around 50ms during the SlotPool's resource
requirement declaration to the ResourceManager, similar to the
Zhanghao Chen created FLINK-33205:
-
Summary: Replace Akka with Pekko in the description of
"pekko.ssl.enabled"
Key: FLINK-33205
URL: https://issues.apache.org/jira/browse/FLINK-33205
Project: Flink
Zhanghao Chen created FLINK-33204:
-
Summary: Add description for missing options in the all
jobmanager/taskmanager options section in document
Key: FLINK-33204
URL:
Hello Martijn,
I do not see the images on Dockerhub yet. Is there an alternative source we
can use in the meantime?
Thanks,
Frans
-Original Message-
From: Martijn Visser
Sent: Tuesday, September 26, 2023 5:34 AM
To: dev@flink.apache.org
Subject: Re: [ANNOUNCE] Apache Flink Stateful
Thanks for all your comments.
@Rui
Thanks for the reminder. The "env.java.opts.all" has already taken effect now.
Best,
Yangze Guo
On Sat, Oct 7, 2023 at 6:45 PM Benchao Li wrote:
>
> Thanks Yangze for preparing this FLIP, it's good to have this ability
> for gateway since we already have it
Thanks Yangze for preparing this FLIP, it's good to have this ability
for gateway since we already have it for other JVM processes
(client/JM/TM) as Rui mentioned.
Rui Fan <1996fan...@gmail.com> 于2023年10月7日周六 18:02写道：
>
> Thanks to Yangze driving this proposal.
>
> `env.java.opts.xxx` is already
Thanks to Yangze driving this proposal.
`env.java.opts.xxx` is already supported for client, historyserver,
jobmanager and taskmanager. And it's very useful for troubleshooting.
So +1 for `env.java.opts.sql-gateway`.
I have a minor question: doesn't the `env.java.opts.all` support
sql-gateway?
It's quite intuitive to provide such a configuration for sql gateway.
Thanks Yangze for bringing this up and looking forward to it.
Best,
Zakelly
On Sat, Oct 7, 2023 at 4:35 PM xiangyu feng wrote:
>
> Thanks for initiating this discussion. Within the development towards
> Streaming
Hi Yangze,
Thanks for your quick response!
Sorry, I re-read the 2.2.2 part[1] about the Waiting Mechanism, I found
it isn't clear. The root cause of introducing the waiting mechanism is
that the slot requests are sent from JobMaster to SlotPool is
one by one instead of one whole batch. I have
Thanks for initiating this discussion. Within the development towards
Streaming Warehousing, SQL Gateway will become more and more important.
Big +1 to specify Java Options separately for SQL Gateway.
Regards,
Xiangyu
Yangze Guo 于2023年10月7日周六 15:24写道：
> Hi, there,
>
> We'd like to start a
Thanks for the clarification, Rui.
I believe the root cause of this issue is that in the current
DefaultResourceAllocationStrategy, slot allocation begins before the
decision to PendingTaskManagers requesting is made. That can be fixed
within the strategy without introducing another waiting
Hi Yangze,
> 2. From my understanding, if user enable the
> cluster.evenly-spread-out-slots,
> LeastUtilizationResourceMatchingStrategy will be used to determine the
> slot distribution and the slot allocation in the three TM will be
> (taskmanager.numberOfTaskSlots=3):
> TM1: 3 slot
> TM2: 2
Hi Jing,
Sorry for the late reply! I agree with you that we do not expect users
to do anything with Flink and we won't "bother" them with those
exceptions. However, users can still catch the `Throwable` and perform
any necessary logging activities, similar to how they use Java
Collection
Hi Shammon,
IIUC, you want more flexibility in controlling the two-phase strategy,
right?
> I want this because we would like to add a new slot to TM strategy such
as SLOTS_NUM in the future for OLAP to improve the performance for olap
jobs, which will use TASKS strategy for task to slot. cc
Yangze Guo created FLINK-33203:
--
Summary: FLIP-374: Adding a separate configuration for specifying
Java Options of the SQL Gateway
Key: FLINK-33203
URL: https://issues.apache.org/jira/browse/FLINK-33203
Hi, there,
We'd like to start a discussion thread on "FLIP-374: Adding a separate
configuration for specifying Java Options of the SQL Gateway"[1],
where we propose adding a separate configuration option to specify the
Java options for the SQL Gateway. This would allow users to fine-tune
the
Hi Martijn,
Sorry for the late reply. I don't think it is feasible to always
enable object reuse. If I understand correctly, object reuse is
disabled by default to guarantee correctness because we cannot assume
that the custom operator/function is safe to enable object reuse.
The method proposed
Xuannan Su created FLINK-33202:
--
Summary: FLIP-327: Support switching from batch to stream mode to
improve throughput when processing backlog data
Key: FLINK-33202
URL:
18 matches
Mail list logo
