I'm reading the Pulsar PIP and noticed another thing to take into account:
multiple applications (with each a different parallelism) that all write
into the same topic.
On Mon, 20 Dec 2021, 10:45 Niels Basjes, wrote:
> Hi Till,
>
> This morning I also realized what you call an 'effective
Hi all,
Really like the discussion on this topic moving forward. I really think
this feature will be much appreciated by the Flink users. What I still have
left to answer/reply to:
-- Good point. If for whatever reason the different taskmanagers can't get
the latest rule, the Operator
Timo Walther created FLINK-25387:
Summary: Introduce ExecNodeMetadata annotation and tooling
Key: FLINK-25387
URL: https://issues.apache.org/jira/browse/FLINK-25387
Project: Flink
Issue
Piotr Nowojski created FLINK-25382:
--
Summary: Failure in "Upload Logs" task
Key: FLINK-25382
URL: https://issues.apache.org/jira/browse/FLINK-25382
Project: Flink
Issue Type: Bug
tanjialiang created FLINK-25384:
---
Summary: Job can not migrate from flink-1.12 to flink-1.13
Key: FLINK-25384
URL: https://issues.apache.org/jira/browse/FLINK-25384
Project: Flink
Issue Type:
Yun Gao created FLINK-25383:
---
Summary: Skip waiting for the final checkpoint if the task do not
contains 2-pc operators
Key: FLINK-25383
URL: https://issues.apache.org/jira/browse/FLINK-25383
Project:
Timo Walther created FLINK-25386:
Summary: Harden table serialization in JSON plan
Key: FLINK-25386
URL: https://issues.apache.org/jira/browse/FLINK-25386
Project: Flink
Issue Type: Sub-task
Hi devs,
I would like to start a discussion about a previously announced follow up
of the FLIP-193 [1], namely allowing savepoints to be in native format and
incremental. The changes do not seem invasive. The full proposal is
written down as FLIP-203: Incremental savepoints [2]. Please take a
Konstantin Knauf created FLINK-25393:
Summary: Make ConfigMap Name for Leader Election Configurable
Key: FLINK-25393
URL: https://issues.apache.org/jira/browse/FLINK-25393
Project: Flink
Hi Till,
This morning I also realized what you call an 'effective watermark' is
indeed what is needed.
I'm going to read up on what Pulsar has planned.
What I realized is that the consuming application must be aware of the
parallelism of the producing application, which is independent of the
Hi everyone,
The voting time for FLIP-190: Support Version Upgrades for Table API &
SQL Programs[1] has passed. I'm closing the vote now.
There were 6 +1 votes, 3 of which are binding:
- Jing Zhang (binding)
- Ingo Bürk (binding)
- Godfrey He (binding)
- Martijn Visser (non-binding)
-
Timo Walther created FLINK-25391:
Summary: Updating existing table factories for mutable table
options
Key: FLINK-25391
URL: https://issues.apache.org/jira/browse/FLINK-25391
Project: Flink
Timo Walther created FLINK-25385:
Summary: Harden function serialization in JSON plan
Key: FLINK-25385
URL: https://issues.apache.org/jira/browse/FLINK-25385
Project: Flink
Issue Type:
Timo Walther created FLINK-25389:
Summary: Update JSON plan with new ExecNodeMetadata
Key: FLINK-25389
URL: https://issues.apache.org/jira/browse/FLINK-25389
Project: Flink
Issue Type:
Timo Walther created FLINK-25388:
Summary: Add annotation to all StreamExec nodes
Key: FLINK-25388
URL: https://issues.apache.org/jira/browse/FLINK-25388
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-25390:
Summary: Support mutable options in DynamicTableFactory
Key: FLINK-25390
URL: https://issues.apache.org/jira/browse/FLINK-25390
Project: Flink
Issue Type:
Timo Walther created FLINK-25392:
Summary: Add new STATEMENT SET syntax
Key: FLINK-25392
URL: https://issues.apache.org/jira/browse/FLINK-25392
Project: Flink
Issue Type: Sub-task
Hi Niels,
if you have multiple inputs going into a single Kafka partition then you
have to calculate the effective watermark by looking at the min watermark
from all inputs. You could insert a Flink operator that takes care of it
and then writes to a set of partitions in 1:n relationship.
Hi Nicholas, Hi Junfeng,
about the concerns around consistency raised by Martijn: I think a lot of
those can be mitigated by using an event time timestamp from which the
rules take effect. The reprocessing scenario, for example, is covered by
this. If a pattern processor should become active as
Till Rohrmann created FLINK-25381:
-
Summary: JdbcExactlyOnceSinkE2eTest.testInsert fails on AZP
Key: FLINK-25381
URL: https://issues.apache.org/jira/browse/FLINK-25381
Project: Flink
Issue
I just wanted to check in on this, is there any interest in a PR to fix
this issue? Looking at the master branch, it still looks to be a problem.
Thanks.
On Mon, Dec 6, 2021 at 9:52 AM Galen Warren (Jira) wrote:
> Galen Warren created FLINK-25197:
>
>
>
Hi everyone,
Please review and vote on the release candidate #2 for the version 3.1.1 of
Apache Flink Stateful Functions, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
This release updates the Flink version to fix the log4j CVEs
Hi Konstantin, Martijn
Thanks for the detailed feedback in the discussion. What I still have left to
answer/reply to:
-- Martijn: Just to be sure, this indeed would mean that if for whatever reason
the heartbeat timeout, it would crash the job, right?
IMO, if for whatever reason the heartbeat
Junfan Zhang created FLINK-25398:
Summary: Show complete stacktrace when requesting thread dump
Key: FLINK-25398
URL: https://issues.apache.org/jira/browse/FLINK-25398
Project: Flink
Issue
+1 (binding)
- Checked hash and signatures
- Checked diff contains Flink upgrade
- mvn clean install with e2e
Thanks,
Gordon
On Mon, Dec 20, 2021, 13:55 Seth Wiesman wrote:
> +1 (non-binding)
>
> - Verified signatures
> - Checked diff
> - Checked site PR
> - Build from source and ran e2e
HunterHunter created FLINK-25396:
Summary: lookupjoin source table for pre-partitioning
Key: FLINK-25396
URL: https://issues.apache.org/jira/browse/FLINK-25396
Project: Flink
Issue Type:
ZhuoYu Chen created FLINK-25397:
---
Summary: grouped_execution
Key: FLINK-25397
URL: https://issues.apache.org/jira/browse/FLINK-25397
Project: Flink
Issue Type: Improvement
Hi Galen,
Sorry for the late reply, a lot of people are on sporadic schedules with
the holidays. A PR would be very welcome! I've gone ahead and assigned you
the Jira. And I'll watch the repo to review your pr.
cheers,
Seth
On Mon, Dec 20, 2021 at 8:24 AM Galen Warren
wrote:
> I just wanted
Hi Piotr,
Thanks a lot for starting the discussion. Big +1.
In my understanding, this FLIP introduces the snapshot format as a *really*
user facing concept. IMO it is important that we document
a) that it is not longer the checkpoint/savepoint characteristics that
determines the kind of changes
I think this problem should be tackled inside Kafka, not Flink.
Kafka already has internal control messages to write transaction
markers. Those could be extended to carry watermark information. It
would be best to generalize those as "user control messages" and
watermarks could just be one
Thanks for getting back to me and Happy Holidays! I'll open a PR soon, it's
a simple fix.
On Mon, Dec 20, 2021 at 9:42 AM Seth Wiesman wrote:
> Hi Galen,
>
> Sorry for the late reply, a lot of people are on sporadic schedules with
> the holidays. A PR would be very welcome! I've gone ahead and
Abdelrahman created FLINK-25394:
---
Summary: [Flink-ML] Upgrade log4j to 2.17.0 to address
CVE-2021-45105
Key: FLINK-25394
URL: https://issues.apache.org/jira/browse/FLINK-25394
Project: Flink
Roman Khachatryan created FLINK-25395:
-
Summary: Incremental shared state might be discarded by TM
Key: FLINK-25395
URL: https://issues.apache.org/jira/browse/FLINK-25395
Project: Flink
+1 (non-binding)
- Verified signatures
- Checked diff
- Checked site PR
- Build from source and ran e2e tests
Seth
On Mon, Dec 20, 2021 at 10:59 AM Igal Shilman wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #2 for the version 3.1.1 of
> Apache Flink Stateful
34 matches
Mail list logo
