Chesnay Schepler created FLINK-35159:
Summary: CreatingExecutionGraph can leak CheckpointCoordinator and
cause JM crash
Key: FLINK-35159
URL: https://issues.apache.org/jira/browse/FLINK-35159
Chesnay Schepler created FLINK-34672:
Summary: HA deadlock between JobMasterServiceLeadershipRunner and
DefaultLeaderElectionService
Key: FLINK-34672
URL: https://issues.apache.org/jira/browse/FLINK-34672
Chesnay Schepler created FLINK-34640:
Summary: Replace DummyMetricGroup usage with
UnregisteredMetricsGroup
Key: FLINK-34640
URL: https://issues.apache.org/jira/browse/FLINK-34640
Project: Flink
Moving classes (== keep the same package) to a module higher up in the
dependency tree should not be a breaking change and can imo be done
anytime without any risk to users.
On 27/02/2024 17:01, Lincoln Lee wrote:
Hi Zakelly,
Thanks for letting us 1.19 RMs know about this!
This change has
+1
- pom contents
- source contents
- Website PR
On 19/02/2024 18:33, Etienne Chauchot wrote:
Hi everyone,
Please review and vote on the release candidate #2 for the version
1.1.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
Chesnay Schepler created FLINK-34499:
Summary: Configuration#toString should hide sensitive values
Key: FLINK-34499
URL: https://issues.apache.org/jira/browse/FLINK-34499
Project: Flink
Chesnay Schepler created FLINK-34498:
Summary: GSFileSystemFactory logs full Flink config
Key: FLINK-34498
URL: https://issues.apache.org/jira/browse/FLINK-34498
Project: Flink
Issue
Chesnay Schepler created FLINK-34496:
Summary: Classloading deadlock between ExecNodeMetadataUtil and
JsonSerdeUtil
Key: FLINK-34496
URL: https://issues.apache.org/jira/browse/FLINK-34496
Project
Chesnay Schepler created FLINK-34485:
Summary: Token delegation doesn't work with Presto S3 filesystem
Key: FLINK-34485
URL: https://issues.apache.org/jira/browse/FLINK-34485
Project: Flink
n Thu, Feb 1, 2024 at 11:19 PM Chesnay Schepler wrote:
- checked source/maven pom contents
Please file a ticket to exclude tools/release from the source release.
+1 (binding)
On 29/01/2024 15:59, Maximilian Michels wrote:
- Inspected the source for licenses and corresponding headers
-
Chesnay Schepler created FLINK-34431:
Summary: Move static BlobWriter methods to separate util
Key: FLINK-34431
URL: https://issues.apache.org/jira/browse/FLINK-34431
Project: Flink
Chesnay Schepler created FLINK-34422:
Summary: BatchTestBase doesn't actually use MiniClusterExtension
Key: FLINK-34422
URL: https://issues.apache.org/jira/browse/FLINK-34422
Project: Flink
Chesnay Schepler created FLINK-34421:
Summary: Skip post-compile checks in compile.sh if fast profile is
active
Key: FLINK-34421
URL: https://issues.apache.org/jira/browse/FLINK-34421
Project
From: Piotr Nowojski
Sent: Thursday, February 8, 2024 14:50
To: Zhanghao Chen
Cc: Chesnay Schepler ; dev@flink.apache.org
; Yu Chen
Subject: Re: [DISCUSS] FLIP 411: Chaining-agnostic Operator ID generation for
improved state compatibility on parallelism change
Hey
AFAIK
The FLIP is a bit weird to be honest. It only applies in cases where
users haven't set uids, but that goes against best-practices and as far
as I'm told SQL also sets UIDs everywhere.
I'm wondering if this is really worth the effort.
On 07/02/2024 10:23, Zhanghao Chen wrote:
After offline
Chesnay Schepler created FLINK-34397:
Summary: Resource wait timeout can't be disabled
Key: FLINK-34397
URL: https://issues.apache.org/jira/browse/FLINK-34397
Project: Flink
Issue Type
Guava CVEs don't apply because it's all about using it's createTempDir
method which we don't use.
Zookeeper CVE doesn't really apply because it's a server-side issue.
On 02/02/2024 09:42, Martijn Visser wrote:
To add to this: we can't upgrade to flink-shaded 18.0, since we've just
reverted
- checked source/maven pom contents
Please file a ticket to exclude tools/release from the source release.
+1 (binding)
On 29/01/2024 15:59, Maximilian Michels wrote:
- Inspected the source for licenses and corresponding headers
- Checksums and signature OK
+1 (binding)
On Tue, Jan 23, 2024
Chesnay Schepler created FLINK-34286:
Summary: Attach cluster config map labels at creation time
Key: FLINK-34286
URL: https://issues.apache.org/jira/browse/FLINK-34286
Project: Flink
Chesnay Schepler created FLINK-34097:
Summary: Remove unused JobMasterGateway#requestJobDetails
Key: FLINK-34097
URL: https://issues.apache.org/jira/browse/FLINK-34097
Project: Flink
Chesnay Schepler created FLINK-34004:
Summary: TestingCheckpointIDCounter can easily lead to NPEs
Key: FLINK-34004
URL: https://issues.apache.org/jira/browse/FLINK-34004
Project: Flink
We could limit the (first) trial run to branches.
PRs wouldn't be affected (avoiding a bunch of concerns about maybe
blocking PRs and misleading people into thinking that CI is green), we'd
have a better handle on how much capacity we are consuming, but
contributors would still get the new
Chesnay Schepler created FLINK-33352:
Summary: OpenAPI spec is lacking mappings for discriminator
properties
Key: FLINK-33352
URL: https://issues.apache.org/jira/browse/FLINK-33352
Project: Flink
IIRC releases MUST be built with JDK 8.
@Yun Tang how did you discover this?
There is supposed to be a check to enforce this, but weirdly enough it
isn't rejecting my JDK 11 installation :/
Ah, we didn't set up the enforcer plugin correctly; while we do set a
required JDK version by default
+1 (binding)
On 13/10/2023 04:12, Junrui Lee wrote:
Hi all,
Thank you to everyone for the feedback on FLIP-366[1]: Support standard
YAML for FLINK configuration in the discussion thread [2].
I would like to start a vote for it. The vote will be open for at least 72
hours (excluding weekends,
Kryo isn't required for this; newer versions do support records but we
want something like a PojoSerializer for records to be performant.
The core challenges are
a) detecting records during type extraction
b) ensuring parameters are passed to the constructor in the right order.
From what I
There is no "monolithic" flink-shaded dependency.
Connectors shouldn't depend on anything that Flink provides, but be
self-contained as Martijn pointed out.
Connectors shouldn't depend on flink-shaded.
The overhead and/or risks of doing/supporting that right now far
outweigh the benefits.
(
It is a unfortunate that we'll need a separate config file but the FLIP
does a good job justifying it.
Personally I'd just name it "config.yaml"; I never quite understood why
there was a flink prefix to begin with, and the current proposal
("flink-configuration.yaml") seems unnecessarily
We need the @Internal annotation for cases where a user-facing class has
methods entirely meant for internal uses.
It's not great that we have these cases to being with, but fixing that
is another story.
The semantics for non-annotated classes is already documented.
Chesnay Schepler created FLINK-32888:
Summary: File upload runs into EndOfDataDecoderException
Key: FLINK-32888
URL: https://issues.apache.org/jira/browse/FLINK-32888
Project: Flink
Chesnay Schepler created FLINK-32834:
Summary: Allow compile.sh to be used manually
Key: FLINK-32834
URL: https://issues.apache.org/jira/browse/FLINK-32834
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32745:
Summary: Add a flag to skip InputSelectable preValidate step
Key: FLINK-32745
URL: https://issues.apache.org/jira/browse/FLINK-32745
Project: Flink
Chesnay Schepler created FLINK-32681:
Summary:
RocksDBStateDownloaderTest.testMultiThreadCleanupOnFailure unstablie
Key: FLINK-32681
URL: https://issues.apache.org/jira/browse/FLINK-32681
Project
On 25/07/2023 04:09, Xintong Song wrote:
In this case, I wonder if it makes sense to bump the REST API version to
V2. That should allow us to gradually phase out the old APIs without the
above mentioned problems.
It's ultimately possible do do that; it just implies duplicating _a lot_
classes
,
Jing
On Mon, Jul 24, 2023 at 11:48 PM Chesnay Schepler
wrote:
The downside to null is that it again forces users to handle this case
themselves.
The reality is that there is no good default value.
Ideally we fix all cases where we return such values, such that the
fallback to 0 isn't even used
at 4:02 PM Matthias Pohl
wrote:
What was the reason you decided to go for 0 as the fallback value instead
of null? Wouldn't that be a more reasonable value for error cases?
On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler
wrote:
There are a number of cases where the REST API can return infinity
:
+1 (binding)
On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler
wrote:
Hello,
I'd like to start a vote on FLIP-340.
Discussion thread:
https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+en
Hello,
I'd like to start a vote on FLIP-340.
Discussion thread:
https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
FLIP:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
Regards,
Chesnay
Hello,
I'd like to start a vote on FLIP-336.
Discussion thread:
https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
FLIP:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
Regards,
Chesnay
Make the REST API more consistent and easier to work with from the UI.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425799
There are a number of cases where the REST API can return infinity or
NaN for certain double fields.
This is problematic because the JSON spec does not allow such values,
and tooling working against that spec may run into issues when
encountering such a value.
Specifically we've seen this
The jar handlers currently accept a variety of parameters both as query
parameters and via the request body.
While it is primarily a problem for sending program args as query
parameters, because it's a nightmare with escaping parameters, the
remaining parameters should follow suite for
To properly reflect the state of the rocksdb statebackend I propose to
move all classes in the state-backend-rocksdb module under the classes
to o.a.f.state.rocksdb package.
On 21/07/2023 11:45, Leonard Xu wrote:
In this way, the user will see the deprecated API firstly but they can not find
a candidate if we can not finish all tasks in one minor version .
i'm not convinced that this matters. There will be a whole bunch of APIs
deprecated in 1.18 (that will
/dashboards. I often
get the question how do I add a tag or label to my Flink metric, and it
would be more intuitive if labels and tags replaced group as a name, to be
more in align with popular metric systems.
Best,
Mason
On Thu, Jul 20, 2023 at 1:41 AM Chesnay Schepler wrote:
environment
of the Flink 2.0 release perse, but starting to
wonder if we'd get more bang for our buck if we started fresh with a v2
REST API vs. one-off cleanups of the current v1 API. @Chesnay Schepler
-- wdyt?
The v1 REST API seemed to grow naturally from its original use case of
supporting the Web UI iiuc
://lists.apache.org/thread/vmhzv8fcw2b33pqxp43486owrxbkd5x9
On Tue, Jul 18, 2023 at 1:10 PM Jing Ge wrote:
+1
On Tue, Jul 18, 2023 at 12:24 PM Xintong Song
wrote:
+1
Best,
Xintong
On Tue, Jul 18, 2023 at 5:02 PM Chesnay Schepler
wrote:
The FLIP number was changed to 342.
On 18/07
be `#addGroup(int)`
Best,
Xintong
On Tue, Jul 18, 2023 at 4:39 PM Chesnay Schepler wrote:
The MetricGroup interface contains methods to create groups and metrics
using an int as a name. The original intention was to allow pattern like
|group.addGroup("subtaskIndex").addGroup(0)| , but t
The FLIP number was changed to 342.
On 18/07/2023 10:56, Chesnay Schepler wrote:
MetricGroup#getAllVariables returns all variables associated with the
metric, for example:
| = abcde|
| = ||0|
The keys are surrounded by brackets for no particular reason.
In virtually every use-case
Something to note is that the UI is using this parameter, and would have
to be changed to the new one.
Since we want to avoid having to split arguments ourselves, this may
imply changes to the UI.
On 18/07/2023 10:18, Chesnay Schepler wrote:
We'll log a warn message when it is used
MetricGroup#getAllVariables returns all variables associated with the
metric, for example:
| = abcde|
| = ||0|
The keys are surrounded by brackets for no particular reason.
In virtually every use-case for this method the user is stripping the
brackets from keys, as done in:
* our datadog
On 18/07/2023 10:33, Wencong Liu wrote:
For FLINK-6912:
There are three implementations of RichFunction that actually use
the Configuration parameter in RichFunction#open:
1. ContinuousFileMonitoringFunction#open: It uses the configuration
to configure the FileInputFormat. [1]
2.
The MetricGroup interface contains methods to create groups and metrics
using an int as a name. The original intention was to allow pattern like
|group.addGroup("subtaskIndex").addGroup(0)| , but this didn't really
work out, with |addGroup(String, String)| serving this use case much
better.
The endpoint hasn't been working for years and was only kept to inform
users about it. Let's finally remove it.
https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
n, Jul 17, 2023 at 5:33 AM Xintong Song wrote:
+1
Best,
Xintong
On Thu, Jul 13, 2023 at 9:34 PM Chesnay Schepler
wrote:
Hello,
The request body for the jar run/plan REST endpoints accepts program
arguments as a string (programArgs) or a list of strings
(programArgsList). The latter wa
I don't understand this bit:/
"One minor Scala change is necessary, though: We need to touch the Scala
implementation of the Pattern class (in flink-cep). Pattern requires a
new method which needs to be implemented in the Scala Pattern class as
well to comply with
https://flink.apache.org/
[3] https://spark.apache.org/
[4] https://kafka.apache.org/
On 7/13/23, 6:25 AM, "Chesnay Schepler" mailto:ches...@apache.org>> wrote:
CAUTION: This email originated from outside of the organization. Do not click
links or open attachments unless you can confir
Hello,
The job cancellation REST endpoint has a terminationMode query
parameter, which in the past could be set to either CANCEL or STOP, but
nowadays the job stop endpoint has subsumed the STOP functionality.
Since then the cancel endpoint rejected requests that specified STOP.
I propose
Hello,
The request body for the jar run/plan REST endpoints accepts program
arguments as a string (programArgs) or a list of strings
(programArgsList). The latter was introduced as kept running into issues
with splitting the string into individual arguments./
/
We ideally force users to use
On 13/07/2023 08:07, Mohan, Deepthi wrote:
However, even these developers when explicitly asked in our conversations often
comment that the website could do with a redesign
Can you go into more detail as to their specific concerns? Are there
functional problems with the page, or is this just
Hello,
Several REST responses contain a timestamp field of the current time
There is no known use-case for said timestamp, it makes caching of
responses technically sketchy (since the response differs on each call)
and it complicates testing since the timestamp field can't be easily
The issue isn't avoiding 1.19.
The issue is that if things aren't deprecated in 1.18 then for every
breaking change we have to start discussing exemptions from the API
deprecation process, which stipulates that all Public APIs must be
deprecated for at least 2 minor releases before they can
Chesnay Schepler created FLINK-32571:
Summary: Prebuild HBase testing docker image
Key: FLINK-32571
URL: https://issues.apache.org/jira/browse/FLINK-32571
Project: Flink
Issue Type
public APIs can be
finalized
and
merged in Flink 2.0 (I will fix the wiki accordingly).
I also agree with Jark that 2.0 is a good chance to rework the
default
value of configurations.
Best
Yuan
On Thu, Jun 29, 2023 at 8:43 PM Chesnay Schepler <
ches...@apache.org>
wrote:
Some
Has there ever been thoughts about changing flink-pythons connector
setup to use the table api connectors underneath?
The wrapping of connectors is a bit of a maintenance nightmare and
doesn't really work with external/custom connectors.
On 04/07/2023 13:35, Dian Fu wrote:
Thanks Ran Tao
Chesnay Schepler created FLINK-32544:
Summary: PythonFunctionFactoryTest fails on Java 17
Key: FLINK-32544
URL: https://issues.apache.org/jira/browse/FLINK-32544
Project: Flink
Issue
There's a whole bunch of metric APIs that would need to be deprecated.
That is of course if the metric FLIPs are being accepted.
Which makes me wonder if we aren't doing things the wrong way around;
shouldn't the decision to deprecate an API be part of the FLIP discussion?
On 05/07/2023
Chesnay Schepler created FLINK-32536:
Summary: Python tests fail with Arrow DirectBuffer exception
Key: FLINK-32536
URL: https://issues.apache.org/jira/browse/FLINK-32536
Project: Flink
o stabilize when a rescale happens".
> You can't express this with only one of the options.
>
> On 20/06/2023 14:41, Etienne Chauchot wrote:
>> Hi Chesnay,
>>
>> Thanks for your feedback. Comments inline
>
-1 (binding)
I feel like this FLIP needs a bit more time in the oven.
It seems to be very light on actual details; you can summarize the
entire changes section as "The enumerator calls this method and then
another checkpoint interval is used."
I would love to know how this is wired into the
Hello,
when using our docker image users can enable certain plugins via an
environment variable, listing all plugins they'd like to enable.
This is a list of plugin jar names, which contain the full Flink
version, like 1.16.1.
This implies that users always need to be aware of the patch
t.html?dev@flink.apache.org
On Mon, Jun 26, 2023 at 6:00 PM Chesnay Schepler <
ches...@apache.org
wrote:
by-and-large I'm quite happy with the list of items.
I'm curious as to why the "Disaggregated State Management" item
is
marked
as a must-have; will it require changes th
Chesnay Schepler created FLINK-32482:
Summary: Add Java 17 to Docker build matrix
Key: FLINK-32482
URL: https://issues.apache.org/jira/browse/FLINK-32482
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-32479:
Summary: Tests revoke leadership too early
Key: FLINK-32479
URL: https://issues.apache.org/jira/browse/FLINK-32479
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32467:
Summary: Move CleanupOnCloseRpcSystem to rpc-core
Key: FLINK-32467
URL: https://issues.apache.org/jira/browse/FLINK-32467
Project: Flink
Issue Type
or your feedback. Comments inline
Le 16/06/2023 à 17:24, Chesnay Schepler a écrit :
1) Options specific to the adaptive scheduler should start with
"jobmanager.adaptive-scheduler".
ok
2)
There isn't /really /a notion of a "scaling event". The scheduler is
informed a
by-and-large I'm quite happy with the list of items.
I'm curious as to why the "Disaggregated State Management" item is
marked as a must-have; will it require changes that break something?
What prevents it from being added in 2.1?
We may want to update the Java 17 item to "Make Java 17 the
Chesnay Schepler created FLINK-32380:
Summary: Serialization of Java records fails
Key: FLINK-32380
URL: https://issues.apache.org/jira/browse/FLINK-32380
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-32379:
Summary: Skip archunit tests in java1X-target profiles
Key: FLINK-32379
URL: https://issues.apache.org/jira/browse/FLINK-32379
Project: Flink
Issue
Chesnay Schepler created FLINK-32378:
Summary: 2.0 Breaking Metric system changes
Key: FLINK-32378
URL: https://issues.apache.org/jira/browse/FLINK-32378
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32377:
Summary: 2.0 Breaking REST API changes
Key: FLINK-32377
URL: https://issues.apache.org/jira/browse/FLINK-32377
Project: Flink
Issue Type: Technical
The master branch now builds and runs with Java 17 out-of-the-box.
Notes:
- a nightly cron build was set up, but I expect it to fail because some
tests only run in the branches and thus weren't covered by my testing.
I'll try to stabilize things over the weekend.
- The Python Kafka connector
1) Options specific to the adaptive scheduler should start with
"jobmanager.adaptive-scheduler".
2)
There isn't /really /a notion of a "scaling event". The scheduler is
informed about new/lost slots and job failures, and reacts accordingly
by maybe rescaling the job.
(sure, you can think of
Chesnay Schepler created FLINK-32370:
Summary: JDBC SQl gateway e2e test is unstable
Key: FLINK-32370
URL: https://issues.apache.org/jira/browse/FLINK-32370
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32369:
Summary: Setup cron build
Key: FLINK-32369
URL: https://issues.apache.org/jira/browse/FLINK-32369
Project: Flink
Issue Type: Sub-task
On 13/06/2023 17:26, Becket Qin wrote:
It would be valuable if we can avoid releasing minor versions for previous
major versions.
On paper, /absolutely /agree, but I'm not sure how viable that is in
practice.
On the current 2.0 agenda is potentially dropping support for Java 8/11,
which
Chesnay Schepler created FLINK-32359:
Summary: AdaptiveSchedulerBuilder shoudl accept executor service
in constructor
Key: FLINK-32359
URL: https://issues.apache.org/jira/browse/FLINK-32359
Chesnay Schepler created FLINK-32358:
Summary: CI may unintentionally use fallback akka loader
Key: FLINK-32358
URL: https://issues.apache.org/jira/browse/FLINK-32358
Project: Flink
Chesnay Schepler created FLINK-32338:
Summary: Add FailsOnJava17 annotation
Key: FLINK-32338
URL: https://issues.apache.org/jira/browse/FLINK-32338
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32336:
Summary: PartitionITCase#ComparablePojo should be public
Key: FLINK-32336
URL: https://issues.apache.org/jira/browse/FLINK-32336
Project: Flink
Chesnay Schepler created FLINK-32330:
Summary: Setup Java 17 in e2e builds
Key: FLINK-32330
URL: https://issues.apache.org/jira/browse/FLINK-32330
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32329:
Summary: Do not overwrite env.java.opts.all in HA e2e test
Key: FLINK-32329
URL: https://issues.apache.org/jira/browse/FLINK-32329
Project: Flink
Chesnay Schepler created FLINK-32328:
Summary: Ensure surefire baseLine is picked up by IntelliJ
Key: FLINK-32328
URL: https://issues.apache.org/jira/browse/FLINK-32328
Project: Flink
Chesnay Schepler created FLINK-32327:
Summary: Python Kafka connector runs into strange
NullPointerException
Key: FLINK-32327
URL: https://issues.apache.org/jira/browse/FLINK-32327
Project: Flink
On 13/06/2023 12:50, Jing Ge wrote:
One major issue we have, afaiu, is caused by the lack of housekeeping/house
cleaning, there are many APIs that were marked as deprecated a few years
ago and still don't get removed. Some APIs should be easy to remove and
others will need some more clear rules,
Chesnay Schepler created FLINK-32314:
Summary: Ignore class-loading errors after RPC system shutdown
Key: FLINK-32314
URL: https://issues.apache.org/jira/browse/FLINK-32314
Project: Flink
Chesnay Schepler created FLINK-32304:
Summary: Reduce rpc-akka jar
Key: FLINK-32304
URL: https://issues.apache.org/jira/browse/FLINK-32304
Project: Flink
Issue Type: Improvement
Chesnay Schepler created FLINK-32302:
Summary: Disable Hbase 2.x tests on Java 17
Key: FLINK-32302
URL: https://issues.apache.org/jira/browse/FLINK-32302
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-32301:
Summary: common.sh#create_ha_config should use set_config_key
Key: FLINK-32301
URL: https://issues.apache.org/jira/browse/FLINK-32301
Project: Flink
Chesnay Schepler created FLINK-32297:
Summary: Use Temurin image in FlinkImageBuilder
Key: FLINK-32297
URL: https://issues.apache.org/jira/browse/FLINK-32297
Project: Flink
Issue Type
1 - 100 of 3156 matches
Mail list logo
