Chesnay Schepler created FLINK-32297:
Summary: Use Temurin image in FlinkImageBuilder
Key: FLINK-32297
URL: https://issues.apache.org/jira/browse/FLINK-32297
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32295:
Summary: Try out Infra-provided Gradle Enterprise
Key: FLINK-32295
URL: https://issues.apache.org/jira/browse/FLINK-32295
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32291:
Summary: Hive E2E test fails consistently
Key: FLINK-32291
URL: https://issues.apache.org/jira/browse/FLINK-32291
Project: Flink
Issue Type
On 08/06/2023 16:06, Kurt Ostfeld wrote:
If I understand correctly, the scenario is resuming from multiple checkpoint
files or from a savepoint and checkpoint files which may be generated by
different versions of Flink
No; it's the same version of Flink, you just didn't do a full migration
Chesnay Schepler created FLINK-32290:
Summary: Enable -XX:+IgnoreUnrecognizedVMOptions
Key: FLINK-32290
URL: https://issues.apache.org/jira/browse/FLINK-32290
Project: Flink
Issue Type
If you were to fork it /and want to redistribute it/ then the short
version is that
1. you have to adhere to the Apache licensing requirements
2. you have to make it clear that your fork does not belong to the
Apache Flink project. (Trademarks and all that)
Neither should be significant hurd
I'm a bit concerned that the last 4 CI runs haven't succeeded in the 3.1
branch.
Has anyone looked into the failing oracle test (both 1.17/1.18)?
https://github.com/apache/flink-connector-jdbc/actions/runs/5058372107/jobs/9078398092
Why is a vote being opened when there's still a blocker ticket
The version in the state is the serializer version, and applies to the
entire state, independent of what it contains.
If you use Kryo2 for reading and Kryo5 for writing (which also implies
writing the new serializer version into state), then I'd assume that a
migration is an all-or-nothing kind
Chesnay Schepler created FLINK-32239:
Summary: Unify TestJvmProcess and TestProcessBuilder
Key: FLINK-32239
URL: https://issues.apache.org/jira/browse/FLINK-32239
Project: Flink
Issue
Chesnay Schepler created FLINK-32238:
Summary: Stable approach for installing libssl
Key: FLINK-32238
URL: https://issues.apache.org/jira/browse/FLINK-32238
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32236:
Summary: Ease YarnTestBase allowlist address regex
Key: FLINK-32236
URL: https://issues.apache.org/jira/browse/FLINK-32236
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32226:
Summary: RestClusterClient leaks jobgraph file if submission fails
Key: FLINK-32226
URL: https://issues.apache.org/jira/browse/FLINK-32226
Project: Flink
Chesnay Schepler created FLINK-32185:
Summary: Remove M2_HOME usages
Key: FLINK-32185
URL: https://issues.apache.org/jira/browse/FLINK-32185
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32184:
Summary: Use revision version property
Key: FLINK-32184
URL: https://issues.apache.org/jira/browse/FLINK-32184
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32183:
Summary: Use maven.multiModuleProjectDirectory property instead of
rootDir plugin
Key: FLINK-32183
URL: https://issues.apache.org/jira/browse/FLINK-32183
Chesnay Schepler created FLINK-32182:
Summary: Use original japicmp plugin
Key: FLINK-32182
URL: https://issues.apache.org/jira/browse/FLINK-32182
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32181:
Summary: Drop support for Maven 3.2.5
Key: FLINK-32181
URL: https://issues.apache.org/jira/browse/FLINK-32181
Project: Flink
Issue Type: Technical
Chesnay Schepler created FLINK-32179:
Summary: Handle more repo names for automatic dist discovery
Key: FLINK-32179
URL: https://issues.apache.org/jira/browse/FLINK-32179
Project: Flink
Chesnay Schepler created FLINK-32169:
Summary: Show allocated slots on TM page
Key: FLINK-32169
URL: https://issues.apache.org/jira/browse/FLINK-32169
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-32168:
Summary: Log required/available resources in RM
Key: FLINK-32168
URL: https://issues.apache.org/jira/browse/FLINK-32168
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32167:
Summary: Log dynamic slot creation on task manager
Key: FLINK-32167
URL: https://issues.apache.org/jira/browse/FLINK-32167
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32166:
Summary: Show unassigned/total TM resources in web ui
Key: FLINK-32166
URL: https://issues.apache.org/jira/browse/FLINK-32166
Project: Flink
Issue
Chesnay Schepler created FLINK-32165:
Summary: Improve observability ofd fine-grained resource management
Key: FLINK-32165
URL: https://issues.apache.org/jira/browse/FLINK-32165
Project: Flink
Chesnay Schepler created FLINK-32162:
Summary: Misleading log message due to missing null check
Key: FLINK-32162
URL: https://issues.apache.org/jira/browse/FLINK-32162
Project: Flink
Chesnay Schepler created FLINK-32154:
Summary: Setup checkstyle rule to forbid mockito/powermock
Key: FLINK-32154
URL: https://issues.apache.org/jira/browse/FLINK-32154
Project: Flink
Chesnay Schepler created FLINK-32153:
Summary: Limit powermock to flink-core/-runtime
Key: FLINK-32153
URL: https://issues.apache.org/jira/browse/FLINK-32153
Project: Flink
Issue Type
Chesnay Schepler created FLINK-32152:
Summary: Consolidate mocking library usage
Key: FLINK-32152
URL: https://issues.apache.org/jira/browse/FLINK-32152
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-32149:
Summary: Remove some unnecessary mocking usages
Key: FLINK-32149
URL: https://issues.apache.org/jira/browse/FLINK-32149
Project: Flink
Issue Type
What happened?
I have just merged the last commits to properly support Maven 3.3+ on
the Flink master branch.
mvnw and CI have been updated to use Maven 3.8.6.
What does this mean for me?
* You can now use Maven versions beyond 3.2.5 (duh).
o Most versions should work, but
Chesnay Schepler created FLINK-31972:
Summary: Remove powermock whitebox usages
Key: FLINK-31972
URL: https://issues.apache.org/jira/browse/FLINK-31972
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-31971:
Summary: Drop
HadoopRecoverableWriterOldHadoopWithNoTruncateSupportTest
Key: FLINK-31971
URL: https://issues.apache.org/jira/browse/FLINK-31971
Project
> --
>> *From:* Jing Ge via user
>> *Sent:* Monday, April 24, 2023 11:15 PM
>> *To:* Chesnay Schepler
>> *Cc:* Piotr Nowojski ; Alexis
Sarda-Espinosa <
>> sarda.espin...@gmail.com>; Martijn Visser
Best,
Lijie
Piotr Nowojski 于2023年4月25日周二 23:02写道：
Ok, thanks for the clarification.
Piotrek
wt., 25 kwi 2023 o 16:38 Chesnay Schepler
napisał(a):
The checkstyle rule would just ban certain imports.
We'd add exclusions for all existing usages as we did when
introducing
other rules.
S
> /Instead of defining compatibility guarantees as "this API won't
change in all 1.x/2.x series", what if we define it as "this API won't
change in the next 2/3 years"./
I can see some benefits to this approach (all APIs having a fixed
minimum lifetime) but it's just gonna be difficult to comm
Chesnay Schepler created FLINK-31940:
Summary: DataStreamCsvITCase#CityPojo should be public
Key: FLINK-31940
URL: https://issues.apache.org/jira/browse/FLINK-31940
Project: Flink
Issue
The checkstyle rule would just ban certain imports.
We'd add exclusions for all existing usages as we did when introducing
other rules.
So far we usually disabled checkstyle rules for a specific files.
On 25/04/2023 16:34, Piotr Nowojski wrote:
+1 to the idea.
How would this checkstyle rule w
This is definitely a good discussion so have.
Some thoughts:
One aspect that wasn't mentioned is what this release means going
forward. I already waited a decade for 2.0; don't really want to wait
another one to see Flink 3.0.
We should discuss how regularly we will ship major releases from no
Chesnay Schepler created FLINK-31934:
Summary: Remove mocking in RocksDB tests
Key: FLINK-31934
URL: https://issues.apache.org/jira/browse/FLINK-31934
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-31933:
Summary: Remove Whitebox usage in ExpressionKeysTest
Key: FLINK-31933
URL: https://issues.apache.org/jira/browse/FLINK-31933
Project: Flink
Issue
As it turns out Kryo isn't a blocker; we ran into a JDK bug.
On 31/03/2023 08:57, Chesnay Schepler wrote:
https://github.com/EsotericSoftware/kryo/wiki/Migration-to-v5#migration-guide
Kroy themselves state that v5 likely can't read v2 data.
However, both versions can be on the
Chesnay Schepler created FLINK-31919:
Summary: Skip ClosureCleaner if object can be serialized
Key: FLINK-31919
URL: https://issues.apache.org/jira/browse/FLINK-31919
Project: Flink
Chesnay Schepler created FLINK-31916:
Summary: Python API only respects deprecated env.java.opts key
Key: FLINK-31916
URL: https://issues.apache.org/jira/browse/FLINK-31916
Project: Flink
Chesnay Schepler created FLINK-31915:
Summary: Python API incorrectly passes env.java.opts as single
argument
Key: FLINK-31915
URL: https://issues.apache.org/jira/browse/FLINK-31915
Project
Chesnay Schepler created FLINK-31913:
Summary: sql-client.sh does not respect env.java.opts.all/client
Key: FLINK-31913
URL: https://issues.apache.org/jira/browse/FLINK-31913
Project: Flink
Chesnay Schepler created FLINK-31912:
Summary: Upgrade bytebuddy
Key: FLINK-31912
URL: https://issues.apache.org/jira/browse/FLINK-31912
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-31911:
Summary: Bad address construction in SqlClientTest
Key: FLINK-31911
URL: https://issues.apache.org/jira/browse/FLINK-31911
Project: Flink
Issue Type
> they require additional hop to serialize Scala objects
This doesn't necessarily mean that we need a Scala API, because a beefed
up type extraction could also solve this.
> This single committer is now with us and ready to maintain it in open
source. The best situation to be :-)
Have you c
Chesnay Schepler created FLINK-31807:
Summary: Test architecture tests don't cover all tests
Key: FLINK-31807
URL: https://issues.apache.org/jira/browse/FLINK-31807
Project: Flink
Chesnay Schepler created FLINK-31806:
Summary: Prod architecture tests didn't detect non-public API usage
Key: FLINK-31806
URL: https://issues.apache.org/jira/browse/FLINK-31806
Project:
Chesnay Schepler created FLINK-31805:
Summary: Cassandra Source shouldn't use IOUtils
Key: FLINK-31805
URL: https://issues.apache.org/jira/browse/FLINK-31805
Project: Flink
Issue
Chesnay Schepler created FLINK-31804:
Summary: ITCase MiniCluster test architecture rule should accept
MiniClusterTestEnvironment
Key: FLINK-31804
URL: https://issues.apache.org/jira/browse/FLINK-31804
Do we have to move Debezium _now_?
There is no hard dependency between debezium-json and the Kafka
connector, nor does the format depend on Kafka afaict.
So is this only about the e2e test that uses debezium-json + kafka
connector?
If so, then I would suggest to put debezium-json issue aside
Chesnay Schepler created FLINK-31744:
Summary: Extend Adaptive Scheduler sparse EG to contain
maxParallelism
Key: FLINK-31744
URL: https://issues.apache.org/jira/browse/FLINK-31744
Project: Flink
Chesnay Schepler created FLINK-31738:
Summary: FlameGraphTypeQueryParameter#Type clashes with
java.reflect.Type in generated clients
Key: FLINK-31738
URL: https://issues.apache.org/jira/browse/FLINK-31738
Chesnay Schepler created FLINK-31735:
Summary: JobDetailsInfo plan incorrectly documented as string
Key: FLINK-31735
URL: https://issues.apache.org/jira/browse/FLINK-31735
Project: Flink
Chesnay Schepler created FLINK-31733:
Summary: Model name clashes in OpenAPI spec
Key: FLINK-31733
URL: https://issues.apache.org/jira/browse/FLINK-31733
Project: Flink
Issue Type: Bug
Chesnay Schepler created FLINK-31728:
Summary: Remove Scala API dependencies from batch/streaming
examples
Key: FLINK-31728
URL: https://issues.apache.org/jira/browse/FLINK-31728
Project: Flink
Chesnay Schepler created FLINK-31711:
Summary: OpenAPI spec omits complete-statement request body
Key: FLINK-31711
URL: https://issues.apache.org/jira/browse/FLINK-31711
Project: Flink
ocker
image.
Could you help re-run the failed cases again?
Best,
Jiabao
2023年3月29日 下午10:08，Chesnay Schepler 写道：
+1 (binding), although I'd add a condition that someone has to look
into
recent test failures on Java 11:
https://github.com/apache/flink-connector-mongodb/actions/runs/
https://github.com/EsotericSoftware/kryo/wiki/Migration-to-v5#migration-guide
Kroy themselves state that v5 likely can't read v2 data.
However, both versions can be on the classpath without classpath as v5
offers a versioned artifact that includes the version in the package.
It probably would
Chesnay Schepler created FLINK-31672:
Summary: Requirement validation does nto take user-specified
maxParallelism into account
Key: FLINK-31672
URL: https://issues.apache.org/jira/browse/FLINK-31672
+1 (binding), although I'd add a condition that someone has to look into
recent test failures on Java 11:
https://github.com/apache/flink-connector-mongodb/actions/runs/4222375371/jobs/7330805980
https://github.com/apache/flink-connector-mongodb/actions/runs/4435527099/jobs/7782784066
* checked
Chesnay Schepler created FLINK-31657:
Summary: ConfigurationInfo generates incorrect openapi schema
Key: FLINK-31657
URL: https://issues.apache.org/jira/browse/FLINK-31657
Project: Flink
https://github.com/apache/flink-web#readme
On Mon, Feb 6, 2023 at 5:09 PM Chesnay Schepler
wrote:
Hello,
Just so more people are aware of it, I recently enabled a staging
environment for the Flink Website.
You can push the rebuilt website to the asf-staging branch in flink-web,
and the changes
Chesnay Schepler created FLINK-31608:
Summary: Re-evaluate 'min-parallelism-increase' option
Key: FLINK-31608
URL: https://issues.apache.org/jira/browse/FLINK-31608
Project: Flink
Chesnay Schepler created FLINK-31591:
Summary: Extend JobGraphWriter to persist requirements
Key: FLINK-31591
URL: https://issues.apache.org/jira/browse/FLINK-31591
Project: Flink
Issue
Chesnay Schepler created FLINK-31590:
Summary: Allow setting JobResourceRequirements through
JobMasterGateway
Key: FLINK-31590
URL: https://issues.apache.org/jira/browse/FLINK-31590
Project
The release of flink-connector-parent 1.0.0 is complete.
Update your connectors everyone. Let's get rid of the strange "zentol"
thing.
We've approved the release.
Votes:
+1 Etienne
+1 Danny (binding)
+1 Martijn (binding)
+1 Gordon (binding)
+1 Mason
On 15/03/2023 16:40, Chesnay Schepler wrote:
Hi everyone,
Please review and vote on the release candidate, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approv
Title correction: the version is 1.0.0.
On 15/03/2023 16:40, Chesnay Schepler wrote:
Hi everyone,
Please review and vote on the release candidate, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
This is the first release of the
Hi everyone,
Please review and vote on the release candidate, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
This is the first release of the flink-connector-parent pom by the Flink
project. This subsumes the previous release that I
Chesnay Schepler created FLINK-31475:
Summary: Allow project to be user-defined in release scripts
Key: FLINK-31475
URL: https://issues.apache.org/jira/browse/FLINK-31475
Project: Flink
Chesnay Schepler created FLINK-31455:
Summary: Add a simple test project for running CI workflow in
shared repo
Key: FLINK-31455
URL: https://issues.apache.org/jira/browse/FLINK-31455
Project
Chesnay Schepler created FLINK-31454:
Summary: Shared CI workflow always caches snapshot binaries
Key: FLINK-31454
URL: https://issues.apache.org/jira/browse/FLINK-31454
Project: Flink
SlotManagerTest that covers some
specific issues that should be ported to the fine grained slot manager.
Yes, we will check all the unit tests related to DeclarativeSlotManager
carefully
to make sure there are no tests lost while migrating from DSM to FGSM.
Best,
Weihua
On Mon, Mar 13, 202
One thing you could potentially look into is integrating the
TaskExecutorManager into the FIneGrainedSlotManager or find another way
to move the timeout handling into another component.
w.r.t. testing, there's the DeclarativeSlotManagerTest that covers some
specific issues that should be porte
/structure
and design. As a first step, I can work with an AWS UX designer and come with a
list of suggestions and wireframes. And while we aren't forming a committee
yet, if anyone wants to collaborate, please reach out.
On 2/21/23, 5:31 AM, "Chesnay Schepler" wrote:
Max is raising some good points.
Ultimately, you are free to propose changes; we don't categorically
reject changes to the website.
On the flip-side, the proposed changes should also be as granular as
possible, because there's no real reason why changing the content
structure should be coupl
FYI, the original mail never actually made it to the dev mailing list
(maybe got stuck in moderation queue).
On 08/02/2023 18:21, Mohsen Rezaei wrote:
Hi,
I was hoping to resurface this issue given that it affects any application
using the REST endpoints to run/submit jobs, causing confusion o
Chesnay Schepler created FLINK-30967:
Summary: Add MongoDB connector documentation
Key: FLINK-30967
URL: https://issues.apache.org/jira/browse/FLINK-30967
Project: Flink
Issue Type: Sub
Chesnay Schepler created FLINK-30963:
Summary: Switch binary downloads to archive.apache.org
Key: FLINK-30963
URL: https://issues.apache.org/jira/browse/FLINK-30963
Project: Flink
Issue
Hello,
Just so more people are aware of it, I recently enabled a staging
environment for the Flink Website.
You can push the rebuilt website to the asf-staging branch in flink-web,
and the changes will be visible at https://flink.staged.apache.org shortly.
This has been documented at
https
Chesnay Schepler created FLINK-30930:
Summary: Automatically determine Flink binary download URL from
version
Key: FLINK-30930
URL: https://issues.apache.org/jira/browse/FLINK-30930
Project
Chesnay Schepler created FLINK-30895:
Summary: SlotSharingSlotAllocator may waste slots
Key: FLINK-30895
URL: https://issues.apache.org/jira/browse/FLINK-30895
Project: Flink
Issue Type
sed.
From:
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/elastic_scaling/#limitations-1
This is a blocker from my side. Why do we have that restriction?
On Thu, Feb 2, 2023 at 5:03 PM Chesnay Schepler wrote:
> If I understand correctly, the adaptive scheduler currently does a
> If I understand correctly, the adaptive scheduler currently does a
full job restart. Is there any work planned to enable in-place rescaling
with the adaptive scheduler?
Nothing concrete. Sure, it's on a wishlist, but it'd require significant
changes to how the runtime works.
Rescaling statef
+1
On 02/02/2023 15:43, Jark Wu wrote:
+1
Best,
Jark
On Thu, 2 Feb 2023 at 21:31, Hang Ruan wrote:
Hi all,
Thanks for all the help about this FLIP. Now let's start the vote again.
Based on the discussion[1], we have come to a consensus, so I would like to
start a vote on FLIP-274: Introduc
ator").
I have updated the FLIP-274. @Chesnay Schepler @Jark
Wu , please help to take a look at the modified FLIP
and make sure that I do not miss something.
If there are no more problems, we plan to start the voting thread later.
Best,
Hang
Chesnay Schepler 于2023年1月31日周二 20:09写道：
II
ll have to decide.
On 31/01/2023 11:29, Teoh, Hong wrote:
Hi @Dong Lin @Chesnay Schepler,
We have deviated from the original purpose of this thread.
Let's keep this thread to the original proposal of externalising
flink-avro-glue-schema-registry and flink-json-glue-schema-registry format
;. That's why I though
"metrics.scope.jm-operator" might be more consistent.
> We would only support operator_name, operator_id, task_name and
task_id (+ everything else from the jm/job groups of course).
+1
Best,
Jark
On Mon, 30 Jan 2023 at 20:09, Chesnay Schepler wrote
major release a major release. It's
expected that things might not work across 1.x and 2.y.
On 31/01/2023 01:06, Dong Lin wrote:
On Mon, Jan 30, 2023 at 8:51 PM Chesnay Schepler wrote:
On 30/01/2023 13:24, Dong Lin wrote:
it should be OK for
apache/flink-connector-aws to use non-
On 30/01/2023 13:24, Dong Lin wrote:
it should be OK for
apache/flink-connector-aws to use non-public API, similar to how classes
inside Flink can use APIs marked with @Internal.
We can mark related classes with @Internal as appropriate, without
requiring any new FLIP.
It's not though, because
a metric reporting
for multiple metric names (e.g. multiple subtasks).
Best,
Jark
On Mon, 30 Jan 2023 at 16:11, Chesnay Schepler wrote:
I hadn't looked in detail in to how the task info can be
introduced, but
given that the coordinates are created by the execution graph,
w
Sorry for coming in so late.
This makes a lot of sense and I'd love to see it happen.
However, the avro glue registry relies heavily on flink-avro interfaces
which are currently not part of the public API.
I think we should sort this out first before externalizing this format.
To add on to tha
list.
How to get the task id/name when reporting a coordinator metric? As we
know, coordinator metrics
are just like JMJobMetricGroup, which don't belong to any tasks/vertexes.
Do you mean reporting
coordinator metrics for every task id under the operator?
Best,
Jark
On Thu, 19 Jan 2023 at 17:33,
docs.
On 27/01/2023 10:33, Gyula Fóra wrote:
Also @David Morávek @Chesnay Schepler
It would be great if you could update the respective docs page before
publishing your improvement FLIPS about the adaptive scheduler:
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/el
scheduler?
If it can do everything, why do we even have 2 schedulers? Why not simply
drop the "old" one?
That would probably clear up all confusionsthen :)
Gyula
On Thu, Jan 26, 2023 at 6:23 PM Chesnay Schepler
wrote:
There's the default and reactive mode; nothing else.
A
is for implementing them.
-Max
On Thu, Jan 26, 2023 at 4:57 PM Chesnay Schepler wrote:
On 26/01/2023 16:18, Maximilian Michels wrote:
I see slightly different goals for the standard and the adaptive
scheduler. The adaptive scheduler's goal is to adapt the Flink job
according to the availab
On 26/01/2023 16:18, Maximilian Michels wrote:
I see slightly different goals for the standard and the adaptive
scheduler. The adaptive scheduler's goal is to adapt the Flink job
according to the available resources.
This is really a misconception that we just have to stomp out.
This statement
Chesnay Schepler created FLINK-30796:
Summary: Make Slf4jReporter less noisy when no/few metrics exist
Key: FLINK-30796
URL: https://issues.apache.org/jira/browse/FLINK-30796
Project: Flink
101 - 200 of 1689 matches
Mail list logo
