Martijn Visser created FLINK-25553:
--
Summary: Remove MapR filesystem
Key: FLINK-25553
URL: https://issues.apache.org/jira/browse/FLINK-25553
Project: Flink
Issue Type: Technical Debt
weibo zhao created FLINK-25552:
--
Summary: Support MinMaxScaler in FlinkML
Key: FLINK-25552
URL: https://issues.apache.org/jira/browse/FLINK-25552
Project: Flink
Issue Type: New Feature
Hi Becket,
I might be missing something but having to define interfaces/formats for
the CEP patterns should be necessary for either approach. The OC approach
needs to receive and understand the pattern data from somewhere as well and
will probably also have to deal with evolving formats. Hence, I
Hi Yun,
Thanks for your feedback!
1. With the cached stream the compile and job submission happens as a
regular job submission. And a job with multiple concurrent cached
DataStream is supported. For your example, a and b are run in the same
job. Thus, all the cached DataStream are created when
I'm happy to announce that we have unanimously approved this release.
There are 6 approving votes, 3 of which are binding:
* Dong Lin (non-binding)
* Zhipeng Zhang (non-binding)
* Xingbo Huang (non-binding)
* Till Rohrmann (binding)
* Dian Fu (binding)
* Becket Qin (binding)
There are no
Very thanks everyone for the verification!
I'll announce the result in a separate thread.
Best,
Yun
--
Sender:Becket Qin
Date:2022/01/05 23:16:26
Recipient:dev
Cc:Yun Gao
Theme:Re: [VOTE] Apache Flink ML Release 2.0.0, release
Huang Xingbo created FLINK-25551:
Summary: Add example and documentation on the usage of Row in
Python UDTF
Key: FLINK-25551
URL: https://issues.apache.org/jira/browse/FLINK-25551
Project: Flink
Hi, Ryan,
Thanks a lot for helping with the migration. Some modules are already
migrated by us, but the code hasn't been merged since we still have some
pending details to discuss.
These modules are flink-runtime, flink-core, flink-test-utils,
flink-runtime-web,
flink-yarn, flink-kuberbetes,
Thanks for the explanation, Till. I like the idea, but have a question
about the first step.
After the first step, would users be able to actually use the dynamic
patterns in CEP?
In the first step you mentioned, the commands and formats for a new CEP
pattern seem related to how users would
Hang Ruan created FLINK-25545:
-
Summary: [JUnit5 Migration] Module: flink-clients
Key: FLINK-25545
URL: https://issues.apache.org/jira/browse/FLINK-25545
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25550:
-
Summary: [JUnit5 Migration] Module: flink-kuberbetes
Key: FLINK-25550
URL: https://issues.apache.org/jira/browse/FLINK-25550
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25549:
-
Summary: [JUnit5 Migration] Module: flink-dstl
Key: FLINK-25549
URL: https://issues.apache.org/jira/browse/FLINK-25549
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25548:
-
Summary: [JUnit5 Migration] Module: flink-sql-parser
Key: FLINK-25548
URL: https://issues.apache.org/jira/browse/FLINK-25548
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25547:
-
Summary: [JUnit5 Migration] Module: flink-optimizer
Key: FLINK-25547
URL: https://issues.apache.org/jira/browse/FLINK-25547
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25546:
-
Summary: [JUnit5 Migration] Module: flink-connector-base
Key: FLINK-25546
URL: https://issues.apache.org/jira/browse/FLINK-25546
Project: Flink
Issue Type:
Hang Ruan created FLINK-25544:
-
Summary: [JUnit5 Migration] Module: flink-streaming-java
Key: FLINK-25544
URL: https://issues.apache.org/jira/browse/FLINK-25544
Project: Flink
Issue Type:
Hang Ruan created FLINK-25543:
-
Summary: [JUnit5 Migration] Module: flink-yarn
Key: FLINK-25543
URL: https://issues.apache.org/jira/browse/FLINK-25543
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25542:
-
Summary: [JUnit5 Migration] Module: flink-runtime-web
Key: FLINK-25542
URL: https://issues.apache.org/jira/browse/FLINK-25542
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25541:
-
Summary: [JUnit5 Migration] Module: flink-test-utils
Key: FLINK-25541
URL: https://issues.apache.org/jira/browse/FLINK-25541
Project: Flink
Issue Type: Sub-task
Hang Ruan created FLINK-25540:
-
Summary: [JUnit5 Migration] Module: flink-runtime
Key: FLINK-25540
URL: https://issues.apache.org/jira/browse/FLINK-25540
Project: Flink
Issue Type: Sub-task
Qingsheng Ren created FLINK-25538:
-
Summary: Migrate flink-connector-kafka to JUnit 5
Key: FLINK-25538
URL: https://issues.apache.org/jira/browse/FLINK-25538
Project: Flink
Issue Type:
王康 created FLINK-25539:
--
Summary:
用flink的BatchTableEnvironment创建连接器去读取oss文件，并行度设为16，读数时有时会出现线程报错：Null IO stream
Key: FLINK-25539
URL: https://issues.apache.org/jira/browse/FLINK-25539
Project: Flink
Qingsheng Ren created FLINK-25537:
-
Summary: Migrate flink-core to JUnit 5
Key: FLINK-25537
URL: https://issues.apache.org/jira/browse/FLINK-25537
Project: Flink
Issue Type: Sub-task
Junfan Zhang created FLINK-25536:
Summary: Minor Fix: Adjust the order of variable declaration and
comment in StateAssignmentOperation
Key: FLINK-25536
URL: https://issues.apache.org/jira/browse/FLINK-25536
Bo Cui created FLINK-25535:
--
Summary: The JVM parameter does not take effect
Key: FLINK-25535
URL: https://issues.apache.org/jira/browse/FLINK-25535
Project: Flink
Issue Type: Bug
Affects
jychen created FLINK-25534:
--
Summary: execute pre-job throws
org.apache.flink.table.api.TableException: Failed to execute sql
Key: FLINK-25534
URL: https://issues.apache.org/jira/browse/FLINK-25534
Project:
Hi Till,
I think it would be great if we could achieve this so that Flink would be
'hardened' by default. Hopefully someone in the community has some ideas
how.
Best regards,
Martijn
On Tue, 4 Jan 2022 at 13:19, Till Rohrmann wrote:
> Hi everyone,
>
> With the latest CVEs around log4j, we
Till Rohrmann created FLINK-25533:
-
Summary: Preferred AllocationIDs are not respected when fulfilling
pending slot requests
Key: FLINK-25533
URL: https://issues.apache.org/jira/browse/FLINK-25533
Hi Xuannnan,
Thanks for the reply.
Regarding whether and how to support cache sideoutput, I agree that the
second option might be better if there do exist a use case that users need
to cache only some certain side outputs.
Xuannan Su 于2022年1月4日周二 15:50写道：
> Hi Zhipeng and Gen,
>
> Thanks for
I think I would scope the effort slightly differently. Note that I might be
missing some requirements or overlook something.
1. Enable Flink to support CEP dynamic patterns
Here I would define the commands and formats for a new CEP pattern. Then I
would extend the CEP operator to understand
+1 (binding)
- Verified the checksum and signature
- Built java code and ran all the tests
- Installed python packages according to the instructions in README.md
(there are some dependency conflicts but it looks they are due to my local
environment issues.)
Regards,
Jiangjie (Becket) Qin
On
Thanks for the detailed answer Xingbo. Quick question on the last figure in
the FLIP. You said that this is a real world Flink stream SQL job. The
title of the graph says UDF(String Upper). So do I understand correctly
that string upper is the real world use case you have measured? What I
wanted
Martijn Visser created FLINK-25532:
--
Summary: Provide Flink SQL CLI as Docker image
Key: FLINK-25532
URL: https://issues.apache.org/jira/browse/FLINK-25532
Project: Flink
Issue Type: New
Hi everyone,
As already mentioned in the previous discussion thread [1] I'm opening up a
parallel discussion thread on moving connectors from Flink to external
connector repositories. If you haven't read up on this discussion before, I
recommend reading that one first.
The goal with the external
+1 (binding)
- Verified the checksums
- Checked the website PR
- Diff'd the NOTICE files comparing it to 14.0 to check for anything
suspicious
- build Flink shaded
Thanks,
Chesnay
On Tue, Dec 14, 2021 at 9:57 AM Chesnay Schepler wrote:
> Hi everyone,
> Please review and vote on the release
Hi Till,
Thanks for the prompt reply. Like you said, we are indeed using the dynamic
CEP pattern use case to test the existing primitives in Flink to see if
they can meet the requirements. I fully understand the concern of
exposing OC as a user interface. Meanwhile I see CEP dynamic patterns as a
Hi Till and Thomas,
Thanks a lot for joining the discussion.
For Till:
>>> Is the slower performance currently the biggest pain point for our
Python users? What else are our Python users mainly complaining about?
PyFlink users are most concerned about two parts, one is better usability,
the
+1 (binding)
- Verified the checksum and signature
- Build the Java code and also run the tests using `mvn clean verify`
- Checked the NOTICE file
- Pip installed the python package in MacOS under Python 3.7
- Reviewed the flink-web PR
Regards,
Dian
On Tue, Jan 4, 2022 at 12:25 AM Dong Lin
Martijn Visser created FLINK-25531:
--
Summary: The test testRetryCommittableOnRetriableError takes one
hour before completing succesfully
Key: FLINK-25531
URL: https://issues.apache.org/jira/browse/FLINK-25531
Ada Wong created FLINK-25530:
Summary: Support Pulsar source connector in Python DataStream API.
Key: FLINK-25530
URL: https://issues.apache.org/jira/browse/FLINK-25530
Project: Flink
Issue
+1 the general direction here seems pretty solid
D.
On Wed, Jan 5, 2022 at 11:57 AM Till Rohrmann wrote:
> If there is no other larger feedback, I would start the vote soonish.
>
> Cheers,
> Till
>
> On Thu, Dec 30, 2021 at 4:28 PM Till Rohrmann
> wrote:
>
> > Hi David,
> >
> > Thanks for
If there is no other larger feedback, I would start the vote soonish.
Cheers,
Till
On Thu, Dec 30, 2021 at 4:28 PM Till Rohrmann wrote:
> Hi David,
>
> Thanks for your feedback.
>
> With the graceful shutdown I mean a way to stop the TaskManager and to
> clean up the working directory. At the
Hi everyone,
I wanted to summarise the email thread and see if there are any open items
that still need to be discussed, before we can finalise the discussion in
this email thread:
1. About having multi connectors in one repo or each connector in its own
repository
As explained by @Arvid Heise
+1 for dropping the MapR FS.
Cheers,
Till
On Wed, Jan 5, 2022 at 10:11 AM Martijn Visser
wrote:
> Hi everyone,
>
> Thanks for your input. I've checked the MapR implementation and it has no
> annotation at all. Given the circumstances that we thought that MapR was
> already dropped, I would
Hello! I can help out with the effort -- I've got a bit of experience with
JUnit 4 and 5 migration, and it looks like even with the AssertJ scripts
there's going to be a lot of mechanical and manual work to be done. The
migration document looks pretty comprehensive!
For the remaining topics to
Thanks for the detailed explanation Becket.
Do you think that an additional dependency is a deal breaker for people to
use dynamic CEP patterns? At the very least people have to operate some
kind of storage/queue system from which the CEP job can read anyway. Maybe
it could be good enough to
Hi Ryan,
Like Till said, your help would be much appreciated. The open tickets are
the most pressing ones for PubSub. The first ticket also has an open PR
that could be interesting to go through. Feel free to reach out to the Dev
mailing list for any questions or review requests.
Best regards,
Hi Dong,
Thanks for making this FLIP. I share the same concern with Martijn. This looks
like a feature that could be shared across all sources so I think it’ll be
great to make it a general one.
Instead of passing the RecordEvaluator to SourceReaderBase, what about
embedding the evaluator
Yuan Zhu created FLINK-25529:
Summary: java.lang.ClassNotFoundException:
org.apache.orc.PhysicalWriter when write bulkly into hive-2.1.1 orc table
Key: FLINK-25529
URL:
Thanks a lot for helping us with the PubSub connector Ryan. This is highly
appreciated! I think going through the open PRs and open issues might be a
good first step.
Cheers,
Till
On Tue, Jan 4, 2022 at 5:42 PM Ryan Skraba
wrote:
> Hello,
>
> I'm familiar with the Pub/Sub connectors from the
+1 (non-binding).
Thanks for driving！
Best regards
Jing
On Wed, Jan 5, 2022 at 4:13 AM Guowei Ma wrote:
> +1(binding).
> Thank you for driving this
> Best,
> Guowei
>
>
> On Wed, Jan 5, 2022 at 5:15 AM Arvid Heise wrote:
>
> > +1 (binding).
> >
> > Thanks for driving!
> >
> > On Tue, Jan 4,
Hi everyone,
Thanks for your input. I've checked the MapR implementation and it has no
annotation at all. Given the circumstances that we thought that MapR was
already dropped, I would propose to immediately remove MapR in Flink 1.15
instead of first marking it as deprecated and removing it in
刘方奇 created FLINK-25528:
---
Summary: state processor api do not support increment checkpoint
Key: FLINK-25528
URL: https://issues.apache.org/jira/browse/FLINK-25528
Project: Flink
Issue Type:
Zhipeng Zhang created FLINK-25527:
-
Summary: Add StringIndexer in FlinkML
Key: FLINK-25527
URL: https://issues.apache.org/jira/browse/FLINK-25527
Project: Flink
Issue Type: New Feature
Hi Xuannan,
Very thanks for drafting the FLIP and initiating the discussion!
I have several issues, sorry if I have misunderstandings:
1. With the cached stream, when would the compile and job submission
happens? Does it happen on calling execute_and_cache() ? If so, could
we support the job
Francesco Guardiani created FLINK-25526:
---
Summary: Deprecate TableSinkFactory, TableSourceFactory and
TableFormatFactory
Key: FLINK-25526
URL: https://issues.apache.org/jira/browse/FLINK-25526
Timo Walther created FLINK-25525:
Summary: flink-examples-table is not runnable in the IDE
Key: FLINK-25525
URL: https://issues.apache.org/jira/browse/FLINK-25525
Project: Flink
Issue Type:
Yun Tang created FLINK-25524:
Summary: If enabled changelog, RocksDB incremental checkpoint
would always be full
Key: FLINK-25524
URL: https://issues.apache.org/jira/browse/FLINK-25524
Project: Flink
Till Rohrmann created FLINK-25523:
-
Summary: KafkaSourceITCase$KafkaSpecificTests.testTimestamp fails
on AZP
Key: FLINK-25523
URL: https://issues.apache.org/jira/browse/FLINK-25523
Project: Flink
Till Rohrmann created FLINK-25522:
-
Summary:
KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime
Key: FLINK-25522
URL: https://issues.apache.org/jira/browse/FLINK-25522
Very thanks @Yingjie for completing the experiments!
Also +1 for changing the default config values. From the experiments,
Changing the default config values would largely increase the open box
experience of the flink batch, thus it seems worth changing from my side
even if it would cause some
61 matches
Mail list logo
