Thanks for all the feedbacks @Hequn @Timo @Godfrey !
We will note your updates and adjust the feature list accordingly in the next
progress announcement.
Best,
Zhijiang
--
From:godfrey he
Send Time:2020 Mar. 11 (Wed.) 17:20
Thanks Aljoscha ~
I agree for most of the query hints, they are optional as an optimizer
instruction, especially for the traditional RDBMS.
But, just like BenChao said, Flink as a computation engine has many different
kind of data sources, thus, dynamic parameters like start_offest can only
Thanks for driving this release, Yu!
+1 to start 1.10.1 release cycle.
>From the Table SQL module, I think we should also try to get in the
following issues:
- FLINK-16441: Allow users to override flink-conf parameters from SQL CLI
environment
this allows users to set e.g. statebackend,
Biao Liu created FLINK-16561:
Summary: Resuming Externalized Checkpoint (rocks, incremental, no
parallelism change) end-to-end test fails on Azure
Key: FLINK-16561
URL:
Zhu Zhu created FLINK-16560:
---
Summary: StreamExecutionEnvironment configuration is empty when
building program via PackagedProgramUtils#createJobGraph
Key: FLINK-16560
URL:
Thanks Andrey for kicking this discussion off.
Regarding "direct" vs. "off-heap", I'm personally in favor of renaming the
"direct" memory in the current FLIP-116[1] to "off-heap" memory, and making
it also account for user native memory usage.
On one hand, I think it would be good that JM & TM
Rui Li created FLINK-16559:
--
Summary: Cannot create Hive avro table in test
Key: FLINK-16559
URL: https://issues.apache.org/jira/browse/FLINK-16559
Project: Flink
Issue Type: Bug
Tzu-Li (Gordon) Tai created FLINK-16558:
---
Summary: Reword Stateful Functions doc's tagline
Key: FLINK-16558
URL: https://issues.apache.org/jira/browse/FLINK-16558
Project: Flink
Issue
Tzu-Li (Gordon) Tai created FLINK-16557:
---
Summary: Document YAML-ized Kafka egresses / ingresses in Stateful
Functions documentation
Key: FLINK-16557
URL: https://issues.apache.org/jira/browse/FLINK-16557
Hi all,
Thanks Danny for bring up this great discussion, generally hints is a great
feature for SQL.
And the discussions are very insightful, I'd like to share some ideas here.
About error handling,
+1 to throw exception by default. IMHO, hints are parts of the query, which
should be validated
Nico Kruber created FLINK-16556:
---
Summary: TopSpeedWindowing should implement checkpointing for its
source
Key: FLINK-16556
URL: https://issues.apache.org/jira/browse/FLINK-16556
Project: Flink
Stephan Ewen created FLINK-16555:
Summary: Preflight check for known unstable hashCodes.
Key: FLINK-16555
URL: https://issues.apache.org/jira/browse/FLINK-16555
Project: Flink
Issue Type:
Hey all,
I was wondering whether for CEP the *AfterMatchSkipStrategy *is applied
during matching or if simply the results are removed after the match. The
question is the result of the experiments I was doing with CEP. Say I have
the readings from some sensor and I want to detect events over some
Roman Khachatryan created FLINK-16554:
-
Summary: Extract static classes from StreamTask
Key: FLINK-16554
URL: https://issues.apache.org/jira/browse/FLINK-16554
Project: Flink
Issue Type:
+1 (non-binding)
Regarding Yu's suggestion about *Roadmap* or *Future Work* section, I think
it's a good idea.
Currently, some MVP limitations are mentioned at the end of the document,
so we can extract and expand it.
As for the recovery speed it's not a priority currently, but we could also
On 09.03.20 06:10, Rong Rong wrote:
- Is this feature (disabling checkpoint and restarting job from Kafka
committed GROUP_OFFSET) not supported?
I believe the Flink community never put much (any?) effort into this
because the Flink Kafka Consumer does its own offset handling. Starting
from
A quick summary that focus of the discussion now shifts to be whether
semantic params like kafka 'starting offset' should be table
hints/properties, and if so, in what form.
I strongly believe the action of setting offset should *not* be part of a
table, neither hints nor properties, for all the
Fabian Paul created FLINK-16553:
---
Summary: KafkaFetcher topic/partition metrics
Key: FLINK-16553
URL: https://issues.apache.org/jira/browse/FLINK-16553
Project: Flink
Issue Type: Improvement
Jason Sinn created FLINK-16552:
--
Summary: Cannot include Option fields in any Table join
Key: FLINK-16552
URL: https://issues.apache.org/jira/browse/FLINK-16552
Project: Flink
Issue Type: Bug
+1 (binding).
As for David's concern of smaller buffers after recovery, I ever had a draft
design [1] to solve this issue.
You can take a look and leave comments if still have concerns. :)
[1]
https://docs.google.com/document/d/16_MOQymzxrKvUHXh6QFr2AAXIKt_2vPUf8vzKy4H_tU/edit
Best,
Zhijiang
Robert Metzger created FLINK-16551:
--
Summary: WebFrontendITCase.getFrontPage fails
Key: FLINK-16551
URL: https://issues.apache.org/jira/browse/FLINK-16551
Project: Flink
Issue Type: Bug
Robert Metzger created FLINK-16550:
--
Summary: HadoopS3* tests fail with NullPointerException exceptions
Key: FLINK-16550
URL: https://issues.apache.org/jira/browse/FLINK-16550
Project: Flink
Robert Metzger created FLINK-16549:
--
Summary: ExecutionContextTest.testCatalogs gets stuck / Out of
memory
Key: FLINK-16549
URL: https://issues.apache.org/jira/browse/FLINK-16549
Project: Flink
hejianchao created FLINK-16548:
--
Summary: [Requirement] Consistent id to identify jm/tm in
different cluster modes
Key: FLINK-16548
URL: https://issues.apache.org/jira/browse/FLINK-16548
Project: Flink
Hi,
I don't understand this discussion. Hints, as I understand them, should
work like this:
- hints are *optional* advice for the optimizer to try and help it to
find a good execution strategy
- hints should not change query semantics, i.e. they should not change
connector properties
+1 (binding).
Piotrek
> On 11 Mar 2020, at 09:19, David Anderson wrote:
>
> +1 I like where this is headed.
>
> One question: during restore, it could happen that a new task manager is
> configured with fewer or smaller buffers than was previously the case. How
> will this be handled?
>
>
Hi All,
As you may have noticed, 1.10 release included an extensive improvements to
memory management and configuration of Task Managers, FLIP-49: [1]. The
memory configuration of Job Managers has not been touched in 1.10.
Although, Job Manager's memory model does not look so sophisticated as
Canbin Zheng created FLINK-16547:
Summary: Corrent the order to write temporary files in
YarnClusterDescriptor#startAppMaster
Key: FLINK-16547
URL: https://issues.apache.org/jira/browse/FLINK-16547
Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225 fix
versions and promoted its priority to Critical. Will also watch the
progress of FLINK-16108/FLINK-16408.
Best Regards,
Yu
On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin wrote:
> Hi Yu,
>
> Thanks for kicking off the
Canbin Zheng created FLINK-16546:
Summary: Fix logging bug in YarnClusterDescriptor#startAppMaster
Key: FLINK-16546
URL: https://issues.apache.org/jira/browse/FLINK-16546
Project: Flink
Chesnay Schepler created FLINK-16545:
Summary: Remove Eclipse-specific plugins
Key: FLINK-16545
URL: https://issues.apache.org/jira/browse/FLINK-16545
Project: Flink
Issue Type:
Angel Barragán created FLINK-16544:
--
Summary: Flink FileSystem for web.uploadDir
Key: FLINK-16544
URL: https://issues.apache.org/jira/browse/FLINK-16544
Project: Flink
Issue Type:
Caizhi Weng created FLINK-16543:
---
Summary: Support setting schedule mode by config for Blink planner
in batch mode
Key: FLINK-16543
URL: https://issues.apache.org/jira/browse/FLINK-16543
Project: Flink
JieFang.He created FLINK-16542:
--
Summary: Nothing on the HistoryServer
Key: FLINK-16542
URL: https://issues.apache.org/jira/browse/FLINK-16542
Project: Flink
Issue Type: Bug
Thanks Timo for summarize the 3 options ~
I agree with Kurt that option2 is too complicated to use because:
• As a Kafka topic consumer, the user must define both the virtual column for
start offset and he must apply a special filter predicate after each query
• And for the internal
Caizhi Weng created FLINK-16541:
---
Summary: Document of table.exec.shuffle-mode is incorrect
Key: FLINK-16541
URL: https://issues.apache.org/jira/browse/FLINK-16541
Project: Flink
Issue Type:
Thanks! I'm reading the document now and will get back to you.
Best,
Aljoscha
Fabian Hueske created FLINK-16540:
-
Summary: Fully specify bugfix version of Docker containers in
Flink Playground docker-compose.yaml files
Key: FLINK-16540
URL: https://issues.apache.org/jira/browse/FLINK-16540
Hi Yu,
Thanks for kicking off the 1.10.1 release discussion!
Apart from
- FLINK-16406 Increase default value for JVM Metaspace to minimise its
OutOfMemoryError
which should be merged soon
I think we should also try to get in the following issues:
- [FLINK-16225] Metaspace Out Of Memory should
Hi Zhijiang and Piotr,
I think we can remove "FLIP-91 Introduce SQL client gateway and provide
JDBC driver" from the list, because we have decided the first step to
support sql gateway and jdbc driver as ecosystem in ververica, we are not
going to put more effort on it now.
Thanks for updating
+1 on removing them.
They are so limited in terms of functionality that I doubt anyone would
be significantly impaired by us removing them.
On 11/03/2020 02:13, Xintong Song wrote:
Thanks for the survey, Aljoscha. I'll also close FLINK-15925.
I second Robert's concern. It sounds a bit weird
Hi Timo,
option 1 & option 3 are almost the same in my opinion. Even though we call
it table hints, the biggest
motivation for now is to modify table properties. I also see other vendor
using syntax like option 1 to
implement table hints, e.g. sql server [1]. It uses syntax like SELECT *
from T
Jun Zhang created FLINK-16539:
-
Summary: sql client set param error
Key: FLINK-16539
URL: https://issues.apache.org/jira/browse/FLINK-16539
Project: Flink
Issue Type: Bug
Components:
Dian Fu created FLINK-16538:
---
Summary: Restructure Python Table API documentation
Key: FLINK-16538
URL: https://issues.apache.org/jira/browse/FLINK-16538
Project: Flink
Issue Type: Improvement
+1 I like where this is headed.
One question: during restore, it could happen that a new task manager is
configured with fewer or smaller buffers than was previously the case. How
will this be handled?
David
On Wed, Mar 11, 2020 at 8:31 AM Arvid Heise wrote:
> Hi Thomas,
>
> it's like you
Hi Zhijiang and Piotr,
from the SQL side we also plan to rework the source and sink interfaces
in 1.11. The FLIP is not yet published but already reserved and
requirement for FLIP-105:
FLIP-95: New TableSource and TableSink interfaces
Thanks for compiling the list!
Regards,
Timo
On
Thanks Zhijiang and Piotr for kicking off the discussion and providing the
detailed list.
This would be very helpful for tracking the features.
BTW, as for PyFlink, it would be great if the feature list can also include
the following features:
- FLIP-112: Support User-Defined Metrics in Python
Hi Danny,
it is true that our DDL is not standard compliant by using the WITH
clause. Nevertheless, we aim for not diverging too much and the LIKE
clause is an example of that. It will solve things like overwriting
WATERMARKs, add additional/modifying properties and inherit schema.
Bowen is
Thanks for driving. Yu. +1 for starting the 1.10.1 release.
Some issues are very important, Users are looking forward to them.
Best,
Jingsong Lee
On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo wrote:
> Thanks for driving this release, Yu!
>
> +1 for starting the 1.10.1 release cycle.
>
> Best,
>
Hi,
Thanks a lot for raising the discussion @Stephan.
+1 to increase the visibilities of the Stateful Functions.
Another option I'm think is adding a section(named Stateful Functions or
Flink Projects?)
under the "Latest Blog Posts". The advantage is we can add a picture and
some descriptions
Hi Thomas,
it's like you said. The first version will not support rescaling and mostly
addresses the concerns about making little to no progress because of
frequent crashes.
The main reason is that we cannot guarantee the ordering of non-keyed data
(and even keyed data in some weird cases) when
Thanks for driving this release, Yu!
+1 for starting the 1.10.1 release cycle.
Best,
Yangze Guo
On Wed, Mar 11, 2020 at 1:42 PM Xintong Song wrote:
>
> Yu,
> Thanks for the explanation.
> I've no concerns. I was just trying to get some inputs for prioritizing
> tasks on my side, and ~1month
Thanks Bowen ~
I agree we should somehow categorize our connector parameters.
For type1, I’m already preparing a solution like the Confluent schema registry
+ Avro schema inference thing, so this may not be a problem in the near future.
For type3, I have some questions:
> "SELECT * FROM
53 matches
Mail list logo
