[jira] [Created] (FLINK-17454) test_configuration.py ConfigurationTests::test_add_all failed on travis

2020-04-28 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-17454: -- Summary: test_configuration.py ConfigurationTests::test_add_all failed on travis Key: FLINK-17454 URL: https://issues.apache.org/jira/browse/FLINK-17454 Project:

[GitHub] [flink-web] klion26 edited a comment on pull request #242: [FLINK-13684][docs-zh] Translate "Code Style - Formatting Guide" page into Chinese

2020-04-28 Thread GitBox
klion26 edited a comment on pull request #242: URL: https://github.com/apache/flink-web/pull/242#issuecomment-620981849 @shining-huang thanks for your contribution. could you please resolve the conflict by rebasing the newly master?

[GitHub] [flink-web] klion26 commented on pull request #242: [FLINK-13684][docs-zh] Translate "Code Style - Formatting Guide" page into Chinese

2020-04-28 Thread GitBox
klion26 commented on pull request #242: URL: https://github.com/apache/flink-web/pull/242#issuecomment-620981849 @shining-huang thanks for your contribution. could you please resolve the conflic by rebasing the newly master?

The use of state ttl incremental cleanup strategy in sql deduplication resulting in significant performance degradation

2020-04-28 Thread 刘大龙
Hi, all! At flink master branch, we have supported state ttl for sql mini-batch deduplication using incremental cleanup strategy on heap backend, refer to FLINK-16581. Because I want to test the performance of this feature, so I compile master branch code and deploy the jar to production

[jira] [Created] (FLINK-17453) KyroSerializer throws IndexOutOfBoundsException type java.util.PriorityQueue

2020-04-28 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-17453: -- Summary: KyroSerializer throws IndexOutOfBoundsException type java.util.PriorityQueue Key: FLINK-17453 URL: https://issues.apache.org/jira/browse/FLINK-17453 Project:

[jira] [Created] (FLINK-17452) Support creating Hive tables with constraints

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17452: -- Summary: Support creating Hive tables with constraints Key: FLINK-17452 URL: https://issues.apache.org/jira/browse/FLINK-17452 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17450) Implement function DDLs for Hive dialect

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17450: -- Summary: Implement function DDLs for Hive dialect Key: FLINK-17450 URL: https://issues.apache.org/jira/browse/FLINK-17450 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17451) Implement view DDLs for Hive dialect

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17451: -- Summary: Implement view DDLs for Hive dialect Key: FLINK-17451 URL: https://issues.apache.org/jira/browse/FLINK-17451 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17449) Implement ADD/DROP partitions

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17449: -- Summary: Implement ADD/DROP partitions Key: FLINK-17449 URL: https://issues.apache.org/jira/browse/FLINK-17449 Project: Flink Issue Type: Sub-task Components:

Re: [DISCUSS] Have foreground processes also create log files

2020-04-28 Thread Yang Wang
Thanks for Chesnay starting this discussion. In FLINK-17166 implementation[1], we are trying to use "tee" instead of introducing the stream redirection(redirect the out/err to files). However, a side effect is that the logging will be duplicated both in .log and .out files. Then it may consume

[jira] [Created] (FLINK-17448) Implement table DDLs for Hive dialect part2

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17448: -- Summary: Implement table DDLs for Hive dialect part2 Key: FLINK-17448 URL: https://issues.apache.org/jira/browse/FLINK-17448 Project: Flink Issue Type: Sub-task

[GitHub] [flink-web] klion26 commented on a change in pull request #235: [FLINK-13344][docs-zh] Translate "How to Contribute" page into Chinese

2020-04-28 Thread GitBox
klion26 commented on a change in pull request #235: URL: https://github.com/apache/flink-web/pull/235#discussion_r417031566 ## File path: contributing/how-to-contribute.zh.md ## @@ -4,136 +4,138 @@ title: "如何参与贡献" -Apache Flink is developed by an open and friendly

[jira] [Created] (FLINK-17447) Flink CEPOperator StateException

2020-04-28 Thread chun111111 (Jira)
chun11 created FLINK-17447: -- Summary: Flink CEPOperator StateException Key: FLINK-17447 URL: https://issues.apache.org/jira/browse/FLINK-17447 Project: Flink Issue Type: Bug

Re: [DISCUSS] Intermediary releases of the flink-docker images

2020-04-28 Thread Yang Wang
Thanks for starting this discussion. In general, i am also in favor of making docker image release could be partly decoupled from Flink release. However, i think it should only happen when we want to make some changes that are independent from Flink majar release(e.g. JDK11, change base image,

[jira] [Created] (FLINK-17446) Blink supports cube and window together in group by

2020-04-28 Thread Liu (Jira)
Liu created FLINK-17446: --- Summary: Blink supports cube and window together in group by Key: FLINK-17446 URL: https://issues.apache.org/jira/browse/FLINK-17446 Project: Flink Issue Type: New Feature

Re: Question about FLIP-66

2020-04-28 Thread Jungtaek Lim
Thanks for confirming! Honestly I support to treat timestamp field as special and restrict modification (the way DataStream API does), but I agree the new approach could be more natural to unify the semantic of SQL for both batch and stream. Thanks again, Jungtaek Lim (HeartSaVioR) On Tue, Apr

[jira] [Created] (FLINK-17445) Allow OperatorTransformation to bootstrapWith a Scala DataSet

2020-04-28 Thread Brandon Bevans (Jira)
Brandon Bevans created FLINK-17445: -- Summary: Allow OperatorTransformation to bootstrapWith a Scala DataSet Key: FLINK-17445 URL: https://issues.apache.org/jira/browse/FLINK-17445 Project: Flink

[jira] [Created] (FLINK-17444) Flink StreamingFileSink Azure HadoopRecoverableWriter is missing.

2020-04-28 Thread Marie May (Jira)
Marie May created FLINK-17444: - Summary: Flink StreamingFileSink Azure HadoopRecoverableWriter is missing. Key: FLINK-17444 URL: https://issues.apache.org/jira/browse/FLINK-17444 Project: Flink

[jira] [Created] (FLINK-17443) Flink's ZK in HA mode setup is unable to start up if any of the zk hosts are unreachable

2020-04-28 Thread Piyush Narang (Jira)
Piyush Narang created FLINK-17443: - Summary: Flink's ZK in HA mode setup is unable to start up if any of the zk hosts are unreachable Key: FLINK-17443 URL: https://issues.apache.org/jira/browse/FLINK-17443

[jira] [Created] (FLINK-17442) Cannot convert String or boxed-primitive arrays to DataStream using TypeInformation

2020-04-28 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-17442: -- Summary: Cannot convert String or boxed-primitive arrays to DataStream using TypeInformation Key: FLINK-17442 URL: https://issues.apache.org/jira/browse/FLINK-17442

[jira] [Created] (FLINK-17441) FlinkKinesisConsumerTest.testPeriodicWatermark: watermark count expected:<2> but was:<1>

2020-04-28 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-17441: -- Summary: FlinkKinesisConsumerTest.testPeriodicWatermark: watermark count expected:<2> but was:<1> Key: FLINK-17441 URL: https://issues.apache.org/jira/browse/FLINK-17441

[DISCUSS] Have foreground processes also create log files

2020-04-28 Thread Chesnay Schepler
Currently, processes started in the foreground (like in the case of Docker) output all logging/stdout directly to the console, without creating any logging files. The downside of this approach, as outlined in FLIP-111, is that the WebUI is not able to display the logs since it relies on these

Re: [DISCUSS] Intermediary releases of the flink-docker images

2020-04-28 Thread Ismaël Mejía
> To me this means that the docker images should be released at the same time > the other artifacts are released. This also includes shapshot releases. So > the build of the docker images should be an integral part of the build. This is already the case since the last release, what this thread

[jira] [Created] (FLINK-17440) Potential Buffer leak in output unspilling for unaligned checkpoints

2020-04-28 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-17440: -- Summary: Potential Buffer leak in output unspilling for unaligned checkpoints Key: FLINK-17440 URL: https://issues.apache.org/jira/browse/FLINK-17440 Project:

[jira] [Created] (FLINK-17439) Setup versioned branches in flink-docker

2020-04-28 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-17439: Summary: Setup versioned branches in flink-docker Key: FLINK-17439 URL: https://issues.apache.org/jira/browse/FLINK-17439 Project: Flink Issue Type:

[jira] [Created] (FLINK-17438) Flink StreamingFileSink chinese garbled

2020-04-28 Thread Jira
颖 created FLINK-17438: - Summary: Flink StreamingFileSink chinese garbled Key: FLINK-17438 URL: https://issues.apache.org/jira/browse/FLINK-17438 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-17437) Use StringData instead of BinaryStringData in code generation

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17437: --- Summary: Use StringData instead of BinaryStringData in code generation Key: FLINK-17437 URL: https://issues.apache.org/jira/browse/FLINK-17437 Project: Flink Issue

Re: [DISCUSS] Should max/min be part of the hierarchy of config option?

2020-04-28 Thread Yangze Guo
Thank you all for the feedback. It seems we reach a consensus: - The naming convention would better to be xyz.[min|max]. - Adding tests/tools checking the pattern of new configuration options - Tickets for Flink 2.0 to migrate the "wrong" configuration options. If there is no objection in the

[jira] [Created] (FLINK-17436) When submitting Python job via "flink run" a IllegalAccessError will be raised due to the package's private access control

2020-04-28 Thread Wei Zhong (Jira)
Wei Zhong created FLINK-17436: - Summary: When submitting Python job via "flink run" a IllegalAccessError will be raised due to the package's private access control Key: FLINK-17436 URL:

Re: [DISCUSS] Intermediary releases of the flink-docker images

2020-04-28 Thread Chesnay Schepler
I agree with Niels that if a Flink release is made it should be accompanied by the Dockerfiles for that version, and release them at once. This makes sense for testing purposes alone, and I view Docker as just another distribution channel like Maven central. The reverse isn't necessarily true

Re: 'Too many files with unapproved license' issue

2020-04-28 Thread Chesnay Schepler
The failure means that a file is missing the Apache license header. This is usually because either a) you have added a file but forgot the header, in which case the solution naturally is to add the header b) you run on Windows, and Maven does not detect that some files are binary files (which

Re: 'Too many files with unapproved license' issue

2020-04-28 Thread Xintong Song
Hi Manish, The error suggests that there are some files in the project without the ASF license included. This usually happens when you added some source files without adding the license, or there are some temporal files (e.g., generated by running a local test) not cleaned up. The error message

'Too many files with unapproved license' issue

2020-04-28 Thread Manish G
Hi, While building the project, I get this error: [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project flink-parent: Too many files with unapproved license Any workaround for it? Manish

[jira] [Created] (FLINK-17435) Hive non-partitioned source support streaming read

2020-04-28 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-17435: Summary: Hive non-partitioned source support streaming read Key: FLINK-17435 URL: https://issues.apache.org/jira/browse/FLINK-17435 Project: Flink Issue

[jira] [Created] (FLINK-17434) Hive partitioned source support streaming read

2020-04-28 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-17434: Summary: Hive partitioned source support streaming read Key: FLINK-17434 URL: https://issues.apache.org/jira/browse/FLINK-17434 Project: Flink Issue Type:

[jira] [Created] (FLINK-17433) Real time hive integration

2020-04-28 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-17433: Summary: Real time hive integration Key: FLINK-17433 URL: https://issues.apache.org/jira/browse/FLINK-17433 Project: Flink Issue Type: Bug

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-04-28 Thread Chesnay Schepler
It would be good if we could nail down what a slim/fat distribution would look like, as there are various ideas floating around in this thread. Like, what is a "slim" distribution? Are we just emptying /opt? Removing everything larger than 1mb? Are we throwing out the Table API from /lib for

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-04-28 Thread Chesnay Schepler
This would likely solve the issues surrounding the SQL client, so I would go along with that. On 17/04/2020 12:16, Aljoscha Krettek wrote: I think having such tools and/or tailor-made distributions can be nice but I also think the discussion is missing the main point: The initial

Re: [DISCUSS] Generating java code?

2020-04-28 Thread Niels Basjes
Hi, Thank you for clarifying the tradeoffs and choices. I've updated my pull request for your review, as far as I can tell it meets the choices. Now there are 3 scenarios: 1) There is no properties file --> everything returns a "default" value. These are the defaults I have chosen: Version

[jira] [Created] (FLINK-17432) Rename Tutorials to Training

2020-04-28 Thread David Anderson (Jira)
David Anderson created FLINK-17432: -- Summary: Rename Tutorials to Training Key: FLINK-17432 URL: https://issues.apache.org/jira/browse/FLINK-17432 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-17431) Implement table DDLs for Hive dialect part 1

2020-04-28 Thread Rui Li (Jira)
Rui Li created FLINK-17431: -- Summary: Implement table DDLs for Hive dialect part 1 Key: FLINK-17431 URL: https://issues.apache.org/jira/browse/FLINK-17431 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17430) Support SupportsPartitioning in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17430: --- Summary: Support SupportsPartitioning in planner Key: FLINK-17430 URL: https://issues.apache.org/jira/browse/FLINK-17430 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17429) Support SupportsOverwrite in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17429: --- Summary: Support SupportsOverwrite in planner Key: FLINK-17429 URL: https://issues.apache.org/jira/browse/FLINK-17429 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17428) Support SupportsProjectionPushDown in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17428: --- Summary: Support SupportsProjectionPushDown in planner Key: FLINK-17428 URL: https://issues.apache.org/jira/browse/FLINK-17428 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17427) Support SupportsPartitionPushDown in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17427: --- Summary: Support SupportsPartitionPushDown in planner Key: FLINK-17427 URL: https://issues.apache.org/jira/browse/FLINK-17427 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17426) Support SupportsLimitPushDown in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17426: --- Summary: Support SupportsLimitPushDown in planner Key: FLINK-17426 URL: https://issues.apache.org/jira/browse/FLINK-17426 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17425) Supports SupportsFilterPushDown in planner

2020-04-28 Thread Jark Wu (Jira)
Jark Wu created FLINK-17425: --- Summary: Supports SupportsFilterPushDown in planner Key: FLINK-17425 URL: https://issues.apache.org/jira/browse/FLINK-17425 Project: Flink Issue Type: Sub-task

Re: Issue

2020-04-28 Thread Till Rohrmann
Hi Pavan, please post these kind of questions to the user ML. I've cross linked it now. Image attachments will be filtered out. Consequently, we cannot see what you have posted. Moreover, it would be good if you could provide the community with a bit more details what the custom way is and what

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-04-28 Thread Till Rohrmann
Hi Yadong, this sounds like a good solution to me. Cheers, Till On Tue, Apr 28, 2020 at 4:18 AM Forward Xu wrote: > +1 > > best, > Forward > > Yadong Xie 于2020年4月28日周二 上午10:03写道: > > > Hi all > > > > sorry for we have an issue that was not discovered in advance > > > > When users run

[jira] [Created] (FLINK-17424) SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1) failed due to download error

2020-04-28 Thread Yu Li (Jira)
Yu Li created FLINK-17424: - Summary: SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1) failed due to download error Key: FLINK-17424 URL: https://issues.apache.org/jira/browse/FLINK-17424

[jira] [Created] (FLINK-17423) Support Python UDTF in blink planner under batch mode

2020-04-28 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-17423: Summary: Support Python UDTF in blink planner under batch mode Key: FLINK-17423 URL: https://issues.apache.org/jira/browse/FLINK-17423 Project: Flink Issue

Re: [DISCUSS] Intermediary releases of the flink-docker images

2020-04-28 Thread Aljoscha Krettek
Hi Niels, I think Robert was referring to the fact that Apache considers only the source release to be "the release", everything else is called convenience release. Best, Aljoscha On 27.04.20 19:43, Niels Basjes wrote: Hi, In my opinion the docker images are essentially simply differently