[jira] [Created] (FLINK-17683) Pojo with no-args constructor will be treated as ROW when returned by ScalarFunction， but not when it is a field of case class.

xiemeilong created FLINK-17683: -- Summary: Pojo with no-args constructor will be treated as ROW when returned by ScalarFunction， but not when it is a field of case class. Key: FLINK-17683 URL:

[jira] [Created] (FLINK-17682) HiveModuleTest.testConstantReturnValue didn't finish

Piotr Nowojski created FLINK-17682: -- Summary: HiveModuleTest.testConstantReturnValue didn't finish Key: FLINK-17682 URL: https://issues.apache.org/jira/browse/FLINK-17682 Project: Flink

[jira] [Created] (FLINK-17681) TableEnvironment fromValues not work with map type and SQL

Jingsong Lee created FLINK-17681: Summary: TableEnvironment fromValues not work with map type and SQL Key: FLINK-17681 URL: https://issues.apache.org/jira/browse/FLINK-17681 Project: Flink

[jira] [Created] (FLINK-17680) Support @ExcludeFromDocumentation in RestAPIDocGenerator

Caizhi Weng created FLINK-17680: --- Summary: Support @ExcludeFromDocumentation in RestAPIDocGenerator Key: FLINK-17680 URL: https://issues.apache.org/jira/browse/FLINK-17680 Project: Flink Issue

[jira] [Created] (FLINK-17679) Fix the bug of encoding bytes in cython coder

Huang Xingbo created FLINK-17679: Summary: Fix the bug of encoding bytes in cython coder Key: FLINK-17679 URL: https://issues.apache.org/jira/browse/FLINK-17679 Project: Flink Issue Type:

[jira] [Created] (FLINK-17678) Support fink-sql-connector-hbase

ShenDa created FLINK-17678: -- Summary: Support fink-sql-connector-hbase Key: FLINK-17678 URL: https://issues.apache.org/jira/browse/FLINK-17678 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-17677) FLINK_LOG_PREFIX recommended in docs is not always available

Xintong Song created FLINK-17677: Summary: FLINK_LOG_PREFIX recommended in docs is not always available Key: FLINK-17677 URL: https://issues.apache.org/jira/browse/FLINK-17677 Project: Flink

[jira] [Created] (FLINK-17676) Is there some way to rollback the .out file of TaskManager

JieFang.He created FLINK-17676: -- Summary: Is there some way to rollback the .out file of TaskManager Key: FLINK-17676 URL: https://issues.apache.org/jira/browse/FLINK-17676 Project: Flink Issue

Re: changing the output files names in Streamfilesink from part-00 to something else

The StreamingFileSink can have a OutputFileConfig [1] to config the prefix and suffix of the part file, does that work for you ? [1] 

[jira] [Created] (FLINK-17675) Resolve CVE-2019-11358 from jquery

Koala Lam created FLINK-17675: - Summary: Resolve CVE-2019-11358 from jquery Key: FLINK-17675 URL: https://issues.apache.org/jira/browse/FLINK-17675 Project: Flink Issue Type: Improvement

Re: [DISCUSS] [FLINK-16824] Creating Temporal Table Function via DDL

Thanks for sharing your opinion. I can see there are some very small divergences we had through your description. I think it would be a good idea to first discuss these first. Let's first put aside table version for now, and only discuss about whether a DDL table should be treated as a DMBS style

Re: Season of Docs - Ex-participant Intro

Hey Kartik, Nice to connect, I am one of the GSoD mentors, along with Aljoscha in cc. Happy to see you're interested in working with the community. Seth On Wed, May 13, 2020 at 4:10 PM Kartik Khare wrote: > Hi, > This is Kartik. I an genuinely interested in working with your organisation >

Season of Docs - Ex-participant Intro

Hi, This is Kartik. I an genuinely interested in working with your organisation for Google season of Docs. I work as data engineer @ walmart labs but also write a lot of blogs on distributed systems on https://medium.com/@kharekartik. I contributed one on flink's website as well:

Re: [Discuss] Breaking API Change in 1.10.1

On Wed, May 13, 2020 at 1:02 PM Piotr Nowojski wrote: > Hi, > > > For the future, it would be great to eliminate the possibility of > > incompatible changes to user-facing classes for patch releases though. > > Running japicmp as part of CI might be a good option. > > I guess the whole point of

Re: [Discuss] Breaking API Change in 1.10.1

Hi, > For the future, it would be great to eliminate the possibility of > incompatible changes to user-facing classes for patch releases though. > Running japicmp as part of CI might be a good option. I guess the whole point of PublicEvolving classes is to allow us for the breaking

[CVE-2020-1960] Apache Flink JMX information disclosure vulnerability

CVE-2020-1960: Apache Flink JMX information disclosure vulnerability Severity: Medium (CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:H/I:L/A:H) Vendor: The Apache Software Foundation Versions Affected: Flink 1.1.0 to 1.1.5 Flink 1.2.0 to 1.2.1 Flink 1.3.0 to 1.3.3 Flink 1.4.0 to 1.4.2 Flink 1.5.0 to

Re: changing the output files names in Streamfilesink from part-00 to something else

Yes we looked at it , The problem is the file name gets generated in a dynamic fashion, based on which organization data we are getting we generate the file name from the coming data. Is there any way we can achieve this ?? On Tue, May 12, 2020 at 8:38 PM Yun Gao wrote: > Hi Dhurandar: > >

[ANNOUNCE] Apache Flink 1.10.1 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.10.1, which is the first bugfix release for the Apache Flink 1.10 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: [DISCUSS] [FLINK-16824] Creating Temporal Table Function via DDL

I think Flink should behave similar to other DBMSs. Other DBMS do not allow to query the history of a table, even though the DBMS has seen all changes of the table (as transactions or directly as a changelog if the table was replicated) and recorded them in its log. You need to declare a table as

Re: [Discuss] Breaking API Change in 1.10.1

+1 to improve and eliminate the possibility of incompatible changes to user-facing classes for patch releases. The PR for 1.10.1 release [1] is already updated, please kindly let me know if any further comments. I plan to merge the PR and officially announce the release in ~ half an hour if no

Re: [Discuss] Breaking API Change in 1.10.1

For release branches, it would be good to not exclude PublicEvolving from checking, by default [1]. There may be instances where it is necessary to make exceptions to fix a critical bug, but those could be whitelisted. Similarly, it might be good to blacklist dependency changes for patch

Re: [Discuss] Breaking API Change in 1.10.1

We are running japicmp in CI; we just don't check anything that is PublicEvolving. On 13/05/2020 18:04, Thomas Weise wrote: +1 for the release note addition Historically Flink X.Y.0 releases have been prone to issues like this. And I suspect that many users would prefer to upgrade to X.Y.1

Re: [Discuss] Breaking API Change in 1.10.1

+1 for the release note addition Historically Flink X.Y.0 releases have been prone to issues like this. And I suspect that many users would prefer to upgrade to X.Y.1 due to other bugs that need to be shaken out. For the future, it would be great to eliminate the possibility of incompatible

Re: [Discuss] Breaking API Change in 1.10.1

Thanks Seth, comment received, will add it into the release note. Best Regards, Yu On Wed, 13 May 2020 at 22:54, Seth Wiesman wrote: > +1 Ufuk's pragmatic solution to update the release notes with a public > notice, I have commented on the release notes PR. > >

Re: [Discuss] Breaking API Change in 1.10.1

+1 Ufuk's pragmatic solution to update the release notes with a public notice, I have commented on the release notes PR. https://github.com/apache/flink-web/pull/330 Seth On Wed, May 13, 2020 at 8:30 AM Chesnay Schepler wrote: > IIRC @PublicEvolving had no /real/ guarantees, from the 1.0.0

[jira] [Created] (FLINK-17674) OperatorCoordinator state in checkpoints should always be a ByteStreamStateHandle

Stephan Ewen created FLINK-17674: Summary: OperatorCoordinator state in checkpoints should always be a ByteStreamStateHandle Key: FLINK-17674 URL: https://issues.apache.org/jira/browse/FLINK-17674

[jira] [Created] (FLINK-17673) Add failure reason to OperatorCoordinator.failTask(...)

Stephan Ewen created FLINK-17673: Summary: Add failure reason to OperatorCoordinator.failTask(...) Key: FLINK-17673 URL: https://issues.apache.org/jira/browse/FLINK-17673 Project: Flink

[jira] [Created] (FLINK-17672) OperatorCoordinators receive failure notifications on task failurenstead of restarts

Stephan Ewen created FLINK-17672: Summary: OperatorCoordinators receive failure notifications on task failurenstead of restarts Key: FLINK-17672 URL: https://issues.apache.org/jira/browse/FLINK-17672

[jira] [Created] (FLINK-17671) Simplify the ManuallyTriggeredScheduledExecutortriggering logic

Stephan Ewen created FLINK-17671: Summary: Simplify the ManuallyTriggeredScheduledExecutortriggering logic Key: FLINK-17671 URL: https://issues.apache.org/jira/browse/FLINK-17671 Project: Flink

[jira] [Created] (FLINK-17670) Savepoint handling of "non-restored" state should also take OperatorCoordinator state into account

Stephan Ewen created FLINK-17670: Summary: Savepoint handling of "non-restored" state should also take OperatorCoordinator state into account Key: FLINK-17670 URL:

[NOTICE] Deletion of older releases from dist.apache.org

Hello, I have just amended the release guide to specify that older releases should only be removed from dist.apache.org after the release process has been completed. Our current approach of deleting them when a new

Re: [Discuss] Breaking API Change in 1.10.1

IIRC @PublicEvolving had no /real/ guarantees, from the 1.0.0 announcement: /"Flink 1.0.0 introduces interface stability annotations for API classes and methods. Interfaces defined as //|@Public|//are guaranteed to remain stable across all releases of the 1.x series. The

[jira] [Created] (FLINK-17669) Use new WatermarkStrategy/WatermarkGenerator in Kafka connector

Aljoscha Krettek created FLINK-17669: Summary: Use new WatermarkStrategy/WatermarkGenerator in Kafka connector Key: FLINK-17669 URL: https://issues.apache.org/jira/browse/FLINK-17669 Project:

Re: [Discuss] Breaking API Change in 1.10.1

A small addendum: The project currently enforces only binary compatibility for classes which are annotated with @Public. The StreamingFileSink is annotated with @PublicEvolving. I am not sure whether the community properly defined what kind of stability/compatibility guarantees we provide for

[jira] [Created] (FLINK-17667) Implement INSERT for Hive dialect

Rui Li created FLINK-17667: -- Summary: Implement INSERT for Hive dialect Key: FLINK-17667 URL: https://issues.apache.org/jira/browse/FLINK-17667 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17668) Fix shortcomings in new data structures

Timo Walther created FLINK-17668: Summary: Fix shortcomings in new data structures Key: FLINK-17668 URL: https://issues.apache.org/jira/browse/FLINK-17668 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17666) Insert into partitioned table can fail with select *

Rui Li created FLINK-17666: -- Summary: Insert into partitioned table can fail with select * Key: FLINK-17666 URL: https://issues.apache.org/jira/browse/FLINK-17666 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17665) Serialize buffer data type of Buffer into BufferResponse

Yingjie Cao created FLINK-17665: --- Summary: Serialize buffer data type of Buffer into BufferResponse Key: FLINK-17665 URL: https://issues.apache.org/jira/browse/FLINK-17665 Project: Flink Issue

[jira] [Created] (FLINK-17664) Introduce print, blackhole connector in table

Jingsong Lee created FLINK-17664: Summary: Introduce print, blackhole connector in table Key: FLINK-17664 URL: https://issues.apache.org/jira/browse/FLINK-17664 Project: Flink Issue Type:

[jira] [Created] (FLINK-17663) CheckpointBarrierUnaligner.getFlattenedChannelIndex can throw ArrayIndexOutOfBoundsException

Piotr Nowojski created FLINK-17663: -- Summary: CheckpointBarrierUnaligner.getFlattenedChannelIndex can throw ArrayIndexOutOfBoundsException Key: FLINK-17663 URL: https://issues.apache.org/jira/browse/FLINK-17663

Re: [Discuss] Breaking API Change in 1.10.1

Thanks for the analysis Till. 1/ I think breaking binary compatibility is acceptable between minor releases (1.10 -> 1.11) as you suggested since the API is marked as @PublicEvolving. 2/ I'm quite torn about how to proceed with the 1.10.x patch release, but slightly leaning towards the

[jira] [Created] (FLINK-17662) testDetachedModeSecureWithPreInstallKeytab: "The JobManager and the TaskManager should both run with Kerberos."

Robert Metzger created FLINK-17662: -- Summary: testDetachedModeSecureWithPreInstallKeytab: "The JobManager and the TaskManager should both run with Kerberos." Key: FLINK-17662 URL:

Re: Reading parquet from S3

Hi Kirill, For DataSet API, yes, we have "ParquetRowInputFormat", "ParquetPojoInputFormat" and "ParquetMapInputFormat" [1]. For file system parquet files in table: - Before 1.11, you can register a ParquetTableSource to batch table environment with legacy planner. - In 1.11, you can create file

GenericType; Actual: LocalDateTime

Dear: Why no stable common examples using flink 1.9.3 about streaming sql with rowtime... Thanks for helping. flink1.9.3 root |-- error_code: STRING |-- window_start: TIMESTAMP(3) *ROWTIME* |-- window_end: TIMESTAMP(3) *ROWTIME* |-- total_num: BIGINT NOT NULL POJO's type is

GenericType; Actual: LocalDateTime

Dear: Why no stable common examples using flink 1.9.3 about streaming sql with rowtime... Thanks for helping. flink1.9.3 root |-- error_code: STRING |-- window_start: TIMESTAMP(3) *ROWTIME* |-- window_end: TIMESTAMP(3) *ROWTIME* |-- total_num: BIGINT NOT NULL POJO's type is TIMESTAMP in

Reading parquet from S3

Hi, I'm writing an article about Apache Flink. I wanted to ask, does Apache Flink supports reading Parquet files in Batching mode? Our team is going to read parquet files from S3. I didn't find such page in flink's tutorials. I will be grateful if you help, thank you -- [image:

[jira] [Created] (FLINK-17661) Add APIs for using new WatermarkStrategy/WatermarkGenerator

Aljoscha Krettek created FLINK-17661: Summary: Add APIs for using new WatermarkStrategy/WatermarkGenerator Key: FLINK-17661 URL: https://issues.apache.org/jira/browse/FLINK-17661 Project: Flink

[jira] [Created] (FLINK-17660) DataTypeExtractor does not check for default constructor

Timo Walther created FLINK-17660: Summary: DataTypeExtractor does not check for default constructor Key: FLINK-17660 URL: https://issues.apache.org/jira/browse/FLINK-17660 Project: Flink

Re: [Discuss] Breaking API Change in 1.10.1

Thanks for reporting this issue Seth. This is indeed a big problem. I looked into the problem and it seems we have the following situation: # 1.9.x --> 1.10.0 There is an API breaking change between 1.9.x and 1.10 due FLINK-13864 because it introduced another generic parameter. I expected only

Re: [DISCUSS] Rework History Server into Global Dashboard

It seems that not everyone can see the screenshot in the email, so here is a link: https://drive.google.com/open?id=1abrlpI976NFqOZSX20k2FoiAfVhBbER9 On Wed, May 13, 2020 at 11:29 AM Gyula Fóra wrote: > Oops I forgot the screenshot, thanks Ufuk :D > > > @Jeff Zhang : Yes we simply call to the

[jira] [Created] (FLINK-17659) Add common watermark strategies and WatermarkStrategies helper

Aljoscha Krettek created FLINK-17659: Summary: Add common watermark strategies and WatermarkStrategies helper Key: FLINK-17659 URL: https://issues.apache.org/jira/browse/FLINK-17659 Project:

Re: [DISCUSS] Rework History Server into Global Dashboard

Oops I forgot the screenshot, thanks Ufuk :D @Jeff Zhang : Yes we simply call to the individual cluster's rest endpoints so it would work with multiple flink versions yes. Gyula On Wed, May 13, 2020 at 10:56 AM Jeff Zhang wrote: > Hi Gyula, > > Big +1 for this, it would be very helpful for

[jira] [Created] (FLINK-17658) Add new TimestampAssigner and WatermarkGenerator interfaces

Aljoscha Krettek created FLINK-17658: Summary: Add new TimestampAssigner and WatermarkGenerator interfaces Key: FLINK-17658 URL: https://issues.apache.org/jira/browse/FLINK-17658 Project: Flink

[jira] [Created] (FLINK-17657) jdbc not support read BIGINT UNSIGNED field

lun zhang created FLINK-17657: - Summary: jdbc not support read BIGINT UNSIGNED field Key: FLINK-17657 URL: https://issues.apache.org/jira/browse/FLINK-17657 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17656) Migrate e2e tests to flink-docker

Chesnay Schepler created FLINK-17656: Summary: Migrate e2e tests to flink-docker Key: FLINK-17656 URL: https://issues.apache.org/jira/browse/FLINK-17656 Project: Flink Issue Type:

[jira] [Created] (FLINK-17655) Remove old and long deprecated TimestampExtractor

Aljoscha Krettek created FLINK-17655: Summary: Remove old and long deprecated TimestampExtractor Key: FLINK-17655 URL: https://issues.apache.org/jira/browse/FLINK-17655 Project: Flink

[jira] [Created] (FLINK-17654) Move Clock classes to flink-core to make them usable outside runtime

Aljoscha Krettek created FLINK-17654: Summary: Move Clock classes to flink-core to make them usable outside runtime Key: FLINK-17654 URL: https://issues.apache.org/jira/browse/FLINK-17654

Re: [DISCUSS] Rework History Server into Global Dashboard

Hi Gyula, Big +1 for this, it would be very helpful for flink jobs and cluster operations. Do you call flink rest api to gather the job info ? I hope this history server could work with multiple versions of flink as long as the flink rest api is compatible. Gyula Fóra 于2020年5月13日周三 下午4:13写道： >

[jira] [Created] (FLINK-17653) FLIP-126: Unify (and separate) Watermark Assigners

Aljoscha Krettek created FLINK-17653: Summary: FLIP-126: Unify (and separate) Watermark Assigners Key: FLINK-17653 URL: https://issues.apache.org/jira/browse/FLINK-17653 Project: Flink

[DISCUSS] Rework History Server into Global Dashboard

Hi All! With the growing number of Flink streaming applications the current HS implementation is starting to lose its value. Users running streaming applications mostly care about what is running right now on the cluster and a centralised view on history is not very useful. We have been

[jira] [Created] (FLINK-17652) Legacy JM heap options should fallback to new JVM_HEAP_MEMORY in standalone

Andrey Zagrebin created FLINK-17652: --- Summary: Legacy JM heap options should fallback to new JVM_HEAP_MEMORY in standalone Key: FLINK-17652 URL: https://issues.apache.org/jira/browse/FLINK-17652

[jira] [Created] (FLINK-17651) DecomposeGroupingSetsRule generates wrong plan when there exist distinct agg and simple agg with same filter

Shuo Cheng created FLINK-17651: -- Summary: DecomposeGroupingSetsRule generates wrong plan when there exist distinct agg and simple agg with same filter Key: FLINK-17651 URL:

[jira] [Created] (FLINK-17650) Impossible to set Security related configs as dynamic properties

Gyula Fora created FLINK-17650: -- Summary: Impossible to set Security related configs as dynamic properties Key: FLINK-17650 URL: https://issues.apache.org/jira/browse/FLINK-17650 Project: Flink

[jira] [Created] (FLINK-17649) Generated hash aggregate code may produce NPE when there exists an aggregate call with Filter.

Shuo Cheng created FLINK-17649: -- Summary: Generated hash aggregate code may produce NPE when there exists an aggregate call with Filter. Key: FLINK-17649 URL: https://issues.apache.org/jira/browse/FLINK-17649