[jira] [Created] (FLINK-17150) Introduce Canal format to support reading canal changelogs

2020-04-14 Thread Jark Wu (Jira)
Jark Wu created FLINK-17150: --- Summary: Introduce Canal format to support reading canal changelogs Key: FLINK-17150 URL: https://issues.apache.org/jira/browse/FLINK-17150 Project: Flink Issue Type:

[jira] [Created] (FLINK-17149) Introduce Debezium format to support reading debezium changelogs

2020-04-14 Thread Jark Wu (Jira)
Jark Wu created FLINK-17149: --- Summary: Introduce Debezium format to support reading debezium changelogs Key: FLINK-17149 URL: https://issues.apache.org/jira/browse/FLINK-17149 Project: Flink

[jira] [Created] (FLINK-17148) Support converting pandas dataframe to flink table

2020-04-14 Thread Dian Fu (Jira)
Dian Fu created FLINK-17148: --- Summary: Support converting pandas dataframe to flink table Key: FLINK-17148 URL: https://issues.apache.org/jira/browse/FLINK-17148 Project: Flink Issue Type:

[jira] [Created] (FLINK-17147) Remove dynamic table options from table source table digest

2020-04-14 Thread Danny Chen (Jira)
Danny Chen created FLINK-17147: -- Summary: Remove dynamic table options from table source table digest Key: FLINK-17147 URL: https://issues.apache.org/jira/browse/FLINK-17147 Project: Flink

[RESULT][VOTE] FLIP-105: Support to Interpret and Emit Changelog in Flink SQL

2020-04-14 Thread Jark Wu
Hi all, The voting time for FLIP-105 has passed. I'm closing the vote now. There were 5 +1 votes, 3 of which are binding: - Benchao (non-binding) - Jark (binding) - Jingsong Li (binding) - zoudan (non-binding) - Kurt (binding) There were no disapproving votes. Thus, FLIP-105 has been

[jira] [Created] (FLINK-17146) Support conversion between PyFlink Table and Pandas DataFrame

2020-04-14 Thread Dian Fu (Jira)
Dian Fu created FLINK-17146: --- Summary: Support conversion between PyFlink Table and Pandas DataFrame Key: FLINK-17146 URL: https://issues.apache.org/jira/browse/FLINK-17146 Project: Flink Issue

Re: [VOTE] FLIP-123: DDL and DML compatibility for Hive connector

2020-04-14 Thread Rui Li
Hey Rong, Thanks for the reminder. I have updated the main page to add a row for FLIP-123. On Tue, Apr 14, 2020 at 11:02 PM Rong Rong wrote: > Great feature, I kinda like how the dialect / conformance is handled. > > +1. also could you please update the FLIP-123 link and add a >

[jira] [Created] (FLINK-17145) AggregateRemoveITCase.testAggregateRemove failed in travis

2020-04-14 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-17145: Summary: AggregateRemoveITCase.testAggregateRemove failed in travis Key: FLINK-17145 URL: https://issues.apache.org/jira/browse/FLINK-17145 Project: Flink

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-14 Thread Till Rohrmann
Thanks for updating the FLIP, Yangze. If ExternalResourceInfo is a marker interface, then ExternalResourceDriver could return a Set. This makes is a bit nicer for the implementor because he can use the concrete subtype. If we assume that users will always cast the ExternalResourceInfo instance

[jira] [Created] (FLINK-17144) Input type generalization does not work correctly for ARRAY and MAP in sql

2020-04-14 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-17144: Summary: Input type generalization does not work correctly for ARRAY and MAP in sql Key: FLINK-17144 URL: https://issues.apache.org/jira/browse/FLINK-17144

[jira] [Created] (FLINK-17143) Blog feed.xml should only contain excerpts, not full contents

2020-04-14 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-17143: --- Summary: Blog feed.xml should only contain excerpts, not full contents Key: FLINK-17143 URL: https://issues.apache.org/jira/browse/FLINK-17143 Project: Flink

Re: flink-orc or flink-orc-nohive

2020-04-14 Thread Sivaprasanna
I have created a ticket to update the ORC version. https://issues.apache.org/jira/browse/FLINK-17142 On Tue, Apr 14, 2020 at 8:18 PM Jingsong Li wrote: > Hi, yes, we can bump orc-core version to a newer. > > Best, > Jingsong Lee > > On Tue, Apr 14, 2020 at 8:16 PM Sivaprasanna > wrote: > > >

[jira] [Created] (FLINK-17142) Bump ORC version

2020-04-14 Thread Sivaprasanna Sethuraman (Jira)
Sivaprasanna Sethuraman created FLINK-17142: --- Summary: Bump ORC version Key: FLINK-17142 URL: https://issues.apache.org/jira/browse/FLINK-17142 Project: Flink Issue Type:

[jira] [Created] (FLINK-17141) Name of SQL Operator is too long

2020-04-14 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-17141: --- Summary: Name of SQL Operator is too long Key: FLINK-17141 URL: https://issues.apache.org/jira/browse/FLINK-17141 Project: Flink Issue Type: Improvement

Re: [VOTE] FLIP-123: DDL and DML compatibility for Hive connector

2020-04-14 Thread Rong Rong
Great feature, I kinda like how the dialect / conformance is handled. +1. also could you please update the FLIP-123 link and add a summary/description row in the main page[1] please? Thanks, Rong -- [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals On Tue, Apr

Re: flink-orc or flink-orc-nohive

2020-04-14 Thread Jingsong Li
Hi, yes, we can bump orc-core version to a newer. Best, Jingsong Lee On Tue, Apr 14, 2020 at 8:16 PM Sivaprasanna wrote: > On a similar note, I just checked that the Flink currently uses orc 1.4.3 > in the dependencies. IMO, it is a little outdated. Can we bump the ORC > version to a slightly

Re: [VOTE] FLIP-123: DDL and DML compatibility for Hive connector

2020-04-14 Thread Jingsong Li
+1 Best, Jingsong Lee On Tue, Apr 14, 2020 at 9:29 PM Kurt Young wrote: > +1 > > Best, > Kurt > > > On Mon, Apr 13, 2020 at 9:26 PM Rui Li wrote: > > > Hi all, > > > > I'd like to start the vote for FLIP-123[1], which has been discussed in > > thread[2]. > > > > The vote will be open for 72h,

[jira] [Created] (FLINK-17140) AsyncLookupJoinITCase.testAsyncJoinTemporalTableOnMultiFieldsWithUdf failed on Azure

2020-04-14 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17140: - Summary: AsyncLookupJoinITCase.testAsyncJoinTemporalTableOnMultiFieldsWithUdf failed on Azure Key: FLINK-17140 URL: https://issues.apache.org/jira/browse/FLINK-17140

Re: [VOTE] FLIP-123: DDL and DML compatibility for Hive connector

2020-04-14 Thread Kurt Young
+1 Best, Kurt On Mon, Apr 13, 2020 at 9:26 PM Rui Li wrote: > Hi all, > > I'd like to start the vote for FLIP-123[1], which has been discussed in > thread[2]. > > The vote will be open for 72h, until April 16 2020 01:00 PM UTC, unless > there's an objection. > > [1] > >

[jira] [Created] (FLINK-17139) The sub-plan reuse optimizer can't reuse sub-plan that from different sql node tree 

2020-04-14 Thread jinfeng (Jira)
jinfeng created FLINK-17139: --- Summary: The sub-plan reuse optimizer can't reuse sub-plan that from different sql node tree  Key: FLINK-17139 URL: https://issues.apache.org/jira/browse/FLINK-17139 Project:

[jira] [Created] (FLINK-17138) LocalExecutorITCase.testParameterizedTypes failed on travis

2020-04-14 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-17138: -- Summary: LocalExecutorITCase.testParameterizedTypes failed on travis Key: FLINK-17138 URL: https://issues.apache.org/jira/browse/FLINK-17138 Project: Flink

[jira] [Created] (FLINK-17137) Support mini batch for WindowOperator in blink planner

2020-04-14 Thread Benchao Li (Jira)
Benchao Li created FLINK-17137: -- Summary: Support mini batch for WindowOperator in blink planner Key: FLINK-17137 URL: https://issues.apache.org/jira/browse/FLINK-17137 Project: Flink Issue

[jira] [Created] (FLINK-17136) Rename toplevel DataSet/DataStream section titles

2020-04-14 Thread Aljoscha Krettek (Jira)
Aljoscha Krettek created FLINK-17136: Summary: Rename toplevel DataSet/DataStream section titles Key: FLINK-17136 URL: https://issues.apache.org/jira/browse/FLINK-17136 Project: Flink

Re: flink-orc or flink-orc-nohive

2020-04-14 Thread Sivaprasanna
On a similar note, I just checked that the Flink currently uses orc 1.4.3 in the dependencies. IMO, it is a little outdated. Can we bump the ORC version to a slightly newer version - maybe 1.5.x or even 1.6.0? - Sivaprasanna On Tue, Apr 14, 2020 at 1:42 PM Jingsong Li wrote: > Hi, > > Maybe

[jira] [Created] (FLINK-17135) PythonCalcSplitRuleTest.testPandasFunctionMixedWithGeneralPythonFunction failed on master

2020-04-14 Thread Dian Fu (Jira)
Dian Fu created FLINK-17135: --- Summary: PythonCalcSplitRuleTest.testPandasFunctionMixedWithGeneralPythonFunction failed on master Key: FLINK-17135 URL: https://issues.apache.org/jira/browse/FLINK-17135

[jira] [Created] (FLINK-17134) Wrong logging information in Kafka010PartitionDiscoverer#L80

2020-04-14 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-17134: -- Summary: Wrong logging information in Kafka010PartitionDiscoverer#L80 Key: FLINK-17134 URL: https://issues.apache.org/jira/browse/FLINK-17134 Project: Flink

[jira] [Created] (FLINK-17133) Support CsvParser Features for CSV format via SQL DDL

2020-04-14 Thread Kai Chen (Jira)
Kai Chen created FLINK-17133: Summary: Support CsvParser Features for CSV format via SQL DDL Key: FLINK-17133 URL: https://issues.apache.org/jira/browse/FLINK-17133 Project: Flink Issue Type:

[jira] [Created] (FLINK-17132) Use the latest version of simpleclient_pushgateway to solve the "Failed to push metrics to PushGateway" problem

2020-04-14 Thread MING LIU (Jira)
MING LIU created FLINK-17132: Summary: Use the latest version of simpleclient_pushgateway to solve the "Failed to push metrics to PushGateway" problem Key: FLINK-17132 URL:

[jira] [Created] (FLINK-17131) Javadocs broken for master

2020-04-14 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-17131: Summary: Javadocs broken for master Key: FLINK-17131 URL: https://issues.apache.org/jira/browse/FLINK-17131 Project: Flink Issue Type: Bug

Re: Javadocs Broken?

2020-04-14 Thread Chesnay Schepler
I'm looking into it. On 10/04/2020 11:27, tison wrote: Hi guys, Right now when I click "JavaDocs" in out docsite[1] it jumps to a page[2] I think is definitely not out api documentation. Any thoughts? Best, tison. [1] https://ci.apache.org/projects/flink/flink-docs-master/ [2]

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-14 Thread Tzu-Li (Gordon) Tai
+1 on creating a new repo, and migrating the flink-benchmarks there. Regarding Chesnay's concern whether this requires a vote: While the flink-benchmarks is indeed an existing external codebase, I'm not entirely sure we do need a vote here. The bylaws state that: "Adoption of large existing

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-14 Thread Chesnay Schepler
Isn't this technically the adoption of a new codebase? On 14/04/2020 10:06, Yun Tang wrote: Hi Flink devs Thanks for the feedbacks! Overall, everyone who replied is positive about creating such a separate repo to host the flink benchmarks. Since such an action does not necessarily need a

[jira] [Created] (FLINK-17130) Web UI: Enable listing JM Logs and displaying Logs by Filename

2020-04-14 Thread Yadong Xie (Jira)
Yadong Xie created FLINK-17130: -- Summary: Web UI: Enable listing JM Logs and displaying Logs by Filename Key: FLINK-17130 URL: https://issues.apache.org/jira/browse/FLINK-17130 Project: Flink

Re: [DISCUSS] FLIP-124: Add open/close and Collector to (De)SerializationSchema

2020-04-14 Thread Dawid Wysakowicz
Hi Xiaogang, I very much agree with Jark's and Aljoscha's responses. On 10/04/2020 17:35, Jark Wu wrote: > Hi Xiaogang, > > I think this proposal doesn't conflict with your use case, you can still > chain a ProcessFunction after a source which emits raw data. > But I'm not in favor of chaining

Re: flink-orc or flink-orc-nohive

2020-04-14 Thread Jingsong Li
Hi, Maybe you should use flink-orc. And use orc-core instead of orc-core with nohive classifier. We can provide nohive version in the future. Because orc and hive are so close, orc still relies on some classes of hive currently. Apache orc with nohive classifier is for create a variant of core

Re: [DISCUSS] Creating a new repo to host Flink benchmarks

2020-04-14 Thread Yun Tang
Hi Flink devs Thanks for the feedbacks! Overall, everyone who replied is positive about creating such a separate repo to host the flink benchmarks. Since such an action does not necessarily need a vote (as defined by the project bylaws), I'll proceed to ask PMC's help to create the repo under

Re: [DISCUSS] FLIP-124: Add open/close and Collector to (De)SerializationSchema

2020-04-14 Thread Aljoscha Krettek
On 10.04.20 17:35, Jark Wu wrote: 1) For correctness, it is necessary to perform the watermark generation as early as possible in order to be close to the actual data generation within a source's data partition. This is also the purpose of per-partition watermark and event-time alignment.

Re: [DISCUSS] FLIP-118: Improve Flink’s ID system

2020-04-14 Thread Zhu Zhu
Thanks for doing this benchmark @Yangze Guo . The result looks promising. So I would +1 to refactor ExecutionAttemptID and IntermediateResultPartitionID. Regarding why 'The size of TDD after serialization become smaller than before', I guess it's because the new IntermediateResultPartitionIDs

flink-orc or flink-orc-nohive

2020-04-14 Thread Sivaprasanna
Hello, I'm working on an implementation of ORC BulkWriter[1]. As of now, I have the entire implementation in a separate module called "flink-orc-compress" under "flink-formats" since I'm not entirely sure whether it should go into the existing ORC modules i.e flink-orc & flink-orc-nohive. So my

Re: [VOTE] FLIP-105: Support to Interpret and Emit Changelog in Flink SQL

2020-04-14 Thread Kurt Young
+1 Best, Kurt On Tue, Apr 14, 2020 at 3:26 PM zoudan wrote: > +1 (non-binding) > > Best, > Dan Zou > > >

Re: [VOTE] FLIP-105: Support to Interpret and Emit Changelog in Flink SQL

2020-04-14 Thread zoudan
+1 (non-binding) Best, Dan Zou

Re: [DISCUSS] FLIP-118: Improve Flink’s ID system

2020-04-14 Thread Yangze Guo
Hi everyone, I've investigated the infect with higher parallelism jobs. The result shows: - The size of TDD after serialization become smaller than before. While I did not meet any issue with Akka framework when the parallelism set to 6000. - There was no significant difference regarding the end

[jira] [Created] (FLINK-17129) 【Flink Kafka Connector】The DDL config(connector.properties.zookeeper.connect) could be optioanal

2020-04-14 Thread zhisheng (Jira)
zhisheng created FLINK-17129: Summary: 【Flink Kafka Connector】The DDL config(connector.properties.zookeeper.connect) could be optioanal Key: FLINK-17129 URL: https://issues.apache.org/jira/browse/FLINK-17129

[jira] [Created] (FLINK-17128) Add isBounded to TableSinkFactory#Context

2020-04-14 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-17128: Summary: Add isBounded to TableSinkFactory#Context Key: FLINK-17128 URL: https://issues.apache.org/jira/browse/FLINK-17128 Project: Flink Issue Type:

Re: [VOTE] FLIP-105: Support to Interpret and Emit Changelog in Flink SQL

2020-04-14 Thread Jingsong Li
+1 Best, Jingsong Lee On Tue, Apr 14, 2020 at 1:46 PM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Sun, 12 Apr 2020 at 09:24, Benchao Li wrote: > > > +1 (non-binding) > > > > Jark Wu 于2020年4月11日周六 上午11:31写道: > > > > > Hi all, > > > > > > I would like to start the vote for FLIP-105