[jira] [Created] (FLINK-16689) kafka connector as source get byte deserialize add support charsets

2020-03-19 Thread xiaodao (Jira)
xiaodao created FLINK-16689: --- Summary: kafka connector as source get byte deserialize add support charsets Key: FLINK-16689 URL: https://issues.apache.org/jira/browse/FLINK-16689 Project: Flink

[DISCUSS] Introduce TableFactory for StatefulSequenceSource

2020-03-19 Thread Jingsong Li
Hi all, I heard some users complain that table is difficult to test. Now with SQL client, users are more and more inclined to use it to test rather than program. The most common example is Kafka source. If users need to test their SQL output and checkpoint, they need to: - 1.Launch a Kafka

[jira] [Created] (FLINK-16688) Hive-connector should set SessionState for hive

2020-03-19 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-16688: Summary: Hive-connector should set SessionState for hive Key: FLINK-16688 URL: https://issues.apache.org/jira/browse/FLINK-16688 Project: Flink Issue Type:

[jira] [Created] (FLINK-16687) PyFlink Cannot determine simple type name "PythonScalarFunction$0"

2020-03-19 Thread mayne wong (Jira)
mayne wong created FLINK-16687: -- Summary: PyFlink Cannot determine simple type name "PythonScalarFunction$0" Key: FLINK-16687 URL: https://issues.apache.org/jira/browse/FLINK-16687 Project: Flink

Re: [DISCUSS] FLIP-108: Add GPU support in Flink

2020-03-19 Thread Xintong Song
Thanks for the feedback, Becket. IMO, eventually an operator should only see info of GPUs that are dedicated for it, instead of all GPUs on the machine/container in the current design. It does not make sense to let the user who writes a UDF to worry about coordination among multiple operators

[jira] [Created] (FLINK-16686) [State TTL] Make user class loader available in native RocksDB compaction thread

2020-03-19 Thread Andrey Zagrebin (Jira)
Andrey Zagrebin created FLINK-16686: --- Summary: [State TTL] Make user class loader available in native RocksDB compaction thread Key: FLINK-16686 URL: https://issues.apache.org/jira/browse/FLINK-16686

Re: Watermark generation in Temporal Table Join

2020-03-19 Thread Dominik Wosiński
Hey, Sure, I have create something that can be called a minimal reproducible example. It's not the prettiest since it uses a lot of *Thread.sleep* but it allows to be sure that the input is exactly what you want. https://github.com/DomWos/FlinkTTF/tree/long-vs-timestamp In the long-vs-timestamp

[jira] [Created] (FLINK-16685) Add an example of how to deploy a multi-language function to Kubernetes.

2020-03-19 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-16685: Summary: Add an example of how to deploy a multi-language function to Kubernetes. Key: FLINK-16685 URL: https://issues.apache.org/jira/browse/FLINK-16685 Project:

[jira] [Created] (FLINK-16684) StreamingFileSink builder does not work with Scala

2020-03-19 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-16684: - Summary: StreamingFileSink builder does not work with Scala Key: FLINK-16684 URL: https://issues.apache.org/jira/browse/FLINK-16684 Project: Flink Issue

Re: Hope to be Flink contributor

2020-03-19 Thread Xingbo Huang
Hi, Welcome you to join the Flink community. Currently, you do not need a JIRA permission. Only committers can assign a Jira ticket. For how to participate in community contributions, you can refer to the official contribution guide[1] and correspoinding Chinese doc[2] Best, Xingbo [1]

[DISCUSS] Introduce a new module 'flink-hadoop-utils'

2020-03-19 Thread Sivaprasanna
Hi, Continuing on an earlier discussion[1] regarding having a separate module for Hadoop related utility components, I have gone through our project briefly and found the following components which I feel could be moved to a separate module for reusability, and better module structure. Module

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

2020-03-19 Thread Robert Metzger
Thank you for your opinions. I filed a ticket for this: https://issues.apache.org/jira/browse/FLINK-16683 On Tue, Mar 17, 2020 at 3:34 PM Till Rohrmann wrote: > +1 for removing them. > > On Wed, Mar 11, 2020 at 10:06 AM Chesnay Schepler > wrote: > > > +1 on removing them. > > > > They are so

[jira] [Created] (FLINK-16683) Remove scripts for starting Flink on Windows

2020-03-19 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-16683: -- Summary: Remove scripts for starting Flink on Windows Key: FLINK-16683 URL: https://issues.apache.org/jira/browse/FLINK-16683 Project: Flink Issue Type:

[jira] [Created] (FLINK-16682) Make batch size configurable for orc reader

2020-03-19 Thread Rui Li (Jira)
Rui Li created FLINK-16682: -- Summary: Make batch size configurable for orc reader Key: FLINK-16682 URL: https://issues.apache.org/jira/browse/FLINK-16682 Project: Flink Issue Type: Task

The question about the FLIP-45

2020-03-19 Thread LakeShen
Hi community, Now I am reading the FLIP-45 Reinforce Job Stop Semantic, I have three questions about it : 1. What the command to use to stop the Flink task, stop or cancel? 2. If use stop command to stop filnk task , but I see the flink source code , the stop command we can set the savepoint dir

Re: [Discussion] Job generation / submission hooks & Atlas integration

2020-03-19 Thread jackylau
Hi: i think flink integrate atlas also need add catalog information such as spark atlas project .https://github.com/hortonworks-spark/spark-atlas-connector when user use catalog such as JDBCCatalog/HiveCatalog, flink atlas project will sync this information to atlas. But i don't find any

[jira] [Created] (FLINK-16681) Jdbc JDBCOutputFormat and JDBCLookupFunction PreparedStatement loss connection, if long time not records to write.

2020-03-19 Thread LakeShen (Jira)
LakeShen created FLINK-16681: Summary: Jdbc JDBCOutputFormat and JDBCLookupFunction PreparedStatement loss connection, if long time not records to write. Key: FLINK-16681 URL:

[jira] [Created] (FLINK-16680) Add documentation for Stateful Function's State Bootstrapping API

2020-03-19 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-16680: --- Summary: Add documentation for Stateful Function's State Bootstrapping API Key: FLINK-16680 URL: https://issues.apache.org/jira/browse/FLINK-16680

Re: [DISCUSS] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-19 Thread Andrey Zagrebin
Alright, thanks for the feedback. I also agree with it. Then this is resolved. > On 19 Mar 2020, at 14:14, Till Rohrmann wrote: > > I agree with Xintong's proposal. If we see that many users run into this > problem, then one could think about escalating the warning message into a > failure. >

Re: [DISCUSS] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-19 Thread Till Rohrmann
I agree with Xintong's proposal. If we see that many users run into this problem, then one could think about escalating the warning message into a failure. Cheers, Till On Thu, Mar 19, 2020 at 4:23 AM Xintong Song wrote: > I think recommend a minimum value in docs and throw a warning if the

Re: [VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-19 Thread Till Rohrmann
+1 (binding) Cheers, Till On Thu, Mar 19, 2020 at 7:29 AM Yang Wang wrote: > +1 (non-binding). > > This FLIP will make jobmanager memory configuration keep up with > taskmanager. > > > Best, > Yang > > Xintong Song 于2020年3月19日周四 上午11:23写道: > > > +1 (non-binding) from my side. > > > > Thank

Hope to be Flink contributor

2020-03-19 Thread 奇天下
Hi Guys: I want to contribute to Apache Flink. Would you please give me the permission as a contributor? My jira username is luck monkey. I am a beginner for Flink, so i want to start with simple ticket. With the deepening of learning, i prefer Flink runtime、 table、sql and so on

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-03-19 Thread Yadong Xie
Hi all I have updated the design of the metric page and FLIP doc, please let me know what you think about it FLIP-102: https://cwiki.apache.org/confluence/display/FLINK/FLIP-102%3A+Add+More+Metrics+to+TaskManager POC web:

[jira] [Created] (FLINK-16679) Investigate more efficient outer join plans

2020-03-19 Thread Timo Walther (Jira)
Timo Walther created FLINK-16679: Summary: Investigate more efficient outer join plans Key: FLINK-16679 URL: https://issues.apache.org/jira/browse/FLINK-16679 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-108: Add GPU support in Flink

2020-03-19 Thread Becket Qin
It probably make sense for us to first agree on the final state. More specifically, will the resource info be exposed through runtime context eventually? If that is the final state and we have a seamless migration story from this FLIP to that final state, Personally I think it is OK to expose the

[jira] [Created] (FLINK-16678) Hbase Connector Doc Error

2020-03-19 Thread sulei (Jira)
sulei created FLINK-16678: - Summary: Hbase Connector Doc Error Key: FLINK-16678 URL: https://issues.apache.org/jira/browse/FLINK-16678 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-16677) Make Stateful Functions more prominent in the Flink website.

2020-03-19 Thread Marta Paes Moreira (Jira)
Marta Paes Moreira created FLINK-16677: -- Summary: Make Stateful Functions more prominent in the Flink website. Key: FLINK-16677 URL: https://issues.apache.org/jira/browse/FLINK-16677 Project:

[jira] [Created] (FLINK-16676) test_pipeline_from_invalid_json failed on Azure

2020-03-19 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-16676: -- Summary: test_pipeline_from_invalid_json failed on Azure Key: FLINK-16676 URL: https://issues.apache.org/jira/browse/FLINK-16676 Project: Flink Issue

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-19 Thread Guowei Ma
Hi, I am very interested in the topic. I would like to join the offline discussion if possible. I think you guys already give many inputs and concerns. I would share some of my thoughts. Correct me if I misunderstand you. Flink is a unified engine. Since that Flink should provide the e2e

[jira] [Created] (FLINK-16675) TableEnvironmentITCase. testClearOperation fails on travis nightly build

2020-03-19 Thread Yu Li (Jira)
Yu Li created FLINK-16675: - Summary: TableEnvironmentITCase. testClearOperation fails on travis nightly build Key: FLINK-16675 URL: https://issues.apache.org/jira/browse/FLINK-16675 Project: Flink

Re: [VOTE] FLIP 116: Unified Memory Configuration for Job Managers

2020-03-19 Thread Yang Wang
+1 (non-binding). This FLIP will make jobmanager memory configuration keep up with taskmanager. Best, Yang Xintong Song 于2020年3月19日周四 上午11:23写道: > +1 (non-binding) from my side. > > Thank you~ > > Xintong Song > > > > On Wed, Mar 18, 2020 at 10:16 PM Andrey Zagrebin > wrote: > > > Hi All, >

[jira] [Created] (FLINK-16674) Add documentation about how to use user-defined metrics for Python UDF

2020-03-19 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16674: --- Summary: Add documentation about how to use user-defined metrics for Python UDF Key: FLINK-16674 URL: https://issues.apache.org/jira/browse/FLINK-16674 Project: Flink

[jira] [Created] (FLINK-16673) Support Counter, Gauge, Meter, Distribution metric type for Python UDTF

2020-03-19 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16673: --- Summary: Support Counter, Gauge, Meter, Distribution metric type for Python UDTF Key: FLINK-16673 URL: https://issues.apache.org/jira/browse/FLINK-16673 Project: Flink

[jira] [Created] (FLINK-16672) Support Counter, Gauge, Meter, Distribution metric type for Python UDF

2020-03-19 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16672: --- Summary: Support Counter, Gauge, Meter, Distribution metric type for Python UDF Key: FLINK-16672 URL: https://issues.apache.org/jira/browse/FLINK-16672 Project: Flink

[jira] [Created] (FLINK-16671) Support define scopes and variables on Python metric group

2020-03-19 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16671: --- Summary: Support define scopes and variables on Python metric group Key: FLINK-16671 URL: https://issues.apache.org/jira/browse/FLINK-16671 Project: Flink

[jira] [Created] (FLINK-16670) Support User-Defined Metrics in Python UDF

2020-03-19 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16670: --- Summary: Support User-Defined Metrics in Python UDF Key: FLINK-16670 URL: https://issues.apache.org/jira/browse/FLINK-16670 Project: Flink Issue Type: New

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-19 Thread Hequn Cheng
Thanks all for the votes. So far, we have - 3 binding +1 votes (Jincheng, Dian, and Hequn) - 1 non-binding +1 votes (Wei) - 0 -1 votes I will move the FLIP to accepted! Best, Hequn On Thu, Mar 19, 2020 at 2:07 PM Hequn Cheng wrote: > +1 (binding) > > Best, Hequn > > On Thu, Mar 19,

Re: [VOTE] FLIP-112: Support User-Defined Metrics for Python UDF

2020-03-19 Thread Hequn Cheng
+1 (binding) Best, Hequn On Thu, Mar 19, 2020 at 10:39 AM Wei Zhong wrote: > +1 (non-binding) > > Best, > Wei > > > 在 2020年3月17日,19:15,Dian Fu 写道: > > > > +1 (binding) > > > > On Tue, Mar 17, 2020 at 10:35 AM jincheng sun > > wrote: > > > >> +1 > >> > >> Best, > >> Jincheng > >> > >> > >> >