[jira] [Created] (FLINK-10901) Jobmanager REST ip binds to ip address instead of hostname

2018-11-15 Thread Jeroen Steggink (JIRA)
Jeroen Steggink created FLINK-10901: --- Summary: Jobmanager REST ip binds to ip address instead of hostname Key: FLINK-10901 URL: https://issues.apache.org/jira/browse/FLINK-10901 Project: Flink

[jira] [Created] (FLINK-10902) Jobmanager in HA setup communicates the ip address instead of hostnames

2018-11-15 Thread Jeroen Steggink (JIRA)
Jeroen Steggink created FLINK-10902: --- Summary: Jobmanager in HA setup communicates the ip address instead of hostnames Key: FLINK-10902 URL: https://issues.apache.org/jira/browse/FLINK-10902

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-15 Thread Piotr Nowojski
Hi, One more thing. I think the Kafka client would be a good example of a connector that could use of this `isBlocked()`/callbacks single threaded API from the “Pattern 2” If we have N threads per N splits, there would be no need for the (N+1)th thread. It could be implemented as a non

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-15 Thread Steven Wu
> And each split has its own (internal) thread for reading from Kafka and putting messages in an internal queue to pull from. This is similar to how the current Kafka source is implemented, which has a separate fetcher thread. Aljoscha, in kafka case, one split may contain multiple kafka

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-15 Thread Piotr Nowojski
Hi Re: Becket > WRT the confusion between advance() / getCurrent(), do you think it would > help if we combine them and have something like: > > CompletableFuture getNext(); > long getWatermark(); > long getCurrentTimestamp(); I think that technically this would work the same as

Re: [DISCUSS] Table API Enhancement Outline

2018-11-15 Thread Piotr Nowojski
Hi, Isn’t the problem of multiple expressions limited only to `flat***` functions and to be more specific only to having two (or more) different table functions passed as an expressions? `.flatAgg(TableAggA('a), scalarFunction1(‘b), scalarFunction2(‘c))` seems to be well defined (duplicate

[jira] [Created] (FLINK-10907) Job recovery on the same JobManager causes JobManager metrics to report stale values

2018-11-15 Thread Mark Cho (JIRA)
Mark Cho created FLINK-10907: Summary: Job recovery on the same JobManager causes JobManager metrics to report stale values Key: FLINK-10907 URL: https://issues.apache.org/jira/browse/FLINK-10907

[jira] [Created] (FLINK-10904) Expose Classloader before Pipeline execution

2018-11-15 Thread Luka Jurukovski (JIRA)
Luka Jurukovski created FLINK-10904: --- Summary: Expose Classloader before Pipeline execution Key: FLINK-10904 URL: https://issues.apache.org/jira/browse/FLINK-10904 Project: Flink Issue

[jira] [Created] (FLINK-10906) docker-entrypoint.sh logs credentails during startup

2018-11-15 Thread Konstantin Knauf (JIRA)
Konstantin Knauf created FLINK-10906: Summary: docker-entrypoint.sh logs credentails during startup Key: FLINK-10906 URL: https://issues.apache.org/jira/browse/FLINK-10906 Project: Flink

Re: JIRA assignment

2018-11-15 Thread Joey Echeverria
One quick follow up on this. The PR has been open for about a month. I don't want to bug the folks to review it, but I'm not sure what the etiquette is for getting PRs reviewed. Any help would be appreciated! Thanks! -Joey On Wed, Nov 7, 2018 at 6:44 AM Joey Echeverria wrote: > Thanks Till!

Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-15 Thread Bowen Li
Thanks for keeping on improving the overall design, Xuefu! It looks quite good to me now. Would be nice that cc-ed Flink committers can help to review and confirm! One minor suggestion: Since the last section of design doc already touches some new sql statements, shall we add another section

[jira] [Created] (FLINK-10903) Shade Internal Akka Dependencies

2018-11-15 Thread Luka Jurukovski (JIRA)
Luka Jurukovski created FLINK-10903: --- Summary: Shade Internal Akka Dependencies Key: FLINK-10903 URL: https://issues.apache.org/jira/browse/FLINK-10903 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-10905) HadoopConfigLoader logs Credentials on DEBUG level

2018-11-15 Thread Konstantin Knauf (JIRA)
Konstantin Knauf created FLINK-10905: Summary: HadoopConfigLoader logs Credentials on DEBUG level Key: FLINK-10905 URL: https://issues.apache.org/jira/browse/FLINK-10905 Project: Flink

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-15 Thread Jamie Grier
Thanks Aljoscha for getting this effort going! There's been plenty of discussion here already and I'll add my big +1 to making this interface very simple to implement for a new Source/SplitReader. Writing a new production quality connector for Flink is very difficult today and requires a lot of

[jira] [Created] (FLINK-10893) Streaming File Sink s3 end-to-end test failed on travis

2018-11-15 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-10893: Summary: Streaming File Sink s3 end-to-end test failed on travis Key: FLINK-10893 URL: https://issues.apache.org/jira/browse/FLINK-10893 Project: Flink

[jira] [Created] (FLINK-10892) Flink Yarn app name is hard coded

2018-11-15 Thread Jeff Zhang (JIRA)
Jeff Zhang created FLINK-10892: -- Summary: Flink Yarn app name is hard coded Key: FLINK-10892 URL: https://issues.apache.org/jira/browse/FLINK-10892 Project: Flink Issue Type: Improvement

Re: How to use RocksDBStateBackend predefined options

2018-11-15 Thread Till Rohrmann
I think it sound like a good idea to be able to specify an options factory in the flink-conf.yaml. Please go ahead with creating the respective JIRA issues. Cheers, Till On Wed, Nov 14, 2018 at 7:33 AM Yun Tang wrote: > Hi all > > We already found the programmatic way to configure RocksDB was

[jira] [Created] (FLINK-10895) TypeSerializerSnapshotMigrationITCase.testSavepoint test failed on travis

2018-11-15 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-10895: Summary: TypeSerializerSnapshotMigrationITCase.testSavepoint test failed on travis Key: FLINK-10895 URL: https://issues.apache.org/jira/browse/FLINK-10895

[jira] [Created] (FLINK-10894) Resuming Externalized Checkpoint (file, async, scale down) end-to-end test failed on travis

2018-11-15 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-10894: Summary: Resuming Externalized Checkpoint (file, async, scale down) end-to-end test failed on travis Key: FLINK-10894 URL:

Re: support/docs for compression in StreamingFileSink

2018-11-15 Thread Till Rohrmann
Hi Addison, I think it is a good idea to add some more details to the documentation. Thus, it would be great if you could contribute how to enable compression. Concerning the RollingPolicy, I've pulled in Klou who might give you more details about the design decisions. Cheers, Till On Wed, Nov

[jira] [Created] (FLINK-10897) Support POJO state schema evolution

2018-11-15 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-10897: --- Summary: Support POJO state schema evolution Key: FLINK-10897 URL: https://issues.apache.org/jira/browse/FLINK-10897 Project: Flink Issue

[jira] [Created] (FLINK-10896) Extend state schema evolution support for more composite types

2018-11-15 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-10896: --- Summary: Extend state schema evolution support for more composite types Key: FLINK-10896 URL: https://issues.apache.org/jira/browse/FLINK-10896

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-15 Thread Aljoscha Krettek
Hi, I thought I had sent this mail a while ago but I must have forgotten to send it. There is another thing we should consider for splits: the range of timestamps that it can contain. For example, the splits of a file source would know what the minimum and maximum timestamp in the splits is,

[jira] [Created] (FLINK-10898) Add yarn.application-attempts-failures-validity-interval in YarnConfigOptions

2018-11-15 Thread xymaqingxiang (JIRA)
xymaqingxiang created FLINK-10898: - Summary: Add yarn.application-attempts-failures-validity-interval in YarnConfigOptions Key: FLINK-10898 URL: https://issues.apache.org/jira/browse/FLINK-10898

[jira] [Created] (FLINK-10899) Don't explicitly set version in flink-end-to-end-tests sub modules

2018-11-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10899: - Summary: Don't explicitly set version in flink-end-to-end-tests sub modules Key: FLINK-10899 URL: https://issues.apache.org/jira/browse/FLINK-10899 Project: Flink

Re: [DISCUSS] Enhancing the functionality and productivity of Table API

2018-11-15 Thread Fabian Hueske
Thanks Jincheng, That makes sense to me. Another differentiation of Table API and DataStream API would be the access to the timer service. The DataStream API can register and act on timers while the Table API would not have this feature. Best, Fabian Am Mi., 14. Nov. 2018 um 02:02 Uhr schrieb

[jira] [Created] (FLINK-10900) Mark Kafka 2.0 connector as beta feature

2018-11-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10900: - Summary: Mark Kafka 2.0 connector as beta feature Key: FLINK-10900 URL: https://issues.apache.org/jira/browse/FLINK-10900 Project: Flink Issue Type: Task

Re: [DISCUSS] Table API Enhancement Outline

2018-11-15 Thread Fabian Hueske
Hi Jincheng, I said before, that I think that the append() method is better than implicitly forwarding keys, but still, I believe it adds unnecessary boiler plate code. Moreover, I haven't seen a convincing argument why map(Expression*) is worse than map(Expression). In either case we need to do