[jira] [Created] (FLINK-9539) Integrate flink-shaded 4.0

2018-06-06 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9539: --- Summary: Integrate flink-shaded 4.0 Key: FLINK-9539 URL: https://issues.apache.org/jira/browse/FLINK-9539 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-6 Problems

2018-06-06 Thread Renjie Liu
Our main use cases are mesos, maybe we can start with mesos support. On Wed, Jun 6, 2018 at 5:00 PM Stephan Ewen wrote: > The FLIP-6 design was specifically such that it allows for separation of > Dispatcher, ResourceManager, and JobManagers. > So that could be another extension at some point. >

[jira] [Created] (FLINK-9540) Apache Flink 1.4.2 S3 Hadooplibrary for Hadoop 2.7 is built for Hadoop 2.8 and fails

2018-06-06 Thread Razvan (JIRA)
Razvan created FLINK-9540: - Summary: Apache Flink 1.4.2 S3 Hadooplibrary for Hadoop 2.7 is built for Hadoop 2.8 and fails Key: FLINK-9540 URL: https://issues.apache.org/jira/browse/FLINK-9540 Project: Flink

[jira] [Created] (FLINK-9543) Expose JobMaster IDs to metric system

2018-06-06 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9543: --- Summary: Expose JobMaster IDs to metric system Key: FLINK-9543 URL: https://issues.apache.org/jira/browse/FLINK-9543 Project: Flink Issue Type: New

Re: [WEBSITE] Proposal to rework the Flink website

2018-06-06 Thread Fabian Hueske
Thanks for the feedback so far. +1 for adding a sitemap.xml and robots.txt to the website. I think we can make this a separate issue. I'll create a JIRA for that. Any other thoughts or feedback? Thanks, Fabian 2018-06-06 9:34 GMT+02:00 Aljoscha Krettek : > Yes, making sure that google search

Need help in understanding flink ordering of records

2018-06-06 Thread Amol S - iProgrammer
Hello Ezmlm, I have implemented code to read mongodb oplog and stream this oplog in flink, But I need all the records in the order the are coming from oplog. first of all clarify me is flink maintains order of insertion? if yes then give me some source document where I can find how flink does

[jira] [Created] (FLINK-9542) ExEnv#registerCachedFile should accept Path

2018-06-06 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9542: --- Summary: ExEnv#registerCachedFile should accept Path Key: FLINK-9542 URL: https://issues.apache.org/jira/browse/FLINK-9542 Project: Flink Issue Type:

[jira] [Created] (FLINK-9541) Add robots.txt and sitemap.xml to Flink website

2018-06-06 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-9541: Summary: Add robots.txt and sitemap.xml to Flink website Key: FLINK-9541 URL: https://issues.apache.org/jira/browse/FLINK-9541 Project: Flink Issue Type:

[jira] [Created] (FLINK-9544) Impossible to downgrade kinesis protocole from CBOR to JSON as required by kinesalite

2018-06-06 Thread Ph.Duveau (JIRA)
Ph.Duveau created FLINK-9544: Summary: Impossible to downgrade kinesis protocole from CBOR to JSON as required by kinesalite Key: FLINK-9544 URL: https://issues.apache.org/jira/browse/FLINK-9544 Project:

[jira] [Created] (FLINK-9537) JobManager isolation in session mode

2018-06-06 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-9537: Summary: JobManager isolation in session mode Key: FLINK-9537 URL: https://issues.apache.org/jira/browse/FLINK-9537 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-6 Problems

2018-06-06 Thread Till Rohrmann
Hi Renjie, there is already an issue for introducing further scheduling constraints (e.g. tags) to achieve TM isolation when using the session mode [1]. What it does not cover is the isolation of the JMs which need to be executed in their own processes. At the moment they share the same process

Re: [WEBSITE] Proposal to rework the Flink website

2018-06-06 Thread Aljoscha Krettek
Yes, making sure that google search results point to the most recent doc would be very good. :+1 Also +1 to the general effort, of course. > On 5. Jun 2018, at 20:19, Ken Krugler wrote: > > Along these lines, it would help to add a sitemap (and the robots.txt > required to reference it) for

[jira] [Created] (FLINK-9538) Make KeyedStateFunction an interface

2018-06-06 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-9538: --- Summary: Make KeyedStateFunction an interface Key: FLINK-9538 URL: https://issues.apache.org/jira/browse/FLINK-9538 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-6 Problems

2018-06-06 Thread Renjie Liu
That's really great! I'll help to contribute to the process. On Wed, Jun 6, 2018 at 3:17 PM Till Rohrmann wrote: > Hi Renjie, > > there is already an issue for introducing further scheduling constraints > (e.g. tags) to achieve TM isolation when using the session mode [1]. What > it does not

[DISCUSS] Flink 1.4 and below STOPS writing to Kinesis after June 12th.

2018-06-06 Thread Bowen Li
Hi, I think the following email thread might have gone lost. Dyana brought up the attention that AWS has informed users that KPL versions < 0.12.6 will *stop working* starting from the 12th of June. Flink 1.4 is using KPL 0.12.5 and Flink 1.5 uses 0.12.6, *so Flink 1.4 and below (1.3, etc) will

Re: [VOTE] Release flink-shaded 4.0, release candidate #2

2018-06-06 Thread Chesnay Schepler
Nice catch, but I wouldn't cancel the vote because of it. The release process is documented in the Wiki and refers to the correct scripts. On 05.06.2018 16:50, Timo Walther wrote: +1 - Checked the commits that went into this release - Build from source I found one minor thing. There are two

[RESULT] [VOTE] Release flink-shaded 4.0, release candidate #2

2018-06-06 Thread Chesnay Schepler
I'm happy to announce that we have unanimously approved this release. These are the 4 approving votes, 3 of which are binding: * Gordong (binding) * Piotr (non-binding) * Timo (binding) * Chesnay (binding) Thanks everyone! On 06.06.2018 08:56, Chesnay Schepler wrote: Nice catch, but I wouldn't

[jira] [Created] (FLINK-9535) log4j.properties specified in env.java.options doesn't get picked.

2018-06-06 Thread SUBRAMANYA SURESH (JIRA)
SUBRAMANYA SURESH created FLINK-9535: Summary: log4j.properties specified in env.java.options doesn't get picked. Key: FLINK-9535 URL: https://issues.apache.org/jira/browse/FLINK-9535 Project:

[jira] [Created] (FLINK-9536) Rolling logs using log4j.properties, prevents them from showing on the UI.

2018-06-06 Thread SUBRAMANYA SURESH (JIRA)
SUBRAMANYA SURESH created FLINK-9536: Summary: Rolling logs using log4j.properties, prevents them from showing on the UI. Key: FLINK-9536 URL: https://issues.apache.org/jira/browse/FLINK-9536

[1.4.2] mvn clean package command takes to much time

2018-06-06 Thread Marvin777
Hi, all. It takes a long time to modify some of the code and recompile it. The process is painful. Is there any method that I can save time. Thanks!

Re: [1.4.2] mvn clean package command takes to much time

2018-06-06 Thread Chesnay Schepler
you only have to compile the module that you changed along with flink-dist to test things locally. On 06.06.2018 10:27, Marvin777 wrote: Hi, all. It takes a long time to modify some of the code and recompile it. The process is painful. Is there any method that I can save time. Thanks!

Re: [DISCUSS] FLIP-6 Problems

2018-06-06 Thread Stephan Ewen
The FLIP-6 design was specifically such that it allows for separation of Dispatcher, ResourceManager, and JobManagers. So that could be another extension at some point. It should be conceptually rather simple, the dispatcher creates per job a new container launch context with the

Re: [1.4.2] mvn clean package command takes to much time

2018-06-06 Thread Marvin777
Thank you for your reply. If I modify the flink-runtime module, then the following command is executed for compilation. 'mvn clean package -pl flink-runtime,flink-dist *-am*' The parameter of '-am' is necessary, it will take a long time, otherwise it will report an error, like 'Failed to

Re: [DISCUSS] Flink 1.4 and below STOPS writing to Kinesis after June 12th.

2018-06-06 Thread Thomas Weise
--> On Wed, Jun 6, 2018 at 1:06 AM, Bowen Li wrote: > Hi, > > I think the following email thread might have gone lost. > > Dyana brought up the attention that AWS has informed users that KPL > versions > < 0.12.6 will *stop working* starting from the 12th of June. Flink 1.4 is > using KPL

FINAL REMINDER: Apache EU Roadshow 2018 in Berlin next week!

2018-06-06 Thread sharan
Hello Apache Supporters and Enthusiasts This is a final reminder that our Apache EU Roadshow will be held in Berlin next week on 13th and 14th June 2018. We will have 28 different sessions running over 2 days that cover some great topics. So if you are interested in Microservices, Internet of

[jira] [Created] (FLINK-9545) Support read file for N times in Flink stream

2018-06-06 Thread Bowen Li (JIRA)
Bowen Li created FLINK-9545: --- Summary: Support read file for N times in Flink stream Key: FLINK-9545 URL: https://issues.apache.org/jira/browse/FLINK-9545 Project: Flink Issue Type: Improvement

Is there a Slack channel?

2018-06-06 Thread Pablo Estrada
Hello there from the Apache Beam world! I was meaning to start learning and hearing about Flink, so I recently subscribed to the mailing lists, and I wonder if there's also a Slack channel that people use? Regards! -P. -- Got feedback? go/pabloem-feedback

Re: Is there a Slack channel?

2018-06-06 Thread Chesnay Schepler
No, there is no slack channel. We had a recent discussion about introducing one but rejected it. On 07.06.2018 01:58, Pablo Estrada wrote: Hello there from the Apache Beam world! I was meaning to start learning and hearing about Flink, so I recently subscribed to the mailing lists, and I

Re: [1.4.2] mvn clean package command takes to much time

2018-06-06 Thread Chesnay Schepler
first run mvn clean install in the flink-shaded-hadoop module. then you only have to run mvn clean package -pl flink-runtime,flink-dist On 07.06.2018 04:48, Marvin777 wrote: Thank you for your reply. If I modify the flink-runtime module, then the following command is executed for compilation.