Re: Dependency convergence

2017-10-04 Thread Piotr Nowojski
I meant for whole flink project. > On Oct 4, 2017, at 8:43 PM, Bowen Li wrote: > > +1. This is great, Piotrek! > > BTW, can you clarify what you mean by 'project wide'? Is it the whole > `flink` project or just `flink-connector-kafka`? I think it's useful to > be

Re: Dependency convergence

2017-10-04 Thread Bowen Li
+1. This is great, Piotrek! BTW, can you clarify what you mean by 'project wide'? Is it the whole `flink` project or just `flink-connector-kafka`? I think it's useful to be applied to the whole flink project. I've seen dependencies conflict problem like this in flink-connector-kinesis. Enabling

Dependency convergence

2017-10-04 Thread Piotr Nowojski
Hi, I have spent last couple of days trying to find and fix Kafka tests instabilities on Travis and I think I have finally found the main reason: dependency conflict on Netty. flakka was pulling in 3.8 and zookeeper 3.10. Effect was very subtle, because rarely in some corner cases (but not

Re: System resource logger

2017-10-04 Thread Greg Hogan
What if we added these as system metrics and added a way to write metrics to a (separate?) log file? > On Oct 4, 2017, at 10:13 AM, Piotr Nowojski wrote: > > Hi, > > Lately I was debugging some weird test failures on Travis and I needed to > look into metrics like:

System resource logger

2017-10-04 Thread Piotr Nowojski
Hi, Lately I was debugging some weird test failures on Travis and I needed to look into metrics like: - User, System, IOWait, IRQ CPU usages (based on CPU ticks since previous check) - System wide memory consumption (including making sure that swap was disabled) - network usage - etc… Without

[jira] [Created] (FLINK-7761) Twitter example is not self-contained

2017-10-04 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7761: --- Summary: Twitter example is not self-contained Key: FLINK-7761 URL: https://issues.apache.org/jira/browse/FLINK-7761 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7760) Restore failing from external checkpointing metadata.

2017-10-04 Thread Shashank Agarwal (JIRA)
Shashank Agarwal created FLINK-7760: --- Summary: Restore failing from external checkpointing metadata. Key: FLINK-7760 URL: https://issues.apache.org/jira/browse/FLINK-7760 Project: Flink

Fwd: Re: Flink Watermark and timing

2017-10-04 Thread Timo Walther
Hi Björn, the behavior of borderlines is defined clearly by the API: "start timestamp (inclusive) and an end timestamp (exclusive)". So it is always [0-4] [5-9]. You could increase the interval by one millisecond to include 5. Regards, Timo Weitergeleitete Nachricht

[jira] [Created] (FLINK-7759) Fix Bug that fieldName/functionName with prefix equals "true" or "false" can't be parsed by ExpressionParser.

2017-10-04 Thread Ruidong Li (JIRA)
Ruidong Li created FLINK-7759: - Summary: Fix Bug that fieldName/functionName with prefix equals "true" or "false" can't be parsed by ExpressionParser. Key: FLINK-7759 URL:

Re: Unable to write snapshots to S3 on EMR

2017-10-04 Thread Till Rohrmann
Hi Andy, this looks to me indeed like a dependency problem. I assume that EMR or something else is pulling in an incompatible version of Hadoop. The classpath you've posted, is this the one logged in the log files (TaskManager log) or did you compile it yourself? In the latter case, it would

[jira] [Created] (FLINK-7758) Fix bug Kafka09Fetcher add offset metrics

2017-10-04 Thread Hai Zhou UTC+8 (JIRA)
Hai Zhou UTC+8 created FLINK-7758: - Summary: Fix bug Kafka09Fetcher add offset metrics Key: FLINK-7758 URL: https://issues.apache.org/jira/browse/FLINK-7758 Project: Flink Issue Type: Bug