[jira] [Created] (FLINK-7768) Load File Systems via Java Service abstraction

2017-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-7768: --- Summary: Load File Systems via Java Service abstraction Key: FLINK-7768 URL: https://issues.apache.org/jira/browse/FLINK-7768 Project: Flink Issue Type:

[jira] [Created] (FLINK-7767) Avoid loading Hadoop conf dynamically at runtime

2017-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-7767: --- Summary: Avoid loading Hadoop conf dynamically at runtime Key: FLINK-7767 URL: https://issues.apache.org/jira/browse/FLINK-7767 Project: Flink Issue Type:

[jira] [Created] (FLINK-7766) Remove obsolete reflection for hflush on HDFS

2017-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-7766: --- Summary: Remove obsolete reflection for hflush on HDFS Key: FLINK-7766 URL: https://issues.apache.org/jira/browse/FLINK-7766 Project: Flink Issue Type:

Re: System resource logger

2017-10-05 Thread Bowen Li
System and processor info, marked as 'logged once' in gist shared by Piotr, should still be logged instead of registered as metrics, right? On Thu, Oct 5, 2017 at 2:38 AM, Till Rohrmann wrote: > Thanks for the proposal Piotr. I like it a lot since it will help people to >

Re: Unable to write snapshots to S3 on EMR

2017-10-05 Thread Andy M.
Hi Till, I believe this is what you are looking for, classpath is much bigger for the task manager. I can also post the whole log file if needed: 2017-10-05 14:17:53,038 INFO org.apache.flink.yarn.YarnTaskManagerRunner - Classpath:

[jira] [Created] (FLINK-7765) Enable dependency convergence

2017-10-05 Thread Piotr Nowojski (JIRA)
Piotr Nowojski created FLINK-7765: - Summary: Enable dependency convergence Key: FLINK-7765 URL: https://issues.apache.org/jira/browse/FLINK-7765 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-7764) FlinkKafkaProducer010 does not accept name, uid, or parallelism

2017-10-05 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-7764: Summary: FlinkKafkaProducer010 does not accept name, uid, or parallelism Key: FLINK-7764 URL: https://issues.apache.org/jira/browse/FLINK-7764 Project: Flink

[jira] [Created] (FLINK-7763) TableSinkITCase fails with "object reuse" enabled

2017-10-05 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-7763: --- Summary: TableSinkITCase fails with "object reuse" enabled Key: FLINK-7763 URL: https://issues.apache.org/jira/browse/FLINK-7763 Project: Flink Issue

Re: Dependency convergence

2017-10-05 Thread Till Rohrmann
+1 for pulling our dependencies straight and guarding it via the maven-enforcer-plugin. On Wed, Oct 4, 2017 at 8:58 PM, Piotr Nowojski wrote: > I meant for whole flink project. > > > On Oct 4, 2017, at 8:43 PM, Bowen Li wrote: > > > > +1. This

Re: Unable to write snapshots to S3 on EMR

2017-10-05 Thread Till Rohrmann
Hi Andy, the CliFrontend is not executed via Yarn, thus, it is not affected by dependencies which are added due to the underlying Yarn cluster. Therefore, it would be helpful to look at the TaskManager logs. Either you have enabled log aggregation on your Yarn cluster, then you can obtain the

Re: System resource logger

2017-10-05 Thread Piotr Nowojski
+1 thanks for pointing this out. It makes sense to just expand those system metrics (I was not aware of them). > On Oct 4, 2017, at 6:07 PM, Greg Hogan wrote: > > What if we added these as system metrics and added a way to write metrics to > a (separate?) log file? > >

[jira] [Created] (FLINK-7762) Make WikipediaEditsSourceTest a proper test

2017-10-05 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-7762: --- Summary: Make WikipediaEditsSourceTest a proper test Key: FLINK-7762 URL: https://issues.apache.org/jira/browse/FLINK-7762 Project: Flink Issue Type: