Re: Akka dissassociated

2017-07-19 Thread Bowen Li
The amount of resource assigned to JobManager looks fine. How much resource (CPU and memory) did you allocate for TaskManager? On Wed, Jul 19, 2017 at 3:07 AM, Stephan Ewen wrote: > Hi Greg! > > Akka disassociation means that the network connection between the > JobManager and

[apache/flink] [Flink-7218] [JobManager] ExecutionVertex.getPreferredLocationsBasedOnInputs() will always return empty

2017-07-19 Thread 周思华
I really like the new PR template, so i use it instead of the default one on github~ You can view, comment on, or merge this pull request online at: #4369 What is the purpose of the change The ExecutionVertex.getPreferredLocationsBasedOnInputs will always return empty, cause sourceSlot

[jira] [Created] (FLINK-7237) Remove DateTimeUtils from Flink once Calcite is upgraded to 1.14

2017-07-19 Thread Haohui Mai (JIRA)
Haohui Mai created FLINK-7237: - Summary: Remove DateTimeUtils from Flink once Calcite is upgraded to 1.14 Key: FLINK-7237 URL: https://issues.apache.org/jira/browse/FLINK-7237 Project: Flink

[jira] [Created] (FLINK-7236) Bump up the Calcite version to 1.14

2017-07-19 Thread Haohui Mai (JIRA)
Haohui Mai created FLINK-7236: - Summary: Bump up the Calcite version to 1.14 Key: FLINK-7236 URL: https://issues.apache.org/jira/browse/FLINK-7236 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7235) Backport CALCITE-1884 to the Flink repository before Calcite 1.14

2017-07-19 Thread Haohui Mai (JIRA)
Haohui Mai created FLINK-7235: - Summary: Backport CALCITE-1884 to the Flink repository before Calcite 1.14 Key: FLINK-7235 URL: https://issues.apache.org/jira/browse/FLINK-7235 Project: Flink

Re: Using native library in Flink

2017-07-19 Thread Mike Accola
Timo/Eron - Thank you for the responses. To answer a few of your questions: - For now, I am just running in a simple, local environment (start-local.sh) - I have this entry in the flink-conf.yaml file: env.java.opts : "-Djava.library.path=/myPathWithTheLibrary". From looking at logs, it

[jira] [Created] (FLINK-7234) Fix CombineHint documentation

2017-07-19 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-7234: - Summary: Fix CombineHint documentation Key: FLINK-7234 URL: https://issues.apache.org/jira/browse/FLINK-7234 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-7233) TaskManagerHeapSizeCalculationJavaBashTest failed on Travis

2017-07-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7233: --- Summary: TaskManagerHeapSizeCalculationJavaBashTest failed on Travis Key: FLINK-7233 URL: https://issues.apache.org/jira/browse/FLINK-7233 Project: Flink

[jira] [Created] (FLINK-7232) Update checkstyle docs regarding test inclusion

2017-07-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7232: --- Summary: Update checkstyle docs regarding test inclusion Key: FLINK-7232 URL: https://issues.apache.org/jira/browse/FLINK-7232 Project: Flink Issue

[jira] [Created] (FLINK-7231) SlotSharingGroups are not always released in time for new restarts

2017-07-19 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-7231: --- Summary: SlotSharingGroups are not always released in time for new restarts Key: FLINK-7231 URL: https://issues.apache.org/jira/browse/FLINK-7231 Project: Flink

Re: Re: AVRO Union type support in Flink

2017-07-19 Thread Vishnu Viswanath
Hi Timo, I just modified AvroOutputFormatTest to test this and it works fine!. I don't plan to use it to key by, but it is a good point. Thanks. Regards, Vishnu On Wed, Jul 19, 2017 at 10:57 AM, Timo Walther wrote: > We have similar checks in our KafkaAvroTableSource, but

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

2017-07-19 Thread Aljoscha Krettek
Hi, Yes! In my opinion, the most critical issues are these: - https://issues.apache.org/jira/browse/FLINK-6964: Fix recovery for incremental checkpoints in StandaloneCompletedCheckpointStore -

Fwd: Re: AVRO Union type support in Flink

2017-07-19 Thread Timo Walther
We have similar checks in our KafkaAvroTableSource, but I could not find such a check in AvroTypeInfo. The field should have a generic type, so you can work with it. If you want to use it as key, you might have to use a mapper before and convert it into a valid key type. Timo

[jira] [Created] (FLINK-7230) Travis sometimes fails due to java version imcompatibilities

2017-07-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7230: --- Summary: Travis sometimes fails due to java version imcompatibilities Key: FLINK-7230 URL: https://issues.apache.org/jira/browse/FLINK-7230 Project: Flink

[VOTE] Release Apache Flink-shaded 1.0 (RC1)

2017-07-19 Thread Chesnay Schepler
Dear Flink community, Please vote on releasing the following candidate as Apache Flink-shaded version 1.0. The commit to be voted in: https://gitbox.apache.org/repos/asf/flink-shaded/commit/fd3033ba9ead310478963bf43e09cd50d1e36d71 Branch: release-1.0-rc1 The release artifacts to be voted on

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

2017-07-19 Thread Shaoxuan Wang
Hi Aljoscha, Glad to see that we have a more thorough testing procedure. Could you please share us what (the critical issues you mentioned) have been broken in 1.3.0 & 1.3.1, and how the new proposed "functional testing section and a combination of systems/configurations" can cover this. This will

[DISCUSS] Release testing procedures, Flink 1.3.2

2017-07-19 Thread Aljoscha Krettek
Hi Everyone, We are on the verge of starting the release process for Flink 1.3.2 and there have been some critical issues in both Flink 1.3.0 and 1.3.1. For Flink 1.3.2 I want to make very sure that we test as much as possible. For this I’m proposing a slightly changed testing procedure [1].

[jira] [Created] (FLINK-7229) Flink doesn't deleted old checkpoint

2017-07-19 Thread Jason Zhou (JIRA)
Jason Zhou created FLINK-7229: - Summary: Flink doesn't deleted old checkpoint Key: FLINK-7229 URL: https://issues.apache.org/jira/browse/FLINK-7229 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7227) OR expression with more than 2 predicates is not pushed into a TableSource

2017-07-19 Thread Usman Younas (JIRA)
Usman Younas created FLINK-7227: --- Summary: OR expression with more than 2 predicates is not pushed into a TableSource Key: FLINK-7227 URL: https://issues.apache.org/jira/browse/FLINK-7227 Project:

[jira] [Created] (FLINK-7226) REST responses contain invalid content-encoding header

2017-07-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7226: --- Summary: REST responses contain invalid content-encoding header Key: FLINK-7226 URL: https://issues.apache.org/jira/browse/FLINK-7226 Project: Flink

[jira] [Created] (FLINK-7225) Cutoff exception message in StateDescriptor

2017-07-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-7225: --- Summary: Cutoff exception message in StateDescriptor Key: FLINK-7225 URL: https://issues.apache.org/jira/browse/FLINK-7225 Project: Flink Issue Type:

Re: Exported metrics by Datadog/Statsd cannot be grouped due to varying metric name

2017-07-19 Thread Chesnay Schepler
If you're using the StatsDReporter there is currently no way to use tags. There is an open PR to add support for tags though: https://github.com/apache/flink/pull/4188 For DataDog specifically we have a separate DatadogReporter that uses tags. For this reporter the metric name can be

Re: Dropping Java 7 support

2017-07-19 Thread Chesnay Schepler
Are the specific things we want to change right away? (build profiles would be one thing) Would be neat to collect them in an umbrella issue. On 18.07.2017 16:49, Timo Walther wrote: Hurray! Finally IntStreams, LongStreams, etc. in our stream processor ;-) Timo Am 18.07.17 um 16:31 schrieb

Re: Akka dissassociated

2017-07-19 Thread Stephan Ewen
Hi Greg! Akka disassociation means that the network connection between the JobManager and TaskManager broke. This can be cause by - actual failures of JobManager / TaskManager (I assume is not the case here) - A limit in the number of open file handles - Network flakeyness - I have seen

Re: Latency Measurement

2017-07-19 Thread Chesnay Schepler
I originally meant startNewChain(), but disableChaining() should work too. Can you rerun the job with the logging level set to DEBUG, and check for any message from org.apache.flink.runtime.metrics? Also looping in Robert, maybe he has an idea. On 17.07.2017 14:23, Paolo Cristofanelli wrote:

Re: [DISCUSS] A more thorough Pull Request check list and template

2017-07-19 Thread Stephan Ewen
@Chesnay: Put text into template => contributor will have to read it Put link to text into template => most contributors will ignore the link Yes, that is pretty much what my observation from the past is. On Tue, Jul 18, 2017 at 11:03 PM, Chesnay Schepler wrote: > I'm

[jira] [Created] (FLINK-7224) Incorrect Javadoc description in all Kafka consumer versions

2017-07-19 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-7224: -- Summary: Incorrect Javadoc description in all Kafka consumer versions Key: FLINK-7224 URL: https://issues.apache.org/jira/browse/FLINK-7224 Project: