[jira] [Created] (FLINK-6206) As an Engineer, I want task state transition log to be warn/error for FAILURE scenarios

Dan Bress created FLINK-6206: Summary: As an Engineer, I want task state transition log to be warn/error for FAILURE scenarios Key: FLINK-6206 URL: https://issues.apache.org/jira/browse/FLINK-6206

Re: Who's hiring, December 2016

The team I work on at Twitter is hiring a Software Engineer: https://careers.twitter.com/en/work-for-twitter/software-engineer-data-products.html Dan On Fri, Dec 16, 2016 at 5:46 AM Kostas Tzoumas wrote: > Hi folks, > > As promised, here is the first thread for

[jira] [Created] (FLINK-5341) Add a metric exposing how many times a job has been restarted

Dan Bress created FLINK-5341: Summary: Add a metric exposing how many times a job has been restarted Key: FLINK-5341 URL: https://issues.apache.org/jira/browse/FLINK-5341 Project: Flink Issue

[jira] [Created] (FLINK-5340) I would like a metric exposing job uptime

Dan Bress created FLINK-5340: Summary: I would like a metric exposing job uptime Key: FLINK-5340 URL: https://issues.apache.org/jira/browse/FLINK-5340 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-5011) TraversableSerializer does not perform a deep copy of the elements it is traversing

Dan Bress created FLINK-5011: Summary: TraversableSerializer does not perform a deep copy of the elements it is traversing Key: FLINK-5011 URL: https://issues.apache.org/jira/browse/FLINK-5011 Project

Re: [DISCUSS] Releasing Flink 1.1.4

+1 for this release, also +1 to Chesnay's suggesting for including this: [FLINK-4875] [metrics] Use correct operator name Dan On Wed, Oct 26, 2016 at 5:06 AM Till Rohrmann wrote: > I'll work on FLINK-3347. Additionally I would like to get in > > -

Re: currentLowWatermark metric not reported for all tasks?

Thank you Robert! On Wed, Oct 12, 2016 at 2:55 AM Robert Metzger <rmetz...@apache.org> wrote: > I added a JIRA for this feature request: > https://issues.apache.org/jira/browse/FLINK-4812 > > On Fri, Sep 30, 2016 at 6:13 PM, dan bress <danbr...@gmail.com> wrote:

Re: Does the job name affect checkpointing?

it belong to? A Job? A TaskManager? A Job Manager? Flink as a whole? Thanks, Dan On Fri, Sep 30, 2016 at 1:20 AM Ufuk Celebi <u...@apache.org> wrote: > Hey Dan! > > On Thu, Sep 29, 2016 at 10:58 PM, dan bress <danbr...@gmail.com> wrote: > > If I deploy a job to Flink called &q

Re: currentLowWatermark metric not reported for all tasks?

pose it through a > > metric. > > > > Regards, > > Chesnay > > > > > > On 29.09.2016 23:03, dan bress wrote: > > > >> I am trying to measure the currentLowWatermark throughout my dataflow, > but > >> I am not seeing it for tasks wit

currentLowWatermark metric not reported for all tasks?

I am trying to measure the currentLowWatermark throughout my dataflow, but I am not seeing it for tasks with sources. For those tasks I see these metrics: lastCheckpointSize numBytesInLocal numBytesInRemote numBytesOut why am I not seeing currentLowWatermark on these tasks? Thanks! Dan

Does the job name affect checkpointing?

If I deploy a job to Flink called "A"(set my streamExecutionEnvironment.execute("A"), that checkpoints state. Then I cancel "A" and deploy the same job but call it "B", will it pick up A's state? Or is checkpointing key'd by the job name? The reason I ask is I would like a way to reflect the

Re: checkpointing a OneInputStreamOperator/AbstractStreamOperator

because they store non-rescalable state, i.e. if you use this > you won't be able to change the parallelism of your job in the future. A > more future proof solution would be to use keyed state, i.e. > getRuntimeContext().getState(). > > Cheers, > Aljoscha > > On Mon,

checkpointing a OneInputStreamOperator/AbstractStreamOperator

Hi Flink Devs, I have an operator that implements both OneInputStreamOperator and AbstractStreamOperator, and I would like to preserve its state. I started by having it implement CheckpointedAsynchronously like my other stateful functions, but this didn't work(in the flink UI under the

Re: Custom Tumbling Window Assigner

t; > [1] > > > https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/windows.html#session-windows > > [2] https://techblog.king.com/rbea-scalable-real-time-analytics-king/ > > > > On Mon, Aug 15, 2016 at 7:05 PM, dan bress <danbr...@gmail.com> wrote:

Custom Tumbling Window Assigner

I am interesting in writing a custom tumbling window assigner to do the following: As soon as the first event for that keyed stream comes in, create a window from that point going forward N milli seconds. Is it possible to do that using flink? What I want to do is get all events that occurred