RE: Kerberos for Streaming & Kafka

2016-03-25 Thread Eron Wright
(fixed bad formatting) Hi, Given the other thread about per-job Kerberos identity, now's a good time to discuss some problems with the current delegation-token approach, since the answer could bear on the per-job enhancement. I see two problems: 1. Delegation tokens expire. For a continuous st

Re: a typical ML algorithm flow

2016-03-25 Thread Dmitriy Lyubimov
Thank you, all :) yes, that's my question. How do we construct such a loop with a concrete example? Let's take something nonsensical yet specific. Say, in samsara terms we do something like that : var avg = Double.PositiveInfinity var drmA = ... (construct elsewhere) do { avg = drmA.colMe

[jira] [Created] (FLINK-3668) Potential null deference in HadoopInputFormatBase#createInputSplits()

2016-03-25 Thread Ted Yu (JIRA)
Ted Yu created FLINK-3668: - Summary: Potential null deference in HadoopInputFormatBase#createInputSplits() Key: FLINK-3668 URL: https://issues.apache.org/jira/browse/FLINK-3668 Project: Flink Issue

Apache Flink: aligning watermark among parallel tasks

2016-03-25 Thread Ozan DENİZ
We are using periodic event time window with watermark. We have currently 4 parallel tasks in our Flink App. During the streaming process, all the 4 tasks' watermark values must be close to trigger window event. For example; Task 1 watermark value = 8 Task 2 watermark value =

Re: Proposal: YARN session per-job Kerberos authentication

2016-03-25 Thread Stefano Baghino
Hi Robert and Max, thanks for giving us some feedback to ponder on. We'll start by opening the issue and some sub-tasks to keep track of the prototyping and and design phase, along with development, testing and documentation. Robert, I thought of the encryption issue and was one of the things I h