QueryableState, unable to register KvStateRegistryListener

2016-07-20 Thread Vishnu Viswanath
Hi, I was trying the QueryableState from the pull request https://github.com/apache/flink/pull/2051 I am doing the following: 1. Make the stream queryable by calling myKeyedStreamd.asQueryableState("my-state",myStateDescriptor) 2. Create a client that takes a job id, conf, query-name and key, nu

Re: [DISCUSS] Enhance Window Evictor in Flink

2016-07-20 Thread Vishnu Viswanath
Hi, When we use RocksDB as state backend, how does the backend state get updated after some elements are evicted from the window? I don't see any update call being made to remove the element from the state stored in RocksDB. It looks like the RocksDBListState is only having get() and add() method

Re: [DISCUSS] FLIP-3 - Organization of Documentation

2016-07-20 Thread Stephan Ewen
I added to the "Application Development" Docs the Section "Types, TypeInformation, Serialization". I think that is an important enough aspect to warrant separate docs. On Mon, Jul 18, 2016 at 3:36 PM, Till Rohrmann wrote: > +1 for the FLIP and making streaming the common case. Very good proposal

[jira] [Created] (FLINK-4238) Only allow/require query for Tuple Stream in CassandraSink

2016-07-20 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4238: --- Summary: Only allow/require query for Tuple Stream in CassandraSink Key: FLINK-4238 URL: https://issues.apache.org/jira/browse/FLINK-4238 Project: Flink

Re: In AbstractRocksDBState, why write a byte 42 between key and namespace?

2016-07-20 Thread Aljoscha Krettek
No, there is no issue for now. It's just not theoretically 100% safe but the way we use it for now is not problematic. On Wed, 20 Jul 2016 at 16:07 Maximilian Michels wrote: > Is there a JIRA issue for this? > > On Mon, Jul 18, 2016 at 12:15 PM, Aljoscha Krettek > wrote: > > Ah I see, Stephan a

Re: In AbstractRocksDBState, why write a byte 42 between key and namespace?

2016-07-20 Thread Maximilian Michels
Is there a JIRA issue for this? On Mon, Jul 18, 2016 at 12:15 PM, Aljoscha Krettek wrote: > Ah I see, Stephan and I had a quick chat and it's for cases where there are > 42s around the edges of the key/namespace. > > On Mon, 18 Jul 2016 at 11:51 Aljoscha Krettek wrote: > >> In which cases is it

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
I will answer Radu's private e-mail here: Sorry to bother you ... I am still running in the same problem and I cannot figure out why. I have download and recompile the last branch of flink 1.1. I also tried using the jar snapshot from the website but I get the same error. What I am doing: I am

[jira] [Created] (FLINK-4237) ClassLoaderITCase.testDisposeSavepointWithCustomKvState fails due to Timeout Futures

2016-07-20 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-4237: - Summary: ClassLoaderITCase.testDisposeSavepointWithCustomKvState fails due to Timeout Futures Key: FLINK-4237 URL: https://issues.apache.org/jira/browse/FLINK-4237

[jira] [Created] (FLINK-4236) Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up

2016-07-20 Thread Gary Yao (JIRA)
Gary Yao created FLINK-4236: --- Summary: Flink Dashboard stops showing list of uploaded jars if main method cannot be looked up Key: FLINK-4236 URL: https://issues.apache.org/jira/browse/FLINK-4236 Project: F

[jira] [Created] (FLINK-4235) ClassLoaderITCase.testDisposeSavepointWithCustomKvState timed out on Travis

2016-07-20 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4235: Summary: ClassLoaderITCase.testDisposeSavepointWithCustomKvState timed out on Travis Key: FLINK-4235 URL: https://issues.apache.org/jira/browse/FLINK-4235 Project: Fl

[jira] [Created] (FLINK-4234) CassandraConnectorTest causes travis build to time out

2016-07-20 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4234: Summary: CassandraConnectorTest causes travis build to time out Key: FLINK-4234 URL: https://issues.apache.org/jira/browse/FLINK-4234 Project: Flink Issue Ty

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
You can always find the latest nightly snapshot version here: http://flink.apache.org/contribute-code.html (at the end of the page) Am 20/07/16 um 14:08 schrieb Radu Tudoran: Hi, I am also using v1.1...with eclipse. i will re-download the source and build it again. Is there also a binary vers

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Radu Tudoran
Hi, I am also using v1.1...with eclipse. i will re-download the source and build it again. Is there also a binary version for version 1.1 (i would like to test also againat that) particularly if the issue persists. otherwise i am downloading and building the version from the main git branch...

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
I also tried it again with the latest 1.1-SNAPSHOT and everything works. This Maven issue has been solved in FLINK-4111. Am 20/07/16 um 13:43 schrieb Suneel Marthi: I am not seeing an issue with this code Radu, this is from present 1.1-Snapshot. This is what I have and it works (running from

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Suneel Marthi
I am not seeing an issue with this code Radu, this is from present 1.1-Snapshot. This is what I have and it works (running from within IntelliJ and not cli) : List> input = new ArrayList<>(); input.add(new Tuple3<>(3L,"test",1)); input.add(new Tuple3<>(5L,"test2",2)); StreamExecutionEnvironment

Re: custom control messages from source

2016-07-20 Thread Stephan Ewen
Hi Chen! If I understand, you want to implement a custom way of triggering checkpoints, based on messages in the input message queue (for example based on Kafka events)? Basically to trigger a checkpoint when you have received a special message through each Kafka partition? Please let me know if

Re: DataStream.partitionCustom() - define parallelism

2016-07-20 Thread Aljoscha Krettek
Hi, I think that was just related to the DataSet API. If I'm not mistaken changing the parallelism should work after a "partitionCustom()". Cheers, Aljoscha On Tue, 19 Jul 2016 at 19:25 Jaromir Vanek wrote: > Aljoscha Krettek-2 wrote > > Hi, > > it should be possible to set the parallelism on t

RE: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Radu Tudoran
Hi, As far as I managed to isolate the cause of the error so far it has to do with some mismatch in the function call val traitDefs:ImmutableList[RelTraitDef[_ <: RelTrait]] = config.getTraitDefs I am not sure thought why it is not working because when I tried to make a dummy test by creating

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Maximilian Michels
CC Timo who I know is working on Table API and SQL. On Tue, Jul 19, 2016 at 6:14 PM, Radu Tudoran wrote: > Hi, > > I am not sure that this problem was solved. I am using the last pom to > compile the table API. > > I was trying to run a simple program. > > > ArrayList> input = new ArrayList St

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

2016-07-20 Thread Maximilian Michels
I think it looks like Beam rather than Hadoop :) What Stephan meant was that he wanted a dedicated output method in the ProcessWindowFunction. I agree with Aljoscha that we shouldn't expose the collector. On Tue, Jul 19, 2016 at 10:45 PM, Aljoscha Krettek wrote: > You mean keep the Collector? I