Re: Reading null value from datasets

2015-10-23 Thread Maximilian Michels
Hi Guido, This depends on your use case but you may read those values as type String and treat them accordingly. Cheers, Max On Fri, Oct 23, 2015 at 1:59 PM, Guido wrote: > Hello, > I would like to ask if there were any particular ways to read or treat > null (e.g. Name,

Error running an hadoop job from web interface

2015-10-23 Thread Flavio Pompermaier
Hi to all, I'm trying to run a job from the web interface but I get this error: java.lang.RuntimeException: java.io.FileNotFoundException: JAR entry core-site.xml not found in /tmp/webclient-jobs/EntitonsJsonizer.jar at

Re: reading csv file from null value

2015-10-23 Thread Maximilian Michels
Hi Philip, How about making the empty field of type String? Then you can read the CSV into a DataSet and treat the empty string as a null value. Not very nice but a workaround. As of now, Flink deliberately doesn't support null values. Regards, Max On Thu, Oct 22, 2015 at 4:30 PM, Philip Lee

Re: Error running an hadoop job from web interface

2015-10-23 Thread Maximilian Michels
Hi Flavio, Which version of Flink are you using? Cheers, Max On Fri, Oct 23, 2015 at 2:45 PM, Flavio Pompermaier wrote: > Hi to all, > I'm trying to run a job from the web interface but I get this error: > > java.lang.RuntimeException: java.io.FileNotFoundException: JAR

Re: Running continuously on yarn with kerberos

2015-10-23 Thread Maximilian Michels
Hi Niels, Thank you for your question. Flink relies entirely on the Kerberos support of Hadoop. So your question could also be rephrased to "Does Hadoop support long-term authentication using Kerberos?". And the answer is: Yes! While Hadoop uses Kerberos tickets to authenticate users with

Re: Reading null value from datasets

2015-10-23 Thread Shiti Saxena
For a similar problem where we wanted to preserve and track null entries, we load the CSV as a DataSet[Array[Object]] and then transform it into DataSet[Row] using a custom RowSerializer( https://gist.github.com/Shiti/d0572c089cc08654019c) which handles null. The Table API(which supports null)

Re: reading csv file from null value

2015-10-23 Thread Shiti Saxena
For a similar problem where we wanted to preserve and track null entries, we load the CSV as a DataSet[Array[Object]] and then transform it into DataSet[Row] using a custom RowSerializer( https://gist.github.com/Shiti/d0572c089cc08654019c) which handles null. The Table API(which supports null)

Re: Session Based Windows

2015-10-23 Thread Aljoscha Krettek
Hi Paul, the key based state should now be fixed in the current 0.10-SNAPSHOT builds if you want to continue playing around with it. Cheers, Aljoscha > On 21 Oct 2015, at 19:40, Aljoscha Krettek wrote: > > Hi Paul, > good to hear that the windowing works for you. > > With