threads, parallelism and task managers

Hi guys, I am trying to test a job that should run a number of tasks to read from a RDBMS using an improved JDBC connector. The connection and the reading run smoothly, but I cannot seem to be able to move above the limit of 8 concurrent threads running. 8 is of course the number of cores of my

Re: Unable to submit flink job that uses Avro data

> On Wed, 23.03.2016 06:59, Chesnay Schepler wrote > Could you be missing the call to execute()? Yes, that was it. Can't believe I missed that ! Thank you Chesnay. Best, Tarandeep On 23.03.2016 01:25, Tarandeep Singh wrote: >> Hi, >> >> I wrote a simple Flink job that uses Avro input format to

DataSet.randomSplit()

All: Does Flink DataSet have a randomSplit(weights:Array[Double], seed: Long): Array[DataSet[T]] function? There is this pull request: https://github.com/apache/flink/pull/921 Does anyone have an update of the progress of this? Thank you. -- *Gna Phetsarath*System Architect // AOL

Re: Flink 1.0.0 reading files from multiple directory with wildcards

Great! I will, once I clear it with the legal team here. On Wed, Mar 23, 2016 at 6:19 AM, Ufuk Celebi wrote: > Nice! Would you like to contribute this to Flink via a pull request? Some > resources about the contribution process can be found here: > >

Re: Valuestate is not saving the state

Never mind I understand what is going on Aljoscha for each unique key the value count is reset to 0. On Wed, Mar 23, 2016 at 4:37 PM, Balaji Rajagopalan < balaji.rajagopa...@olacabs.com> wrote: > (Booking(te7uc4,compact,j...@gmail.com,Mon Feb 29 19:19:40 IST >

Re: Oracle 11g number serialization: classcast problem

Thanks for the clarification. case java.sql.Types.DECIMAL: reuse.setField(resultSet.getBigDecimal(pos + 1).doubleValue(), pos); break; this causes both a nullpointer on null values as well as a double class cast exception when serializing the tuple. For

Re: Oracle 11g number serialization: classcast problem

On 23.03.2016 10:38, Chesnay Schepler wrote: On 23.03.2016 10:04, Stefano Bortoli wrote: I had a look at the JDBC input format, and it does indeed interpret BIGDECIMAL and NUMERIC values as double. This sounds more like a bug actually. Feel free to open a JIRA for this. Actually, this was

Re: Oracle 11g number serialization: classcast problem

On 23.03.2016 10:04, Stefano Bortoli wrote: I had a look at the JDBC input format, and it does indeed interpret BIGDECIMAL and NUMERIC values as double. This sounds more like a bug actually. Feel free to open a JIRA for this. The status of the JDBCInputFormat is not adequate for real world use

Re: TumblingProcessingTimeWindow and ContinuousProcessingTimeTrigger

Hi, the output at 19:44:44.635 is indeed strange. Is this reproducible? As for the removal of windows. That is a pitfall a lot of users have fallen into. The timeWindowAll() call just sets up a window assigner, so in your case the equivalent call would be: .flatMap {

Re: Oracle 11g number serialization: classcast problem

I had a look at the JDBC input format, and it does indeed interpret BIGDECIMAL and NUMERIC values as double. The status of the JDBCInputFormat is not adequate for real world use case, as for example does not deal with NULL values. However, with little effort we fixed few stuff and now we are

Re: Intermediate solution set of delta iteration

Hi Mengqi, if what you are trying to do is output the solution set of every iteration, before the iteration has finished, then that is not possible. i.e. you can not output the solution set to a sink or another operator during the iteration. However, you can add elements to the solution set and

Re: Unable to submit flink job that uses Avro data

Could you be missing the call to execute()? On 23.03.2016 01:25, Tarandeep Singh wrote: Hi, I wrote a simple Flink job that uses Avro input format to read avro file and save the results in avro format. The job does not get submitted and job client exist out immediately. Same thing happens if