Re: Flink first() operator

2016-04-26 Thread Fabian Hueske
mber of lines read in one go as reading a big file would be > a > big problem in terms of memory. > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-first-operator-tp6377p6451.html > Sent from the Apache

Re: Flink first() operator

2016-04-25 Thread Fabian Hueske
Hi Biplop, you can also implement a generic IF that wraps another IF (such as a CsvInputFormat). The wrapping IF forwards all calls to the wrapped IF and in addition counts how many records were emitted (how often InputFormat.nextRecord() was called). Once the count arrives at the threshold, it

Flink first() operator

2016-04-23 Thread Biplob Biswas
Hi, It might be a naive question but I was concerned as I am trying to read from a file. My question is if I have a file with n lines and i want m lines out of that where n << m, would the first operator process only the first m lines or would it go through the entire file? If it does go through