Re: Data Source Generator emits 4 instances of the same tuple

[Biplob Biswas]

Yes Thanks a lot, also the fact that I was using ParallelSourceFunction was
problematic. So as suggested by Fabian and Robert, I used Source Function
and then in the flink job, i set the output of map with a parallelism of 4
to get the desired result.

Thanks again.



--
View this message in context: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Source-Generator-emits-4-instances-of-the-same-tuple-tp7392p7513.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at 
Nabble.com.

Re: Data Source Generator emits 4 instances of the same tuple

[Fabian Hueske]

We solved this problem yesterday at the Flink Hackathon.
The issue was that the source function was started with parallelism 4 and
each function read the whole file.

Cheers, Fabian

2016-06-06 16:53 GMT+02:00 Biplob Biswas :

> Hi,
>
> I tried streaming the source data 2 ways
>
> 1. Is a simple straight forward way of sending data without using the
> serving speed concept
> http://pastebin.com/cTv0Pk5U
>
>
> 2. The one where I use the TaxiRide source which is exactly similar except
> loading the data in the proper data structures.
> http://pastebin.com/NenvXShH
>
>
> I hope to get a solution out of it.
>
> Thanks and Regards
> Biplob Biswas
>
>
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Source-Generator-emits-4-instances-of-the-same-tuple-tp7392p7405.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Re: Data Source Generator emits 4 instances of the same tuple

[Biplob Biswas]

Hi,

I tried streaming the source data 2 ways 

1. Is a simple straight forward way of sending data without using the
serving speed concept
http://pastebin.com/cTv0Pk5U


2. The one where I use the TaxiRide source which is exactly similar except
loading the data in the proper data structures.
http://pastebin.com/NenvXShH


I hope to get a solution out of it.

Thanks and Regards
Biplob Biswas





--
View this message in context: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Source-Generator-emits-4-instances-of-the-same-tuple-tp7392p7405.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at 
Nabble.com.