Hello Rahul,
Please try to use df.filter(df("id").isin(1,2))
Thanks,
On Thu, Mar 30, 2017 at 10:45 PM, Rahul Nandi
wrote:
> Hi,
> I have around 2 million data as parquet file in s3. The file structure is
> somewhat like
> id data
> 1 abc
> 2 cdf
> 3 fas
> Now I want to filter and take the reco
Hi,
I have around 2 million data as parquet file in s3. The file structure is
somewhat like
id data
1 abc
2 cdf
3 fas
Now I want to filter and take the records where the id matches with my
required Id.
val requiredDataId = Array(1,2) //Might go upto 100s of records.
df.filter(requiredDataId.conta
