Hi,
I have one dataset with parameters and another with data that needs to
apply/ filter based on the first dataset (Parameter dataset).
*Scenario is as follows:*
For each row in parameter dataset, I need to apply the parameter row to
the second dataset.I will end up having multiple dataset. for each second
dataset i need to run a bunch of calculation.
How can I achieve this in spark?
*Pseudo code for better understanding:*
Dataset<Parameter> paramsDataset = sparkSession.sql("select * from
paramsview");
Dataset<myData> myDataset = sparkSession.sql("select * from tempview");
Question: For each row in paramsDataset, I need to filter myDataset and run
some calculations on it. Is it possible to do that ? if not whats the best
way to solve it?
Thanks
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
