Have you tried creating a DataFrame from the RDD and join with DataFrame which corresponds to the hive table ?
On Sun, Feb 14, 2016 at 9:53 PM, SRK <swethakasire...@gmail.com> wrote: > Hi, > > How to join an RDD with a hive table and retrieve only the records that I > am > interested. Suppose, I have an RDD that has 1000 records and there is a > Hive > table with 100,000 records, I should be able to join the RDD with the hive > table by an Id and I should be able to load only those 1000 records from > Hive table so that are no memory issues. Also, I was planning on storing > the > data in hive in the form of parquet files. Any help on this is greatly > appreciated. > > Thanks! > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-join-an-RDD-with-a-hive-table-tp26225.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
