Hi Divya,
That's the right way to access a value for a key in a broadcast map.
I'm pretty sure tough that you could do the same (or similar) with
higher-level broadcast Datasets. Try it out!
Pozdrawiam,
Jacek Laskowski
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/
Hi Jacek,
Can you please share example how can I access broacasted map
val pltStnMapBrdcst = sc.broadcast(keyvalueMap )
val df_replacekeys = df_input.withColumn("map_values",
pltStnMapBrdcst.value.get("key"
Is the above the right way to access the broadcasted map ?
Thanks,
Divya
On 18 Ju
See broadcast variable.
Or (just a thought) do join between DataFrames.
Jacek
On 18 Jul 2016 9:24 a.m., "Divya Gehlot" wrote:
> Hi,
>
> I have created a map by reading a text file
> val keyValueMap = file_read.map(t => t.getString(0) ->
> t.getString(4)).collect().toMap
>
> Now I have another
Hi,
I have created a map by reading a text file
val keyValueMap = file_read.map(t => t.getString(0) ->
t.getString(4)).collect().toMap
Now I have another dataframe where I need to dynamically replace all the
keys of Map with values
val df_input = reading the file as dataframe
val df_replacekeys =