In your example, a.attributes.name is a list and is not a string . Run this
to find it out :
a.select($"a.attributes.name").show()
On Mon, Aug 24, 2015 at 2:51 PM, Ilya Karpov wrote:
> Hi, guys
> I'm confused about joining columns in SparkSQL and need your advice.
> I want to join 2 datasets o
Thanks,
but I think this is not the case of multiple spark contexts (never the less I
tried your suggestion - didn’t worked). The problem is join to datasets using
array items value: attribute.value in my case. Has anyone ideas?
> 24 авг. 2015 г., в 15:01, satish chandra j
> написал(а):
>
>
Hi,
If you join logic is correct, it seems to be a similar issue which i faced
recently
Can you try by
*SparkContext(conf).set("spark.driver.allowMultipleContexts","true")*
Regards,
Satish Chandra
On Mon, Aug 24, 2015 at 2:51 PM, Ilya Karpov wrote:
> Hi, guys
> I'm confused about joining colum
Hi, guys
I'm confused about joining columns in SparkSQL and need your advice.
I want to join 2 datasets of profiles. Each profile has name and array of
attributes(age, gender, email etc).
There can be mutliple instances of attribute with the same name, e.g. profile
has 2 emails - so 2 attributes