and then I
>>>> applied toDF() function.
>>>> Something like that in your case:
>>>>
>>>> case class Event(id: String, event: String, entityType: String,
>>>> entityId: String, targetEntityType: String, targetEntityId: String,
>>>&g
ied toDF() function.
>>> Something like that in your case:
>>>
>>> case class Event(id: String, event: String, entityType: String, entityId:
>>> String, targetEntityType: String, targetEntityId: String, properties:
>>> String)
>>>
t; Something like that in your case:
>>
>> case class Event(id: String, event: String, entityType: String, entityId:
>> String, targetEntityType: String, targetEntityId: String, properties:
>> String)
>> eventRDD.map{case (id, event, entityType, entityId, targetEntityType,
targetEntityId: String, properties:
> String)
> eventRDD.map{case (id, event, entityType, entityId, targetEntityType,
> targetEntityId,
> properties) =>
> Event(id, event, entityType, entityId, targetEntityType, targetEntityId,
> properties) }.toDF()
>
> The comparison === sho
Hi, Yin
Thanks for your time. This is the result.
--
scala> eventDF.filter($"entityType" ===
"user").select("entityId").distinct.explain(true)
== Parsed Logical Plan ==
Aggregate [entityId#16], [entityId#16]
Project [entityId#16]
Filter (entityType#15 = user)
Project [_1#0 A
Can you attach the result of eventDF.filter($"entityType" ===
"user").select("entityId").distinct.explain(true)?
Thanks,
Yin
On Thu, Nov 5, 2015 at 1:12 AM, 千成徳 wrote:
> Hi All,
>
> I have data frame like this.
>
> Equality expression is not working in 1.5.1 but, works as expected in 1.4.0
> W
Hi All,
I have data frame like this.
Equality expression is not working in 1.5.1 but, works as expected in 1.4.0
What is the difference?
scala> eventDF.printSchema()
root
|-- id: string (nullable = true)
|-- event: string (nullable = true)
|-- entityType: string (nullable = true)
|-- entityI
