No, that is why I included the link to SPARK-2096
<https://issues.apache.org/jira/browse/SPARK-2096> as well. You'll need to
use HiveQL at this time.
Is it possible or planed to support the "schools.time" format to filter the
>> record that there is an element inside array of schools satisfy time > 2?
>>
>
It would be great to support something like this, but its going to take a
while to hammer out the correct semantics as SQL does not in general have
great support for nested structures. I think different people might
interpret that query to mean there is SOME school.time >2 vs. ALL
school.time > 2, etc.
You can get what you want now using a lateral view:
hql("SELECT DISTINCT name FROM people LATERAL VIEW explode(schools) s as
school WHERE school.time > 2")
