Re: HiveContext fails when querying large external Parquet tables

2015-05-22 Thread Andrew Otto
What is also strange is that this seems to work on external JSON data, but not Parquet. I’ll try to do more verification of that next week. On May 22, 2015, at 16:24, yana yana.kadiy...@gmail.com wrote: There is an open Jira on Spark not pushing predicates to metastore. I have a large

RE: HiveContext fails when querying large external Parquet tables

2015-05-22 Thread yana
There is an open Jira on Spark not pushing predicates to metastore. I have a large dataset with many partitions but doing anything with it 8s very slow...But I am surprised Spark 1.2 worked for you: it has this problem... div Original message /divdivFrom: Andrew Otto