/divdivDate:05/07/2015 7:38 AM (GMT-05:00)
/divdivTo: Olivier Girardot ssab...@gmail.com /divdivCc:
user@spark.apache.org /divdivSubject: Re: Spark 1.3.1 and Parquet
Partitions /divdiv
/divOlivier
Nope. Wildcard extensions don't work I am debugging the code to figure out
what's wrong I know I am
Olivier
Nope. Wildcard extensions don't work I am debugging the code to figure out
what's wrong I know I am using 1.3.1 for sure
Pardon typos...
On May 7, 2015, at 7:06 AM, Olivier Girardot ssab...@gmail.com wrote:
hdfs://some ip:8029/dataset/*/*.parquet doesn't work for you ?
Le jeu. 7
Hi V,
I am assuming that each of the three .parquet paths you mentioned have
multiple partitions in them.
For eg: [/dataset/city=London/data.parquet/part-r-0.parquet,
/dataset/city=London/data.parquet/part-r-1.parquet]
I haven't personally used this with hdfs, but I've worked with a similar
From: Vaxuki
Date:05/07/2015 7:38 AM (GMT-05:00)
To: Olivier Girardot
Cc: user@spark.apache.org
Subject: Re: Spark 1.3.1 and Parquet Partitions
Olivier
Nope. Wildcard extensions don't work I am debugging the code to figure out
what's wrong I know I am using 1.3.1 for sure
Pardon typos
hdfs://some ip:8029/dataset/*/*.parquet doesn't work for you ?
Le jeu. 7 mai 2015 à 03:32, vasuki vax...@gmail.com a écrit :
Spark 1.3.1 -
i have a parquet file on hdfs partitioned by some string looking like this
/dataset/city=London/data.parquet
/dataset/city=NewYork/data.parquet
Spark 1.3.1 -
i have a parquet file on hdfs partitioned by some string looking like this
/dataset/city=London/data.parquet
/dataset/city=NewYork/data.parquet
/dataset/city=Paris/data.paruqet
….
I am trying to get to load it using sqlContext using sqlcontext.parquetFile(
hdfs://some
