Re: [SQL] Write parquet files under partition directories?

, Mingyu Kim < > m...@palantir.com>, Andrew Ash > Subject: Re: [SQL] Write parquet files under partition directories? > > There will be in 1.4. > > df.write.partitionBy("year", "month", "day").parquet("/path/to/output") > > On Mon

Re: [SQL] Write parquet files under partition directories?

Excellent! Where can I find the code, pull request, and Spark ticket where this was introduced? Thanks, -Matt Cheah From: Reynold Xin Date: Monday, June 1, 2015 at 10:25 PM To: Matt Cheah Cc: "dev@spark.apache.org" , Mingyu Kim , Andrew Ash Subject: Re: [SQL] Write parquet f

Re: [SQL] Write parquet files under partition directories?

There will be in 1.4. df.write.partitionBy("year", "month", "day").parquet("/path/to/output") On Mon, Jun 1, 2015 at 10:21 PM, Matt Cheah wrote: > Hi there, > > I noticed in the latest Spark SQL programming guide > , there > is su

[SQL] Write parquet files under partition directories?

Hi there, I noticed in the latest Spark SQL programming guide , there is support for optimized reading of partitioned Parquet files that have a particular directory structure (year=1/month=10/day=3, for example). However, I see no a