Re: How to configure parquet.block.size on Spark 1.6

2016-04-08 Thread nihed mbarek 
I can't write on hadoopConfig in Java

Le vendredi 8 avril 2016, Silvio Fiorito  a
écrit :

> Have you tried:
>
> sc.hadoopConfiguration.setLong(parquet.hadoop.ParquetOutputFormat.BLOCK_SIZE,
> N * 1024 * 1024)
>
> Not sure if it’d work or not, but since it’s getting it from the Hadoop
> config it should do it.
>
>
> From: nihed mbarek  >
> Date: Friday, April 8, 2016 at 12:01 PM
> To: "User@spark.apache.org
> " <
> User@spark.apache.org
> >
> Subject: How to configure parquet.block.size on Spark 1.6
>
> Hi
> How to configure parquet.block.size on Spark 1.6 ?
>
> Thank you
> Nihed MBAREK
>
>
> --
>
> M'BAREK Med Nihed,
> Fedora Ambassador, TUNISIA, Northern Africa
> http://www.nihed.com
>
> <http://tn.linkedin.com/in/nihed>
>
>
>
>
>

-- 

M'BAREK Med Nihed,
Fedora Ambassador, TUNISIA, Northern Africa
http://www.nihed.com

<http://tn.linkedin.com/in/nihed>

Re: How to configure parquet.block.size on Spark 1.6

2016-04-08 Thread Silvio Fiorito 
Have you tried:

sc.hadoopConfiguration.setLong(parquet.hadoop.ParquetOutputFormat.BLOCK_SIZE, N 
* 1024 * 1024)

Not sure if it’d work or not, but since it’s getting it from the Hadoop config 
it should do it.


From: nihed mbarek mailto:nihe...@gmail.com>>
Date: Friday, April 8, 2016 at 12:01 PM
To: "User@spark.apache.org<mailto:User@spark.apache.org>" 
mailto:User@spark.apache.org>>
Subject: How to configure parquet.block.size on Spark 1.6

Hi
How to configure parquet.block.size on Spark 1.6 ?

Thank you
Nihed MBAREK


--

M'BAREK Med Nihed,
Fedora Ambassador, TUNISIA, Northern Africa
http://www.nihed.com

[http://www.linkedin.com/img/webpromo/btn_myprofile_160x33_fr_FR.png]<http://tn.linkedin.com/in/nihed>

Re: How to configure parquet.block.size on Spark 1.6

2016-04-08 Thread Ted Yu 
I searched 1.6.1 code base but didn't find how this can be configured
(within Spark).

On Fri, Apr 8, 2016 at 9:01 AM, nihed mbarek  wrote:

> Hi
> How to configure parquet.block.size on Spark 1.6 ?
>
> Thank you
> Nihed MBAREK
>
>
> --
>
> M'BAREK Med Nihed,
> Fedora Ambassador, TUNISIA, Northern Africa
> http://www.nihed.com
>
> <http://tn.linkedin.com/in/nihed>
>
>
>

How to configure parquet.block.size on Spark 1.6

2016-04-08 Thread nihed mbarek 
Hi
How to configure parquet.block.size on Spark 1.6 ?

Thank you
Nihed MBAREK


-- 

M'BAREK Med Nihed,
Fedora Ambassador, TUNISIA, Northern Africa
http://www.nihed.com

<http://tn.linkedin.com/in/nihed>