Re: Control the stdout and stderr streams in a executor JVM

You can create log4j.properties for executors, and use "--files log4j.properties" when submitting spark jobs. On Mon, Feb 29, 2016 at 1:50 PM, Niranda Perera wrote: > Hi all, > > Is there any possibility to control the stdout and stderr streams in an > executor JVM? > > I understand that there a

Control the stdout and stderr streams in a executor JVM

Hi all, Is there any possibility to control the stdout and stderr streams in an executor JVM? I understand that there are some configurations provided from the spark conf as follows spark.executor.logs.rolling.maxRetainedFiles spark.executor.logs.rolling.maxSize spark.executor.logs.rolling.strate

Re: Is spark.driver.maxResultSize used correctly ?

data skew might be possible, but not the common case. I think we should design for the common case, for the skew case, we may can set some parameter of fraction to allow user to tune it. On Sat, Feb 27, 2016 at 4:51 PM, Reynold Xin wrote: > But sometimes you might have skew and almost all the re

Implementing Bagging ensemble method using spark.mlLib

Hi, I found out that ml Lib supports two ensemble algorithms, GBT and Random Forest. I want to implement Bagging method using ml Lib features. Can you give me the location that contains the implementation of GBT and Random Forest methods in the repository. Also I'm grateful if you can give me some