scalastyle violation on mvn install but not on mvn package

2017-05-17 Thread yiskylee
./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package works, but ./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean install triggers scalastyle violation error. Is the scalastyle check not used on package but only on install? To install, should

scalastyle violation on mvn install but not on mvn package

2017-05-04 Thread yiskylee
./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package works, but ./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean install triggers scalastyle violation error. Is the scalastyle check not used on package but only on install? To install, should I

SparkPi running slower with more cores on each worker

2015-11-30 Thread yiskylee
Hi, I was running the SparkPi example code and studying their performance difference using different number of cores per worker. I change the number of cores by using start-slave.sh -c CORES on the worker machine for distributed computation. I also use spark-submit --master local[CORES] for the