spark git commit: Tighten up field/method visibility in Executor and made some code more clear to read.

2015-03-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master f17d43b03 -> 0745a305f Tighten up field/method visibility in Executor and made some code more clear to read. I was reading Executor just now and found that some latest changes introduced some weird code path with too much monadic chaining

spark git commit: [SPARK-6219] [Build] Check that Python code compiles

2015-03-19 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 3b5aaa6a5 -> f17d43b03 [SPARK-6219] [Build] Check that Python code compiles This PR expands the Python lint checks so that they check for obvious compilation errors in our Python code. For example: ``` $ ./dev/lint-python Python lint che

spark git commit: [Core][minor] remove unused `visitedStages` in `DAGScheduler.stageDependsOn`

2015-03-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8cb23a1f9 -> 3b5aaa6a5 [Core][minor] remove unused `visitedStages` in `DAGScheduler.stageDependsOn` We define and update `visitedStages` in `DAGScheduler.stageDependsOn`, but never read it. So we can safely remove it. Author: Wenchen Fan

spark git commit: [SPARK-5313][Project Infra]: Create simple framework for highlighting changes introduced in a PR

2015-03-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master dda4dedca -> 8cb23a1f9 [SPARK-5313][Project Infra]: Create simple framework for highlighting changes introduced in a PR Built a simple framework with a `dev/tests` directory to house all pull request related tests. I've moved the two orig

spark git commit: [SPARK-6291] [MLLIB] GLM toString & toDebugString

2015-03-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 3c4e486b9 -> dda4dedca [SPARK-6291] [MLLIB] GLM toString & toDebugString GLM toString prints out intercept, numFeatures. For LogisticRegression and SVM model, toString also prints out numClasses, threshold. GLM toDebugString prints out the

spark git commit: [SPARK-5843] [API] Allowing map-side combine to be specified in Java.

2015-03-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 797f8a000 -> 3c4e486b9 [SPARK-5843] [API] Allowing map-side combine to be specified in Java. Specifically, when calling JavaPairRDD.combineByKey(), there is a new six-parameter method that exposes the map-side-combine boolean as the fifth p

spark git commit: [SPARK-6402][DOC] - Remove some refererences to shark in docs and ec2

2015-03-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2c3f83c34 -> 797f8a000 [SPARK-6402][DOC] - Remove some refererences to shark in docs and ec2 EC2 script and job scheduling documentation still refered to Shark. I removed these references. I also removed a remaining `SHARK_VERSION` variabl