spark git commit: [MINOR][BUILD] Add modernizr MIT license; specify "2014 and onwards" in license copyright

2016-06-04 Thread srowen
ense. Specify "2014 and onwards" in copyright statement. ## How was this patch tested? (none required) Author: Sean Owen Closes #13510 from srowen/ModernizrLicense. (cherry picked from commit 681387b2dc9a094cfba84188a1dd1ac9192bb99c) Signed-off-by: Sean Owen Project: http://git-

spark git commit: [SPARK-15707][SQL] Make Code Neat - Use map instead of if check.

2016-06-04 Thread srowen
Repository: spark Updated Branches: refs/heads/master 091f81e1f -> 0f307db5e [SPARK-15707][SQL] Make Code Neat - Use map instead of if check. ## What changes were proposed in this pull request? In forType function of object RandomDataGenerator, the code following: if (maybeSqlTypeGenerator.isD

spark git commit: [SPARK-15707][SQL] Make Code Neat - Use map instead of if check.

2016-06-04 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 7e4c9dd55 -> 32a64d8fc [SPARK-15707][SQL] Make Code Neat - Use map instead of if check. ## What changes were proposed in this pull request? In forType function of object RandomDataGenerator, the code following: if (maybeSqlTypeGenerator

spark git commit: [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is …

2016-06-05 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0f307db5e -> 4e767d0f9 [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is … ## What changes were proposed in this pull request? Stop using the abbreviated and ambiguous timezone "EST" in a test, since it

spark git commit: [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is …

2016-06-05 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 32a64d8fc -> 8c0ec85e6 [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is … ## What changes were proposed in this pull request? Stop using the abbreviated and ambiguous timezone "EST" in a test, since

spark git commit: [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is …

2016-06-05 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 a0cf7d0b2 -> 6a9f19dd5 [SPARK-15723] Fixed local-timezone-brittle test where short-timezone form "EST" is … ## What changes were proposed in this pull request? Stop using the abbreviated and ambiguous timezone "EST" in a test, since

spark git commit: [MINOR] Fix Typos 'an -> a'

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/master 32f2f95db -> fd8af3971 [MINOR] Fix Typos 'an -> a' ## What changes were proposed in this pull request? `an -> a` Use cmds like `find . -name '*.R' | xargs -i sh -c "grep -in ' an [^aeiou]' {} && echo {}"` to generate candidates, and revi

spark git commit: [SPARK-15771][ML][EXAMPLES] Use 'accuracy' rather than 'precision' in many ML examples

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/master fd8af3971 -> a95252823 [SPARK-15771][ML][EXAMPLES] Use 'accuracy' rather than 'precision' in many ML examples ## What changes were proposed in this pull request? Since [SPARK-15617](https://issues.apache.org/jira/browse/SPARK-15617) depre

spark git commit: [MINOR] Fix Typos 'an -> a'

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 7d10e4bdd -> 90e94b826 [MINOR] Fix Typos 'an -> a' ## What changes were proposed in this pull request? `an -> a` Use cmds like `find . -name '*.R' | xargs -i sh -c "grep -in ' an [^aeiou]' {} && echo {}"` to generate candidates, and

spark git commit: [SPARK-15771][ML][EXAMPLES] Use 'accuracy' rather than 'precision' in many ML examples

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 90e94b826 -> 86a35a229 [SPARK-15771][ML][EXAMPLES] Use 'accuracy' rather than 'precision' in many ML examples ## What changes were proposed in this pull request? Since [SPARK-15617](https://issues.apache.org/jira/browse/SPARK-15617) d

spark git commit: [SPARK-14900][ML][PYSPARK] Add accuracy and deprecate precison, recall, f1

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/master a95252823 -> 00ad4f054 [SPARK-14900][ML][PYSPARK] Add accuracy and deprecate precison,recall,f1 ## What changes were proposed in this pull request? 1, add accuracy for MulticlassMetrics 2, deprecate overall precision,recall,f1 and recommend

spark git commit: [SPARK-14900][ML][PYSPARK] Add accuracy and deprecate precison, recall, f1

2016-06-06 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 86a35a229 -> e38ff70e6 [SPARK-14900][ML][PYSPARK] Add accuracy and deprecate precison,recall,f1 ## What changes were proposed in this pull request? 1, add accuracy for MulticlassMetrics 2, deprecate overall precision,recall,f1 and recom

svn commit: r1747061 - in /spark: downloads.md js/downloads.js site/downloads.html site/js/downloads.js

2016-06-06 Thread srowen
Author: srowen Date: Mon Jun 6 19:56:07 2016 New Revision: 1747061 URL: http://svn.apache.org/viewvc?rev=1747061&view=rev Log: SPARK-15778 add spark-2.0.0-preview release to options and other minor related updates Modified: spark/downloads.md spark/js/downloads.js spark/

svn commit: r1747076 - in /spark: js/downloads.js site/js/downloads.js

2016-06-06 Thread srowen
Author: srowen Date: Mon Jun 6 20:59:54 2016 New Revision: 1747076 URL: http://svn.apache.org/viewvc?rev=1747076&view=rev Log: SPARK-15778 part 2: group preview/stable releases in download version dropdown Modified: spark/js/downloads.js spark/site/js/downloads.js Modified: spar

spark git commit: [SPARK-12655][GRAPHX] GraphX does not unpersist RDDs

2016-06-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 6a9f19dd5 -> 5830828ef [SPARK-12655][GRAPHX] GraphX does not unpersist RDDs Some VertexRDD and EdgeRDD are created during the intermediate step of g.connectedComponents() but unnecessarily left cached after the method is done. The fix

spark git commit: [MINOR] fix typo in documents

2016-06-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 57dd4efcd -> a7e9e60df [MINOR] fix typo in documents ## What changes were proposed in this pull request? I use spell check tools checks typo in spark documents and fix them. ## How was this patch tested? N/A Author: WeichenXu Clos

spark git commit: [MINOR] fix typo in documents

2016-06-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master 5f731d685 -> 1e2c93118 [MINOR] fix typo in documents ## What changes were proposed in this pull request? I use spell check tools checks typo in spark documents and fix them. ## How was this patch tested? N/A Author: WeichenXu Closes #

spark git commit: [SPARK-15793][ML] Add maxSentenceLength for ml.Word2Vec

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 91fbc880b -> 87706eb66 [SPARK-15793][ML] Add maxSentenceLength for ml.Word2Vec ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-15793 Word2vec in ML package should have maxSentenceLength meth

spark git commit: [SPARK-15793][ML] Add maxSentenceLength for ml.Word2Vec

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 141e910af -> a790ac579 [SPARK-15793][ML] Add maxSentenceLength for ml.Word2Vec ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-15793 Word2vec in ML package should have maxSentenceLength

svn commit: r1747385 - in /spark: ./ site/ site/docs/ site/docs/2.0.0-preview/ site/docs/2.0.0-preview/api/ site/docs/2.0.0-preview/api/R/ site/docs/2.0.0-preview/api/java/ site/docs/2.0.0-preview/api

2016-06-08 Thread srowen
Author: srowen Date: Wed Jun 8 12:04:28 2016 New Revision: 1747385 URL: http://svn.apache.org/viewvc?rev=1747385&view=rev Log: Uploaded Spark 2.0.0 preview docs and added preview docs section on site [This commit notification would consist of 1214 parts, which exceeds the limit of 50 ones

spark git commit: [MINOR] Fix Java Lint errors introduced by #13286 and #13280

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 a790ac579 -> 5e9a8e715 [MINOR] Fix Java Lint errors introduced by #13286 and #13280 ## What changes were proposed in this pull request? revived #13464 Fix Java Lint errors introduced by #13286 and #13280 Before: ``` Using `mvn` from p

spark git commit: [MINOR] Fix Java Lint errors introduced by #13286 and #13280

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 87706eb66 -> f958c1c3e [MINOR] Fix Java Lint errors introduced by #13286 and #13280 ## What changes were proposed in this pull request? revived #13464 Fix Java Lint errors introduced by #13286 and #13280 Before: ``` Using `mvn` from path:

spark git commit: [DOCUMENTATION] Fixed target JAR path

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master f958c1c3e -> ca70ab27c [DOCUMENTATION] Fixed target JAR path ## What changes were proposed in this pull request? Mentioned Scala version in the sbt configuration file is 2.11, so the path of the target JAR should be `/target/scala-2.11/si

spark git commit: [DOCUMENTATION] Fixed target JAR path

2016-06-08 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 5e9a8e715 -> b2778c8bb [DOCUMENTATION] Fixed target JAR path ## What changes were proposed in this pull request? Mentioned Scala version in the sbt configuration file is 2.11, so the path of the target JAR should be `/target/scala-2.1

spark git commit: [SPARK-15818][BUILD] Upgrade to Hadoop 2.7.2

2016-06-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master 921fa40b1 -> 147c02082 [SPARK-15818][BUILD] Upgrade to Hadoop 2.7.2 ## What changes were proposed in this pull request? Updating the Hadoop version from 2.7.0 to 2.7.2 if we use the Hadoop-2.7 build profile ## How was this patch tested?

spark git commit: [SPARK-15818][BUILD] Upgrade to Hadoop 2.7.2

2016-06-09 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 8ee93eed9 -> 77c08d224 [SPARK-15818][BUILD] Upgrade to Hadoop 2.7.2 ## What changes were proposed in this pull request? Updating the Hadoop version from 2.7.0 to 2.7.2 if we use the Hadoop-2.7 build profile ## How was this patch test

spark git commit: [SPARK-15823][PYSPARK][ML] Add @property for 'accuracy' in MulticlassMetrics

2016-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 675a73715 -> 16ca32eac [SPARK-15823][PYSPARK][ML] Add @property for 'accuracy' in MulticlassMetrics ## What changes were proposed in this pull request? `accuracy` should be decorated with `property` to keep step with other methods in `pysp

spark git commit: [SPARK-15823][PYSPARK][ML] Add @property for 'accuracy' in MulticlassMetrics

2016-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 84a8421e5 -> 6709ce1ae [SPARK-15823][PYSPARK][ML] Add @property for 'accuracy' in MulticlassMetrics ## What changes were proposed in this pull request? `accuracy` should be decorated with `property` to keep step with other methods in `

spark git commit: [SPARK-15837][ML][PYSPARK] Word2vec python add maxsentence parameter

2016-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 16ca32eac -> cdd7f5a57 [SPARK-15837][ML][PYSPARK] Word2vec python add maxsentence parameter ## What changes were proposed in this pull request? Word2vec python add maxsentence parameter. ## How was this patch tested? Existing test. Auth

spark git commit: [SPARK-15837][ML][PYSPARK] Word2vec python add maxsentence parameter

2016-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 6709ce1ae -> 54b4763d2 [SPARK-15837][ML][PYSPARK] Word2vec python add maxsentence parameter ## What changes were proposed in this pull request? Word2vec python add maxsentence parameter. ## How was this patch tested? Existing test.

spark git commit: [SPARK-15879][DOCS][UI] Update logo in UI and docs to add "Apache"

2016-06-11 Thread srowen
old unreferenced logo files. ## How was this patch tested? Manual check of generated HTML site and Spark UI. I searched for references to the deleted files to make sure they were not used. Author: Sean Owen Closes #13609 from srowen/SPARK-15879. Project: http://git-wip-us.apache.org/repo

spark git commit: [SPARK-15879][DOCS][UI] Update logo in UI and docs to add "Apache"

2016-06-11 Thread srowen
emove old unreferenced logo files. ## How was this patch tested? Manual check of generated HTML site and Spark UI. I searched for references to the deleted files to make sure they were not used. Author: Sean Owen Closes #13609 from srowen/SPARK-15879. (cherry picked from commit 3761330dd0151d736

spark git commit: [SPARK-15883][MLLIB][DOCS] Fix broken links in mllib documents

2016-06-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master 3761330dd -> ad102af16 [SPARK-15883][MLLIB][DOCS] Fix broken links in mllib documents ## What changes were proposed in this pull request? This issue fixes all broken links on Spark 2.0 preview MLLib documents. Also, this contains some edi

spark git commit: [SPARK-15883][MLLIB][DOCS] Fix broken links in mllib documents

2016-06-11 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 4c29c55f2 -> 8cf33fb8a [SPARK-15883][MLLIB][DOCS] Fix broken links in mllib documents ## What changes were proposed in this pull request? This issue fixes all broken links on Spark 2.0 preview MLLib documents. Also, this contains some

spark git commit: [SPARK-15878][CORE][TEST] fix cleanup in EventLoggingListenerSuite and ReplayListenerSuite

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9e204c62c -> 8cc22b008 [SPARK-15878][CORE][TEST] fix cleanup in EventLoggingListenerSuite and ReplayListenerSuite ## What changes were proposed in this pull request? These tests weren't properly using `LocalSparkContext` so weren't cleani

spark git commit: [SPARK-15878][CORE][TEST] fix cleanup in EventLoggingListenerSuite and ReplayListenerSuite

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 d494a483a -> 879e8fd09 [SPARK-15878][CORE][TEST] fix cleanup in EventLoggingListenerSuite and ReplayListenerSuite ## What changes were proposed in this pull request? These tests weren't properly using `LocalSparkContext` so weren't cl

spark git commit: [SPARK-15781][DOCUMENTATION] remove deprecated environment variable doc

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 879e8fd09 -> 8c294f4ad [SPARK-15781][DOCUMENTATION] remove deprecated environment variable doc ## What changes were proposed in this pull request? Like `SPARK_JAVA_OPTS` and `SPARK_CLASSPATH`, we will remove the document for `SPARK_WO

spark git commit: [SPARK-15781][DOCUMENTATION] remove deprecated environment variable doc

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8cc22b008 -> 3fd3ee038 [SPARK-15781][DOCUMENTATION] remove deprecated environment variable doc ## What changes were proposed in this pull request? Like `SPARK_JAVA_OPTS` and `SPARK_CLASSPATH`, we will remove the document for `SPARK_WORKER

spark git commit: [SPARK-15806][DOCUMENTATION] update doc for SPARK_MASTER_IP

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 3fd3ee038 -> 50248dcff [SPARK-15806][DOCUMENTATION] update doc for SPARK_MASTER_IP ## What changes were proposed in this pull request? SPARK_MASTER_IP is a deprecated environment variable. It is replaced by SPARK_MASTER_HOST according to

spark git commit: [SPARK-15806][DOCUMENTATION] update doc for SPARK_MASTER_IP

2016-06-12 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 8c294f4ad -> b75d1c201 [SPARK-15806][DOCUMENTATION] update doc for SPARK_MASTER_IP ## What changes were proposed in this pull request? SPARK_MASTER_IP is a deprecated environment variable. It is replaced by SPARK_MASTER_HOST according

spark git commit: [SPARK-15813] Improve Canceling log message to make it less ambiguous

2016-06-13 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 b96e7f6aa -> 41f309bfb [SPARK-15813] Improve Canceling log message to make it less ambiguous ## What changes were proposed in this pull request? Add new desired executor number to make the log message less ambiguous. ## How was this pa

spark git commit: [SPARK-15813] Improve Canceling log message to make it less ambiguous

2016-06-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master e2ab79d5e -> d681742b2 [SPARK-15813] Improve Canceling log message to make it less ambiguous ## What changes were proposed in this pull request? Add new desired executor number to make the log message less ambiguous. ## How was this patch

spark git commit: [DOCUMENTATION] fixed typos in python programming guide

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 688b6ef9d -> a87a56f5c [DOCUMENTATION] fixed typos in python programming guide ## What changes were proposed in this pull request? minor typo ## How was this patch tested? minor typo in the doc, should be self explanatory Author: Mortad

spark git commit: [DOCUMENTATION] fixed typos in python programming guide

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 974be6241 -> cf52375b9 [DOCUMENTATION] fixed typos in python programming guide ## What changes were proposed in this pull request? minor typo ## How was this patch tested? minor typo in the doc, should be self explanatory Author: Mo

spark git commit: [SPARK-15821][DOCS] Include parallel build info

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 96c3500c6 -> a431e3f1f [SPARK-15821][DOCS] Include parallel build info ## What changes were proposed in this pull request? We should mention that users can build Spark using multiple threads to decrease build times; either here or in "Bui

spark git commit: [SPARK-15821][DOCS] Include parallel build info

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 d59859d38 -> 0d80bc291 [SPARK-15821][DOCS] Include parallel build info ## What changes were proposed in this pull request? We should mention that users can build Spark using multiple threads to decrease build times; either here or in

spark git commit: doc fix of HiveThriftServer

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master a431e3f1f -> 53bb03084 doc fix of HiveThriftServer ## What changes were proposed in this pull request? Just minor doc fix. \cc yhuai Author: Jeff Zhang Closes #13659 from zjffdu/doc_fix. Project: http://git-wip-us.apache.org/repos/as

spark git commit: doc fix of HiveThriftServer

2016-06-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 0d80bc291 -> e90ba2287 doc fix of HiveThriftServer ## What changes were proposed in this pull request? Just minor doc fix. \cc yhuai Author: Jeff Zhang Closes #13659 from zjffdu/doc_fix. (cherry picked from commit 53bb03084796231f

spark git commit: [MINOR] Clean up several build warnings, mostly due to internal use of old accumulators

2016-06-14 Thread srowen
API usage in tests that are straightforward to update. I think this qualifies as "minor". Jenkins Author: Sean Owen Closes #13642 from srowen/BuildWarnings. (cherry picked from commit 6151d2641f91c8e3ec0c324e78afb46cdb2ef111) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.

spark git commit: [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock`

2016-06-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master f9bf15d9b -> 36110a830 [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock` ## What changes were proposed in this pull request? SPARK-15922 reports the following scenario throwing an exception du

spark git commit: [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock`

2016-06-16 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 5b003c9bc -> 579268426 [SPARK-15922][MLLIB] `toIndexedRowMatrix` should consider the case `cols < offset+colsPerBlock` ## What changes were proposed in this pull request? SPARK-15922 reports the following scenario throwing an exceptio

spark git commit: [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config

2016-06-16 Thread srowen
der to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14 ## How was this patch tested? Jenkins tests. Author: Sean Owen Closes #13618 from srowen/SPARK-15796. Project: http:/

spark git commit: [SPARK-15796][CORE] Reduce spark.memory.fraction default to avoid overrunning old gen in JVM default config

2016-06-16 Thread srowen
in order to make it fit within default JVM old generation size (2/3 heap). See JIRA discussion. This means a full cache doesn't spill into the new gen. CC andrewor14 ## How was this patch tested? Jenkins tests. Author: Sean Owen Closes #13618 from srowen/SPARK-15796. (cherry picked fro

spark git commit: [SPARK-15942][REPL] Unblock `:reset` command in REPL.

2016-06-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 001a58960 -> 1b3a9b966 [SPARK-15942][REPL] Unblock `:reset` command in REPL. ## What changes were proposed in this pull (Paste from JIRA issue.) As a follow up for SPARK-15697, I have following semantics for `:reset` command. On `:reset` we

spark git commit: [SPARK-15942][REPL] Unblock `:reset` command in REPL.

2016-06-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 dc85bd0a0 -> 2c1c337ba [SPARK-15942][REPL] Unblock `:reset` command in REPL. ## What changes were proposed in this pull (Paste from JIRA issue.) As a follow up for SPARK-15697, I have following semantics for `:reset` command. On `:reset

spark git commit: [SPARK-16040][MLLIB][DOC] spark.mllib PIC document extra line of refernece

2016-06-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 2c1c337ba -> 80c6d4e3a [SPARK-16040][MLLIB][DOC] spark.mllib PIC document extra line of refernece ## What changes were proposed in this pull request? In the 2.0 document, Line "A full example that produces the experiment described in

spark git commit: [SPARK-16040][MLLIB][DOC] spark.mllib PIC document extra line of refernece

2016-06-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1b3a9b966 -> 5930d7a2e [SPARK-16040][MLLIB][DOC] spark.mllib PIC document extra line of refernece ## What changes were proposed in this pull request? In the 2.0 document, Line "A full example that produces the experiment described in the

spark git commit: [MINOR] Closing stale pull requests.

2016-06-20 Thread srowen
Repository: spark Updated Branches: refs/heads/master 359c2e827 -> 92514232e [MINOR] Closing stale pull requests. Closes #13114 Closes #10187 Closes #13432 Closes #13550 Author: Sean Owen Closes #13781 from srowen/CloseStalePR. Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table

2016-06-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master a58f40239 -> f3a768b7b [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table ## What changes were proposed in this pull request? 1. FORMATTED is actually supported, but partition is not supported; 2. Remove parenthesis as it is not

spark git commit: [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table

2016-06-21 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 0499ed961 -> 34a8e23c7 [SPARK-16084][SQL] Minor comments update for "DESCRIBE" table ## What changes were proposed in this pull request? 1. FORMATTED is actually supported, but partition is not supported; 2. Remove parenthesis as it is

spark git commit: [SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery

2016-06-22 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 d98fb19c1 -> 4fdac3c27 [SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery ## What changes were proposed in this pull request? Because this test extracts data from `DStream.generatedRDDs` b

spark git commit: [SPARK-15660][CORE] Update RDD `variance/stdev` description and add popVariance/popStdev

2016-06-23 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4374a46bf -> 5eef1e6c6 [SPARK-15660][CORE] Update RDD `variance/stdev` description and add popVariance/popStdev ## What changes were proposed in this pull request? In Spark-11490, `variance/stdev` are redefined as the **sample** `varianc

spark git commit: [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite

2016-06-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2d2f607bf -> f4fd7432f [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite ## What changes were proposed in this pull request? Since SPARK-13220(Deprecate "yarn-client" and "yarn-cluster"), YarnClusterSuite do

spark git commit: [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite

2016-06-24 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 3ccdd6b9c -> b6420db9e [SPARK-16125][YARN] Fix not test yarn cluster mode correctly in YarnClusterSuite ## What changes were proposed in this pull request? Since SPARK-13220(Deprecate "yarn-client" and "yarn-cluster"), YarnClusterSuite

spark git commit: [SPARK-16129][CORE][SQL] Eliminate direct use of commons-lang classes in favor of commons-lang3

2016-06-24 Thread srowen
and forbid the former via scalastyle; remove `NotImplementedException` from `comons-lang` in favor of JDK `UnsupportedOperationException` ## How was this patch tested? Jenkins tests Author: Sean Owen Closes #13843 from srowen/SPARK-16129. Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-16129][CORE][SQL] Eliminate direct use of commons-lang classes in favor of commons-lang3

2016-06-24 Thread srowen
and forbid the former via scalastyle; remove `NotImplementedException` from `comons-lang` in favor of JDK `UnsupportedOperationException` ## How was this patch tested? Jenkins tests Author: Sean Owen Closes #13843 from srowen/SPARK-16129. (cherry picked from com

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

2016-06-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master a7d29499d -> a3c7b4187 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the size o

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

2016-06-25 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 d079b5de7 -> cbfcdcfb6 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the si

spark git commit: [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution.

2016-06-25 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 b7acc1b71 -> 24d59fb64 [MLLIB] org.apache.spark.mllib.util.SVMDataGenerator generates ArrayIndexOutOfBoundsException. I have found the bug and tested the solution. ## What changes were proposed in this pull request? Just adjust the si

spark git commit: [SPARK-15958] Make initial buffer size for the Sorter configurable

2016-06-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master a3c7b4187 -> bf665a958 [SPARK-15958] Make initial buffer size for the Sorter configurable ## What changes were proposed in this pull request? Currently the initial buffer size in the sorter is hard coded inside the code and is too small f

spark git commit: [SPARK-1301][WEB UI] Added anchor links to Accumulators and Tasks on StagePage

2016-06-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master bf665a958 -> 3ee9695d1 [SPARK-1301][WEB UI] Added anchor links to Accumulators and Tasks on StagePage ## What changes were proposed in this pull request? Sometimes the "Aggregated Metrics by Executor" table on the Stage page can get very

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-25 Thread srowen
of stages that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen Closes #13896 from srowen/SPARK-16193. (cherry picked from commit e87741589a24821b5fe73e5d9ee2164247998580) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/repo Com

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-25 Thread srowen
of stages that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen Closes #13896 from srowen/SPARK-16193. (cherry picked from commit e87741589a24821b5fe73e5d9ee2164247998580) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/repo Com

spark git commit: [SPARK-16193][TESTS] Address flaky ExternalAppendOnlyMapSuite spilling tests

2016-06-25 Thread srowen
ges that spilled ## How was this patch tested? Existing Jenkins tests. Author: Sean Owen Closes #13896 from srowen/SPARK-16193. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e8774158 Tree: http://git-wip-us.apache.org/re

spark git commit: [SPARK-16214][EXAMPLES] fix the denominator of SparkPi

2016-06-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 e01776395 -> efce6e17c [SPARK-16214][EXAMPLES] fix the denominator of SparkPi ## What changes were proposed in this pull request? reduce the denominator of SparkPi by 1 ## How was this patch tested? integration tests Author: 杨æµ

spark git commit: [SPARK-16214][EXAMPLES] fix the denominator of SparkPi

2016-06-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master 30b182bcc -> b45202632 [SPARK-16214][EXAMPLES] fix the denominator of SparkPi ## What changes were proposed in this pull request? reduce the denominator of SparkPi by 1 ## How was this patch tested? integration tests Author: 杨浩

spark git commit: [SPARK-16214][EXAMPLES] fix the denominator of SparkPi

2016-06-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 60e095b9b -> 22a496d2a [SPARK-16214][EXAMPLES] fix the denominator of SparkPi ## What changes were proposed in this pull request? reduce the denominator of SparkPi by 1 ## How was this patch tested? integration tests Author: 杨æµ

spark git commit: [MINOR][CORE] Fix display wrong free memory size in the log

2016-06-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master b45202632 -> 52d4fe057 [MINOR][CORE] Fix display wrong free memory size in the log ## What changes were proposed in this pull request? Free memory size displayed in the log is wrong (used memory), fix to make it correct. ## How was this

spark git commit: [MINOR][CORE] Fix display wrong free memory size in the log

2016-06-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 efce6e17c -> ea8d419c1 [MINOR][CORE] Fix display wrong free memory size in the log ## What changes were proposed in this pull request? Free memory size displayed in the log is wrong (used memory), fix to make it correct. ## How was t

svn commit: r1750410 [2/2] - in /spark: ./ _plugins/ mllib/ releases/_posts/ site/ site/mllib/ site/news/ site/releases/ site/sql/ site/streaming/ sql/ streaming/

2016-06-27 Thread srowen
Modified: spark/site/releases/spark-release-1-1-0.html URL: http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-1-0.html?rev=1750410&r1=1750409&r2=1750410&view=diff == --- spark/site/releases/spark-release-1-1

svn commit: r1750410 [1/2] - in /spark: ./ _plugins/ mllib/ releases/_posts/ site/ site/mllib/ site/news/ site/releases/ site/sql/ site/streaming/ sql/ streaming/

2016-06-27 Thread srowen
Author: srowen Date: Mon Jun 27 20:31:41 2016 New Revision: 1750410 URL: http://svn.apache.org/viewvc?rev=1750410&view=rev Log: Remove Spark site plugins (not used/working); fix jekyll build warning and one bad heading tag; remove inactive {% extra %} tag; commit current output of jekyll

spark git commit: [SPARK-15858][ML] Fix calculating error by tree stack over flow prob…

2016-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master 21385d02a -> 393db655c [SPARK-15858][ML] Fix calculating error by tree stack over flow prob… ## What changes were proposed in this pull request? What changes were proposed in this pull request? Improving evaluateEachIteration function i

spark git commit: [SPARK-16257][BUILD] Update spark_ec2.py to support Spark 1.6.2 and 1.6.3.

2016-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 1ac830aca -> ccc7fa357 [SPARK-16257][BUILD] Update spark_ec2.py to support Spark 1.6.2 and 1.6.3. ## What changes were proposed in this pull request? - Adds 1.6.2 and 1.6.3 as supported Spark versions within the bundled spark-ec2 scri

spark git commit: [SPARK-16182][CORE] Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-07-01 Thread srowen
d of a custom implementation. ## How was this patch tested? Existing tests, which cover the force-kill case, and Amplab tests, which will cover both Java 7 and Java 8 eventually. However I tested locally on Java 8 and the PR builder will try Java 7 here. Author: Sean Owen Closes #13973 from

spark git commit: [SPARK-16182][CORE] Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-07-01 Thread srowen
instead of a custom implementation. ## How was this patch tested? Existing tests, which cover the force-kill case, and Amplab tests, which will cover both Java 7 and Java 8 eventually. However I tested locally on Java 8 and the PR builder will try Java 7 here. Author: Sean Owen Closes #1397

spark git commit: [SPARK-16182][CORE] Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails

2016-07-01 Thread srowen
instead of a custom implementation. ## How was this patch tested? Existing tests, which cover the force-kill case, and Amplab tests, which will cover both Java 7 and Java 8 eventually. However I tested locally on Java 8 and the PR builder will try Java 7 here. Author: Sean Owen Closes #1397

spark git commit: [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3

2016-07-01 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 83f860448 -> 1026aba16 [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3 ## What changes were proposed in this pull request? I would like to use IPython with Python 3.5. It is annoying when it fails with IPytho

spark git commit: [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3

2016-07-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2075bf8ef -> 66283ee0b [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3 ## What changes were proposed in this pull request? I would like to use IPython with Python 3.5. It is annoying when it fails with IPython re

spark git commit: [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3

2016-07-01 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 972106dd3 -> 0b64543c5 [SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3 ## What changes were proposed in this pull request? I would like to use IPython with Python 3.5. It is annoying when it fails with IPytho

spark git commit: [SPARK-16222][SQL] JDBC Sources - Handling illegal input values for `fetchsize` and `batchsize`

2016-07-01 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 0b64543c5 -> 3665927c6 [SPARK-16222][SQL] JDBC Sources - Handling illegal input values for `fetchsize` and `batchsize` What changes were proposed in this pull request? For JDBC data sources, users can specify `batchsize` for multi

spark git commit: [SPARK-16222][SQL] JDBC Sources - Handling illegal input values for `fetchsize` and `batchsize`

2016-07-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master 66283ee0b -> 0ad6ce7e5 [SPARK-16222][SQL] JDBC Sources - Handling illegal input values for `fetchsize` and `batchsize` What changes were proposed in this pull request? For JDBC data sources, users can specify `batchsize` for multi-row

spark git commit: [GRAPHX][EXAMPLES] move graphx test data directory and update graphx document

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master bad0f7dbb -> 192d1f9cf [GRAPHX][EXAMPLES] move graphx test data directory and update graphx document ## What changes were proposed in this pull request? There are two test data files used for graphx examples existing in directory "graphx/

spark git commit: [GRAPHX][EXAMPLES] move graphx test data directory and update graphx document

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 ab4303800 -> f3a359939 [GRAPHX][EXAMPLES] move graphx test data directory and update graphx document ## What changes were proposed in this pull request? There are two test data files used for graphx examples existing in directory "gra

spark git commit: [SPARK-16345][DOCUMENTATION][EXAMPLES][GRAPHX] Extract graphx programming guide example snippets from source files instead of hard code them

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master 192d1f9cf -> 0bd7cd18b [SPARK-16345][DOCUMENTATION][EXAMPLES][GRAPHX] Extract graphx programming guide example snippets from source files instead of hard code them ## What changes were proposed in this pull request? I extract 6 example pr

spark git commit: [SPARK-16345][DOCUMENTATION][EXAMPLES][GRAPHX] Extract graphx programming guide example snippets from source files instead of hard code them

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 f3a359939 -> 0d0b41609 [SPARK-16345][DOCUMENTATION][EXAMPLES][GRAPHX] Extract graphx programming guide example snippets from source files instead of hard code them ## What changes were proposed in this pull request? I extract 6 exampl

spark git commit: [MINOR][BUILD] Fix Java linter errors

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0bd7cd18b -> 3000b4b29 [MINOR][BUILD] Fix Java linter errors ## What changes were proposed in this pull request? This PR fixes the minor Java linter errors like the following. ``` -public int read(char cbuf[], int off, int len) throws

spark git commit: [MINOR][BUILD] Fix Java linter errors

2016-07-02 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 0d0b41609 -> 0c6fd03fa [MINOR][BUILD] Fix Java linter errors This PR fixes the minor Java linter errors like the following. ``` -public int read(char cbuf[], int off, int len) throws IOException { +public int read(char[] cbuf, i

spark git commit: [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure

2016-07-04 Thread srowen
hor: Sean Owen Closes #14029 from srowen/RemoveCompressImages. (cherry picked from commit 18fb57f58a04685823408f3a174a8722f155fd4d) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ecbb4470 Tree: h

spark git commit: [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure

2016-07-04 Thread srowen
ean Owen Closes #14029 from srowen/RemoveCompressImages. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/18fb57f5 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/18fb57f5 Diff: http://git-wip-us.apache.org/repos/asf/sp

svn commit: r1751226 - in /spark: _includes/ images/ site/images/

2016-07-04 Thread srowen
Author: srowen Date: Mon Jul 4 08:31:32 2016 New Revision: 1751226 URL: http://svn.apache.org/viewvc?rev=1751226&view=rev Log: Remove unused images from Spark site; crush large PNGs; remove obsolete .html _includes Removed: spark/_includes/footer.html spark/_includes/navbar.

<    1   2   3   4   5   6   7   8   9   10   >