spark git commit: [SPARK-18497][SS] Make ForeachSink support watermark

2016-11-18 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 4b1df0e89 -> b4bad04c5 [SPARK-18497][SS] Make ForeachSink support watermark ## What changes were proposed in this pull request? The issue in ForeachSink is the new created DataSet still uses the old QueryExecution. When

spark git commit: [SPARK-18497][SS] Make ForeachSink support watermark

2016-11-18 Thread tdas
Repository: spark Updated Branches: refs/heads/master 6f7ff7509 -> 2a40de408 [SPARK-18497][SS] Make ForeachSink support watermark ## What changes were proposed in this pull request? The issue in ForeachSink is the new created DataSet still uses the old QueryExecution. When

spark git commit: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread rxin
Repository: spark Updated Branches: refs/heads/master e5f5c29e0 -> 6f7ff7509 [SPARK-18505][SQL] Simplify AnalyzeColumnCommand ## What changes were proposed in this pull request? I'm spending more time at the design & code level for cost-based optimizer now, and have found a number of issues

spark git commit: [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog

2016-11-18 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 6717981e4 -> 136f687c6 [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog ## What changes were proposed in this pull request? HDFS `write` may just hang until timeout if some network error happens. It's better to enable

spark git commit: [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog

2016-11-18 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.0 da9d51661 -> 9dad3a7b0 [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog ## What changes were proposed in this pull request? HDFS `write` may just hang until timeout if some network error happens. It's better to enable

spark git commit: [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog

2016-11-18 Thread tdas
Repository: spark Updated Branches: refs/heads/master 40d59ff5e -> e5f5c29e0 [SPARK-18477][SS] Enable interrupts for HDFS in HDFSMetadataLog ## What changes were proposed in this pull request? HDFS `write` may just hang until timeout if some network error happens. It's better to enable

spark git commit: [SPARK-18422][CORE] Fix wholeTextFiles test to pass on Windows in JavaAPISuite

2016-11-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master 795e9fc92 -> 40d59ff5e [SPARK-18422][CORE] Fix wholeTextFiles test to pass on Windows in JavaAPISuite ## What changes were proposed in this pull request? This PR fixes the test `wholeTextFiles` in `JavaAPISuite.java`. This is failed due

spark git commit: [SPARK-18422][CORE] Fix wholeTextFiles test to pass on Windows in JavaAPISuite

2016-11-18 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.1 ec622eb7e -> 6717981e4 [SPARK-18422][CORE] Fix wholeTextFiles test to pass on Windows in JavaAPISuite ## What changes were proposed in this pull request? This PR fixes the test `wholeTextFiles` in `JavaAPISuite.java`. This is failed

spark git commit: [SPARK-18457][SQL] ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-18 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 5912c19e7 -> ec622eb7e [SPARK-18457][SQL] ORC and other columnar formats using HiveShim read all columns when doing a simple count ## What changes were proposed in this pull request? When reading zero columns (e.g., count(*)) from

spark git commit: [SPARK-18457][SQL] ORC and other columnar formats using HiveShim read all columns when doing a simple count

2016-11-18 Thread rxin
Repository: spark Updated Branches: refs/heads/master 51baca221 -> 795e9fc92 [SPARK-18457][SQL] ORC and other columnar formats using HiveShim read all columns when doing a simple count ## What changes were proposed in this pull request? When reading zero columns (e.g., count(*)) from ORC or

spark git commit: [SPARK-18187][SQL] CompactibleFileStreamLog should not use "compactInterval" direcly with user setting.

2016-11-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master d9dd979d1 -> 51baca221 [SPARK-18187][SQL] CompactibleFileStreamLog should not use "compactInterval" direcly with user setting. ## What changes were proposed in this pull request? CompactibleFileStreamLog relys on "compactInterval" to

spark-website git commit: Expand guidance on SO and mailing lists, per discussion

2016-11-18 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 8781cd3c4 -> 80a543b56 Expand guidance on SO and mailing lists, per discussion Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/80a543b5 Tree: