[GitHub] spark issue #20133: [SPARK-22934] [SQL] Make optional clauses order insensit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20133 **[Test build #85576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85576/testReport)** for PR 20133 at commit [`8ae8f18`](https://github.com/apache/spark/commit/8ae8f1832a62caf10a62511f339402c0d94f89ea). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20133: [SPARK-22934] [SQL] Make optional clauses order i...
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20133 [SPARK-22934] [SQL] Make optional clauses order insensitive for CREATE TABLE SQL statement ## What changes were proposed in this pull request? Currently, our CREATE TABLE syntax require the EXACT order of clauses. It is pretty hard to remember the exact order. Thus, this PR is to make optional clauses order insensitive for `CREATE TABLE` SQL statement. ``` CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [db_name.]table_name [(col_name1 col_type1 [COMMENT col_comment1], ...)] USING datasource [OPTIONS (key1=val1, key2=val2, ...)] [PARTITIONED BY (col_name1, col_name2, ...)] [CLUSTERED BY (col_name3, col_name4, ...) INTO num_buckets BUCKETS] [LOCATION path] [COMMENT table_comment] [TBLPROPERTIES (key1=val1, key2=val2, ...)] [AS select_statement] ``` The proposal is to make the following clauses order insensitive. ``` [OPTIONS (key1=val1, key2=val2, ...)] [PARTITIONED BY (col_name1, col_name2, ...)] [CLUSTERED BY (col_name3, col_name4, ...) INTO num_buckets BUCKETS] [LOCATION path] [COMMENT table_comment] [TBLPROPERTIES (key1=val1, key2=val2, ...)] ``` The same idea is also applicable to Create Hive Table. ``` CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name [(col_name1[:] col_type1 [COMMENT col_comment1], ...)] [COMMENT table_comment] [PARTITIONED BY (col_name2[:] col_type2 [COMMENT col_comment2], ...)] [ROW FORMAT row_format] [STORED AS file_format] [LOCATION path] [TBLPROPERTIES (key1=val1, key2=val2, ...)] [AS select_statement] ``` The proposal is to make the following clauses order insensitive. ``` [COMMENT table_comment] [PARTITIONED BY (col_name2[:] col_type2 [COMMENT col_comment2], ...)] [ROW FORMAT row_format] [STORED AS file_format] [LOCATION path] [TBLPROPERTIES (key1=val1, key2=val2, ...)] ``` ## How was this patch tested? Added test cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark createDataSourceTableDDL Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20133 commit 8ae8f1832a62caf10a62511f339402c0d94f89ea Author: gatorsmile Date: 2018-01-01T07:03:01Z fix --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20106: [SPARK-21616][SPARKR][DOCS] update R migration guide and...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20106 Maybe. Other test files have that extra empty line though ;) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85570/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85570/testReport)** for PR 20127 at commit [`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user MrBago commented on the issue: https://github.com/apache/spark/pull/20095 @jkbradley I pushed changes in response to your comments. I think we should split the `TrainValidationSplit` memory split into another PR, I may have time to work on it tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20095 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20095 **[Test build #85572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85572/testReport)** for PR 20095 at commit [`8c7c8e3`](https://github.com/apache/spark/commit/8c7c8e3e14c6c2d7f64c5a3f54c129e794176672). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20095 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85572/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18277 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85574/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18277 **[Test build #85574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85574/testReport)** for PR 18277 at commit [`8c88595`](https://github.com/apache/spark/commit/8c88595125fbd328a3ed2383a9e96db7ad96f0e9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18277 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #85575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85575/testReport)** for PR 19789 at commit [`7700bf0`](https://github.com/apache/spark/commit/7700bf02c06d6fe3909053a1e146569a67b21de3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85575/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20106: [SPARK-21616][SPARKR][DOCS] update R migration gu...
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20106#discussion_r159149323 --- Diff: R/pkg/tests/fulltests/test_Windows.R --- @@ -14,6 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. # + --- End diff -- We could remove this back tho :). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20131 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #85575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85575/testReport)** for PR 19789 at commit [`7700bf0`](https://github.com/apache/spark/commit/7700bf02c06d6fe3909053a1e146569a67b21de3). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20130: [BUILD] Close stale PRs
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20130 I saw this one 19520. Might be good to add too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19789 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19693: [MINOR][CORE] Improved statistical shuffle write time
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19693 Shall we leave this closed for now @heary-cao? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18424 **[Test build #85573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85573/testReport)** for PR 18424 at commit [`62f273b`](https://github.com/apache/spark/commit/62f273b0b0e7fec6b80d95c39f49eba32de92c43). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18424 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85573/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18424 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18424 **[Test build #85573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85573/testReport)** for PR 18424 at commit [`62f273b`](https://github.com/apache/spark/commit/62f273b0b0e7fec6b80d95c39f49eba32de92c43). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18277 **[Test build #85574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85574/testReport)** for PR 18277 at commit [`8c88595`](https://github.com/apache/spark/commit/8c88595125fbd328a3ed2383a9e96db7ad96f0e9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18277: [SPARK-20947][PYTHON] Fix encoding/decoding error in pip...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18277 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18424 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20095 **[Test build #85572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85572/testReport)** for PR 20095 at commit [`8c7c8e3`](https://github.com/apache/spark/commit/8c7c8e3e14c6c2d7f64c5a3f54c129e794176672). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20131 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85568/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20131 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20131 **[Test build #85568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85568/testReport)** for PR 20131 at commit [`8eea4de`](https://github.com/apache/spark/commit/8eea4de9c005f52f4e83eaec647cd7e67b44dc2d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20095: [SPARK-22126][ML] Added fitMultiple method with default ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20095 **[Test build #85571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85571/testReport)** for PR 20095 at commit [`4d10c19`](https://github.com/apache/spark/commit/4d10c19e004e6f7b265f7028accbda6d83af8ae4). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85570/testReport)** for PR 20127 at commit [`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20127 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85564/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85564/testReport)** for PR 20129 at commit [`2d26bd8`](https://github.com/apache/spark/commit/2d26bd84d75cb3e3d7159ae192b014313a0f6fc8). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20132 @jkbradley Thanks for this follow-up! I've noticed that first issue in original PR. But don't have enough time to discuss with you further. I'll go through this soon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85569/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85566/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85566/testReport)** for PR 20129 at commit [`73e0def`](https://github.com/apache/spark/commit/73e0defb92d9d1f7fc8f7bb1f2bf5b637ac83113). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20130: [BUILD] Close stale PRs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20130 **[Test build #85565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85565/testReport)** for PR 20130 at commit [`d710a54`](https://github.com/apache/spark/commit/d710a543f9c1fdc362b3e4c7adde1b2a8debabf8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20130: [BUILD] Close stale PRs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20130 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20130: [BUILD] Close stale PRs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20130 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85565/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20132 **[Test build #85569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85569/testReport)** for PR 20132 at commit [`9bf045d`](https://github.com/apache/spark/commit/9bf045da1adeaa08deeb96eaa0289d8d4cb74bc1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20132 @viirya This basically has 2 changes: * configedCategorySize: my mistake! * encoder: clarify what I meant before --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19527 Thank you for all the work in this PR! Here's the follow-up: https://github.com/apache/spark/pull/20132 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEn...
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/20132 [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEstimator ## What changes were proposed in this pull request? Follow-up cleanups for the OneHotEncoderEstimator PR. See some discussion in the original PR: https://github.com/apache/spark/pull/19527 or read below for what this PR includes: * configedCategorySize: I reverted this to return an Array. I realized the original setup (which I had recommended in the original PR) caused the whole model to be serialized in the UDF. * encoder: I reorganized the logic to show what I meant in the comment in the previous PR. I think it's simpler but am open to suggestions. I also made some small style cleanups based on IntelliJ warnings. ## How was this patch tested? Existing unit tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/jkbradley/spark viirya-SPARK-13030 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20132.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20132 commit 9bf045da1adeaa08deeb96eaa0289d8d4cb74bc1 Author: Joseph K. Bradley Date: 2017-12-31T23:25:45Z updates for final PR --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19527 @jkbradley Thanks for reviewing and merging this. Thanks for all helping this too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19527 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19527 Thanks for the updates! I still think there's some confusion, but since I think this code is correct & it doesn't affect APIs, I'll go ahead and merge this. I'll ping you on a follow-up PR to show what I had in mind. LGTM Merging with master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20131 **[Test build #85568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85568/testReport)** for PR 20131 at commit [`8eea4de`](https://github.com/apache/spark/commit/8eea4de9c005f52f4e83eaec647cd7e67b44dc2d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20131 **[Test build #85567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85567/testReport)** for PR 20131 at commit [`a762e19`](https://github.com/apache/spark/commit/a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85567/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20131 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20131: [MINOR] Fix a bunch of typos
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20131 **[Test build #85567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85567/testReport)** for PR 20131 at commit [`a762e19`](https://github.com/apache/spark/commit/a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20131: [MINOR] Fix a bunch of typos
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/20131 [MINOR] Fix a bunch of typos ## What changes were proposed in this pull request? Fix a bunch of typos, to head off more typo PRs. ## How was this patch tested? Existing tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark Spelling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20131.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20131 commit a762e19411cd6632e3f683ebaa4dcba3c3fe5bdf Author: Sean Owen Date: 2017-12-31T23:00:41Z Fix a bunch of typos --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85566/testReport)** for PR 20129 at commit [`73e0def`](https://github.com/apache/spark/commit/73e0defb92d9d1f7fc8f7bb1f2bf5b637ac83113). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85563/testReport)** for PR 20129 at commit [`3843395`](https://github.com/apache/spark/commit/3843395391281f74278346bd518e543a4b86a66f). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85563/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20130: [BUILD] Close stale PRs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20130 **[Test build #85565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85565/testReport)** for PR 20130 at commit [`d710a54`](https://github.com/apache/spark/commit/d710a543f9c1fdc362b3e4c7adde1b2a8debabf8). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20129 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20130: [BUILD] Close stale PRs
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/20130 [BUILD] Close stale PRs Closes #18916 Closes #19613 Closes #19739 Closes #19936 Closes #19919 Closes #19933 Closes #19917 Closes #20027 Closes #19035 Closes #20044 Closes #20104 You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark StalePRs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20130.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20130 commit d710a543f9c1fdc362b3e4c7adde1b2a8debabf8 Author: Sean Owen Date: 2017-12-31T21:45:22Z Close stale PRs. Closes #18916 Closes #19613 Closes #19739 Closes #19936 Closes #19919 Closes #19933 Closes #19917 Closes #20027 Closes #19035 Closes #20044 Closes #20104 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19968 Ping @KaiXinXiaoLei to update or close --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20108: [MINOR][DOCS] s/It take/It takes/g
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20108 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20108 This time I'll just merge it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85564/testReport)** for PR 20129 at commit [`2d26bd8`](https://github.com/apache/spark/commit/2d26bd84d75cb3e3d7159ae192b014313a0f6fc8). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20129: [SPARK-22933][SPARKR] R Structured Streaming API for wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20129 **[Test build #85563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85563/testReport)** for PR 20129 at commit [`3843395`](https://github.com/apache/spark/commit/3843395391281f74278346bd518e543a4b86a66f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20129: [SPARK-22933][SPARKR] R Structured Streaming API ...
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/20129 [SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, trigger, partitionBy ## What changes were proposed in this pull request? R Structured Streaming API for withWatermark, trigger, partitionBy ## How was this patch tested? manual, unit tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/felixcheung/spark rwater Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20129.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20129 commit 9c980e3987c44c6df87e4d39524e4689babcabda Author: Felix Cheung Date: 2017-08-21T00:09:57Z withWatermark commit e440706bfeb819faa1579f27fdf93d740aba0e3e Author: Felix Cheung Date: 2017-12-31T20:48:21Z trigger, patitionBy, tests commit 3843395391281f74278346bd518e543a4b86a66f Author: Felix Cheung Date: 2017-12-31T20:57:17Z more test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19715: [SPARK-22397][ML]add multiple columns support to Quantil...
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/19715 Thank you all for your help!! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85562/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85562 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85562/testReport)** for PR 20127 at commit [`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20128 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85561/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20128 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20128 **[Test build #85561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85561/testReport)** for PR 20128 at commit [`3b1533a`](https://github.com/apache/spark/commit/3b1533a400ecf277cf0e18b8cb76a14305b0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20108 **[Test build #4032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4032/testReport)** for PR 20108 at commit [`daae351`](https://github.com/apache/spark/commit/daae3511d04f5dab1a32790f09b05c9188c4ac5b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20108: [MINOR][DOCS] s/It take/It takes/g
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20108 **[Test build #4032 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4032/testReport)** for PR 20108 at commit [`daae351`](https://github.com/apache/spark/commit/daae3511d04f5dab1a32790f09b05c9188c4ac5b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85562/testReport)** for PR 20127 at commit [`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20070#discussion_r159143833 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala --- @@ -45,7 +45,7 @@ object QuantileDiscretizerExample { .setNumBuckets(3) val result = discretizer.fit(df).transform(df) -result.show() +result.show(false) --- End diff -- Which other examples? most do not set this, and the Java equivalent doesn't either. If there's a good reason that the output needs to be untruncated, that's fine, just also change the Java example. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20070#discussion_r159143849 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala --- @@ -46,7 +46,10 @@ object LatentDirichletAllocationExample { val topics = ldaModel.topicsMatrix for (topic <- Range(0, 3)) { print(s"Topic $topic :") - for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, topic)}") } + for (word <- Range(0, ldaModel.vocabSize)) + { --- End diff -- No, we put the open brace on the preceding line with for, and don't triple indent. See any other for loop in the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20127 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85558/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20126 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20126: [DO-NOT-MERGE] Investigate if changes in flume.py actual...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20126 **[Test build #85558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85558/testReport)** for PR 20126 at commit [`85639dd`](https://github.com/apache/spark/commit/85639dd220e8fcb0489febc0414b51d22c0e41a9). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20128 Ah I probably missed that when moving these behind a profile. I wasn't fully aware of the dependency mechanism. If it resolves the problem, yes go for it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159142765 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -323,11 +323,13 @@ object SQLConf { .createWithDefault(false) val PARQUET_COMPRESSION = buildConf("spark.sql.parquet.compression.codec") -.doc("Sets the compression codec use when writing Parquet files. Acceptable values include: " + - "uncompressed, snappy, gzip, lzo.") +.doc("Sets the compression codec use when writing Parquet files. If other compression codec " + --- End diff -- s/use when/used when --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159142783 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -364,7 +366,9 @@ object SQLConf { .createWithDefault(true) val ORC_COMPRESSION = buildConf("spark.sql.orc.compression.codec") -.doc("Sets the compression codec use when writing ORC files. Acceptable values include: " + +.doc("Sets the compression codec use when writing ORC files. If other compression codec " + --- End diff -- s/use when/used when --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159142760 --- Diff: docs/sql-programming-guide.md --- @@ -953,8 +953,10 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession spark.sql.parquet.compression.codec snappy -Sets the compression codec use when writing Parquet files. Acceptable values include: -uncompressed, snappy, gzip, lzo. +Sets the compression codec use when writing Parquet files. If other compression codec --- End diff -- s/use when/used when --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85557/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20127 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][TESTS][FOLLOWUP] Enables PySp...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20128 **[Test build #85561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85561/testReport)** for PR 20128 at commit [`3b1533a`](https://github.com/apache/spark/commit/3b1533a400ecf277cf0e18b8cb76a14305b0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20127: [SPARK-22932] [SQL] Refactor AnalysisContext
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20127 **[Test build #85557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85557/testReport)** for PR 20127 at commit [`f158a95`](https://github.com/apache/spark/commit/f158a951b779e56e06d2c73234bac5c79055b2f5). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20128: [SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySp...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20128 cc @srowen, @vanzin, @yhuai and @zsxwing who I believe are primarily related with this change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20128: [SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enabl...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20128 [SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySpark tests for Flume and Kafka in Jenkins ## What changes were proposed in this pull request? This PR proposes to enable PySpark tests for Flume and Kafka in Jenkins by explicitly setting the environment variables in `modules.py`. Seems we are not taking the dependencies into account when calculating environment variables: https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L554-L561 ## How was this patch tested? Manual tests with Jenkins in https://github.com/apache/spark/pull/20126. **Before** - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/consoleFull ``` [info] Setup the following environment variables for tests: ... ``` **After** - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/consoleFull ``` [info] Setup the following environment variables for tests: ENABLE_KAFKA_0_8_TESTS=1 ENABLE_FLUME_TESTS=1 ... ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-21893 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20128.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20128 commit 3b1533a400ecf277cf0e18b8cb76a14305b0 Author: hyukjinkwon Date: 2017-12-31T15:35:54Z Enables PySpark tests for Flume and Kafka in Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20126: [DO-NOT-MERGE] Investigate if changes in flume.py...
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/20126 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org