[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58615814 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/344/consoleFull) for PR 2716 at commit [`29e94d5`](https://github.com/apache/spark/commit/29e94d5764d6b9d1877fd16a9041f6b0ad61b347). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class NullType(PrimitiveType):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58615995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21572/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58615984 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-58615990 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21572/consoleFull) for PR 2716 at commit [`e48d7fb`](https://github.com/apache/spark/commit/e48d7fb0800946a50922caae0062805d0fd4c371). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class NullType(PrimitiveType):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58616220 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21576/consoleFull) for PR 2750 at commit [`d9785c3`](https://github.com/apache/spark/commit/d9785c384c4c300a5ce85a1e61cff7173bf88a4f). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58616217 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21577/consoleFull) for PR 2344 at commit [`0df6ea1`](https://github.com/apache/spark/commit/0df6ea1964710e6376daac282686da7fb35c29b2). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3896] checkSpeculatableTasks fask quit ...
GitHub user shijinkui opened a pull request: https://github.com/apache/spark/pull/2751 [SPARK-3896] checkSpeculatableTasks fask quit loop; checkSpeculatableTasks is expensive 1. invoking checkSpeculatableTasks is expensive, once true then return true 2. import section formatting You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/spark fast_quit_loop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2751.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2751 commit 63f60d5ac25b6ad37bfbb63bd93b5072f1f955a1 Author: çç jinkui@alibaba-inc.com Date: 2014-10-10T06:20:51Z checkSpeculatableTasks fask quit loop; checkSpeculatableTasks is expensive --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3896] Pool#checkSpeculatableTasks fask ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2751#issuecomment-58617197 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3896] Pool#checkSpeculatableTasks fask ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2751#issuecomment-58617192 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3739] [SQL] Update the split num base o...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2589#issuecomment-58617264 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21575/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: fixed dynamic pagerank vertex update computati...
Github user planvin commented on the pull request: https://github.com/apache/spark/pull/2738#issuecomment-58617277 I am clearly missing something here. Sorry for that. I am closing the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3739] [SQL] Update the split num base o...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2589#issuecomment-58617261 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21575/consoleFull) for PR 2589 at commit [`c78a045`](https://github.com/apache/spark/commit/c78a0450abde75355b901d5adb78a5a2f73aec64). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: fixed dynamic pagerank vertex update computati...
Github user planvin closed the pull request at: https://github.com/apache/spark/pull/2738 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/2570#discussion_r18690123
--- Diff:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala
---
@@ -211,7 +211,11 @@ class HiveCompatibilitySuite extends HiveQueryFileTest
with BeforeAndAfter {
describe_comment_indent,
// Limit clause without a ordering, which causes failure.
-orc_predicate_pushdown
+orc_predicate_pushdown,
+
+// Sort with Limit clause causes failure.
--- End diff --
Yes, the reason it failed part of due to the #2859. I will keep updating
the black list once #2859 be merged.
Currently, the added unit test in `SQLQuerySuite` may works for the same
purpose.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3807: SparkSql does not work for tables ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2674#issuecomment-58617474 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21573/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3807: SparkSql does not work for tables ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2674#issuecomment-58617467 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21573/consoleFull) for PR 2674 at commit [`1f26805`](https://github.com/apache/spark/commit/1f26805e714049e83ed9399c8ca8361da459777c). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/2752 [SPARK-2805] Upgrade Akka to 2.3.4 This is a second rev of the Akka upgrade (earlier merged, but reverted). I made a slight modification which is that I also upgrade Hive to deal with a compatibility issue related to the protocol buffers library. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pwendell/spark akka-upgrade Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2752 commit 2a551d36bd36a9b79715b1581d7d52a9bb142e9a Author: Anand Avati av...@redhat.com Date: 2014-07-27T02:20:34Z SPARK-1812: core - upgrade to akka 2.3.4 Signed-off-by: Anand Avati av...@redhat.com commit 57a23158973872f17d577730b85cab808fc99142 Author: Anand Avati av...@redhat.com Date: 2014-07-31T02:22:48Z SPARK-1812: streaming - remove tests which depend on akka.actor.IO Not available in 2.3.x Signed-off-by: Anand Avati av...@redhat.com commit 4c7ca3fd78091b460dc01da9e42f9142637da479 Author: Patrick Wendell pwend...@gmail.com Date: 2014-10-10T06:34:45Z Upgrading to new hive-protobuf version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3888] [PySpark] limit the memory used b...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2743#issuecomment-58617749 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/343/consoleFull)** for PR 2743 at commit [`52fab51`](https://github.com/apache/spark/commit/52fab51c37daf7997b1770620a16169081c30567) after a configured wait of `120m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58617755 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58617902 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21578/consoleFull) for PR 2752 at commit [`4c7ca3f`](https://github.com/apache/spark/commit/4c7ca3fd78091b460dc01da9e42f9142637da479). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58618269 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21574/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58618260 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21574/consoleFull) for PR 2750 at commit [`d9785c3`](https://github.com/apache/spark/commit/d9785c384c4c300a5ce85a1e61cff7173bf88a4f). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58619034 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21577/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58619031 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21577/consoleFull) for PR 2344 at commit [`0df6ea1`](https://github.com/apache/spark/commit/0df6ea1964710e6376daac282686da7fb35c29b2). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58619871 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21579/consoleFull) for PR 2673 at commit [`285be82`](https://github.com/apache/spark/commit/285be82218d5f52145c4dc019aebfc80d112aa3d). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58620288 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21576/consoleFull) for PR 2750 at commit [`d9785c3`](https://github.com/apache/spark/commit/d9785c384c4c300a5ce85a1e61cff7173bf88a4f). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2924] Required by scala 2.11, only one ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2750#issuecomment-58620295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21576/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
GitHub user aarondav opened a pull request: https://github.com/apache/spark/pull/2753 [SPARK-3453] Netty-based BlockTransferService, extracted from Spark core This PR encapsulates #2330, which is itself a continuation of #2240. The first goal of this PR is to provide an alternate, simpler implementation of the ConnectionManager which is based on Netty. In addition to this goal, however, we want to resolve [SPARK-3796](https://issues.apache.org/jira/browse/SPARK-3796), which calls for a standalone shuffle service which can be integrated into the YARN NodeManager, Standalone Worker, or on its own. This PR makes the first step in this direction by ensuring that the actual Netty service is as small as possible and extracted from Spark core. Given this, we should be able to construct this standalone jar which can be included in other JVMs without incurring significant dependency or runtime issues. The actual work to ensure that such a standalone shuffle service would work in Spark will be left for a future PR, however. In order to minimize dependencies and allow for the service to be long-running (possibly much longer-running than Spark, and possibly having to support multiple version of Spark simultaneously), the entire service has been ported to Java, where we have full control over the binary compatibility of the components and do not depend on the Scala runtime or version. These issues: have been addressed by folding in #2330: SPARK-3453: Refactor Netty module to use BlockTransferService interface SPARK-3018: Release all buffers upon task completion/failure SPARK-3002: Create a connection pool and reuse clients across different threads SPARK-3017: Integration tests and unit tests for connection failures SPARK-3049: Make sure client doesn't block when server/connection has error(s) SPARK-3502: SO_RCVBUF and SO_SNDBUF should be bootstrap childOption, not option SPARK-3503: Disable thread local cache in PooledByteBufAllocator TODO before mergeable: [ ] Implement uploadBlock() [ ] Unit tests for RPC side of code [ ] Performance testing [ ] Turn OFF by default (currently on for unit testing) You can merge this pull request into a Git repository by running: $ git pull https://github.com/aarondav/spark netty Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2753.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2753 commit 165eab1518f5184ef9609f26d374c5ccefd05472 Author: Reynold Xin r...@apache.org Date: 2014-09-09T07:29:33Z [SPARK-3453] Refactor Netty module to use BlockTransferService. Also includes some partial support for uploading blocks. commit 1760d3292ecf262e4c77c9e3b28bfd2900d25840 Author: Reynold Xin r...@apache.org Date: 2014-09-09T07:42:37Z Use Epoll.isAvailable in BlockServer as well. commit 2b44cf1b7547919bbe7386e954fe2f56be046790 Author: Reynold Xin r...@apache.org Date: 2014-09-09T21:36:31Z Added more documentation. commit 064747b50a591acb132b2c750957e79f54dfa88f Author: Reynold Xin r...@apache.org Date: 2014-09-10T06:38:38Z Reference count buffers and clean them up properly. commit b5c8d1fca6d3cf5c2b95395310200c8149a7eb16 Author: Reynold Xin r...@apache.org Date: 2014-09-10T08:09:44Z Fixed ShuffleBlockFetcherIteratorSuite. commit 108c9edaed06c5e046a21c9a8e54c50390da9a0b Author: Reynold Xin r...@apache.org Date: 2014-09-10T08:10:04Z Forgot to add TestSerializer to the commit list. commit 1be4e8ee7d932821c789cb974310e5d59df4ff84 Author: Reynold Xin r...@apache.org Date: 2014-09-10T08:11:40Z Shorten NioManagedBuffer and NettyManagedBuffer class names. commit cb589ec7b6d3758498249b63b395634efb83d8ba Author: Reynold Xin r...@apache.org Date: 2014-09-11T02:01:23Z Added more test cases covering cleanup when fault happens in ShuffleBlockFetcherIteratorSuite commit 5cd33d7798ae742e76107bb976d8478ab9476ae7 Author: Reynold Xin r...@apache.org Date: 2014-09-11T02:55:54Z Fixed style violation. commit 9e0cb8736be6d38e3f30766271d28875ceca1ae8 Author: Reynold Xin r...@apache.org Date: 2014-09-11T04:04:56Z Fixed BlockClientHandlerSuite commit d23ed7bfd912770ace7eed7cd0dff2db6ac826e3 Author: Reynold Xin r...@apache.org Date: 2014-09-12T01:28:45Z Incorporated feedback from Norman: - use same pool for boss and worker - remove ioratio - disable caching of byte buf allocator - childoption sendbuf/receivebuf - fire exception through pipeline In addition: - fire failure handler BlockFetchingListener at least once per block. - enabled a bunch of ignored tests commit b2f3281d0de540d38ea5b4c7bf576b775405d56d Author: Reynold Xin r...@apache.org Date: 2014-09-12T05:12:08Z Added connection
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58620672 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21580/consoleFull) for PR 2753 at commit [`c0cd242`](https://github.com/apache/spark/commit/c0cd242f375e939e1422e30d4b230a8a78b13b88). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58620770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21580/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58620769 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21580/consoleFull) for PR 2753 at commit [`c0cd242`](https://github.com/apache/spark/commit/c0cd242f375e939e1422e30d4b230a8a78b13b88). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
GitHub user shijinkui opened a pull request: https://github.com/apache/spark/pull/2754 [SPARK-3897]format example code and explicit val instead of var You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/spark example_improve Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2754.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2754 commit b2009bc04ed750b2ef03e76b5fdae2bffd958967 Author: çç jinkui@alibaba-inc.com Date: 2014-10-10T07:19:37Z format example code and explicit val instead of var --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2754#issuecomment-58621250 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2754#issuecomment-58621255 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58621380 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58621442 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21581/consoleFull) for PR 2753 at commit [`29c6dcf`](https://github.com/apache/spark/commit/29c6dcfaacb2e8b1f0582c6d5e435349c52e29af). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3887] Send stracktrace in ConnectionMan...
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/2741#issuecomment-58621671 It's really helpful :+1: Just a small question: Should specify charset when converting between bytes and String, or Spark always assumes the environment uses UTF8? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2742#issuecomment-58621877 Added a non-configurable version of the memory map pathway, with the threshold you suggested (2MB, the size of a hugepage). Note that this fix will also be included in https://github.com/apache/spark/pull/2753. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2742#issuecomment-58622268 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21583/consoleFull) for PR 2742 at commit [`a152065`](https://github.com/apache/spark/commit/a152065971d6045170137ce1358e16e52b0d9cc9). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58622482 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21582/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58622515 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21578/consoleFull) for PR 2752 at commit [`4c7ca3f`](https://github.com/apache/spark/commit/4c7ca3fd78091b460dc01da9e42f9142637da479). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58622520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21578/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58622871 Okay let's see if this does better with the nightly maven builds. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2752 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/2570#discussion_r18692067
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -117,8 +117,10 @@ case class InsertIntoTable(
case class CreateTableAsSelect(
databaseName: Option[String],
tableName: String,
-child: LogicalPlan) extends UnaryNode {
- override def output = child.output
+child: LogicalPlan,
+allowExisting: Boolean,
+extra: AnyRef = null) extends UnaryNode {
--- End diff --
What about to make the extra as generic type? `CTAS` probably widely
supported by different SQL dialects, creating specialized version maybe lead to
duplicated code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2754#discussion_r18692509
--- Diff: examples/src/main/scala/org/apache/spark/examples/LocalALS.scala
---
@@ -31,9 +31,12 @@ import cern.jet.math._
*/
object LocalALS {
// Parameters set through command line arguments
- var M = 0 // Number of movies
- var U = 0 // Number of users
- var F = 0 // Number of features
+ var M = 0
+ // Number of movies
--- End diff --
This is not a good change. It makes it looks like the comment pertains to
the following line. Leave it inline.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2754#discussion_r18692540
--- Diff:
examples/src/main/scala/org/apache/spark/examples/streaming/ActorWordCount.scala
---
@@ -127,12 +127,12 @@ object FeederActor {
* A sample word count program demonstrating the use of plugging in
* Actor as Receiver
* Usage: ActorWordCount hostname port
- * hostname and port describe the AkkaSystem that Spark Sample
feeder is running on.
+ * hostname and port describe the AkkaSystem that Spark Sample feeder
is running on.
--- End diff --
These are not helpful either. The continuation indent improves readability
and is on purpose, I presume.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2750] support https in spark web ui
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/1980#discussion_r18692553
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -205,10 +231,74 @@ private[spark] object JettyUtils extends Logging {
ServerInfo(server, boundPort, collection)
}
+ // to generate a new url string scheme://server:port+path
--- End diff --
Ok, updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2750] support https in spark web ui
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-58624892 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21584/consoleFull) for PR 1980 at commit [`3b01d3a`](https://github.com/apache/spark/commit/3b01d3a66d21854b5e053de89f3d301fee58b315). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs
Github user viper-kun commented on the pull request: https://github.com/apache/spark/pull/2471#issuecomment-58624839 @mattf @vanzin is this ok to go ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58624889 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58625132 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21579/consoleFull) for PR 2673 at commit [`285be82`](https://github.com/apache/spark/commit/285be82218d5f52145c4dc019aebfc80d112aa3d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58625140 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21579/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58625852 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21586/consoleFull) for PR 2570 at commit [`366e758`](https://github.com/apache/spark/commit/366e758c1d2ad2e793936b7e6976fc215923a15a). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58626122 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21585/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58626811 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21587/consoleFull) for PR 2570 at commit [`3774bd4`](https://github.com/apache/spark/commit/3774bd4617cb4dec3f78a08bdf42653b682102fd). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user shijinkui commented on a diff in the pull request:
https://github.com/apache/spark/pull/2754#discussion_r18693582
--- Diff: examples/src/main/scala/org/apache/spark/examples/LocalALS.scala
---
@@ -31,9 +31,12 @@ import cern.jet.math._
*/
object LocalALS {
// Parameters set through command line arguments
- var M = 0 // Number of movies
- var U = 0 // Number of users
- var F = 0 // Number of features
+ var M = 0
+ // Number of movies
--- End diff --
standard usage is two line.
code separate with annotation as fa as possible, as scala can omit
semicolon, not likely java
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user shijinkui commented on a diff in the pull request:
https://github.com/apache/spark/pull/2754#discussion_r18693711
--- Diff:
examples/src/main/scala/org/apache/spark/examples/streaming/ActorWordCount.scala
---
@@ -127,12 +127,12 @@ object FeederActor {
* A sample word count program demonstrating the use of plugging in
* Actor as Receiver
* Usage: ActorWordCount hostname port
- * hostname and port describe the AkkaSystem that Spark Sample
feeder is running on.
+ * hostname and port describe the AkkaSystem that Spark Sample feeder
is running on.
--- End diff --
agree.
the problem is i cannot find where to set two space which line start with
in intellij IDEA
i think we can tolerate this less readability place
:)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2742#issuecomment-58627794 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21583/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2742#issuecomment-58627787 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21583/consoleFull) for PR 2742 at commit [`a152065`](https://github.com/apache/spark/commit/a152065971d6045170137ce1358e16e52b0d9cc9). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2742#issuecomment-58628192 LGTM. Merged. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2574#issuecomment-58628221 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21588/consoleFull) for PR 2574 at commit [`304c4fc`](https://github.com/apache/spark/commit/304c4fc6a9acc9a65516966b990590d3bb7247df). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user shijinkui commented on the pull request: https://github.com/apache/spark/pull/2754#issuecomment-58628292 this format only in example module, cannot effect the other module our coding habit is formatting after modify by the way. if there are several difference after formatting, it will effect the current modify. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3897]format example code and explicit v...
Github user shijinkui commented on the pull request: https://github.com/apache/spark/pull/2754#issuecomment-58628485 @srowen can u review the other PR #2751 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3889] Attempt to avoid SIGBUS by not mm...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2742 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58628677 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21581/consoleFull) for PR 2753 at commit [`29c6dcf`](https://github.com/apache/spark/commit/29c6dcfaacb2e8b1f0582c6d5e435349c52e29af). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-58628684 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21581/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3887] Send stracktrace in ConnectionMan...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2741#issuecomment-58628876 Changes lgtm overall. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58628891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21586/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58628884 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21586/consoleFull) for PR 2570 at commit [`366e758`](https://github.com/apache/spark/commit/366e758c1d2ad2e793936b7e6976fc215923a15a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class CreateTableAsSelect[T](` * ` logDebug(Found class for $serdeName)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3121] Wrong implementation of implicit ...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/2712#issuecomment-58629065 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58630045 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21587/consoleFull) for PR 2570 at commit [`3774bd4`](https://github.com/apache/spark/commit/3774bd4617cb4dec3f78a08bdf42653b682102fd). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class CreateTableAsSelect[T](` * ` logDebug(Found class for $serdeName)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-58630054 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21587/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58631408 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21589/consoleFull) for PR 2673 at commit [`242a24e`](https://github.com/apache/spark/commit/242a24e6dc942ac7baebe5439dea41238b5770a1). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58631368 @pwendell I tried maven shade plugin to somehow work as effective pom generator, but that does not happen unless we have dependencies apart from the project's itself to put in uber jar. Please see the above commit and their commit messages for reasons. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18695986
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -220,20 +220,44 @@ trait HiveTypeCoercion {
case a: BinaryArithmetic if a.right.dataType == StringType =
a.makeCopy(Array(a.left, Cast(a.right, DoubleType)))
+ // we should cast all timestamp/date/string compare into string
compare
+ case p: BinaryPredicate if p.left.dataType == StringType
+ p.right.dataType == DateType =
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == StringType =
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType == TimestampType =
-p.makeCopy(Array(Cast(p.left, TimestampType), p.right))
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
case p: BinaryPredicate if p.left.dataType == TimestampType
p.right.dataType == StringType =
-p.makeCopy(Array(p.left, Cast(p.right, TimestampType)))
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
+ case p: BinaryPredicate if p.left.dataType == TimestampType
+ p.right.dataType == DateType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == TimestampType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
--- End diff --
How about turning `Date`/`Timestamp` comparison to `Long` comparison?
String and long representations of `Timestamp` are both accurate to seconds.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18696379
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -220,20 +220,44 @@ trait HiveTypeCoercion {
case a: BinaryArithmetic if a.right.dataType == StringType =
a.makeCopy(Array(a.left, Cast(a.right, DoubleType)))
+ // we should cast all timestamp/date/string compare into string
compare
+ case p: BinaryPredicate if p.left.dataType == StringType
+ p.right.dataType == DateType =
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == StringType =
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType == TimestampType =
-p.makeCopy(Array(Cast(p.left, TimestampType), p.right))
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
case p: BinaryPredicate if p.left.dataType == TimestampType
p.right.dataType == StringType =
-p.makeCopy(Array(p.left, Cast(p.right, TimestampType)))
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
+ case p: BinaryPredicate if p.left.dataType == TimestampType
+ p.right.dataType == DateType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == TimestampType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType != StringType =
p.makeCopy(Array(Cast(p.left, DoubleType), p.right))
case p: BinaryPredicate if p.left.dataType != StringType
p.right.dataType == StringType =
p.makeCopy(Array(p.left, Cast(p.right, DoubleType)))
- case i @ In(a,b) if a.dataType == TimestampType
b.forall(_.dataType == StringType) =
-i.makeCopy(Array(a,b.map(Cast(_,TimestampType
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== StringType) =
+i.makeCopy(Array(Cast(a, StringType), b))
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == StringType) =
+i.makeCopy(Array(Cast(a, StringType), b))
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== TimestampType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == DateType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== DateType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == TimestampType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
--- End diff --
The 4 case clauses above can also use long instead of string (and they
probably should be moved to some rule other than `PromoteStrings`).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2574#issuecomment-58634418 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21588/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3719][CORE]:complete/failed stages is...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2574#issuecomment-58634416 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21588/consoleFull) for PR 2574 at commit [`304c4fc`](https://github.com/apache/spark/commit/304c4fc6a9acc9a65516966b990590d3bb7247df). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2750] support https in spark web ui
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-58636687 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21584/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2750] support https in spark web ui
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-58636685 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21584/consoleFull)** for PR 1980 at commit [`3b01d3a`](https://github.com/apache/spark/commit/3b01d3a66d21854b5e053de89f3d301fee58b315) after a configured wait of `120m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18697748
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
---
@@ -56,7 +60,9 @@ case class Cast(child: Expression, dataType: DataType)
extends UnaryExpression {
case StringType =
buildCast[String](_, _.length() != 0)
case TimestampType =
- buildCast[Timestamp](_, b = b.getTime() != 0 || b.getNanos() != 0)
+ buildCast[Timestamp](_, t = t.getTime() != 0 || t.getNanos() != 0)
+case DateType =
+ buildCast[Date](_, d = null)
--- End diff --
Leaving a comment here would be good. It's really unintuitive here to see a
timestamp can be casted to a boolean while a date has to be null.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58637193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21589/Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58637190 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21589/consoleFull) for PR 2673 at commit [`242a24e`](https://github.com/apache/spark/commit/242a24e6dc942ac7baebe5439dea41238b5770a1). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18698063
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnStats.scala ---
@@ -190,6 +190,24 @@ private[sql] class StringColumnStats extends
ColumnStats {
def collectedStatistics = Row(lower, upper, nullCount)
}
+private[sql] class DateColumnStats extends ColumnStats {
+ var upper: Date = null
+ var lower: Date = null
+ var nullCount = 0
+
+ override def gatherStats(row: Row, ordinal: Int) {
+if (!row.isNullAt(ordinal)) {
+ val value = row(ordinal).asInstanceOf[Date]
+ if (upper == null || value.toString.compareTo(upper.toString) 0)
upper = value
+ if (lower == null || value.toString.compareTo(lower.toString) 0)
lower = value
--- End diff --
Using `.getTime` to do the comparison would be much more efficient. When
caching a `DateType` column, this function is really a critical path.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18698218
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala ---
@@ -376,8 +395,8 @@ private[sql] sealed abstract class
ByteArrayColumnType[T : DataType](
}
}
-private[sql] object BINARY extends ByteArrayColumnType[BinaryType.type](9,
16) {
- override def setField(row: MutableRow, ordinal: Int, value:
Array[Byte]): Unit = {
+private[sql] object BINARY extends
ByteArrayColumnType[BinaryType.type](10, 16) {
+ override def setField(row: MutableRow, ordinal: Int, value: Array[Byte])
{
--- End diff --
Nit: Hmm, actually `: Unit =` is more recommended :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58639081 Left some minor comments, otherwise LGTM. Also, do we want to add Python API in this PR or a separate one? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58646718 I can see most of the reviews are focused on comparing and ordering. I'd like to fix those comparing rules in a separated PR. I tested what you declared here the other day, that was also different from what Hive did. Also Python API is can be a separated PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3807: SparkSql does not work for tables ...
Github user chiragaggarwal commented on the pull request: https://github.com/apache/spark/pull/2674#issuecomment-58647328 Incorporated the review comments and also added a test case --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3807: SparkSql does not work for tables ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2674#issuecomment-58647819 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21590/consoleFull) for PR 2674 at commit [`370c31b`](https://github.com/apache/spark/commit/370c31b58d15c0549176939509ddc44e9f5f677e). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58651729 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21591/consoleFull) for PR 2344 at commit [`00fe81f`](https://github.com/apache/spark/commit/00fe81f5586118694642422f94846f48e6ca50fc). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user adrian-wang commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18704345
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -220,20 +220,44 @@ trait HiveTypeCoercion {
case a: BinaryArithmetic if a.right.dataType == StringType =
a.makeCopy(Array(a.left, Cast(a.right, DoubleType)))
+ // we should cast all timestamp/date/string compare into string
compare
+ case p: BinaryPredicate if p.left.dataType == StringType
+ p.right.dataType == DateType =
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == StringType =
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType == TimestampType =
-p.makeCopy(Array(Cast(p.left, TimestampType), p.right))
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
case p: BinaryPredicate if p.left.dataType == TimestampType
p.right.dataType == StringType =
-p.makeCopy(Array(p.left, Cast(p.right, TimestampType)))
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
+ case p: BinaryPredicate if p.left.dataType == TimestampType
+ p.right.dataType == DateType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == TimestampType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType != StringType =
p.makeCopy(Array(Cast(p.left, DoubleType), p.right))
case p: BinaryPredicate if p.left.dataType != StringType
p.right.dataType == StringType =
p.makeCopy(Array(p.left, Cast(p.right, DoubleType)))
- case i @ In(a,b) if a.dataType == TimestampType
b.forall(_.dataType == StringType) =
-i.makeCopy(Array(a,b.map(Cast(_,TimestampType
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== StringType) =
+i.makeCopy(Array(Cast(a, StringType), b))
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == StringType) =
+i.makeCopy(Array(Cast(a, StringType), b))
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== TimestampType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == DateType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == DateType b.forall(_.dataType
== DateType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
+ case i @ In(a, b) if a.dataType == TimestampType
b.forall(_.dataType == TimestampType) =
+i.makeCopy(Array(Cast(a, StringType), b.map(Cast(_, StringType
--- End diff --
Using long value of `java.sql.Date` is way too far from Hive. I have
discussed with @chenghao-intel and We think we can leave it as it is and fix
the ordering and comparing in a separate PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58652251 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21592/consoleFull) for PR 2344 at commit [`2038085`](https://github.com/apache/spark/commit/203808565ccb6f1e503b4feb2a8eac4fe9b59408). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58654642 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21593/consoleFull) for PR 2344 at commit [`f15074a`](https://github.com/apache/spark/commit/f15074a614281d3fe4de4f0529ddc53994b4c0d9). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user adrian-wang commented on a diff in the pull request:
https://github.com/apache/spark/pull/2344#discussion_r18705174
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -220,20 +220,44 @@ trait HiveTypeCoercion {
case a: BinaryArithmetic if a.right.dataType == StringType =
a.makeCopy(Array(a.left, Cast(a.right, DoubleType)))
+ // we should cast all timestamp/date/string compare into string
compare
+ case p: BinaryPredicate if p.left.dataType == StringType
+ p.right.dataType == DateType =
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == StringType =
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
case p: BinaryPredicate if p.left.dataType == StringType
p.right.dataType == TimestampType =
-p.makeCopy(Array(Cast(p.left, TimestampType), p.right))
+p.makeCopy(Array(p.left, Cast(p.right, StringType)))
case p: BinaryPredicate if p.left.dataType == TimestampType
p.right.dataType == StringType =
-p.makeCopy(Array(p.left, Cast(p.right, TimestampType)))
+p.makeCopy(Array(Cast(p.left, StringType), p.right))
+ case p: BinaryPredicate if p.left.dataType == TimestampType
+ p.right.dataType == DateType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
+ case p: BinaryPredicate if p.left.dataType == DateType
+ p.right.dataType == TimestampType =
+p.makeCopy(Array(Cast(p.left, StringType), Cast(p.right,
StringType)))
--- End diff --
It seems cast('1970-01-01' as date) cast('1970-01-01 00:00:00' as
timestamp)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2805] Upgrade Akka to 2.3.4
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/2752#issuecomment-58655305 This will cause some bug. It seems to be related to `akka.remote.retry-gate-closed-for` configuration the log: ``` 14/10/10 20:56:03 WARN ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkyar...@10dian118.domain.test:20412] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 14/10/10 20:56:03 INFO TaskSetManager: Starting task 60.0 in stage 5427.0 (TID 35496, 10dian71.domain.test, PROCESS_LOCAL, 8062 bytes) 14/10/10 20:56:03 INFO TaskSetManager: Finished task 39.0 in stage 5427.0 (TID 35480) in 539 ms on 10dian71.domain.test (63/72) 14/10/10 20:56:03 INFO TaskSetManager: Finished task 50.0 in stage 5427.0 (TID 35488) in 279 ms on 10dian72.domain.test (64/72) 14/10/10 20:56:03 ERROR YarnClientSchedulerBackend: Yarn application has already exited with state FINISHED! 14/10/10 20:56:03 INFO SparkUI: Stopped Spark web UI at http://10dian71.domain.test:4040 14/10/10 20:56:03 INFO DAGScheduler: Stopping DAGScheduler 14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Shutting down all executors 14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Asking each executor to shut down 14/10/10 20:56:03 INFO DAGScheduler: Job 218 failed: count at TopicModeling.scala:475, took 2.400066 s 14/10/10 20:56:03 INFO YarnClientSchedulerBackend: Stopped ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58655469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21591/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58655457 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21591/consoleFull) for PR 2344 at commit [`00fe81f`](https://github.com/apache/spark/commit/00fe81f5586118694642422f94846f48e6ca50fc). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Cast(child: Expression, dataType: DataType) extends UnaryExpression with Logging ` * `public class DateType extends DataType ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58656118 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21592/consoleFull) for PR 2344 at commit [`2038085`](https://github.com/apache/spark/commit/203808565ccb6f1e503b4feb2a8eac4fe9b59408). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Cast(child: Expression, dataType: DataType) extends UnaryExpression with Logging ` * `public class DateType extends DataType ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3407][SQL]Add Date type support
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2344#issuecomment-58656123 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21592/Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
