[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22001 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94644/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22001 **[Test build #94644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94644/testReport)** for PR 22001 at commit [`8b16c57`](https://github.com/apache/spark/commit/8b16c57dd6c58361b4ff40dbaf644b4f22d10808). * This patch **fails from timeout after a configured wait of \`340m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94652/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #94652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94652/testReport)** for PR 18906 at commit [`cdd16a9`](https://github.com/apache/spark/commit/cdd16a9b232b58d96b925405c27fd61182eb3b7a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #94652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94652/testReport)** for PR 18906 at commit [`cdd16a9`](https://github.com/apache/spark/commit/cdd16a9b232b58d96b925405c27fd61182eb3b7a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21439 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21439#discussion_r209461516
--- Diff: sql/core/src/test/resources/sql-tests/inputs/json-functions.sql
---
@@ -39,3 +39,8 @@ select from_json('{"a":1, "b":"2"}',
'struct');
-- infer schema of json literal
select schema_of_json('{"c1":0, "c2":[1]}');
select from_json('{"c1":[1, 2, 3]}', schema_of_json('{"c1":[0]}'));
+
+-- from_json - array type
+select from_json('[1, 2, 3]', 'array');
--- End diff --
Add more cases ?
select from_json('[3, null, 4]', 'array')
select from_json('[3, "str", 4]', 'array')
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21439#discussion_r209461334
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
---
@@ -101,6 +102,21 @@ class JacksonParser(
}
}
+ private def makeArrayRootConverter(at: ArrayType): JsonParser =>
Seq[InternalRow] = {
+val elemConverter = makeConverter(at.elementType)
+(parser: JsonParser) => parseJsonToken[Seq[InternalRow]](parser, at) {
+ case START_ARRAY => Seq(InternalRow(convertArray(parser,
elemConverter)))
+ case START_OBJECT if at.elementType.isInstanceOf[StructType] =>
+// This handles the case when an input JSON object is a structure
but
+// the specified schema is an array of structures. In that case,
the input JSON is
--- End diff --
Could you add an example here, like what we did in `makeStructRootConverter
`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22053 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22053 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94645/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21939: [SPARK-23874][SQL][PYTHON] Upgrade Apache Arrow to 0.10....
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21939 Nice! Thanks for getting that running @shaneknapp . So what are peoples thoughts about merging this for 2.4 since it passes normal tests with pyarrow 0.8.0 and we've also shown it passes with 0.10.0? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22053 **[Test build #94645 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94645/testReport)** for PR 22053 at commit [`d95d357`](https://github.com/apache/spark/commit/d95d35794528702a2de5523ca00334d479598c57). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22075 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2105/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22075 **[Test build #94651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94651/testReport)** for PR 22075 at commit [`388c2d3`](https://github.com/apache/spark/commit/388c2d3d812bf749ddf9de029432eab729bcc932). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22001: [SPARK-24819][CORE] Fail fast when no enough slot...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r209460397
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -929,11 +963,38 @@ class DAGScheduler(
// HadoopRDD whose underlying HDFS files have been deleted.
finalStage = createResultStage(finalRDD, func, partitions, jobId,
callSite)
} catch {
+ case e: Exception if e.getMessage.contains(
+
DAGScheduler.ERROR_MESSAGE_BARRIER_REQUIRE_MORE_SLOTS_THAN_CURRENT_TOTAL_NUMBER)
=>
+logWarning(s"The job $jobId requires to run a barrier stage that
requires more slots " +
+ "than the total number of slots in the cluster currently.")
+jobIdToNumTasksCheckFailures.compute(jobId, new BiFunction[Int,
Int, Int] {
+ override def apply(key: Int, value: Int): Int = value + 1
+})
+val numCheckFailures = jobIdToNumTasksCheckFailures.get(jobId)
+if (numCheckFailures <= maxFailureNumTasksCheck) {
+ messageScheduler.schedule(
+new Runnable {
+ override def run(): Unit =
eventProcessLoop.post(JobSubmitted(jobId, finalRDD, func,
+partitions, callSite, listener, properties))
+},
+timeIntervalNumTasksCheck * 1000,
--- End diff --
minor: how about removing `1000` and changing the time unit to `SECONDS`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22001: [SPARK-24819][CORE] Fail fast when no enough slot...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r209460279
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -929,11 +963,38 @@ class DAGScheduler(
// HadoopRDD whose underlying HDFS files have been deleted.
finalStage = createResultStage(finalRDD, func, partitions, jobId,
callSite)
} catch {
+ case e: Exception if e.getMessage.contains(
+
DAGScheduler.ERROR_MESSAGE_BARRIER_REQUIRE_MORE_SLOTS_THAN_CURRENT_TOTAL_NUMBER)
=>
+logWarning(s"The job $jobId requires to run a barrier stage that
requires more slots " +
+ "than the total number of slots in the cluster currently.")
+jobIdToNumTasksCheckFailures.compute(jobId, new BiFunction[Int,
Int, Int] {
--- End diff --
minor: Should have an inline comment that mentions the implicit
conversation from `null` to `0: Int` to handle new keys.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22001: [SPARK-24819][CORE] Fail fast when no enough slot...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r209460309
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -929,11 +963,38 @@ class DAGScheduler(
// HadoopRDD whose underlying HDFS files have been deleted.
finalStage = createResultStage(finalRDD, func, partitions, jobId,
callSite)
} catch {
+ case e: Exception if e.getMessage.contains(
+
DAGScheduler.ERROR_MESSAGE_BARRIER_REQUIRE_MORE_SLOTS_THAN_CURRENT_TOTAL_NUMBER)
=>
+logWarning(s"The job $jobId requires to run a barrier stage that
requires more slots " +
+ "than the total number of slots in the cluster currently.")
+jobIdToNumTasksCheckFailures.compute(jobId, new BiFunction[Int,
Int, Int] {
+ override def apply(key: Int, value: Int): Int = value + 1
+})
+val numCheckFailures = jobIdToNumTasksCheckFailures.get(jobId)
--- End diff --
minor: this is the return value from `compute`. we don't need `get`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2104/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22085 **[Test build #94650 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94650/testReport)** for PR 22085 at commit [`7b48829`](https://github.com/apache/spark/commit/7b488299709f715d344e5c38956577f31718ab34). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22085: [SPARK-25095][PySpark] Python support for Barrier...
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/22085 [SPARK-25095][PySpark] Python support for BarrierTaskContext ## What changes were proposed in this pull request? Add method `barrier()` and `getTaskInfos()` in python TaskContext, these two methods are only allowed for barrier tasks. ## How was this patch tested? TBD You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiangxb1987/spark python.barrier Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22085.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22085 commit 7b488299709f715d344e5c38956577f31718ab34 Author: Xingbo Jiang Date: 2018-08-12T16:04:20Z implement python barrier taskcontext --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22001 **[Test build #94649 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94649/testReport)** for PR 22001 at commit [`8b16c57`](https://github.com/apache/spark/commit/8b16c57dd6c58361b4ff40dbaf644b4f22d10808). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22001 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2103/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/22001 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22084: [SPARK-25025][BUILD] Binary releases should contain some...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2102/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22084: [SPARK-25025][BUILD] Binary releases should contain some...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22084: [SPARK-25025][BUILD] Binary releases should contain some...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22084 **[Test build #94648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94648/testReport)** for PR 22084 at commit [`5b5c4f5`](https://github.com/apache/spark/commit/5b5c4f5bd3d12410da416a3253dad31508e48ce6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22084: [SPARK-25025][BUILD] Binary releases should conta...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/22084 [SPARK-25025][BUILD] Binary releases should contain some copy of compiled external integration modules ## What changes were proposed in this pull request? Copy external/ modules' JAR files into external/jars in binary release ## How was this patch tested? N/A; manual testing You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark SPARK-25026 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22084.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22084 commit 5b5c4f5bd3d12410da416a3253dad31508e48ce6 Author: Sean Owen Date: 2018-08-12T15:19:42Z Copy external/ modules' JAR files into external/jars in binary release --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94646/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21912 **[Test build #94646 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94646/testReport)** for PR 21912 at commit [`32dfcb5`](https://github.com/apache/spark/commit/32dfcb57b280884e57e464c3d718498510c9e307). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ArraySetLike ` * `trait ArrayBinaryLike extends BinaryArrayExpressionWithImplicitCast with ArraySetLike ` * `case class ArrayUnion(left: Expression, right: Expression) extends ArrayBinaryLike` * `case class ArrayIntersect(left: Expression, right: Expression) extends ArrayBinaryLike` * `case class ArrayExcept(left: Expression, right: Expression) extends ArrayBinaryLike` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94642/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22038 **[Test build #94642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94642/testReport)** for PR 22038 at commit [`4fd2143`](https://github.com/apache/spark/commit/4fd21436f50130fb493d155aa3566268e67c5963). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94643/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22082 **[Test build #94643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94643/testReport)** for PR 22082 at commit [`2666500`](https://github.com/apache/spark/commit/266650006ed1f5d19d6eaf24d7058ca341457039). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22076 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22076 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94647/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22076 **[Test build #94647 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94647/testReport)** for PR 22076 at commit [`1727bf0`](https://github.com/apache/spark/commit/1727bf0c467edec0646e5055f6348813ac33671a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22076 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22076 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2101/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22071: [SPARK-25088][CORE][MESOS][DOCS] Update Rest Server docs...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22071 **[Test build #4245 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4245/testReport)** for PR 22071 at commit [`b4ca224`](https://github.com/apache/spark/commit/b4ca224095cb7fda6822c431465bfb7f48a4bb2d). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22076: [SPARK-25090][ML] Enforce implicit type coercion in Para...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22076 **[Test build #94647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94647/testReport)** for PR 22076 at commit [`1727bf0`](https://github.com/apache/spark/commit/1727bf0c467edec0646e5055f6348813ac33671a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22069: [MINOR][DOC] Fix Java example code in Column's co...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22069 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22069: [MINOR][DOC] Fix Java example code in Column's comments
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22069
I'm going to merge this, as the additional example quoted above should
actually work, assuming one imported the 'col' function. In Java,
`people("foo")` would be wrong but `col("foo")` is not (necessarily).
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22081: [SPARK-23654][BUILD] remove jets3t as a dependency of sp...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22081 Hm @steveloughran looks like the Kinesis tests fail reliably. That makes me suspicious that jets3t is needed, given that's the AWS dependency here. WDYT? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22016: Fix typos
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22016 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22016: Fix typos
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22016 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22081: [SPARK-23654][BUILD] remove jets3t as a dependenc...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22081#discussion_r209454764
--- Diff: pom.xml ---
@@ -984,24 +987,15 @@
-
+
-net.java.dev.jets3t
-jets3t
-${jets3t.version}
+javax.activation
+activation
+1.1.1
--- End diff --
Yeah really the change is to remove jets3t; I think Steve also thought it
necessary to add back javax.activation that it brought in but wasn't otherwise
depended-on by Hadoop. That is I think this patches a gap in the Hadoop pom?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22076: [SPARK-25090][ML] Enforce implicit type coercion ...
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22076#discussion_r209454605 --- Diff: python/pyspark/ml/tests.py --- @@ -950,6 +950,13 @@ def test_fit_maximize_metric(self): "Best model should have zero induced error") self.assertEqual(1.0, bestModelMetric, "Best model has R-squared of 1") +def test_param_grid_type_coercion(self): +lr = LogisticRegression(maxIter=10) +paramGrid = ParamGridBuilder().addGrid(lr.regParam, [0.5, 1]).build() +for param in paramGrid: +for v in param.values(): +assert(type(v) is float) --- End diff -- nit: `type(v) == float ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20838 Hm, why https://github.com/apache/spark/pull/20838/commits/b178587909be038b8bc5c7254fb2a384ae61a7e4 was reverted? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21912 **[Test build #94646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94646/testReport)** for PR 21912 at commit [`32dfcb5`](https://github.com/apache/spark/commit/32dfcb57b280884e57e464c3d718498510c9e307). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2100/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22053 **[Test build #94645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94645/testReport)** for PR 22053 at commit [`d95d357`](https://github.com/apache/spark/commit/d95d35794528702a2de5523ca00334d479598c57). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22053 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2099/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22053 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22053: [SPARK-25069][CORE]Using UnsafeAlignedOffset to make the...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22053 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22001 **[Test build #94644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94644/testReport)** for PR 22001 at commit [`8b16c57`](https://github.com/apache/spark/commit/8b16c57dd6c58361b4ff40dbaf644b4f22d10808). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2098/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22001 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22001: [SPARK-24819][CORE] Fail fast when no enough slots to la...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/22001 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22083: [SQL][Test][Minor] Add missing codes to ParquetCo...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22083 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22083 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22079: [SPARK-23207][SQL][BACKPORT-2.2] Shuffle+Repartition on ...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/22079 We shall also include #20088 in this backport PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22078: [SPARK-25085][SQL] Insert overwrite a non-partitioned ta...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22078 I am just worried/concerned by the performance regression this might introduce in the case the table is made of many files. Can we have a benchmark using different file system (at least HDFS, S3) with hundreds and thousands of files? Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2097/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22082 **[Test build #94643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94643/testReport)** for PR 22082 at commit [`2666500`](https://github.com/apache/spark/commit/266650006ed1f5d19d6eaf24d7058ca341457039). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22082 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22038 **[Test build #94642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94642/testReport)** for PR 22038 at commit [`4fd2143`](https://github.com/apache/spark/commit/4fd21436f50130fb493d155aa3566268e67c5963). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22083 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94641/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2096/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22083 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22083 **[Test build #94641 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94641/testReport)** for PR 22083 at commit [`974224b`](https://github.com/apache/spark/commit/974224b495fc8155b9a6a73673ea478ccdb164d6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22038 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/22077 Thanks @wangyum for triggerring test again and again. Now all tests passed, cc @cloud-fan @gatorsmile @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22077 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22077 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94640/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22077 **[Test build #94640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94640/testReport)** for PR 22077 at commit [`49c9060`](https://github.com/apache/spark/commit/49c9060e7f9426aa4cf67b64446b9c71e877d687). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22010: [SPARK-21436][CORE] Take advantage of known parti...
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22010#discussion_r209450199
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -396,7 +396,16 @@ abstract class RDD[T: ClassTag](
* Return a new RDD containing the distinct elements in this RDD.
*/
def distinct(numPartitions: Int)(implicit ord: Ordering[T] = null):
RDD[T] = withScope {
-map(x => (x, null)).reduceByKey((x, y) => x, numPartitions).map(_._1)
+// If the data is already approriately partitioned with a known
partitioner we can work locally.
+def removeDuplicatesInPartition(itr: Iterator[T]): Iterator[T] = {
+ val set = new mutable.HashSet[T]()
+ itr.filter(set.add(_))
--- End diff --
yes, it is not a big deal, but if you check the implementation in the scala
lib you can see that the hash and the index for the key is computed despite it
not needed (since `addElem` is called anyway). Probably it doesn't change much,
but we could save this computation...
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22083 **[Test build #94641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94641/testReport)** for PR 22083 at commit [`974224b`](https://github.com/apache/spark/commit/974224b495fc8155b9a6a73673ea478ccdb164d6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22083 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2095/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22083: [SQL][Test][Minor] Add missing codes to ParquetCompressi...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22083 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22077 **[Test build #94640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94640/testReport)** for PR 22077 at commit [`49c9060`](https://github.com/apache/spark/commit/49c9060e7f9426aa4cf67b64446b9c71e877d687). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22083: [SQL][Test][Minor] Add missing codes to ParquetCo...
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22083 [SQL][Test][Minor] Add missing codes to ParquetCompressionCodecPrecedenceSuite ## What changes were proposed in this pull request? This PR adds codes to ``"Test `spark.sql.parquet.compression.codec` config"` in `ParquetCompressionCodecPrecedenceSuite`. ## How was this patch tested? Existing UTs You can merge this pull request into a Git repository by running: $ git pull https://github.com/kiszk/spark ParquetCompressionCodecPrecedenceSuite Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22083.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22083 commit 974224b495fc8155b9a6a73673ea478ccdb164d6 Author: Kazuaki Ishizaki Date: 2018-08-12T07:08:42Z update --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22077 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94638/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94637/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22077 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22038: [SPARK-25056][SQL] Unify the InConversion and BinaryComp...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22038 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22082 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94639/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22082: [SPARK-24420][Build][FOLLOW-UP] Upgrade ASM6 APIs
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22082 **[Test build #94639 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94639/testReport)** for PR 22082 at commit [`2666500`](https://github.com/apache/spark/commit/266650006ed1f5d19d6eaf24d7058ca341457039). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22077: [SPARK-25084][SQL][BACKPORT-2.3] "distribute by" on mult...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22077 **[Test build #94637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94637/testReport)** for PR 22077 at commit [`49c9060`](https://github.com/apache/spark/commit/49c9060e7f9426aa4cf67b64446b9c71e877d687). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
