[GitHub] [spark] huaxingao commented on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
huaxingao commented on pull request #29043:
URL: https://github.com/apache/spark/pull/29043#issuecomment-657179299
@TJX2014
I am still not convinced that it's a good idea to map TimeStamp to DateTime
in MySQLDialect. Using an example similar to yours:
```
sql("select cast('1970-01-01 00:00:01' as
timestamp)").toDF("ts").write.mode("append").jdbc("jdbc:mysql://localhost:3306/test",
"ts_test3",prop)
```
Since user explicitly cast string to timestamp, I would think that the user
wants to insert 1970-01-0 00:00:01 as a TimeStamp data type. Suppose the
current time zone on mysql server is america/los_angeles. After the data is
inserted to mysql, if the user changes the time zone setting
SET TIME_ZONE = "america/new_york";
Then the user would expect to get 1970-01-01 03:00:01 when retrieving the
timestamp. But with this patch, we silently change the data type from TimeStamp
to DateTime, after user insert the timestamp and change the time zone setting,
user will still get 1970-01-0 00:00:01 which is not the correct value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657178823 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657178823 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
SparkQA removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657154729 **[Test build #125699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125699/testReport)** for PR 28898 at commit [`da85920`](https://github.com/apache/spark/commit/da859203e91a0bc90b017a1557bcf3646733982a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657178726 **[Test build #125699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125699/testReport)** for PR 28898 at commit [`da85920`](https://github.com/apache/spark/commit/da859203e91a0bc90b017a1557bcf3646733982a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable
imback82 commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-657170656 > (3).We are seeing in production, coalescing might hurt the parallelism, if the number of buckets are too few. Another way to avoid shuffle and sort, is to split/divide the table with less buckets. E.g. joining tables with t1 (8 buckets) and t2 (32 buckets), we can keep number of tasks to be 32, and each task for reading t1 table will have a filter at run-time to only keep its portion of table (divide the table with less buckets). This has downside of reading the t1 more than once from multiple tasks, but if the size of t1 is not big, it's a good trade off to have more parallelism (and may be better than shuffling t1 directly). I had a rough [POC](https://github.com/apache/spark/compare/master...imback82:bucketing_poc2) few months back, where each row is filtered out based on its bucket id, but I never got a chance to run benchmarks. @c21 Do you have some numbers to share? I am wondering how reading multiple copies impacts the overall runtime. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection
imback82 commented on a change in pull request #29074:
URL: https://github.com/apache/spark/pull/29074#discussion_r453261237
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
##
@@ -171,19 +176,50 @@ case class EnsureRequirements(conf: SQLConf) extends
Rule[SparkPlan] {
leftPartitioning: Partitioning,
rightPartitioning: Partitioning): (Seq[Expression], Seq[Expression]) = {
if (leftKeys.forall(_.deterministic) && rightKeys.forall(_.deterministic))
{
- (leftPartitioning, rightPartitioning) match {
-case (HashPartitioning(leftExpressions, _), _) =>
- reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
leftExpressions, leftKeys)
-case (_, HashPartitioning(rightExpressions, _)) =>
- reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
rightExpressions, rightKeys)
-case _ =>
- (leftKeys, rightKeys)
- }
+ reorderJoinKeysRecursively(leftKeys, rightKeys, leftPartitioning,
rightPartitioning)
+.getOrElse((leftKeys, rightKeys))
} else {
(leftKeys, rightKeys)
}
}
+ /**
+ * Recursively reorders the join keys based on partitioning. It starts
reordering the
+ * join keys to match HashPartitioning on either side, followed by
PartitioningCollection.
+ */
+ private def reorderJoinKeysRecursively(
+ leftKeys: Seq[Expression],
+ rightKeys: Seq[Expression],
+ leftPartitioning: Partitioning,
+ rightPartitioning: Partitioning): Option[(Seq[Expression],
Seq[Expression])] = {
+(leftPartitioning, rightPartitioning) match {
+ case (HashPartitioning(leftExpressions, _), _) =>
+reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
leftExpressions, leftKeys)
+ .orElse(reorderJoinKeysRecursively(
+leftKeys, rightKeys, UnknownPartitioning(0), rightPartitioning))
+ case (_, HashPartitioning(rightExpressions, _)) =>
+reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
rightExpressions, rightKeys)
+ .orElse(reorderJoinKeysRecursively(
Review comment:
This can be also implemented by looking at left partitioning first then
move to the right partitionoing:
```scala
(leftPartitioning, rightPartitioning) match {
case (HashPartitioning(leftExpressions, _), _) =>
reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
leftExpressions, leftKeys)
.orElse(reorderJoinKeysRecursively(
leftKeys, rightKeys, UnknownPartitioning(0), rightPartitioning))
case (PartitioningCollection(partitionings), _) =>
partitionings.foreach { p =>
reorderJoinKeysRecursively(leftKeys, rightKeys, p,
rightPartitioning).map { k =>
return Some(k)
}
}
reorderJoinKeysRecursively(leftKeys, rightKeys,
UnknownPartitioning(0), rightPartitioning)
case (_, HashPartitioning(rightExpressions, _)) =>
reorder(leftKeys.toIndexedSeq, rightKeys.toIndexedSeq,
rightExpressions, rightKeys)
case (_, PartitioningCollection(partitionings)) =>
partitionings.foreach { p =>
reorderJoinKeysRecursively(leftKeys, rightKeys, leftPartitioning,
p).map { k =>
return Some(k)
}
}
None
case _ =>
None
}
```
However, I chose this way so that the behavior remains the same. If you have
`leftPartitioning = PartitioningCollection` and `rightPartitioning =
HashPartitioning`, it will match the `rightPartitioning` first, which is the
existing behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection
AmplabJenkins removed a comment on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-657167465 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection
AmplabJenkins commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-657167465 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection
SparkQA commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-657167295 **[Test build #125704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125704/testReport)** for PR 29074 at commit [`99493e4`](https://github.com/apache/spark/commit/99493e422f6d3540cf342a4ff23686d1aee4ac14). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable
maropu commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-657166984 Thanks for your interest, @c21 > (1).Is there a reason why we don't cover ShuffledHashJoin as well? (we are seeing in production, people also use ShuffledHashJoin a lot for joining bucketed tables when one side is small) As you said in (3), too, I think that's because there is a concern where coalescing might hurt the parallelism. You can see the related discussion in the history: https://github.com/apache/spark/pull/28123#discussion_r427073319 As for (1) and (3), IMO its worth digging into it for more improvements. > (2).Per this PR, the ordering property of coalesced bucket files does not preserve (https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala#L317), and the ordering can be preserved through a sort-merge-way read of all sorted buckets file. This can help when reading multiple partitions of bucketed table. I think that's the long-standing issue we have. Have you checked the discussion in SPARK-24528? If you're interested in the issue, you can revisit it there. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 opened a new pull request #29074: [SPARK-xxx][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection
imback82 opened a new pull request #29074:
URL: https://github.com/apache/spark/pull/29074
### What changes were proposed in this pull request?
This PR proposes to improve `EnsureRquirement.reorderJoinKeys` to handle
the following scenarios:
1. If the keys cannot be reordered to match the left-side
`HashPartitioning`, consider the right-side `HashPartitioning`.
2. Handle `PartitioningCollection`, which may contain `HashPartitioning`
### Why are the changes needed?
1. For the scenario 1), the current behavior matches either the left-side
`HashPartitioning` or the right-side `HashPartitioning`. This means that if
both sides are `HashPartitioning`, it will try to match only the left side.
The following will not consider the right-side `HashPartitioning`:
```
val df1 = (0 until 10).map(i => (i % 5, i % 13)).toDF("i1", "j1")
val df2 = (0 until 10).map(i => (i % 7, i % 11)).toDF("i2", "j2")
df1.write.format("parquet").bucketBy(4, "i1",
"j1").saveAsTable("t1")df2.write.format("parquet").bucketBy(4, "i2",
"j2").saveAsTable("t2")
val t1 = spark.table("t1")
val t2 = spark.table("t2")
val join = t1.join(t2, t1("i1") === t2("j2") && t1("i1") === t2("i2"))
join.explain
== Physical Plan ==
*(5) SortMergeJoin [i1#26, i1#26], [j2#31, i2#30], Inner
:- *(2) Sort [i1#26 ASC NULLS FIRST, i1#26 ASC NULLS FIRST], false, 0
: +- Exchange hashpartitioning(i1#26, i1#26, 4), true, [id=#69]
: +- *(1) Project [i1#26, j1#27]
:+- *(1) Filter isnotnull(i1#26)
: +- *(1) ColumnarToRow
: +- FileScan parquet default.t1[i1#26,j1#27] Batched: true,
DataFilters: [isnotnull(i1#26)], Format: Parquet, Location:
InMemoryFileIndex[..., PartitionFilters: [], PushedFilters: [IsNotNull(i1)],
ReadSchema: struct, SelectedBucketsCount: 4 out of 4
+- *(4) Sort [j2#31 ASC NULLS FIRST, i2#30 ASC NULLS FIRST], false, 0.
+- Exchange hashpartitioning(j2#31, i2#30, 4), true, [id=#79].
<= This can be removed
+- *(3) Project [i2#30, j2#31]
+- *(3) Filter (((j2#31 = i2#30) AND isnotnull(j2#31)) AND
isnotnull(i2#30))
+- *(3) ColumnarToRow
+- FileScan parquet default.t2[i2#30,j2#31] Batched: true,
DataFilters: [(j2#31 = i2#30), isnotnull(j2#31), isnotnull(i2#30)], Format:
Parquet, Location: InMemoryFileIndex[..., PartitionFilters: [], PushedFilters:
[IsNotNull(j2), IsNotNull(i2)], ReadSchema: struct,
SelectedBucketsCount: 4 out of 4
```
2. For the scenario 2), the current behavior does not handle
`PartitioningCollection`:
```
val df1 = (0 until 100).map(i => (i % 5, i % 13)).toDF("i1", "j1")
val df2 = (0 until 100).map(i => (i % 7, i % 11)).toDF("i2", "j2")
val df3 = (0 until 100).map(i => (i % 5, i % 13)).toDF("i3", "j3")
val join = df1.join(df2, df1("i1") === df2("i2") && df1("j1") === df2("j2"))
// PartitioningCollection
val join2 = join.join(df3, join("j1") === df3("j3") && join("i1") ===
df3("i3"))
join2.explain
== Physical Plan ==
*(9) SortMergeJoin [j1#8, i1#7], [j3#30, i3#29], Inner
:- *(6) Sort [j1#8 ASC NULLS FIRST, i1#7 ASC NULLS FIRST], false, 0.
<= This can be removed
: +- Exchange hashpartitioning(j1#8, i1#7, 5), true, [id=#58]
<= This can be removed
: +- *(5) SortMergeJoin [i1#7, j1#8], [i2#18, j2#19], Inner
::- *(2) Sort [i1#7 ASC NULLS FIRST, j1#8 ASC NULLS FIRST], false, 0
:: +- Exchange hashpartitioning(i1#7, j1#8, 5), true, [id=#45]
:: +- *(1) Project [_1#2 AS i1#7, _2#3 AS j1#8]
::+- *(1) LocalTableScan [_1#2, _2#3]
:+- *(4) Sort [i2#18 ASC NULLS FIRST, j2#19 ASC NULLS FIRST], false, 0
: +- Exchange hashpartitioning(i2#18, j2#19, 5), true, [id=#51]
: +- *(3) Project [_1#13 AS i2#18, _2#14 AS j2#19]
: +- *(3) LocalTableScan [_1#13, _2#14]
+- *(8) Sort [j3#30 ASC NULLS FIRST, i3#29 ASC NULLS FIRST], false, 0
+- Exchange hashpartitioning(j3#30, i3#29, 5), true, [id=#64]
+- *(7) Project [_1#24 AS i3#29, _2#25 AS j3#30]
+- *(7) LocalTableScan [_1#24, _2#25]
```
### Does this PR introduce _any_ user-facing change?
Yes, now from the above examples, the shuffle/sort nodes pointed by `This
can be removed` are now removed:
1. Senario 1):
```
== Physical Plan ==
*(4) SortMergeJoin [i1#26, i1#26], [i2#30, j2#31], Inner
:- *(2) Sort [i1#26 ASC NULLS FIRST, i1#26 ASC NULLS FIRST], false, 0
: +- Exchange hashpartitioning(i1#26, i1#26, 4), true, [id=#67]
: +- *(1) Project [i1#26, j1#27]
:+- *(1) Filter isnotnull(i1#26)
: +- *(1) ColumnarToRow
: +- FileScan parquet default.t1[i1#26,j1#27] Batched: true,
DataFilters: [isnotnull(i1#26)], Format: Parquet, Location:
InMemoryFileIndex[..., PartitionFilters: [], PushedFilters:
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657165941 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657163429 **[Test build #125702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125702/testReport)** for PR 29069 at commit [`611ba1b`](https://github.com/apache/spark/commit/611ba1b882cceb8a9657a6b8a175b6aa27aea6b3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657165941 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657165900 **[Test build #125702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125702/testReport)** for PR 29069 at commit [`611ba1b`](https://github.com/apache/spark/commit/611ba1b882cceb8a9657a6b8a175b6aa27aea6b3). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class GetCatalogsOperationMock extends GetCatalogsOperation ` * `public class OperationManagerMock extends OperationManager ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] kiszk commented on a change in pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable
kiszk commented on a change in pull request #29067:
URL: https://github.com/apache/spark/pull/29067#discussion_r453257863
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala
##
@@ -474,12 +484,8 @@ class InMemoryColumnarQuerySuite extends QueryTest with
SharedSparkSession {
test("SPARK-22249: buildFilter should not throw exception when In contains
an empty list") {
val attribute = AttributeReference("a", IntegerType)()
-val localTableScanExec = LocalTableScanExec(Seq(attribute), Nil)
-val testRelation = InMemoryRelation(false, 1, MEMORY_ONLY,
localTableScanExec, None,
- LocalRelation(Seq(attribute), Nil))
-val tableScanExec = InMemoryTableScanExec(Seq(attribute),
- Seq(In(attribute, Nil)), testRelation)
-assert(tableScanExec.partitionFilters.isEmpty)
+val testSerializer = new TestCachedBatchSerializer(false, 1)
+testSerializer.buildFilter(Seq(In(attribute, Nil)), Seq(attribute))
Review comment:
Why is `assert` removed from this test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657163573 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
AmplabJenkins removed a comment on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-657163572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657163573 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
AmplabJenkins commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-657163572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
SparkQA commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-657163441 **[Test build #125703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125703/testReport)** for PR 28715 at commit [`d6f8459`](https://github.com/apache/spark/commit/d6f8459556d864b3d2249f45f73332812659dba0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657163429 **[Test build #125702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125702/testReport)** for PR 29069 at commit [`611ba1b`](https://github.com/apache/spark/commit/611ba1b882cceb8a9657a6b8a175b6aa27aea6b3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] kiszk commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
kiszk commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-657162781 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657162290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657162290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657142312 **[Test build #125695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125695/testReport)** for PR 29072 at commit [`7588799`](https://github.com/apache/spark/commit/75887992b679c7e32b048a465bbab72c4cb55e30). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657162153 **[Test build #125695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125695/testReport)** for PR 29072 at commit [`7588799`](https://github.com/apache/spark/commit/75887992b679c7e32b048a465bbab72c4cb55e30). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657161975 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657161975 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657143178 **[Test build #125696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125696/testReport)** for PR 29072 at commit [`00f4202`](https://github.com/apache/spark/commit/00f4202c52ba3c9283d52d712f6e3e4c81996196). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657161855 **[Test build #125696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125696/testReport)** for PR 29072 at commit [`00f4202`](https://github.com/apache/spark/commit/00f4202c52ba3c9283d52d712f6e3e4c81996196). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657160470 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125701/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657159524 **[Test build #125701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125701/testReport)** for PR 29069 at commit [`6771504`](https://github.com/apache/spark/commit/6771504eb87c3cf8a6f23f1657c93509c3932041). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657160466 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657160461 **[Test build #125701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125701/testReport)** for PR 29069 at commit [`6771504`](https://github.com/apache/spark/commit/6771504eb87c3cf8a6f23f1657c93509c3932041). * This patch **fails Java style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class GetCatalogsOperationMock extends GetCatalogsOperation ` * `public class OperationManagerMock extends OperationManager ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657160466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] kiszk commented on pull request #29073: Update ml-datasource.md
kiszk commented on pull request #29073: URL: https://github.com/apache/spark/pull/29073#issuecomment-657160205 Thank you for your contribution. - Could you update the description based on the template (like [this](https://github.com/apache/spark/pull/28958))? - Could you apply this change to other places (at least I can find the similar typo twice in the same file)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657159620 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657159620 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657159524 **[Test build #125701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125701/testReport)** for PR 29069 at commit [`6771504`](https://github.com/apache/spark/commit/6771504eb87c3cf8a6f23f1657c93509c3932041). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28971: [SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13
AmplabJenkins removed a comment on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-657157966 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28971: [SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13
AmplabJenkins commented on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-657157966 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible
SparkQA commented on pull request #29013: URL: https://github.com/apache/spark/pull/29013#issuecomment-657157955 **[Test build #125700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125700/testReport)** for PR 29013 at commit [`21c5262`](https://github.com/apache/spark/commit/21c5262e7025a7e1453c1aeaa901ada0781e875d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28971: [SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13
SparkQA removed a comment on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-657133192 **[Test build #125693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125693/testReport)** for PR 28971 at commit [`8f5af5f`](https://github.com/apache/spark/commit/8f5af5f20db18ce0ea1fa02bc854f710a9a71dff). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28971: [SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13
SparkQA commented on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-657157739 **[Test build #125693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125693/testReport)** for PR 28971 at commit [`8f5af5f`](https://github.com/apache/spark/commit/8f5af5f20db18ce0ea1fa02bc854f710a9a71dff). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes
AmplabJenkins removed a comment on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-657155924 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125690/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes
SparkQA removed a comment on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-657128198 **[Test build #125690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125690/testReport)** for PR 29066 at commit [`6fea82b`](https://github.com/apache/spark/commit/6fea82b34621843844f9fde8a70b113f170c74b2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
SparkQA removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-657135885 **[Test build #125694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125694/testReport)** for PR 28746 at commit [`4c8ceec`](https://github.com/apache/spark/commit/4c8ceec09911ea3f62efc3d2add222636bffc2fd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes
AmplabJenkins removed a comment on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-657155922 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible
AmplabJenkins removed a comment on pull request #29013: URL: https://github.com/apache/spark/pull/29013#issuecomment-657157242 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-657157265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29056: [SPARK-31753][SQL][DOCS][WIP]Add missing keywords
maropu commented on pull request #29056: URL: https://github.com/apache/spark/pull/29056#issuecomment-657157467 > LATERAL VIEW，PIVOT I am not sure if them shoud belong to function group or our key words group,such as SELECT So i just leave them for furture. I will add it if needed. If possible, I think we'd better to add them in this PR. How about adding them in the select doc? https://github.com/apache/spark/blob/master/docs/sql-ref-syntax-qry-select.md https://github.com/apache/spark/blob/004aea8155d3b768684ab20cd29b3a820a204f22/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4#L574 WDYT? @huaxingao @dilipbiswal This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
AmplabJenkins commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-657157265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible
AmplabJenkins commented on pull request #29013: URL: https://github.com/apache/spark/pull/29013#issuecomment-657157242 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
SparkQA commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-657157152 **[Test build #125694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125694/testReport)** for PR 28746 at commit [`4c8ceec`](https://github.com/apache/spark/commit/4c8ceec09911ea3f62efc3d2add222636bffc2fd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes
AmplabJenkins commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-657155922 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes
SparkQA commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-657155858 **[Test build #125690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125690/testReport)** for PR 29066 at commit [`6fea82b`](https://github.com/apache/spark/commit/6fea82b34621843844f9fde8a70b113f170c74b2). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class Distributions ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29073: Update ml-datasource.md
AmplabJenkins removed a comment on pull request #29073: URL: https://github.com/apache/spark/pull/29073#issuecomment-657155463 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29073: Update ml-datasource.md
AmplabJenkins commented on pull request #29073: URL: https://github.com/apache/spark/pull/29073#issuecomment-65716 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE
LantaoJin commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-657155507 > Can you keep the original PR template? Like "What changes were proposed in this pull request?" was removed. Please describe why the issue happened and how this fixes it in the PR description. Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29073: Update ml-datasource.md
AmplabJenkins commented on pull request #29073: URL: https://github.com/apache/spark/pull/29073#issuecomment-657155463 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ChuliangXiao opened a new pull request #29073: Update ml-datasource.md
ChuliangXiao opened a new pull request #29073: URL: https://github.com/apache/spark/pull/29073 change `true` to `True` in the python code ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type
maropu commented on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-657155312 Thanks! Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu closed pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type
maropu closed pull request #28979: URL: https://github.com/apache/spark/pull/28979 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657154893 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #29056: [SPARK-31753][SQL][DOCS][WIP]Add missing keywords
maropu commented on pull request #29056: URL: https://github.com/apache/spark/pull/29056#issuecomment-657154982 > @maropu EXPLAIN LOGICAL seesm disallowed, so i did not add it in the ref-docs AFTER key words, i have no ideas about its usage , Do you have any ideas? LATERAL VIEW，PIVOT I am not sure if them shoud belong to function group or our key words group,such as SELECT So i just leave them for furture. I will add it if needed. Yea, its okay to add valid keywords only. I just listed up the missing keywords in the jira by using a `grep` command, so we need to check if they are valid or not word-by-word. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657154893 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657154729 **[Test build #125699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125699/testReport)** for PR 28898 at commit [`da85920`](https://github.com/apache/spark/commit/da859203e91a0bc90b017a1557bcf3646733982a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
maropu commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657154379 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 commented on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153936 @srowen @huaxingao Thank you for your response, the main issue resolved by this PR is `` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 closed pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 closed pull request #29043: URL: https://github.com/apache/spark/pull/29043 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 removed a comment on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 removed a comment on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153936 @srowen @huaxingao Thank you for your response, the main issue resolved by this PR is `` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 opened a new pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 opened a new pull request #29043:
URL: https://github.com/apache/spark/pull/29043
### What changes were proposed in this pull request?
1. Change timestamp to mysql to datetime in
`org.apache.spark.sql.jdbc.MySQLDialect#getJDBCType`
2. Add UT in `org.apache.spark.sql.test.SQLTestUtils#test` .
### Why are the changes needed?
Because write spark timestamp to mysql should has a '1000-01-01 00:00:00' to
'-12-31 23:59:59' range.
see https://dev.mysql.com/doc/refman/5.7/en/datetime.html
While the date type in mysql should be datetime rather than timestamp.
**Before this patch**, when we use timestamp data type in mysql by auto
created table：
`sql("select cast('-01-01 00:00:01' as
timestamp)").toDF("ts").write.mode("append").jdbc("jdbc:mysql://localhost:3306/test",
"ts_test3",prop)`
we will get an exception：
`com.mysql.jdbc.MysqlDataTruncation: Data truncation: Incorrect datetime
value: '-01-01 00:00:01' for column 'ts' at row`
### Does this PR introduce _any_ user-facing change?
Yes, **after this patch**, people could insert '1000-01-01 00:00:00' to
'-12-31 23:59:59' range timestamp to mysql by table auto created.
### How was this patch tested?
Unit test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #26674: [SPARK-30059][CORE]Stop AsyncEventQueue when interrupted in dispatch
github-actions[bot] commented on pull request #26674: URL: https://github.com/apache/spark/pull/26674#issuecomment-657153865 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 edited a comment on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 edited a comment on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657024230 @huaxingao Thank you for your detail response. Actually, `Timestamp` in spark has the range '0001-01-01T00:00:00.00Z, -12-31T23:59:59.99Z', see `org.apache.spark.sql.types.TimestampType`(8 bytes), and the relative date type seems should be `datetime` (8 bytes) in mysql, not the timestamp, only 4 bytes and have a smaller range '1970-01-01 00:00:01.00' to '2038-01-19 03:14:07.99'. In real test： We can not insert '-01-01 00:00:01' from spark into TIMESTAMP column of mysql currently because of out of range. but we can insert the former '-01-01 00:00:01'into DATETIME column of mysql for `datetime` contains the range of `timestamp` in mysql. See: https://dev.mysql.com/doc/refman/5.7/en/datetime.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 edited a comment on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 edited a comment on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153038 Hi @srowen , The zone info is according to local zone of mysql, it seems we should not consider it as timezone loss because we have considered it when write. The different between DATETIME and TIMESTAMP is zone index rather than loss it and it belongs to mysql design. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 edited a comment on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 edited a comment on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153038 @srowen The zone info is according to local zone of mysql, it seems we should not consider it as timezone loss because we have consider it when writting. The different between DATETIME and TIMESTAMP is zone index rather than loss it and it belongs to mysql design. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 edited a comment on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 edited a comment on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153038 @srowen The zone info is according to local zone of mysql, it seems we should not consider it as timezone loss because we have considered it when write. The different between DATETIME and TIMESTAMP is zone index rather than loss it and it belongs to mysql design. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
TJX2014 commented on pull request #29043: URL: https://github.com/apache/spark/pull/29043#issuecomment-657153038 @srowen The zone info is according to local zone of mysql, it seems we should not consider it as timezone loss. The different between DATETIME and TIMESTAMP is zone index rather than loss it and it belongs to mysql design. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657152582 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125698/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657152579 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657151458 **[Test build #125698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125698/testReport)** for PR 29069 at commit [`9067f21`](https://github.com/apache/spark/commit/9067f217c6707da54be56000d25ceda691b38eb6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657152579 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657152573 **[Test build #125698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125698/testReport)** for PR 29069 at commit [`9067f21`](https://github.com/apache/spark/commit/9067f217c6707da54be56000d25ceda691b38eb6). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class GetCatalogsOperationMock extends GetCatalogsOperation ` * `public class OperationManagerMock extends OperationManager ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657151510 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125691/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657151632 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657151632 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657151508 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use subclasses for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657151458 **[Test build #125698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125698/testReport)** for PR 29069 at commit [`9067f21`](https://github.com/apache/spark/commit/9067f217c6707da54be56000d25ceda691b38eb6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
AmplabJenkins commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657151508 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA commented on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657151270 **[Test build #125691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125691/testReport)** for PR 29072 at commit [`4976e6a`](https://github.com/apache/spark/commit/4976e6a2d8b762ba12768d870e527320f1f73463). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29072: [WIP] New GitHub Action PR Builder Health Check
SparkQA removed a comment on pull request #29072: URL: https://github.com/apache/spark/pull/29072#issuecomment-657131268 **[Test build #125691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125691/testReport)** for PR 29072 at commit [`4976e6a`](https://github.com/apache/spark/commit/4976e6a2d8b762ba12768d870e527320f1f73463). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150638 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125697/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150686 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
AmplabJenkins removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150634 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
SparkQA removed a comment on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150528 **[Test build #125697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125697/testReport)** for PR 29069 at commit [`7c668fe`](https://github.com/apache/spark/commit/7c668fe60ed37570465880fb8b778fb3a9dd5449). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] frankyin-factual commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans
frankyin-factual commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-657150642 @viirya @maropu Looks like the build env stabilizes. Can I get a retest? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
SparkQA commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150629 **[Test build #125697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125697/testReport)** for PR 29069 at commit [`7c668fe`](https://github.com/apache/spark/commit/7c668fe60ed37570465880fb8b778fb3a9dd5449). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `public class GetCatalogsOperationMock extends GetCatalogsOperation ` * `public class OperationManagerMock extends OperationManager ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29069: [SPARK-31831][SQL][TESTS] Use constructor for mock in HiveSessionImplSuite
AmplabJenkins commented on pull request #29069: URL: https://github.com/apache/spark/pull/29069#issuecomment-657150634 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
