Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/17120#discussion_r105080626
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -309,6 +315,10 @@ object FileStreamSource {
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/17120#discussion_r105080572
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -75,7 +77,7 @@ class FileStreamSource(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17202
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74232/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17202
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17202
**[Test build #74232 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74232/testReport)**
for PR 17202 at commit
[`10a949d`](https://github.com/apache/spark/commit/1
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/17220
Ok, I have modified the title.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabl
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17179
**[Test build #74235 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74235/testReport)**
for PR 17179 at commit
[`7b3e0a2`](https://github.com/apache/spark/commit/7b
Github user witgo commented on the issue:
https://github.com/apache/spark/pull/17139
@kayousterhout The test report has been updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/17220
Can you please fix the title like what other PR did.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/17113
@tgravescs , I just added a configuration to turn off this feature by
default.
Do you have any further comments on it?
---
If your project is set up for it, you can reply to this email a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17113
**[Test build #74234 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74234/testReport)**
for PR 17113 at commit
[`7ba0623`](https://github.com/apache/spark/commit/7b
Github user hustfxj commented on a diff in the pull request:
https://github.com/apache/spark/pull/17189#discussion_r105076371
--- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
---
@@ -577,13 +582,17 @@ private[deploy] class Worker(
val shouldCleanup
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17221
**[Test build #74233 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74233/testReport)**
for PR 17221 at commit
[`2c2c806`](https://github.com/apache/spark/commit/2c
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17202#discussion_r105075392
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -576,6 +576,8 @@ class Dataset[T] private[sql](
val parsedDelay =
GitHub user uncleGen opened a pull request:
https://github.com/apache/spark/pull/17221
[SPARK-19859][SS][Follow-up] The new watermark should override the old one.
## What changes were proposed in this pull request?
A follow up to SPARK-19859:
- extract the calculati
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17217
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17217
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74229/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17217
**[Test build #74229 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74229/testReport)**
for PR 17217 at commit
[`7d21ec3`](https://github.com/apache/spark/commit/7
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/17214
Thank you @gatorsmile @cloud-fan @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/17122
LGTM cc @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/17203
With cached KafkaConsumer, by default the largest connections per executor
would be 64, in your case 64 may not enough. Your fix looks not so solid and
cannot explain why shifting to uncached one
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17219
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74231/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17219
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17219
**[Test build #74231 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74231/testReport)**
for PR 17219 at commit
[`682eb1a`](https://github.com/apache/spark/commit/6
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/17220
In spark 1.4.1, you delete the parameter of 'hash'.I think it should be
deleted.In the spark website indicated in the documents, should not keep this
logic in the code.
*
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17202#discussion_r105069930
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -576,6 +576,11 @@ class Dataset[T] private[sql](
val parsedDelay =
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17216#discussion_r105069281
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -380,7 +382,20 @@ class StreamExecution(
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17214
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/17220
This was an exposed parameter, we cannot remove it - irrespective of the
duplication.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/17200#discussion_r105069175
--- Diff: python/pyspark/sql/types.py ---
@@ -189,7 +189,7 @@ def toInternal(self, dt):
if dt is not None:
seconds = (calendar.t
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17214
Thanks! Merging to master/2.1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled an
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16592
ping @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17220
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15363#discussion_r105068816
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -20,19 +20,340 @@ package org.apache.spark.sql.catalyst
GitHub user guoxiaolongzte opened a pull request:
https://github.com/apache/spark/pull/17220
remove tungsten-sort.Because it is not represent 'org.apache.spark.shâ¦
JIRA Issue: https://github.com/guoxiaolongzte/spark/tree/SPARK-19862
In SparkEnv.scala,remove tungsten-sort.B
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17202
**[Test build #74232 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74232/testReport)**
for PR 17202 at commit
[`10a949d`](https://github.com/apache/spark/commit/10
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/17202#discussion_r105005343
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -576,6 +576,11 @@ class Dataset[T] private[sql](
val parsedDelay =
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17214
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17202#discussion_r105067664
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -576,6 +576,11 @@ class Dataset[T] private[sql](
val parsedDelay =
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17214
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74228/
Test PASSed.
---
Github user lvdongr commented on the issue:
https://github.com/apache/spark/pull/17203
In our case,we deploy a streaming application whose data source are 20
topics with 30 partitions in kafka cluster(3 brokers). Then the amount of
connection with kafka is very large,up to a thousand,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17214
**[Test build #74228 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74228/testReport)**
for PR 17214 at commit
[`0dee6dd`](https://github.com/apache/spark/commit/0
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17138
@nsyca This algorithm supports bushy tree, I also had a test case for it.
About your second comment, yes you are right, we had a similar discussion
[here](https://github.com/apache/spark/pull/17138
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/16592
greate~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16944
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74225/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16944
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16944
**[Test build #74225 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74225/testReport)**
for PR 16944 at commit
[`56fbc7e`](https://github.com/apache/spark/commit/5
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17216
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17216
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74226/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17216
**[Test build #74226 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74226/testReport)**
for PR 17216 at commit
[`9ff4d29`](https://github.com/apache/spark/commit/9
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15363#discussion_r105066059
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -389,6 +389,18 @@ object SQLConf {
.booleanConf
.c
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17214
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the fe
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17213
cc @dgingrich who I guess the reporter of SPARK-19507 - what do you think
about this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as w
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16592
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16592
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74223/
Test PASSed.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17213
Could we deal with SPARK-19507 together if it looks easy to fix it up
together? Also, I think we should run `./dev/lint-python`. It seems some lines
does not comply pep8 here. As a bonus, we cou
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16592
**[Test build #74223 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74223/testReport)**
for PR 16592 at commit
[`9d4211c`](https://github.com/apache/spark/commit/9
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17216
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17216
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74224/
Test PASSed.
---
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17213#discussion_r105063798
--- Diff: python/pyspark/sql/types.py ---
@@ -1249,7 +1249,7 @@ def _infer_schema_type(obj, dataType):
}
-def _verify_type(obj, data
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17216
**[Test build #74224 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74224/testReport)**
for PR 17216 at commit
[`12f5fd3`](https://github.com/apache/spark/commit/1
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105062302
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/Trigger.scala ---
@@ -38,6 +38,26 @@ sealed trait Trigger
/**
* :: Experi
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105061613
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetCommitLog.scala
---
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apac
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105062818
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -377,17 +385,25 @@ class StreamExecution(
p
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105062498
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
---
@@ -284,6 +291,7 @@ class StreamExecution(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105061689
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetCommitLog.scala
---
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apac
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/17219#discussion_r105062343
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetCommitLog.scala
---
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apac
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17219
**[Test build #74231 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74231/testReport)**
for PR 17219 at commit
[`682eb1a`](https://github.com/apache/spark/commit/68
GitHub user tcondie opened a pull request:
https://github.com/apache/spark/pull/17219
[SPARK-19876][SS][WIP] OneTime Trigger Executor
## What changes were proposed in this pull request?
An additional trigger and trigger executor that will execute a single
trigger only. One
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17218
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17218
**[Test build #74230 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74230/testReport)**
for PR 17218 at commit
[`3b10a30`](https://github.com/apache/spark/commit/3
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17218
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74230/
Test PASSed.
---
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105059417
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -422,7 +422,7 @@ object FunctionRegistry {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105059294
--- Diff:
sql/core/src/test/resources/sql-tests/results/json-functions.sql.out ---
@@ -32,32 +34,40 @@ Usage: to_json(expr[, options]) - Returns a json
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105059270
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -422,7 +422,7 @@ object FunctionRegistry {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17218
**[Test build #74230 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74230/testReport)**
for PR 17218 at commit
[`3b10a30`](https://github.com/apache/spark/commit/3b
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17110
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74227/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17110
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17110
**[Test build #74227 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74227/testReport)**
for PR 17110 at commit
[`19fa02a`](https://github.com/apache/spark/commit/1
GitHub user zero323 opened a pull request:
https://github.com/apache/spark/pull/17218
[SPARK-19281][WIP][PYTHON][ML] spark.ml Python API for FPGrowth
## What changes were proposed in this pull request?
- Add `HasSupport` and `HasConfidence` to `pyspark.ml.param.shared`.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105056816
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -422,7 +422,7 @@ object FunctionRegistry {
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105055922
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala ---
@@ -220,4 +242,5 @@ class JsonFunctionsSuite extends QueryTest with
Shar
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105055860
--- Diff:
sql/core/src/test/resources/sql-tests/results/json-functions.sql.out ---
@@ -32,32 +34,40 @@ Usage: to_json(expr[, options]) - Returns a json strin
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105054224
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala ---
@@ -220,4 +242,5 @@ class JsonFunctionsSuite extends QueryTest with
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105052901
--- Diff:
sql/core/src/test/resources/sql-tests/results/json-functions.sql.out ---
@@ -32,32 +34,40 @@ Usage: to_json(expr[, options]) - Returns a json
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17192#discussion_r105052696
--- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R ---
@@ -1339,6 +1339,11 @@ test_that("column functions", {
expect_equal(collect(select(df, bro
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17217
**[Test build #74229 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74229/testReport)**
for PR 17217 at commit
[`7d21ec3`](https://github.com/apache/spark/commit/7d
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/17216#discussion_r105050897
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeq.scala
---
@@ -71,7 +71,10 @@ object OffsetSeq {
* @param batchTi
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/17217
[SPARK-19874][BUILD]Hide API docs for org.apache.spark.sql.internal
## What changes were proposed in this pull request?
The API docs should not include the "org.apache.spark.sql.internal" p
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17214
**[Test build #74228 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74228/testReport)**
for PR 17214 at commit
[`0dee6dd`](https://github.com/apache/spark/commit/0d
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/16002
@yanboliang Sorry for missing earlier discussion. I'm OK with declaring
defeat here, though I still disagree about using exceptions. I agree that
passing an obscure error code up is not ideal.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/17120#discussion_r105047840
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -75,7 +77,7 @@ class FileStreamSource(
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/17120#discussion_r105048802
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -309,6 +315,10 @@ object FileStreamSource {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17216
**[Test build #74226 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74226/testReport)**
for PR 17216 at commit
[`9ff4d29`](https://github.com/apache/spark/commit/9f
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17110
**[Test build #74227 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74227/testReport)**
for PR 17110 at commit
[`19fa02a`](https://github.com/apache/spark/commit/19
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/17110
I just reversed my opinion about a shared "Statistics" object. See
https://github.com/apache/spark/pull/17108#issuecomment-285200613 for details.
I pushed an update per your review @imati
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/17178#discussion_r105049248
--- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R ---
@@ -1342,28 +1342,52 @@ test_that("column functions", {
df <- read.json(mapTypeJsonPath)
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/17108
Given further thought, I'd prefer we stick to the API specified in the
design doc, with a Correlations object instead of a generic Statistics object.
In the future, we may want optional Params su
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/17138
You are right. The plans generated at n-join level comes from the join of
the plans in (n-1)-join level as well as (n-2)-join level and so on. So it
should be able to generate {A,B} join {C,D} plan.
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/17214#discussion_r105047697
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -46,9 +46,13 @@ class QueryExecution(val sparkSession: Spar
101 - 200 of 533 matches
Mail list logo
