[GitHub] spark issue #22599: [SPARK-25581][SQL] Rename method `benchmark` as `runBenc...

Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22599 Thanks @gengliangwang I agree with you. It’s a good change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22603: SPARK-25062: clean up BlockLocations in InMemoryFileInde...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22603 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark issue #22603: SPARK-25062: clean up BlockLocations in InMemoryFileInde...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22603 **[Test build #96856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96856/testReport)** for PR 22603 at commit [`45f0c81`](https://github.com/apache/spark/commit/45

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221913364 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Lice

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221913801 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Lice

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22609 Hi @mridulm , Can't we limit the task information by setting 'spark.ui.retainedTasks' lesser, to avoid OOM? correct me if I am wrong. --- ---

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22605 **[Test build #96848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96848/testReport)** for PR 22605 at commit [`8e9a0b1`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96848/ Test PASSed. ---

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user mridulm commented on the issue: https://github.com/apache/spark/pull/22609 This is a regression introduced in 2.3; unfortunately we did not notice it until now. On Tue, Oct 2, 2018 at 4:56 AM Shahid wrote: > Hi @mridulm

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22609 **[Test build #96846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96846/testReport)** for PR 22609 at commit [`c2f6b4a`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22609 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96846/ Test FAILed. ---

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22609 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22607: [SPARK-24530][followup] run Sphinx with python 3 in dock...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22607 **[Test build #96844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96844/testReport)** for PR 22607 at commit [`e76463d`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #22607: [SPARK-24530][followup] run Sphinx with python 3 in dock...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22607 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96844/ Test PASSed. ---

[GitHub] spark issue #22607: [SPARK-24530][followup] run Sphinx with python 3 in dock...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22607 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22602 **[Test build #96852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96852/testReport)** for PR 22602 at commit [`6b84b41`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96852/ Test PASSed. ---

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221934230 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Licen

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

Github user peter-toth commented on the issue: https://github.com/apache/spark/pull/22318 @cloud-fan could you please help me with this PR and take it one step forward? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3629/

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22602 **[Test build #96857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96857/testReport)** for PR 22602 at commit [`64f4ed0`](https://github.com/apache/spark/commit/64

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user samdvr commented on the issue: https://github.com/apache/spark/pull/22596 @dongjoon-hyun merged, thank you, did not know how to skip lint rule there. --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22596 **[Test build #96858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96858/testReport)** for PR 22596 at commit [`07156d4`](https://github.com/apache/spark/commit/07

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22609 Hi @mridulm , it seems that the changes will affect the metrics in `SparkStatusTracker`, e.g. number of active tasks in `getExecutorInfos` or `getStageInfo` . Maybe we should

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/22524 Thanks @mgaido91. Resetting a flag in blocking operators is feasible solution I think for now to solve this issue that limit operator consumes all inputs. For current blocking operators, we should ad

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22606 **[Test build #96847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96847/testReport)** for PR 22606 at commit [`a0467da`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221955172 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Lice

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22606 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22606 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96847/ Test PASSed. ---

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221960064 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Licen

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #10989: [SPARK-12798] [SQL] generated BroadcastHashJoin

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10989#discussion_r221961271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala --- @@ -117,6 +120,87 @@ case class BroadcastHashJoin(

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3630/

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22602 **[Test build #96859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96859/testReport)** for PR 22602 at commit [`72b7c5c`](https://github.com/apache/spark/commit/72

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22602 LGTM, pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #10989: [SPARK-12798] [SQL] generated BroadcastHashJoin

Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/10989#discussion_r221966919 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala --- @@ -117,6 +120,87 @@ case class BroadcastHashJoin(

[GitHub] spark pull request #10989: [SPARK-12798] [SQL] generated BroadcastHashJoin

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10989#discussion_r221972056 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala --- @@ -117,6 +120,87 @@ case class BroadcastHashJoin(

[GitHub] spark issue #21816: [SPARK-24794][CORE] Driver launched through rest should ...

Github user bsikander commented on the issue: https://github.com/apache/spark/pull/21816 >> So is there no behavior change if your master specifies only one master? If user specifies only 1 master then there is no behavioral change. --- ---

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22605 Thanks, @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22603: [SPARK-25062][SQL] Clean up BlockLocations in InMemoryFi...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22603 **[Test build #96856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96856/testReport)** for PR 22603 at commit [`45f0c81`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #22603: [SPARK-25062][SQL] Clean up BlockLocations in InMemoryFi...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22603 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96856/ Test PASSed. ---

[GitHub] spark issue #22603: [SPARK-25062][SQL] Clean up BlockLocations in InMemoryFi...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22603 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22601 Thank you, @shahidki31 . Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...

Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22601 Thanks a lot @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark pull request #22601: [SPARK-25583][DOC]Add history-server related conf...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22601 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22601 `spark.history.fs.endEventReparseChunkSize` and `spark.history.fs.inProgressOptimization.enabled` exist since `branch-2.4`. So, this PR lands at master/2.4. If you want to fix some in `branch-

[GitHub] spark pull request #21916: [SPARK-24958][WIP] Report executors' process tree...

Github user rezasafi closed the pull request at: https://github.com/apache/spark/pull/21916 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21916: [SPARK-24958][WIP] Report executors' process tree total ...

Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/21916 I will close this and open the actual Pr shortly. Thank you everyone for the great reviews. --- - To unsubscribe, e-mail: revi

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

GitHub user rezasafi opened a pull request: https://github.com/apache/spark/pull/22612 [SPARK-24958] Add executors' process tree total memory information to heartbeat signals. Spark executors' process tree total memory information can be really useful. Currently such information ar

[GitHub] spark pull request #22613: [SPARK-25583][DOC][BRANCH-2.3]Add history-server ...

GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/22613 [SPARK-25583][DOC][BRANCH-2.3]Add history-server related configuration in the documentation. ## What changes were proposed in this pull request? This is a follow up PR for the PR, https://g

[GitHub] spark issue #22613: [SPARK-25583][DOC][BRANCH-2.3]Add history-server related...

Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/22613 cc @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22612 Previously I opened a WIP PR [here](https://github.com/apache/spark/pull/21916) and [here](https://github.com/rezasafi/spark/pull/1). I have applied all the comments that I received there to this

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22612 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22613: [SPARK-25583][DOC][BRANCH-2.3]Add history-server related...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22613 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22613: [SPARK-25583][DOC][BRANCH-2.3]Add history-server related...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22613 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22612 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/22318 For a query similar to the one in the PR description: `a.join(b, a("id") === b("id"), "inner").join(c, a("id") === b("id"), "inner"` It is interpreted as a cartesian products now. Bu

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22612 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22613: [SPARK-25583][DOC][BRANCH-2.3]Add history-server related...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22613 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3631/

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22524 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22524: [SPARK-25497][SQL] Limit operation within whole stage co...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22524 **[Test build #96860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96860/testReport)** for PR 22524 at commit [`1b2ab61`](https://github.com/apache/spark/commit/1b

[GitHub] spark issue #22599: [SPARK-25581][SQL] Rename method `benchmark` as `runBenc...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22599 **[Test build #96855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96855/testReport)** for PR 22599 at commit [`1c15c25`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #22599: [SPARK-25581][SQL] Rename method `benchmark` as `runBenc...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22599: [SPARK-25581][SQL] Rename method `benchmark` as `runBenc...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96855/ Test PASSed. ---

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22610: [WIP][SPARK-25461][PySpark][SQL] Print warning wh...

Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22610#discussion_r222007837 --- Diff: python/pyspark/worker.py --- @@ -84,13 +84,36 @@ def wrap_scalar_pandas_udf(f, return_type): arrow_return_type = to_arrow_type(return_

[GitHub] spark pull request #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22606 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22610: [WIP][SPARK-25461][PySpark][SQL] Print warning when retu...

Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22610 Thanks for looking into this @viirya ! I agree that there are lots of cases where casting to another type is intentional and works fine, so this isn't a bug. The only other idea I have is to pr

[GitHub] spark pull request #22610: [WIP][SPARK-25461][PySpark][SQL] Print warning wh...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22610#discussion_r222015287 --- Diff: python/pyspark/worker.py --- @@ -84,13 +84,36 @@ def wrap_scalar_pandas_udf(f, return_type): arrow_return_type = to_arrow_type(return_type)

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22596 **[Test build #96858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96858/testReport)** for PR 22596 at commit [`07156d4`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96858/ Test PASSed. ---

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22602 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22612 @squito --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22602 **[Test build #96857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96857/testReport)** for PR 22602 at commit [`64f4ed0`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96857/ Test PASSed. ---

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r222032210 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala --- @@ -0,0 +1,48 @@ +/* + *

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22606 Great, @gatorsmile ! Could you update Spark website properly, too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

Github user peter-toth commented on the issue: https://github.com/apache/spark/pull/22318 Thanks @viirya, your analysis is correct. Unfortunately an attribute doesn't have a reference to its dataset so I don't think this scenario can be solved easily. I believe the good soluti

[GitHub] spark issue #22609: [SPARK-25594] [Core] Avoid maintaining task information ...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22609 @gengliangwang is right; there are still things that need to be done when the UI is disabled; job, stage and executor info are all still exposed even if the UI is disabled, and the task even callback

[GitHub] spark pull request #22599: [SPARK-25581][SQL] Rename method `benchmark` as `...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22599 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22607: [SPARK-24530][followup] run Sphinx with python 3 in dock...

Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22607 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark pull request #22607: [SPARK-24530][followup] run Sphinx with python 3 ...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22607 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22596 BTW, I chose the most official one, `Sam Davarnia `. But, it seems that your GitHub is not happy with that. Next time, you had better be consistent in your commit message. ``` Lead-aut

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22596 Since it's merged into `branch-2.2`, could you close this PR? GitHub only closes the PRs against `master` branch. :) --- -

[GitHub] spark issue #22606: [SPARK-25592] Setting version to 3.0.0-SNAPSHOT

Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22606 @dongjoon-hyun Could you help me fix the website? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addit

[GitHub] spark pull request #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user samdvr closed the pull request at: https://github.com/apache/spark/pull/22596 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22596: [SPARK-25576][BUILD][BRANCH-2.2] Fix lint failure

Github user samdvr commented on the issue: https://github.com/apache/spark/pull/22596 Thanks will do! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22605 Rebased to `master` due to https://github.com/apache/spark/pull/22599. --- - To unsubscribe, e-mail: reviews-unsubscr...@sp

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22605 **[Test build #96861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96861/testReport)** for PR 22605 at commit [`fb116f0`](https://github.com/apache/spark/commit/fb

[GitHub] spark issue #22500: [SPARK-25488][TEST] Refactor MiscBenchmark to use main m...

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22500 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22491: [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to...

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22491 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22503: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV da...

Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22503 @HyukjinKwon is this ready to be merged in, or is there more feedback to be addressed? --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark issue #22488: [SPARK-25479][TEST] Refactor DatasetBenchmark to use mai...

Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22488 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22605: [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3632/

<    1   2   3   4   5   >