[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95718514 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,96 @@ object CaseKeyWhen {

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95718502 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,555 @@ +/* +

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95718465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,96 @@ object CaseKeyWhen {

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95718174 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelationSuite.scala --- @@ -31,7 +31,7 @@ class

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95717900 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,555 @@ +/* +

[GitHub] spark issue #16550: [SPARK-19178][SQL] convert string of large numbers to in...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16550 **[Test build #71241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71241/testReport)** for PR 16550 at commit

[GitHub] spark issue #16553: [SPARK-9435][SQL] Reuse function in Java UDF to correctl...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71235/ Test PASSed. ---

[GitHub] spark issue #16553: [SPARK-9435][SQL] Reuse function in Java UDF to correctl...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16553 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16553: [SPARK-9435][SQL] Reuse function in Java UDF to correctl...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16553 **[Test build #71235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71235/testReport)** for PR 16553 at commit

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95717498 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala --- @@ -609,6 +609,98 @@ class ParquetMetastoreSuite extends

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95717399 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala --- @@ -609,6 +609,98 @@ class ParquetMetastoreSuite extends

[GitHub] spark pull request #16550: [SPARK-19178][SQL] convert string of large number...

Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16550#discussion_r95717360 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -835,6 +835,185 @@ public UTF8String translate(Map

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95717322 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala --- @@ -609,6 +609,98 @@ class ParquetMetastoreSuite extends

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95717298 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala --- @@ -52,3 +56,12 @@ object

[GitHub] spark pull request #16550: [SPARK-19178][SQL] convert string of large number...

Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16550#discussion_r95717275 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -835,6 +835,185 @@ public UTF8String translate(Map

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95717157 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -392,7 +392,9 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to internal c...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16554 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to internal c...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16554 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71233/ Test PASSed. ---

[GitHub] spark pull request #16308: [SPARK-18936][SQL] Infrastructure for session loc...

Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/16308#discussion_r95716928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -111,7 +112,8 @@ case class CatalogTablePartition(

[GitHub] spark issue #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to internal c...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16554 **[Test build #71233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71233/testReport)** for PR 16554 at commit

[GitHub] spark pull request #16549: [SPARK-19151][SQL]DataFrameWriter.saveAsTable sup...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16549#discussion_r95716715 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1340,8 +1348,8 @@ class HiveDDLSuite val e3 =

[GitHub] spark pull request #16549: [SPARK-19151][SQL]DataFrameWriter.saveAsTable sup...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16549#discussion_r95716599 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1314,6 +1314,14 @@ class HiveDDLSuite

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95716353 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,555 @@ +/* +

[GitHub] spark pull request #16549: [SPARK-19151][SQL]DataFrameWriter.saveAsTable sup...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16549#discussion_r95716225 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -383,14 +383,19 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/15837 ping @cloud-fan Since 2.1 is released now, can you have time to review this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/15837 @kiszk I am hesitant to set a threshold for that. One reason is it will increase the complexity of subexpression elimination implementation and the benefit can be low or uncertain. Another reason is

[GitHub] spark pull request #16549: [SPARK-19151][SQL]DataFrameWriter.saveAsTable sup...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16549#discussion_r95716115 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/MetastoreRelation.scala --- @@ -49,8 +49,7 @@ private[hive] case class MetastoreRelation(

[GitHub] spark issue #16341: [SQL] [WIP] Switch internal catalog types to use URI ins...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16341 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71236/ Test FAILed. ---

[GitHub] spark issue #16341: [SQL] [WIP] Switch internal catalog types to use URI ins...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16341 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16341: [SQL] [WIP] Switch internal catalog types to use URI ins...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16341 **[Test build #71236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71236/testReport)** for PR 16341 at commit

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95715947 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -175,7 +178,7 @@ case class

[GitHub] spark issue #16552: [WIP][SPARK-19152][SQL]DataFrameWriter.saveAsTable suppo...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16552 **[Test build #71239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71239/testReport)** for PR 16552 at commit

[GitHub] spark pull request #16481: [SPARK-19092] [SQL] Save() API of DataFrameWriter...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16481#discussion_r95715805 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -413,17 +413,22 @@ case class DataSource(

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15837 **[Test build #71240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71240/testReport)** for PR 15837 at commit

[GitHub] spark issue #16556: [SPARK-19184][MLlib] Improve numerical stability for met...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16556 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16556: [SPARK-19184][MLlib] Improve numerical stability ...

GitHub user hl475 opened a pull request: https://github.com/apache/spark/pull/16556 [SPARK-19184][MLlib] Improve numerical stability for method tallSkinnyQR. ## What changes were proposed in this pull request? In method tallSkinnyQR, the final Q is calculated by A * inv(R).

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95715663 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala --- @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16547: [SPARK-19168][Structured Streaming] Improvement: filter ...

Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16547 @zsxwing thanks for the comments. I'll update this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16541: [SPARK-19088][SQL] Optimize sequence type deseria...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16541#discussion_r95715108 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -589,6 +590,171 @@ case class MapObjects

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95714934 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala --- @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16474: [SPARK-19082][SQL] Make ignoreCorruptFiles work for Parq...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/16474 ping @rxin I do address the previous comments. Can you review again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14847: [SPARK-17254][SQL] Add StopAfter physical plan fo...

Github user viirya closed the pull request at: https://github.com/apache/spark/pull/14847 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/14847 Close this for now. Re-open if anyone is interested in this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95713575 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -175,7 +178,7 @@ case class

[GitHub] spark issue #16528: [SPARK-19148][SQL] do not expose the external table conc...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16528 **[Test build #71238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71238/testReport)** for PR 16528 at commit

[GitHub] spark issue #16528: [SPARK-19148][SQL] do not expose the external table conc...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16528 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16528: [SPARK-19148][SQL] do not expose the external table conc...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16528 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71238/ Test FAILed. ---

[GitHub] spark issue #16528: [SPARK-19148][SQL] do not expose the external table conc...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16528 **[Test build #71238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71238/testReport)** for PR 16528 at commit

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16233: [SPARK-18801][SQL] Support resolve a nested view

Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/16233#discussion_r95711504 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -50,6 +50,36 @@ object SimpleAnalyzer extends

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71231/ Test PASSed. ---

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #71231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71231/testReport)** for PR 14702 at commit

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95710436 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark issue #16502: Branch 2.1

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16502 Hi @bupt2012, could you just click the "Close pull request" button below? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14451: [SPARK-16848][SQL] Check schema validation for user-spec...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14451 I just simply rebased this. Could this be merged by any change after the build? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14451: [SPARK-16848][SQL] Check schema validation for user-spec...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14451 **[Test build #71237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71237/testReport)** for PR 14451 at commit

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95709024 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95707722 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95707748 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark issue #16341: [SQL] [WIP] Switch internal catalog types to use URI ins...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16341 **[Test build #71236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71236/testReport)** for PR 16341 at commit

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95707644 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16441 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71232/ Test PASSed. ---

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16441 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16441 **[Test build #71232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71232/testReport)** for PR 16441 at commit

[GitHub] spark issue #16553: [SPARK-9435][SQL] Reuse function in Java UDF to correctl...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16553 **[Test build #71235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71235/testReport)** for PR 16553 at commit

[GitHub] spark pull request #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cach...

Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16500#discussion_r95705420 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -392,7 +392,9 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request #16553: [SPARK-9435][SQL] Reuse function in Java UDF to c...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16553#discussion_r95705218 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala --- @@ -109,9 +109,10 @@ class UDFRegistration private[sql]

[GitHub] spark issue #16500: [SPARK-19120] [SPARK-19121] Refresh Metadata Cache After...

Github user ericl commented on the issue: https://github.com/apache/spark/pull/16500 I guess the only purpose of the cache now is to associate file-status caches with specific table names. If we removed that, then tables would have to find their file-status cache by path, or we could

[GitHub] spark issue #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to internal c...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16554 **[Test build #71234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71234/testReport)** for PR 16554 at commit

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95704748 --- Diff: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala --- @@ -112,6 +113,15 @@ abstract class FileCommitProtocol { *

[GitHub] spark issue #16555: [SPARK-19180][SQL] the offset of short should be 4 in Of...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16555 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95702468 --- Diff: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala --- @@ -112,6 +113,15 @@ abstract class FileCommitProtocol { * just

[GitHub] spark issue #16555: [SPARK-19180][SQL] the offset of short should be 4 in Of...

Github user yucai commented on the issue: https://github.com/apache/spark/pull/16555 @chenghao-intel @adrian-wang kindly help review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to internal c...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16554 **[Test build #71233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71233/testReport)** for PR 16554 at commit

[GitHub] spark pull request #16555: [SPARK-19180][SQL] the offset of short should be ...

GitHub user yucai opened a pull request: https://github.com/apache/spark/pull/16555 [SPARK-19180][SQL] the offset of short should be 4 in OffHeapColumn ## What changes were proposed in this pull request? the offset of short is 4 in OffHeapColumnVector's putShorts, but

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16554#discussion_r95702064 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala --- @@ -462,16 +463,19 @@ class ParquetIOSuite

[GitHub] spark pull request #16554: [SPARK-19183] [SQL] Add deleteWithJob hook to int...

GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/16554 [SPARK-19183] [SQL] Add deleteWithJob hook to internal commit protocol API ## What changes were proposed in this pull request? Currently in SQL we implement overwrites by calling fs.delete()

[GitHub] spark issue #16429: [SPARK-19019][PYTHON] Fix hijacked `collections.namedtup...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16429 gentle ping @joshrosen and @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16441 @sethah thank you for doing the review! I've updated based on your latest comments. @jkbradley would you be able to take a look as well? --- If your project is set up for it, you can reply

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16441 **[Test build #71232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71232/testReport)** for PR 16441 at commit

[GitHub] spark pull request #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict ...

Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16441#discussion_r95697903 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -275,18 +316,30 @@ class GBTClassificationModel

[GitHub] spark pull request #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict ...

Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16441#discussion_r95697512 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -193,6 +199,8 @@ object GBTClassifier extends

[GitHub] spark pull request #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict ...

Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16441#discussion_r95697253 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -159,14 +158,21 @@ class GBTClassifier @Since("1.4.0") (

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95695560 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,555 @@ +/* +

[GitHub] spark issue #16387: [SPARK-18986][Core] ExternalAppendOnlyMap shouldn't fail...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/16387 @vanzin ok, thanks for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #71231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71231/testReport)** for PR 14702 at commit

[GitHub] spark pull request #16541: [SPARK-19088][SQL] Optimize sequence type deseria...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16541#discussion_r95694769 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -589,6 +590,171 @@ case class MapObjects

[GitHub] spark pull request #16553: [SPARK-9435][SQL] Reuse function in Java UDF to c...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16553#discussion_r95694410 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala --- @@ -488,219 +488,241 @@ class UDFRegistration private[sql]

[GitHub] spark pull request #16551: [SPARK-19132] [SQL] Add test cases for row size e...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16551 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16551: [SPARK-19132] [SQL] Add test cases for row size estimati...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/16551 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95691485 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95689935 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -245,6 +245,16 @@ class

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95691071 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -517,6 +518,32 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95690174 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -517,6 +518,32 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95690660 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -517,6 +518,32 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r95690239 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -0,0 +1,173 @@ +/* + *

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95688659 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -52,7 +55,43 @@ private[spark] class TaskDescription(

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95687698 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -52,7 +55,36 @@ private[spark] class TaskDescription(

[GitHub] spark pull request #16544: [SPARK-19149][SQL] Follow-up: simplify cache impl...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16544 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16544: [SPARK-19149][SQL] Follow-up: simplify cache implementat...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/16544 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

<    1   2   3   4   5   >