[jira] [Resolved] (SPARK-20980) Rename the option `wholeFile` to `multiLine` for JSON and CSV

[ https://issues.apache.org/jira/browse/SPARK-20980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20980. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18202 [https://githu

[jira] [Resolved] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18016. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18075 [https://githu

[jira] [Assigned] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18016: --- Assignee: Aleksander Eskilson > Code Generation: Constant Pool Past Limit for Wide/Nested Da

[jira] [Assigned] (SPARK-19900) [Standalone] Master registers application again when driver relaunched

[ https://issues.apache.org/jira/browse/SPARK-19900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19900: --- Assignee: Li Yichao > [Standalone] Master registers application again when driver relaunched

[jira] [Resolved] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16251. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 2.0.3 I

[jira] [Resolved] (SPARK-20200) Flaky Test: org.apache.spark.rdd.LocalCheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-20200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20200. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 2.0.3 I

[jira] [Assigned] (SPARK-16251) LocalCheckpointSuite's - missing checkpoint block fails with informative message is flaky.

[ https://issues.apache.org/jira/browse/SPARK-16251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-16251: --- Assignee: Jiang Xingbo > LocalCheckpointSuite's - missing checkpoint block fails with inform

[jira] [Assigned] (SPARK-20200) Flaky Test: org.apache.spark.rdd.LocalCheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-20200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20200: --- Assignee: Jiang Xingbo > Flaky Test: org.apache.spark.rdd.LocalCheckpointSuite > ---

[jira] [Resolved] (SPARK-21112) ALTER TABLE SET TBLPROPERTIES should not overwrite COMMENT

[ https://issues.apache.org/jira/browse/SPARK-21112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21112. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18318 [https://githu

[jira] [Resolved] (SPARK-21072) `TreeNode.mapChildren` should only apply to the children node.

[ https://issues.apache.org/jira/browse/SPARK-21072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21072. - Resolution: Fixed Assignee: coneyliu Fix Version/s: 2.2.0 2.1.2

[jira] [Resolved] (SPARK-21114) Test failure in Spark 2.1 due to name mismatch

[ https://issues.apache.org/jira/browse/SPARK-21114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21114. - Resolution: Fixed Fix Version/s: 2.1.2 > Test failure in Spark 2.1 due to name mismatch >

[jira] [Created] (SPARK-21119) unset table properties should keep the table comment

Wenchen Fan created SPARK-21119: --- Summary: unset table properties should keep the table comment Key: SPARK-21119 URL: https://issues.apache.org/jira/browse/SPARK-21119 Project: Spark Issue Type

[jira] [Resolved] (SPARK-20994) Alleviate memory pressure in StreamManager

[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20994. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18231 [https://githu

[jira] [Assigned] (SPARK-20994) Alleviate memory pressure in StreamManager

[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20994: --- Assignee: jin xing > Alleviate memory pressure in StreamManager > --

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052057#comment-16052057 ] Wenchen Fan commented on SPARK-18016: - cc [~aeskilson] do you wanna send a new PR to

[jira] [Resolved] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21090. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18296 [https://githu

[jira] [Assigned] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21090: --- Assignee: liuxian > Optimize the unified memory manager code >

[jira] [Updated] (SPARK-21090) Optimize the unified memory manager code

[ https://issues.apache.org/jira/browse/SPARK-21090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21090: Fix Version/s: (was: 2.3.0) 2.2.0 > Optimize the unified memory manager cod

[jira] [Resolved] (SPARK-21132) DISTINCT modifier of function arguments should not be silently ignored

[ https://issues.apache.org/jira/browse/SPARK-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21132. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18340 [https://githu

[jira] [Resolved] (SPARK-21133) HighlyCompressedMapStatus#writeExternal throws NPE

[ https://issues.apache.org/jira/browse/SPARK-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21133. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18343 [https://githu

[jira] [Assigned] (SPARK-21133) HighlyCompressedMapStatus#writeExternal throws NPE

[ https://issues.apache.org/jira/browse/SPARK-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21133: --- Assignee: Yuming Wang > HighlyCompressedMapStatus#writeExternal throws NPE > ---

[jira] [Resolved] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20989. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18290 [https://githu

[jira] [Assigned] (SPARK-20989) Fail to start multiple workers on one host if external shuffle service is enabled in standalone mode

[ https://issues.apache.org/jira/browse/SPARK-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20989: --- Assignee: Jiang Xingbo > Fail to start multiple workers on one host if external shuffle serv

[jira] [Resolved] (SPARK-20640) Make rpc timeout and retry for shuffle registration configurable

[ https://issues.apache.org/jira/browse/SPARK-20640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20640. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18092 [https://githu

[jira] [Assigned] (SPARK-20640) Make rpc timeout and retry for shuffle registration configurable

[ https://issues.apache.org/jira/browse/SPARK-20640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20640: --- Assignee: Li Yichao > Make rpc timeout and retry for shuffle registration configurable > ---

[jira] [Created] (SPARK-21163) DataFrame.toPandas should respect the data type

Wenchen Fan created SPARK-21163: --- Summary: DataFrame.toPandas should respect the data type Key: SPARK-21163 URL: https://issues.apache.org/jira/browse/SPARK-21163 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18016: Fix Version/s: (was: 2.3.0) 2.2.0 2.1.2 > Code Generation

[jira] [Resolved] (SPARK-21163) DataFrame.toPandas should respect the data type

[ https://issues.apache.org/jira/browse/SPARK-21163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21163. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18378 [https://githu

[jira] [Resolved] (SPARK-20832) Standalone master should explicitly inform drivers of worker deaths and invalidate external shuffle service outputs

[ https://issues.apache.org/jira/browse/SPARK-20832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20832. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18362 [https://githu

[jira] [Assigned] (SPARK-20832) Standalone master should explicitly inform drivers of worker deaths and invalidate external shuffle service outputs

[ https://issues.apache.org/jira/browse/SPARK-20832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20832: --- Assignee: Jiang Xingbo > Standalone master should explicitly inform drivers of worker deaths

[jira] [Resolved] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-13534. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 15821 [https://githu

[jira] [Assigned] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-13534: --- Assignee: Bryan Cutler > Implement Apache Arrow serializer for Spark DataFrame for use in >

[jira] [Assigned] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20923: --- Assignee: Thomas Graves > TaskMetrics._updatedBlockStatuses uses a lot of memory > -

[jira] [Resolved] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20923. - Resolution: Fixed Fix Version/s: 2.3.0 > TaskMetrics._updatedBlockStatuses uses a lot of m

[jira] [Updated] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20923: Labels: releasenotes (was: ) > TaskMetrics._updatedBlockStatuses uses a lot of memory > --

[jira] [Commented] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060283#comment-16060283 ] Wenchen Fan commented on SPARK-20923: - This patch changes the public behavior and we

[jira] [Resolved] (SPARK-21174) Validate sampling fraction in logical operator level

[ https://issues.apache.org/jira/browse/SPARK-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21174. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18387 [https://githu

[jira] [Assigned] (SPARK-21174) Validate sampling fraction in logical operator level

[ https://issues.apache.org/jira/browse/SPARK-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21174: --- Assignee: Gengliang Wang > Validate sampling fraction in logical operator level > --

[jira] [Assigned] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21047: --- Assignee: jin xing > Add test suites for complicated cases in ColumnarBatchSuite > -

[jira] [Resolved] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21047. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18327 [https://githu

[jira] [Resolved] (SPARK-21165) Fail to write into partitioned hive table due to attribute reference not working with cast on partition column

[ https://issues.apache.org/jira/browse/SPARK-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21165. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18386 [https://githu

[jira] [Assigned] (SPARK-21115) If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

[ https://issues.apache.org/jira/browse/SPARK-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21115: --- Assignee: eaton > If the cores left is less than the coresPerExecutor,the cores left will no

[jira] [Resolved] (SPARK-21115) If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

[ https://issues.apache.org/jira/browse/SPARK-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21115. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18322 [https://githu

[jira] [Assigned] (SPARK-21193) Specify Pandas version in setup.py

[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21193: --- Assignee: Hyukjin Kwon > Specify Pandas version in setup.py > --

[jira] [Resolved] (SPARK-21193) Specify Pandas version in setup.py

[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21193. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18403 [https://githu

[jira] [Assigned] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21159: --- Assignee: Marcelo Vanzin > Cluster mode, driver throws connection refused exception submitte

[jira] [Resolved] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21159. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18397 [https://githu

[jira] [Updated] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21159: Fix Version/s: (was: 2.3.0) 2.2.0 2.1.2 > Cluster mode, d

[jira] [Resolved] (SPARK-21203) Wrong results of insertion of Array of Struct

[ https://issues.apache.org/jira/browse/SPARK-21203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21203. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.2 > Wrong results of inserti

[jira] [Updated] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18016: Fix Version/s: (was: 2.1.2) > Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1606#comment-1606 ] Wenchen Fan commented on SPARK-18016: - ok reverted > Code Generation: Constant Pool

[jira] [Assigned] (SPARK-21196) Split codegen info of query plan into sequence

[ https://issues.apache.org/jira/browse/SPARK-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21196: --- Assignee: Gengliang Wang > Split codegen info of query plan into sequence >

[jira] [Resolved] (SPARK-21196) Split codegen info of query plan into sequence

[ https://issues.apache.org/jira/browse/SPARK-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21196. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18409 [https://githu

[jira] [Created] (SPARK-21229) remove QueryPlan.preCanonicalized

Wenchen Fan created SPARK-21229: --- Summary: remove QueryPlan.preCanonicalized Key: SPARK-21229 URL: https://issues.apache.org/jira/browse/SPARK-21229 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19104. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18418 [https://githu

[jira] [Assigned] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19104: --- Assignee: Liang-Chi Hsieh > CompileException with Map and Case Class in Spark 2.1.0 > -

[jira] [Resolved] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21155. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18369 [https://githu

[jira] [Assigned] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21155: --- Assignee: Eric Vandenberg > Add (? running tasks) into Spark UI progress > -

[jira] [Created] (SPARK-21238) allow nested SQL execution

Wenchen Fan created SPARK-21238: --- Summary: allow nested SQL execution Key: SPARK-21238 URL: https://issues.apache.org/jira/browse/SPARK-21238 Project: Spark Issue Type: Improvement Co

[jira] [Assigned] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

[ https://issues.apache.org/jira/browse/SPARK-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21222: --- Assignee: Gengliang Wang > Move elimination of Distinct clause from analyzer to optimizer >

[jira] [Resolved] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

[ https://issues.apache.org/jira/browse/SPARK-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21222. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18429 [https://githu

[jira] [Resolved] (SPARK-21229) remove QueryPlan.preCanonicalized

[ https://issues.apache.org/jira/browse/SPARK-21229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21229. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18440 [https://githu

[jira] [Assigned] (SPARK-21237) Invalidate stats once table data is changed

[ https://issues.apache.org/jira/browse/SPARK-21237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21237: --- Assignee: Zhenhua Wang > Invalidate stats once table data is changed > -

[jira] [Resolved] (SPARK-21237) Invalidate stats once table data is changed

[ https://issues.apache.org/jira/browse/SPARK-21237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21237. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18449 [https://githu

[jira] [Resolved] (SPARK-3577) Add task metric to report spill time

[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-3577. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17471 [https://github.c

[jira] [Assigned] (SPARK-3577) Add task metric to report spill time

[ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-3577: -- Assignee: Sital Kedia > Add task metric to report spill time >

[jira] [Resolved] (SPARK-21238) allow nested SQL execution

[ https://issues.apache.org/jira/browse/SPARK-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21238. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18450 [https://githu

[jira] [Resolved] (SPARK-21225) decrease the Mem using for variable 'tasks' in function resourceOffers

[ https://issues.apache.org/jira/browse/SPARK-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21225. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18435 [https://githu

[jira] [Assigned] (SPARK-21225) decrease the Mem using for variable 'tasks' in function resourceOffers

[ https://issues.apache.org/jira/browse/SPARK-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21225: --- Assignee: yangZhiguo > decrease the Mem using for variable 'tasks' in function resourceOffer

[jira] [Assigned] (SPARK-21052) Add hash map metrics to join

[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21052: --- Assignee: Liang-Chi Hsieh > Add hash map metrics to join > > >

[jira] [Resolved] (SPARK-21052) Add hash map metrics to join

[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21052. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18301 [https://githu

[jira] [Resolved] (SPARK-21253) Cannot fetch big blocks to disk

[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21253. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18472 [https://githu

[jira] [Resolved] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21176. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Issue resolved by pull req

[jira] [Assigned] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21176: --- Assignee: Ingo Schuster > Master UI hangs with spark.ui.reverseProxy=true if the master node

[jira] [Resolved] (SPARK-21258) Window result incorrect using complex object with spilling

[ https://issues.apache.org/jira/browse/SPARK-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21258. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Issue resolved by pull req

[jira] [Updated] (SPARK-17528) data should be copied properly before saving into InternalRow

[ https://issues.apache.org/jira/browse/SPARK-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17528: Summary: data should be copied properly before saving into InternalRow (was: MutableProjection sho

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069665#comment-16069665 ] Wenchen Fan commented on SPARK-21190: - For aggregate, I think it makes more sense to

[jira] [Resolved] (SPARK-18294) Implement commit protocol to support `mapred` package's committer

[ https://issues.apache.org/jira/browse/SPARK-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18294. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18438 [https://githu

[jira] [Assigned] (SPARK-18294) Implement commit protocol to support `mapred` package's committer

[ https://issues.apache.org/jira/browse/SPARK-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18294: --- Assignee: Jiang Xingbo > Implement commit protocol to support `mapred` package's committer >

[jira] [Commented] (SPARK-17924) Consolidate streaming and batch write path

[ https://issues.apache.org/jira/browse/SPARK-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070019#comment-16070019 ] Wenchen Fan commented on SPARK-17924: - cc [~rxin] can we resolve this ticket? all sub

[jira] [Resolved] (SPARK-17528) data should be copied properly before saving into InternalRow

[ https://issues.apache.org/jira/browse/SPARK-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17528. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18483 [https://githu

[jira] [Commented] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070944#comment-16070944 ] Wenchen Fan commented on SPARK-21271: - We do have this regulation for var-length part

[jira] [Assigned] (SPARK-21127) Update statistics after data changing commands

[ https://issues.apache.org/jira/browse/SPARK-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21127: --- Assignee: Zhenhua Wang > Update statistics after data changing commands > --

[jira] [Resolved] (SPARK-21127) Update statistics after data changing commands

[ https://issues.apache.org/jira/browse/SPARK-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21127. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18334 [https://githu

[jira] [Commented] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070963#comment-16070963 ] Wenchen Fan commented on SPARK-21271: - For word-aligned I mean 8-bytes aligned, so th

[jira] [Commented] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070969#comment-16070969 ] Wenchen Fan commented on SPARK-21271: - yea we should. BTW the code seems wrong to be,

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16071002#comment-16071002 ] Wenchen Fan commented on SPARK-21190: - Thanks for your proposal! I have 2 thoughts:

[jira] [Resolved] (SPARK-21282) Fix test failure in 2.0

[ https://issues.apache.org/jira/browse/SPARK-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21282. - Resolution: Fixed Fix Version/s: 2.0.3 Issue resolved by pull request 18506 [https://githu

[jira] [Resolved] (SPARK-21250) Add a url in the table of 'Running Executors' in worker page to visit job page

[ https://issues.apache.org/jira/browse/SPARK-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21250. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18464 [https://githu

[jira] [Assigned] (SPARK-21250) Add a url in the table of 'Running Executors' in worker page to visit job page

[ https://issues.apache.org/jira/browse/SPARK-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21250: --- Assignee: guoxiaolongzte > Add a url in the table of 'Running Executors' in worker page to

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072019#comment-16072019 ] Wenchen Fan commented on SPARK-21190: - > I think we can get away with doing windowing

[jira] [Created] (SPARK-21284) rename SessionCatalog.registerFunction parameter name

Wenchen Fan created SPARK-21284: --- Summary: rename SessionCatalog.registerFunction parameter name Key: SPARK-21284 URL: https://issues.apache.org/jira/browse/SPARK-21284 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-21137) Spark reads many small files slowly off local filesystem

[ https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21137. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18441 [https://githu

[jira] [Assigned] (SPARK-21137) Spark reads many small files slowly off local filesystem

[ https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21137: --- Assignee: Sean Owen > Spark reads many small files slowly off local filesystem > ---

[jira] [Assigned] (SPARK-21283) FileOutputStream should be created as append mode

[ https://issues.apache.org/jira/browse/SPARK-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21283: --- Assignee: liuxian > FileOutputStream should be created as append mode >

[jira] [Resolved] (SPARK-21283) FileOutputStream should be created as append mode

[ https://issues.apache.org/jira/browse/SPARK-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21283. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18507 [https://githu

[jira] [Resolved] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19507. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18521 [https://githu

[jira] [Resolved] (SPARK-21296) Avoid per-record type dispatch in PySpark createDataFrame schema verification

[ https://issues.apache.org/jira/browse/SPARK-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21296. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18521 [https://githu

[jira] [Assigned] (SPARK-21296) Avoid per-record type dispatch in PySpark createDataFrame schema verification

[ https://issues.apache.org/jira/browse/SPARK-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21296: --- Assignee: Hyukjin Kwon > Avoid per-record type dispatch in PySpark createDataFrame schema ve

[jira] [Assigned] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19507: --- Assignee: Hyukjin Kwon > pyspark.sql.types._verify_type() exceptions too broad to debug coll

  1   2   3   4   5   6   7   8   9   10   >