[jira] [Created] (SPARK-21043) Add unionByName API to Dataset

Reynold Xin created SPARK-21043: --- Summary: Add unionByName API to Dataset Key: SPARK-21043 URL: https://issues.apache.org/jira/browse/SPARK-21043 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-21042) Document Dataset.union is resolution by position, not name

[ https://issues.apache.org/jira/browse/SPARK-21042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-21042. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 > Document Dataset.union

[jira] [Created] (SPARK-21042) Document Dataset.union is resolution by position, not name

Reynold Xin created SPARK-21042: --- Summary: Document Dataset.union is resolution by position, not name Key: SPARK-21042 URL: https://issues.apache.org/jira/browse/SPARK-21042 Project: Spark

[jira] [Commented] (SPARK-20953) Add hash map metrics to aggregate and join

[ https://issues.apache.org/jira/browse/SPARK-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043581#comment-16043581 ] Reynold Xin commented on SPARK-20953: - I'd show the avg in the UI if possible. As a matter of fact

[jira] [Created] (SPARK-20953) Add hash map metrics to aggregate and join

Reynold Xin created SPARK-20953: --- Summary: Add hash map metrics to aggregate and join Key: SPARK-20953 URL: https://issues.apache.org/jira/browse/SPARK-20953 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-20953) Add hash map metrics to aggregate and join

[ https://issues.apache.org/jira/browse/SPARK-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033470#comment-16033470 ] Reynold Xin commented on SPARK-20953: - cc [~viirya] want to give this a try? > Add hash map metrics

[jira] [Closed] (SPARK-2183) Avoid loading/shuffling data twice in self-join query

[ https://issues.apache.org/jira/browse/SPARK-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-2183. -- Resolution: Fixed Assignee: Reynold Xin This shouldn't be an issue anymore with reuse exchange in

[jira] [Closed] (SPARK-20895) Support fast execution based on an optimized plan and parameter placeholders

[ https://issues.apache.org/jira/browse/SPARK-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-20895. --- Resolution: Later > Support fast execution based on an optimized plan and parameter placeholders >

[jira] [Commented] (SPARK-20895) Support fast execution based on an optimized plan and parameter placeholders

[ https://issues.apache.org/jira/browse/SPARK-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027972#comment-16027972 ] Reynold Xin commented on SPARK-20895: - Can you create a separate ticket to discuss prepare and

[jira] [Commented] (SPARK-20895) Support fast execution based on an optimized plan and parameter placeholders

[ https://issues.apache.org/jira/browse/SPARK-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026028#comment-16026028 ] Reynold Xin commented on SPARK-20895: - Have you seen a specific case in practice in which the

[jira] [Commented] (SPARK-20895) Support fast execution based on an optimized plan and parameter placeholders

[ https://issues.apache.org/jira/browse/SPARK-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025970#comment-16025970 ] Reynold Xin commented on SPARK-20895: - What's the benefit? > Support fast execution based on an

[jira] [Commented] (SPARK-20843) Cannot gracefully kill drivers which take longer than 10 seconds to die

[ https://issues.apache.org/jira/browse/SPARK-20843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025042#comment-16025042 ] Reynold Xin commented on SPARK-20843: - cc [~joshrosen] and [~marmbrus] > Cannot gracefully kill

[jira] [Created] (SPARK-20867) Move individual hints from Statistics into HintInfo class

Reynold Xin created SPARK-20867: --- Summary: Move individual hints from Statistics into HintInfo class Key: SPARK-20867 URL: https://issues.apache.org/jira/browse/SPARK-20867 Project: Spark

[jira] [Resolved] (SPARK-20857) Generic resolved hint node

[ https://issues.apache.org/jira/browse/SPARK-20857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20857. - Resolution: Fixed Fix Version/s: 2.2.0 > Generic resolved hint node >

[jira] [Updated] (SPARK-20857) Turn BroadcastHint into a more generic hint node

[ https://issues.apache.org/jira/browse/SPARK-20857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20857: Summary: Turn BroadcastHint into a more generic hint node (was: Make BroadcastHint a more

[jira] [Created] (SPARK-20857) Make BroadcastHint a more ResolvedHint node

Reynold Xin created SPARK-20857: --- Summary: Make BroadcastHint a more ResolvedHint node Key: SPARK-20857 URL: https://issues.apache.org/jira/browse/SPARK-20857 Project: Spark Issue Type:

[jira] [Updated] (SPARK-20857) Generic resolved hint node

[ https://issues.apache.org/jira/browse/SPARK-20857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20857: Summary: Generic resolved hint node (was: Turn BroadcastHint into a more generic hint node) >

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013540#comment-16013540 ] Reynold Xin commented on SPARK-12297: - I don't think the CSV example you gave make sense. It is still

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013540#comment-16013540 ] Reynold Xin edited comment on SPARK-12297 at 5/17/17 5:12 AM: -- I don't think

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010967#comment-16010967 ] Reynold Xin commented on SPARK-12297: - Can you clarify what you mean that with other formats (e.g.

[jira] [Commented] (SPARK-20703) Add an operator for writing data out

[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007738#comment-16007738 ] Reynold Xin commented on SPARK-20703: - That and also Hive. We can do them one by one though. > Add

[jira] [Commented] (SPARK-13486) Move SQLConf into an internal package

[ https://issues.apache.org/jira/browse/SPARK-13486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007428#comment-16007428 ] Reynold Xin commented on SPARK-13486: - Why is this troubling? SQLConf was previously package visible

[jira] [Created] (SPARK-20703) Add an operator for writing data out

Reynold Xin created SPARK-20703: --- Summary: Add an operator for writing data out Key: SPARK-20703 URL: https://issues.apache.org/jira/browse/SPARK-20703 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-20703) Add an operator for writing data out

[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005765#comment-16005765 ] Reynold Xin commented on SPARK-20703: - cc [~viirya] want to give this a try? > Add an operator for

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:40 PM: - I looked at the

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:39 PM: - I looked at the

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin commented on SPARK-12297: - I looked at the issue again and reverted the patch. If we want

[jira] [Updated] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12297: Fix Version/s: (was: 2.3.0) > Add work-around for Parquet/Hive int96 timestamp bug. >

[jira] [Reopened] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-12297: - Assignee: (was: Imran Rashid) > Add work-around for Parquet/Hive int96 timestamp bug. >

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001214#comment-16001214 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:35 PM: - Sorry I'm going

[jira] [Created] (SPARK-20674) Support registering UserDefinedFunction as named UDF

Reynold Xin created SPARK-20674: --- Summary: Support registering UserDefinedFunction as named UDF Key: SPARK-20674 URL: https://issues.apache.org/jira/browse/SPARK-20674 Project: Spark Issue

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001214#comment-16001214 ] Reynold Xin edited comment on SPARK-12297 at 5/8/17 7:41 PM: - Sorry I'm going

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001214#comment-16001214 ] Reynold Xin commented on SPARK-12297: - Sorry I'm going to revert this. I think this requires further

[jira] [Commented] (SPARK-18350) Support session local timezone

[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000173#comment-16000173 ] Reynold Xin commented on SPARK-18350: - [~srowen] why was this reopened? > Support session local

[jira] [Resolved] (SPARK-20616) RuleExecutor logDebug of batch results should show diff to start of batch

[ https://issues.apache.org/jira/browse/SPARK-20616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20616. - Resolution: Fixed Assignee: Juliusz Sompolski Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-20584) Python generic hint support

[ https://issues.apache.org/jira/browse/SPARK-20584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20584. - Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.2.0 > Python

[jira] [Resolved] (SPARK-20583) Scala/Java generic hint support

[ https://issues.apache.org/jira/browse/SPARK-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20583. - Resolution: Fixed Fix Version/s: 2.2.0 > Scala/Java generic hint support >

[jira] [Created] (SPARK-20584) Python generic hint support

Reynold Xin created SPARK-20584: --- Summary: Python generic hint support Key: SPARK-20584 URL: https://issues.apache.org/jira/browse/SPARK-20584 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-20583) Scala/Java generic hint support

Reynold Xin created SPARK-20583: --- Summary: Scala/Java generic hint support Key: SPARK-20583 URL: https://issues.apache.org/jira/browse/SPARK-20583 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-20585) R generic hint support

Reynold Xin created SPARK-20585: --- Summary: R generic hint support Key: SPARK-20585 URL: https://issues.apache.org/jira/browse/SPARK-20585 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-20576) Support generic hint function in Dataset/DataFrame

Reynold Xin created SPARK-20576: --- Summary: Support generic hint function in Dataset/DataFrame Key: SPARK-20576 URL: https://issues.apache.org/jira/browse/SPARK-20576 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-20474) OnHeapColumnVector realocation may not copy existing data

[ https://issues.apache.org/jira/browse/SPARK-20474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20474. - Resolution: Fixed Assignee: Michal Szafranski Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-20473) ColumnVector.Array is missing accessors for some types

[ https://issues.apache.org/jira/browse/SPARK-20473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20473. - Resolution: Fixed Assignee: Michal Szafranski Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-20453) Bump master branch version to 2.3.0-SNAPSHOT

[ https://issues.apache.org/jira/browse/SPARK-20453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20453. - Resolution: Fixed Fix Version/s: 2.3.0 > Bump master branch version to 2.3.0-SNAPSHOT >

[jira] [Resolved] (SPARK-20420) Add events to the external catalog

[ https://issues.apache.org/jira/browse/SPARK-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20420. - Resolution: Fixed Fix Version/s: 2.2.0 > Add events to the external catalog >

[jira] [Created] (SPARK-20405) Dataset.withNewExecutionId should be private

Reynold Xin created SPARK-20405: --- Summary: Dataset.withNewExecutionId should be private Key: SPARK-20405 URL: https://issues.apache.org/jira/browse/SPARK-20405 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13534: Shepherd: Reynold Xin Affects Version/s: 2.1.0 > Implement Apache Arrow serializer

[jira] [Resolved] (SPARK-20398) range() operator should include cancellation reason when killed

[ https://issues.apache.org/jira/browse/SPARK-20398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20398. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.2.0 > range() operator

[jira] [Updated] (SPARK-20349) ListFunctions returns duplicate functions after using persistent functions

[ https://issues.apache.org/jira/browse/SPARK-20349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20349: Fix Version/s: (was: 2.1.2) > ListFunctions returns duplicate functions after using persistent

[jira] [Resolved] (SPARK-17647) SQL LIKE does not handle backslashes correctly

[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17647. - Resolution: Fixed Assignee: Xiangrui Meng Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-20302) Short circuit cast when from and to types are structurally the same

[ https://issues.apache.org/jira/browse/SPARK-20302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20302. - Resolution: Fixed Fix Version/s: 2.2.0 > Short circuit cast when from and to types are

[jira] [Created] (SPARK-20304) AssertNotNull should not include path in string representation

Reynold Xin created SPARK-20304: --- Summary: AssertNotNull should not include path in string representation Key: SPARK-20304 URL: https://issues.apache.org/jira/browse/SPARK-20304 Project: Spark

[jira] [Updated] (SPARK-20302) Short circuit cast when from and to types are structurally the same

[ https://issues.apache.org/jira/browse/SPARK-20302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20302: Summary: Short circuit cast when from and to types are structurally the same (was: Optimize cast

[jira] [Created] (SPARK-20302) Optimize cast when from and to types are structurally the same

Reynold Xin created SPARK-20302: --- Summary: Optimize cast when from and to types are structurally the same Key: SPARK-20302 URL: https://issues.apache.org/jira/browse/SPARK-20302 Project: Spark

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965220#comment-15965220 ] Reynold Xin commented on SPARK-20202: - There are no currently targeted version, are there? > Remove

[jira] [Resolved] (SPARK-20289) Use StaticInvoke rather than NewInstance for boxing primitive types

[ https://issues.apache.org/jira/browse/SPARK-20289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20289. - Resolution: Fixed Fix Version/s: 2.2.0 > Use StaticInvoke rather than NewInstance for

[jira] [Created] (SPARK-20289) Use StaticInvoke rather than NewInstance for boxing primitive types

Reynold Xin created SPARK-20289: --- Summary: Use StaticInvoke rather than NewInstance for boxing primitive types Key: SPARK-20289 URL: https://issues.apache.org/jira/browse/SPARK-20289 Project: Spark

[jira] [Resolved] (SPARK-17564) Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay

[ https://issues.apache.org/jira/browse/SPARK-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17564. - Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20283. - Resolution: Fixed Fix Version/s: 2.2.0 > Add preOptimizationBatches >

[jira] [Updated] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20283: Description: We currently have postHocOptimizationBatches, but not preOptimizationBatches. This

[jira] [Created] (SPARK-20283) Add preOptimizationBatches

Reynold Xin created SPARK-20283: --- Summary: Add preOptimizationBatches Key: SPARK-20283 URL: https://issues.apache.org/jira/browse/SPARK-20283 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-20264) asm should be non-test dependency in sql/core

Reynold Xin created SPARK-20264: --- Summary: asm should be non-test dependency in sql/core Key: SPARK-20264 URL: https://issues.apache.org/jira/browse/SPARK-20264 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-20262) AssertNotNull should throw NullPointerException

Reynold Xin created SPARK-20262: --- Summary: AssertNotNull should throw NullPointerException Key: SPARK-20262 URL: https://issues.apache.org/jira/browse/SPARK-20262 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-20255) FileIndex hierarchy inconsistency

[ https://issues.apache.org/jira/browse/SPARK-20255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20255. - Resolution: Fixed Assignee: Adrian Ionescu Fix Version/s: 2.2.0 > FileIndex

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957843#comment-15957843 ] Reynold Xin commented on SPARK-20202: - I've created a ticket on the Hive side to publish 1.2.x:

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957811#comment-15957811 ] Reynold Xin commented on SPARK-20202: - Yes this is really important. The proper way to do this is to

[jira] [Updated] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20202: Priority: Major (was: Blocker) > Remove references to org.spark-project.hive >

[jira] [Resolved] (SPARK-20204) remove SimpleCatalystConf and CatalystConf type alias

[ https://issues.apache.org/jira/browse/SPARK-20204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20204. - Resolution: Fixed Fix Version/s: 2.2.0 > remove SimpleCatalystConf and CatalystConf type

[jira] [Resolved] (SPARK-20145) "SELECT * FROM range(1)" works, but "SELECT * FROM RANGE(1)" doesn't

[ https://issues.apache.org/jira/browse/SPARK-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20145. - Resolution: Fixed Assignee: sam elamin Fix Version/s: 2.2.0 > "SELECT * FROM

[jira] [Resolved] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20151. - Resolution: Fixed Fix Version/s: 2.2.0 > Account for partition pruning in scan

[jira] [Updated] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20151: Summary: Account for partition pruning in scan metadataTime metrics (was: Take partition pruning

[jira] [Created] (SPARK-20151) Take partition pruning timing into account in scan metadataTime metrics

Reynold Xin created SPARK-20151: --- Summary: Take partition pruning timing into account in scan metadataTime metrics Key: SPARK-20151 URL: https://issues.apache.org/jira/browse/SPARK-20151 Project: Spark

[jira] [Updated] (SPARK-20151) Account for partition pruning in scan metadataTime metrics

[ https://issues.apache.org/jira/browse/SPARK-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20151: Description: After SPARK-20136, we report metadata timing metrics in scan operator. However, that

[jira] [Resolved] (SPARK-20148) Extend the file commit interface to allow subscribing to task commit messages

[ https://issues.apache.org/jira/browse/SPARK-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20148. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.2.0 > Extend the file

[jira] [Resolved] (SPARK-20136) Add num files and metadata operation timing to scan metrics

[ https://issues.apache.org/jira/browse/SPARK-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20136. - Resolution: Fixed Fix Version/s: 2.2.0 > Add num files and metadata operation timing to

[jira] [Resolved] (SPARK-20146) Column comment information is missing for Thrift Server's TableSchema

[ https://issues.apache.org/jira/browse/SPARK-20146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20146. - Resolution: Fixed Assignee: Bo Meng Fix Version/s: 2.2.0 > Column comment

[jira] [Updated] (SPARK-20136) Add num files and metadata operation timing to scan metrics

[ https://issues.apache.org/jira/browse/SPARK-20136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20136: Description: We currently do not include explicitly metadata operation timing and number of files

[jira] [Created] (SPARK-20136) Add num files and metadata operation timing to scan metrics

Reynold Xin created SPARK-20136: --- Summary: Add num files and metadata operation timing to scan metrics Key: SPARK-20136 URL: https://issues.apache.org/jira/browse/SPARK-20136 Project: Spark

[jira] [Resolved] (SPARK-20134) SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates

[ https://issues.apache.org/jira/browse/SPARK-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20134. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 >

[jira] [Created] (SPARK-20134) SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates

Reynold Xin created SPARK-20134: --- Summary: SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates Key: SPARK-20134 URL: https://issues.apache.org/jira/browse/SPARK-20134 Project:

[jira] [Resolved] (SPARK-19846) Add a flag to disable constraint propagation

[ https://issues.apache.org/jira/browse/SPARK-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19846. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0 > Add a flag to

[jira] [Resolved] (SPARK-18847) PageRank gives incorrect results for graphs with sinks

[ https://issues.apache.org/jira/browse/SPARK-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18847. - Resolution: Fixed Assignee: Andrew Ray Fix Version/s: 2.2.0 > PageRank gives

[jira] [Updated] (SPARK-19987) Pass all filters into FileIndex

[ https://issues.apache.org/jira/browse/SPARK-19987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19987: Description: This is a tiny teeny refactoring to pass data filters also to the FileIndex, so

[jira] [Created] (SPARK-19987) Pass all filters into FileIndex

Reynold Xin created SPARK-19987: --- Summary: Pass all filters into FileIndex Key: SPARK-19987 URL: https://issues.apache.org/jira/browse/SPARK-19987 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-19960) Move `SparkHadoopWriter` to `internal/io/`

[ https://issues.apache.org/jira/browse/SPARK-19960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19960. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.2.0 > Move

[jira] [Updated] (SPARK-19952) Remove specialized catalog related analysis exceptions

[ https://issues.apache.org/jira/browse/SPARK-19952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19952: Description: We introduce catalog specific analysis exceptions (that extends AnalysisException)

[jira] [Updated] (SPARK-19952) Remove specialized catalog related analysis exceptions

[ https://issues.apache.org/jira/browse/SPARK-19952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19952: Description: We introduce catalog specific analysis exceptions (that extends AnalysisException) in

[jira] [Commented] (SPARK-19416) Dataset.schema is inconsistent with Dataset in handling columns with periods

[ https://issues.apache.org/jira/browse/SPARK-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924646#comment-15924646 ] Reynold Xin commented on SPARK-19416: - We probably can't change any of them now, unless we introduce

[jira] [Created] (SPARK-19944) Move SQLConf from sql/core to sql/catalyst

Reynold Xin created SPARK-19944: --- Summary: Move SQLConf from sql/core to sql/catalyst Key: SPARK-19944 URL: https://issues.apache.org/jira/browse/SPARK-19944 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite from integration tests into unit tests

[ https://issues.apache.org/jira/browse/SPARK-19856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19856: Summary: Turn partitioning related test cases in FileSourceStrategySuite from integration tests

[jira] [Created] (SPARK-19855) Create an internal FilePartitionStrategy interface

Reynold Xin created SPARK-19855: --- Summary: Create an internal FilePartitionStrategy interface Key: SPARK-19855 URL: https://issues.apache.org/jira/browse/SPARK-19855 Project: Spark Issue Type:

[jira] [Created] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite into unit tests

Reynold Xin created SPARK-19856: --- Summary: Turn partitioning related test cases in FileSourceStrategySuite into unit tests Key: SPARK-19856 URL: https://issues.apache.org/jira/browse/SPARK-19856

[jira] [Created] (SPARK-19854) Refactor file partitioning strategy to make it easier to extend / unit test

Reynold Xin created SPARK-19854: --- Summary: Refactor file partitioning strategy to make it easier to extend / unit test Key: SPARK-19854 URL: https://issues.apache.org/jira/browse/SPARK-19854 Project:

[jira] [Updated] (SPARK-18138) More officially deprecate support for Python 2.6, Java 7, and Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-18138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18138: Labels: releasenotes (was: ) > More officially deprecate support for Python 2.6, Java 7, and

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898005#comment-15898005 ] Reynold Xin commented on SPARK-17495: - We should probably create subtickets next time for this ...

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893900#comment-15893900 ] Reynold Xin commented on SPARK-19659: - Rather than this tracking the max for each N/2000 consecutive

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889432#comment-15889432 ] Reynold Xin commented on SPARK-17495: - Is it possible to figure out the hashing function based on

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883394#comment-15883394 ] Reynold Xin commented on SPARK-17495: - Let me put some thoughts here Please let me know if I

[jira] [Commented] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883189#comment-15883189 ] Reynold Xin commented on SPARK-17495: - Ah yes. I kept doing it ... :) > Hive hash implementation >

[jira] [Resolved] (SPARK-17495) Hive hash implementation

[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17495. - Resolution: Fixed Fix Version/s: 2.2.0 > Hive hash implementation >

<    1   2   3   4   5   6   7   8   9   10   >