[jira] [Commented] (SPARK-19351) Support for obtaining file splits from underlying InputFormat

2017-02-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883177#comment-15883177 ] Reynold Xin commented on SPARK-19351: - Approach 1 should be supported today. I actually think our

[jira] [Created] (SPARK-19669) Open up visibility for sharedState, sessionState, and a few other functions

2017-02-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19669: --- Summary: Open up visibility for sharedState, sessionState, and a few other functions Key: SPARK-19669 URL: https://issues.apache.org/jira/browse/SPARK-19669 Project:

[jira] [Commented] (SPARK-19493) Remove Java 7 support

2017-02-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873769#comment-15873769 ] Reynold Xin commented on SPARK-19493: - [~srowen] Are there more to be done here? > Remove Java 7

[jira] [Updated] (SPARK-19160) Decorator for UDF creation.

2017-02-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19160: Labels: releasenotes (was: ) > Decorator for UDF creation. > --- > >

[jira] [Commented] (SPARK-19611) Spark 2.1.0 breaks some Hive tables backed by case-sensitive data files

2017-02-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870164#comment-15870164 ] Reynold Xin commented on SPARK-19611: - Rather than this fix, can we just save the case sensitive

[jira] [Resolved] (SPARK-19607) Finding QueryExecution that matches provided executionId

2017-02-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19607. - Resolution: Fixed Assignee: Ala Luszczak Fix Version/s: 2.2.0 > Finding

[jira] [Commented] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867810#comment-15867810 ] Reynold Xin commented on SPARK-19598: - Actually you can even remove the alias parameter from

[jira] [Commented] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866410#comment-15866410 ] Reynold Xin commented on SPARK-19598: - cc [~windpiger] are you interested in working on this? >

[jira] [Created] (SPARK-19598) Remove the alias parameter in UnresolvedRelation

2017-02-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19598: --- Summary: Remove the alias parameter in UnresolvedRelation Key: SPARK-19598 URL: https://issues.apache.org/jira/browse/SPARK-19598 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-19549) Allow providing reasons for stage/job cancelling

2017-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19549. - Resolution: Fixed Assignee: Ala Luszczak Fix Version/s: 2.2.0 > Allow providing

[jira] [Updated] (SPARK-19512) codegen for compare structs fails

2017-02-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19512: Fix Version/s: 2.1.1 > codegen for compare structs fails > - > >

[jira] [Resolved] (SPARK-19514) Range is not interruptible

2017-02-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19514. - Resolution: Fixed Assignee: Ala Luszczak Fix Version/s: 2.2.0 > Range is not

[jira] [Deleted] (SPARK-19494) Remove java7 pom profile

2017-02-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin deleted SPARK-19494: > Remove java7 pom profile > > > Key: SPARK-19494 >

[jira] [Commented] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858156#comment-15858156 ] Reynold Xin commented on SPARK-19489: - The ticket doesn't dictate either. > Stable serialization

[jira] [Resolved] (SPARK-19495) Make SQLConf slightly more extensible

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19495. - Resolution: Fixed Fix Version/s: 2.2.0 > Make SQLConf slightly more extensible >

[jira] [Created] (SPARK-19495) Make SQLConf slightly more extensible

2017-02-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19495: --- Summary: Make SQLConf slightly more extensible Key: SPARK-19495 URL: https://issues.apache.org/jira/browse/SPARK-19495 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-19493) Remove Java 7 support

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856141#comment-15856141 ] Reynold Xin commented on SPARK-19493: - OK great. Maybe wait a bit to see feedback from others

[jira] [Updated] (SPARK-19493) Remove Java 7 support

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19493: Description: Spark deprecated Java 7 support in 2.0, and the goal of the ticket is to officially

[jira] [Updated] (SPARK-19493) Remove Java 7 support

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19493: Target Version/s: (was: 2.2.0) > Remove Java 7 support > - > >

[jira] [Created] (SPARK-19494) Remove java7 pom profile

2017-02-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19494: --- Summary: Remove java7 pom profile Key: SPARK-19494 URL: https://issues.apache.org/jira/browse/SPARK-19494 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-19493) Remove Java 7 support

2017-02-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19493: --- Summary: Remove Java 7 support Key: SPARK-19493 URL: https://issues.apache.org/jira/browse/SPARK-19493 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-19447) Fix input metrics for range operator

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-19447: --- Assignee: Ala Luszczak > Fix input metrics for range operator >

[jira] [Resolved] (SPARK-19447) Fix input metrics for range operator

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19447. - Resolution: Fixed Fix Version/s: 2.2.0 > Fix input metrics for range operator >

[jira] [Updated] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19489: Target Version/s: 2.3.0 > Stable serialization format for external & native code integration >

[jira] [Created] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19489: --- Summary: Stable serialization format for external & native code integration Key: SPARK-19489 URL: https://issues.apache.org/jira/browse/SPARK-19489 Project: Spark

[jira] [Updated] (SPARK-19398) Log in TaskSetManager is not correct

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19398: Fix Version/s: (was: 2.2) 2.2.0 > Log in TaskSetManager is not correct >

[jira] [Commented] (SPARK-18967) Locality preferences should be used when scheduling even when delay scheduling is turned off

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855615#comment-15855615 ] Reynold Xin commented on SPARK-18967: - [~imranr] please don't use "2.2" as the version. It should be

[jira] [Updated] (SPARK-18967) Locality preferences should be used when scheduling even when delay scheduling is turned off

2017-02-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18967: Fix Version/s: (was: 2.2) 2.2.0 > Locality preferences should be used when

[jira] [Created] (SPARK-19480) Higher order functions in SQL

2017-02-06 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19480: --- Summary: Higher order functions in SQL Key: SPARK-19480 URL: https://issues.apache.org/jira/browse/SPARK-19480 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-19447) Fix input metrics for range operator

2017-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19447: --- Summary: Fix input metrics for range operator Key: SPARK-19447 URL: https://issues.apache.org/jira/browse/SPARK-19447 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-19411) Remove the metadata used to mark optional columns in merged Parquet schema for filter predicate pushdown

2017-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19411. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0 > Remove the

[jira] [Commented] (SPARK-19351) Support for obtaining file splits from underlying InputFormat

2017-02-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849736#comment-15849736 ] Reynold Xin commented on SPARK-19351: - [~vinothchandar] are you saying approach 1 already works in

[jira] (SPARK-19409) Upgrade Parquet to 1.8.2

2017-01-31 Thread Reynold Xin (JIRA)
Title: Message Title Reynold Xin resolved as Fixed

[jira] (SPARK-19403) pyspark.sql.column exports non-existent names

2017-01-30 Thread Reynold Xin (JIRA)
Title: Message Title Reynold Xin resolved as Fixed

[jira] [Resolved] (SPARK-18495) Web UI should document meaning of green dot in DAG visualization

2017-01-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18495. - Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 > Web UI should

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2017-01-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833216#comment-15833216 ] Reynold Xin commented on SPARK-16693: - We should absolutely remove the RDD methods. > Remove R

[jira] [Resolved] (SPARK-18917) Dataframe - Time Out Issues / Taking long time in append mode on object stores

2017-01-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18917. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.2.0 > Dataframe - Time

[jira] [Updated] (SPARK-19164) Remove unused UserDefinedFunction._broadcast

2017-01-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19164: Assignee: Maciej Szymkiewicz > Remove unused UserDefinedFunction._broadcast >

[jira] [Updated] (SPARK-19164) Remove unused UserDefinedFunction._broadcast

2017-01-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19164: Summary: Remove unused UserDefinedFunction._broadcast (was: Review of

[jira] [Resolved] (SPARK-19164) Remove unused UserDefinedFunction._broadcast

2017-01-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19164. - > Remove unused UserDefinedFunction._broadcast > > >

[jira] [Commented] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819820#comment-15819820 ] Reynold Xin commented on SPARK-19164: - Which one should I review? I see that you opened a bunch of

[jira] [Resolved] (SPARK-19132) Add test cases for row size estimation

2017-01-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19132. - Resolution: Fixed Fix Version/s: 2.2.0 > Add test cases for row size estimation >

[jira] [Commented] (SPARK-19132) Add test cases for row size estimation

2017-01-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817434#comment-15817434 ] Reynold Xin commented on SPARK-19132: - We should also fix the comment in

[jira] [Resolved] (SPARK-19149) Unify two sets of statistics in LogicalPlan

2017-01-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19149. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.2.0 > Unify two sets

[jira] [Commented] (SPARK-18871) New test cases for IN/NOT IN subquery

2017-01-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816068#comment-15816068 ] Reynold Xin commented on SPARK-18871: - Yes I mean just submit them under this jira. > New test

[jira] [Resolved] (SPARK-19020) Cardinality estimation of aggregate operator

2017-01-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19020. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.2.0 > Cardinality

[jira] [Created] (SPARK-19132) Add test cases for row size estimation

2017-01-08 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19132: --- Summary: Add test cases for row size estimation Key: SPARK-19132 URL: https://issues.apache.org/jira/browse/SPARK-19132 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-17077) Cardinality estimation of project operator

2017-01-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17077. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.2.0 > Cardinality

[jira] [Resolved] (SPARK-19127) Inconsistencies in dense_rank and rank documentation

2017-01-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19127. - Resolution: Fixed Assignee: Bill Chambers Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-18871) New test cases for IN/NOT IN subquery

2017-01-05 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803681#comment-15803681 ] Reynold Xin commented on SPARK-18871: - Yea let's just reuse this pr, since they are all the same set

[jira] [Commented] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-01-05 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803057#comment-15803057 ] Reynold Xin commented on SPARK-18874: - Thanks. Where is the doc? > First phase: Deferring the

[jira] [Resolved] (SPARK-18975) Add an API to remove SparkListener from SparkContext

2016-12-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18975. - Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.2.0 > Add an API to

[jira] [Created] (SPARK-18973) Remove SortPartitions and RedistributeData

2016-12-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18973: --- Summary: Remove SortPartitions and RedistributeData Key: SPARK-18973 URL: https://issues.apache.org/jira/browse/SPARK-18973 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-18969) PullOutNondeterministic should work for Aggregate operator

2016-12-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18969: --- Summary: PullOutNondeterministic should work for Aggregate operator Key: SPARK-18969 URL: https://issues.apache.org/jira/browse/SPARK-18969 Project: Spark

[jira] [Updated] (SPARK-18949) Add recoverPartitions API to Catalog

2016-12-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18949: Fix Version/s: 2.2.0 > Add recoverPartitions API to Catalog >

[jira] [Updated] (SPARK-18949) Add recoverPartitions API to Catalog

2016-12-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18949: Fix Version/s: (was: 2.11) 2.1.1 > Add recoverPartitions API to Catalog >

[jira] [Updated] (SPARK-18949) Add recoverPartitions API to Catalog

2016-12-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18949: Fix Version/s: (was: 2.2.0) > Add recoverPartitions API to Catalog >

[jira] [Resolved] (SPARK-18949) Add recoverPartitions API to Catalog

2016-12-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18949. - Resolution: Fixed Fix Version/s: 2.2.0 2.11 > Add recoverPartitions

[jira] [Resolved] (SPARK-18576) Expose basic TaskContext info in PySpark

2016-12-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18576. - Resolution: Fixed Assignee: holdenk Fix Version/s: 2.2.0 > Expose basic

[jira] [Commented] (SPARK-18939) Timezone support in partition values.

2016-12-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763378#comment-15763378 ] Reynold Xin commented on SPARK-18939: - FWIW it's probably unlikely timestamp is used as a partition

[jira] [Commented] (SPARK-18939) Timezone support in partition values.

2016-12-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763372#comment-15763372 ] Reynold Xin commented on SPARK-18939: - This only impacts timestamp data type right? > Timezone

[jira] [Created] (SPARK-18937) Timezone support in CSV/JSON parsing

2016-12-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18937: --- Summary: Timezone support in CSV/JSON parsing Key: SPARK-18937 URL: https://issues.apache.org/jira/browse/SPARK-18937 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-18936) Infrastructure for session local timezone support

2016-12-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18936: --- Summary: Infrastructure for session local timezone support Key: SPARK-18936 URL: https://issues.apache.org/jira/browse/SPARK-18936 Project: Spark Issue Type:

[jira] [Closed] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2016-12-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17997. --- Resolution: Later > Aggregation function for counting distinct values for multiple intervals >

[jira] [Updated] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2016-12-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17997: Issue Type: Sub-task (was: New Feature) Parent: SPARK-16026 > Aggregation function for

[jira] [Resolved] (SPARK-18892) Alias percentile_approx approx_percentile

2016-12-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18892. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Alias percentile_approx

[jira] [Created] (SPARK-18892) Alias percentile_approx approx_percentile

2016-12-15 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18892: --- Summary: Alias percentile_approx approx_percentile Key: SPARK-18892 URL: https://issues.apache.org/jira/browse/SPARK-18892 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-17910) Allow users to update the comment of a column

2016-12-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17910. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.2.0 > Allow users to

[jira] [Commented] (SPARK-18455) General support for correlated subquery processing

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750638#comment-15750638 ] Reynold Xin commented on SPARK-18455: - Thanks for sharing the doc. This is a really well written

[jira] [Resolved] (SPARK-18869) Add TreeNode.p that returns BaseType

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18869. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 > Add TreeNode.p that

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17822: Fix Version/s: 2.0.3 > JVMObjectTracker.objMap may leak JVM objects >

[jira] [Updated] (SPARK-18793) SparkR vignette update: random forest

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18793: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18865) SparkR vignettes MLP and LDA updates

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18865: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18751) Deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18751: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18840: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18835) Do not expose shaded types in JavaTypeInference API

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18835: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 > Do

[jira] [Updated] (SPARK-18349) Update R API documentation on ml model summary

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18349: Fix Version/s: (was: 2.1.1) 2.1.0 > Update R API documentation on ml model

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17822: Fix Version/s: (was: 2.1.1) (was: 2.0.3) (was:

[jira] [Updated] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18681: Fix Version/s: (was: 2.1.1) 2.1.0 > Throw Filtering is supported only on

[jira] [Updated] (SPARK-18797) Update spark.logit in sparkr-vignettes

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18797: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18812) Clarify "Spark ML"

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18812: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18816) executor page fails to show log links if executors are added after an app is launched

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18816: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18811) Stream Source resolution should happen in StreamExecution thread, not main thread

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18811: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18760) Provide consistent format output for all file formats

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18760: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18325) SparkR 2.1 QA: Check for new R APIs requiring example code

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18325: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18590: Fix Version/s: (was: 2.1.1) 2.1.0 > R - Include package vignettes and help

[jira] [Updated] (SPARK-18815) NPE when collecting column stats for string/binary column having only null values

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18815: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 > NPE

[jira] [Updated] (SPARK-18794) SparkR vignette update: gbt

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18794: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18795) SparkR vignette update: ksTest

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18795: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18807) Should suppress output print for calls to JVM methods with void return values

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18807: Fix Version/s: (was: 2.1.1) 2.1.0 > Should suppress output print for calls

[jira] [Updated] (SPARK-18628) Update handle invalid documentation string

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18628: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18810) SparkR install.spark does not work for RCs, snapshots

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18810: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18774) Ignore non-existing files when ignoreCorruptFiles is enabled

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18774: Fix Version/s: (was: 2.1.1) (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-18790) Keep a general offset history of stream batches

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18790: Fix Version/s: (was: 2.1.1) 2.1.0 > Keep a general offset history of stream

[jira] [Updated] (SPARK-18745) java.lang.IndexOutOfBoundsException running query 68 Spark SQL on (100TB)

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18745: Fix Version/s: (was: 2.1.1) 2.1.0 > java.lang.IndexOutOfBoundsException

[jira] [Updated] (SPARK-16589) Chained cartesian produces incorrect number of records

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16589: Fix Version/s: (was: 2.1.1) 2.1.0 > Chained cartesian produces incorrect

[jira] [Updated] (SPARK-18843) Fix timeout in awaitResultInForkJoinSafely

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18843: Fix Version/s: (was: 2.1.1) 2.1.0 > Fix timeout in

[jira] [Resolved] (SPARK-18856) Newly created catalog table assumed to have 0 rows and 0 bytes

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18856. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.1.0 > Newly created

[jira] [Updated] (SPARK-18869) Add TreeNode.p that returns BaseType

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18869: Description: After the bug fix in SPARK-18854, TreeNode.apply now returns TreeNode[_] rather than

[jira] [Updated] (SPARK-18869) Add TreeNode.p that returns BaseType

2016-12-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18869: Summary: Add TreeNode.p that returns BaseType (was: Add lp and pp to plan nodes for getting

<    1   2   3   4   5   6   7   8   9   10   >