[jira] [Commented] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037349#comment-15037349 ] Davies Liu commented on SPARK-12110: [~aedwip] How do you launch the EC2 cluster usin

[jira] [Created] (SPARK-12132) Cltr-C should clear current line in pyspark shell

Davies Liu created SPARK-12132: -- Summary: Cltr-C should clear current line in pyspark shell Key: SPARK-12132 URL: https://issues.apache.org/jira/browse/SPARK-12132 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12089. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull reque

[jira] [Resolved] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

[ https://issues.apache.org/jira/browse/SPARK-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12032. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10073 [https://github.

[jira] [Resolved] (SPARK-12132) Cltr-C should clear current line in pyspark shell

[ https://issues.apache.org/jira/browse/SPARK-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12132. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull reque

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046014#comment-15046014 ] Davies Liu commented on SPARK-12179: This may be related to https://issues.apache.org

[jira] [Updated] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12179: --- Priority: Critical (was: Minor) > Spark SQL get different result with the same code > --

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047186#comment-15047186 ] Davies Liu commented on SPARK-12179: Could you also test 1.6-RC1? I'm just wondering

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047193#comment-15047193 ] Davies Liu commented on SPARK-12179: There are two direction to narrow down the probl

[jira] [Comment Edited] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047193#comment-15047193 ] Davies Liu edited comment on SPARK-12179 at 12/8/15 6:24 PM: -

[jira] [Created] (SPARK-12213) Query with only one distinct should not having on expand

Davies Liu created SPARK-12213: -- Summary: Query with only one distinct should not having on expand Key: SPARK-12213 URL: https://issues.apache.org/jira/browse/SPARK-12213 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-12222) deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-1. Resolution: Fixed Fix Version/s: 1.6.0 2.0.0 Issue resolved by pull reque

[jira] [Assigned] (SPARK-12213) Query with only one distinct should not having on expand

[ https://issues.apache.org/jira/browse/SPARK-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12213: -- Assignee: Davies Liu > Query with only one distinct should not having on expand >

[jira] [Resolved] (SPARK-11713) Initial RDD for updateStateByKey for pyspark

[ https://issues.apache.org/jira/browse/SPARK-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11713. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10082 [https://github.

[jira] [Assigned] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-11885: -- Assignee: Davies Liu (was: Yin Huai) > UDAF may nondeterministically generate wrong results >

[jira] [Commented] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053528#comment-15053528 ] Davies Liu commented on SPARK-11885: The root cause is that we generate ExprId for Sc

[jira] [Resolved] (SPARK-11885) UDAF may nondeterministically generate wrong results

[ https://issues.apache.org/jira/browse/SPARK-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11885. Resolution: Fixed Fix Version/s: 1.5.3 > UDAF may nondeterministically generate wrong result

[jira] [Created] (SPARK-12283) Use UnsafeRow as the buffer in SortBasedAggregation to avoid Unsafe/Safe conversion

Davies Liu created SPARK-12283: -- Summary: Use UnsafeRow as the buffer in SortBasedAggregation to avoid Unsafe/Safe conversion Key: SPARK-12283 URL: https://issues.apache.org/jira/browse/SPARK-12283 Proje

[jira] [Created] (SPARK-12284) Output UnsafeRow from window function

Davies Liu created SPARK-12284: -- Summary: Output UnsafeRow from window function Key: SPARK-12284 URL: https://issues.apache.org/jira/browse/SPARK-12284 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-12286) Support UnsafeRow in all SparkPlan (if possible)

Davies Liu created SPARK-12286: -- Summary: Support UnsafeRow in all SparkPlan (if possible) Key: SPARK-12286 URL: https://issues.apache.org/jira/browse/SPARK-12286 Project: Spark Issue Type: Epic

[jira] [Assigned] (SPARK-12286) Support UnsafeRow in all SparkPlan (if possible)

[ https://issues.apache.org/jira/browse/SPARK-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12286: -- Assignee: Davies Liu > Support UnsafeRow in all SparkPlan (if possible) >

[jira] [Created] (SPARK-12287) Support UnsafeRow in MapPartitions/MapGroups/CoGroup

Davies Liu created SPARK-12287: -- Summary: Support UnsafeRow in MapPartitions/MapGroups/CoGroup Key: SPARK-12287 URL: https://issues.apache.org/jira/browse/SPARK-12287 Project: Spark Issue Type:

[jira] [Updated] (SPARK-12287) Support UnsafeRow in MapPartitions/MapGroups/CoGroup

[ https://issues.apache.org/jira/browse/SPARK-12287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12287: --- Issue Type: Improvement (was: Epic) > Support UnsafeRow in MapPartitions/MapGroups/CoGroup > ---

[jira] [Created] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

Davies Liu created SPARK-12288: -- Summary: Support UnsafeRow in Coalesce/Except/Intersect Key: SPARK-12288 URL: https://issues.apache.org/jira/browse/SPARK-12288 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-12289) Support UnsafeRow in TakeOrderedAndProject/Limit

Davies Liu created SPARK-12289: -- Summary: Support UnsafeRow in TakeOrderedAndProject/Limit Key: SPARK-12289 URL: https://issues.apache.org/jira/browse/SPARK-12289 Project: Spark Issue Type: Impr

[jira] [Created] (SPARK-12290) Change the default value in SparkPlan

Davies Liu created SPARK-12290: -- Summary: Change the default value in SparkPlan Key: SPARK-12290 URL: https://issues.apache.org/jira/browse/SPARK-12290 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

Davies Liu created SPARK-12291: -- Summary: Support UnsafeRow in BroadcastLeftSemiJoinHash Key: SPARK-12291 URL: https://issues.apache.org/jira/browse/SPARK-12291 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-12292) Support UnsafeRow in Generate

Davies Liu created SPARK-12292: -- Summary: Support UnsafeRow in Generate Key: SPARK-12292 URL: https://issues.apache.org/jira/browse/SPARK-12292 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-12293) Support UnsafeRow in LocalTableScan

Davies Liu created SPARK-12293: -- Summary: Support UnsafeRow in LocalTableScan Key: SPARK-12293 URL: https://issues.apache.org/jira/browse/SPARK-12293 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-12294) Support UnsafeRow in HiveTableScan

Davies Liu created SPARK-12294: -- Summary: Support UnsafeRow in HiveTableScan Key: SPARK-12294 URL: https://issues.apache.org/jira/browse/SPARK-12294 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-12295) Manage the memory used by window function

Davies Liu created SPARK-12295: -- Summary: Manage the memory used by window function Key: SPARK-12295 URL: https://issues.apache.org/jira/browse/SPARK-12295 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-12016) word2vec load model can't use findSynonyms to get words

[ https://issues.apache.org/jira/browse/SPARK-12016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12016. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10100 [https://github.

[jira] [Resolved] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12288. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10285 [https://github.

[jira] [Assigned] (SPARK-8745) Remove GenerateProjection

[ https://issues.apache.org/jira/browse/SPARK-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-8745: - Assignee: Davies Liu > Remove GenerateProjection > - > >

[jira] [Updated] (SPARK-12288) Support UnsafeRow in Coalesce/Except/Intersect

[ https://issues.apache.org/jira/browse/SPARK-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12288: --- Assignee: Xiao Li > Support UnsafeRow in Coalesce/Except/Intersect >

[jira] [Updated] (SPARK-12293) Support UnsafeRow in LocalTableScan

[ https://issues.apache.org/jira/browse/SPARK-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12293: --- Assignee: Liang-Chi Hsieh (was: Apache Spark) > Support UnsafeRow in LocalTableScan > --

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060213#comment-15060213 ] Davies Liu commented on SPARK-12179: I think this UDF is not thread safe, rowNum and

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060217#comment-15060217 ] Davies Liu commented on SPARK-12179: Which version of Spark are you using? > Spark

[jira] [Comment Edited] (SPARK-12179) Spark SQL get different result with the same code

[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060217#comment-15060217 ] Davies Liu edited comment on SPARK-12179 at 12/16/15 4:04 PM: -

[jira] [Resolved] (SPARK-8745) Remove GenerateProjection

[ https://issues.apache.org/jira/browse/SPARK-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8745. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10316 [https://github.com

[jira] [Created] (SPARK-12380) MLLib should use existing SQLContext instead create new one

Davies Liu created SPARK-12380: -- Summary: MLLib should use existing SQLContext instead create new one Key: SPARK-12380 URL: https://issues.apache.org/jira/browse/SPARK-12380 Project: Spark Issu

[jira] [Created] (SPARK-12385) Push projection into Join

Davies Liu created SPARK-12385: -- Summary: Push projection into Join Key: SPARK-12385 URL: https://issues.apache.org/jira/browse/SPARK-12385 Project: Spark Issue Type: Improvement Compo

[jira] [Resolved] (SPARK-12380) MLLib should use existing SQLContext instead create new one

[ https://issues.apache.org/jira/browse/SPARK-12380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12380. Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull reque

[jira] [Created] (SPARK-12388) Change default compressor to LZ4

Davies Liu created SPARK-12388: -- Summary: Change default compressor to LZ4 Key: SPARK-12388 URL: https://issues.apache.org/jira/browse/SPARK-12388 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-12388) Change default compressor to LZ4

[ https://issues.apache.org/jira/browse/SPARK-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12388: --- Description: According the benchmark [1], LZ4-java could be 80% (or 30%) faster than Snappy. After c

[jira] [Created] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

Davies Liu created SPARK-12395: -- Summary: Result of DataFrame.join(usingColumns) could be wrong for outer join Key: SPARK-12395 URL: https://issues.apache.org/jira/browse/SPARK-12395 Project: Spark

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Priority: Critical (was: Major) > Result of DataFrame.join(usingColumns) could be wrong for outer jo

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Priority: Blocker (was: Critical) > Result of DataFrame.join(usingColumns) could be wrong for outer

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Affects Version/s: 1.6.0 > Result of DataFrame.join(usingColumns) could be wrong for outer join > ---

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Priority: Blocker (was: Critical) > Result of DataFrame.join(usingColumns) could be wrong for outer

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Component/s: SQL > Result of DataFrame.join(usingColumns) could be wrong for outer join > ---

[jira] [Updated] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12395: --- Priority: Critical (was: Blocker) > Result of DataFrame.join(usingColumns) could be wrong for outer

[jira] [Assigned] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12395: -- Assignee: Davies Liu > Result of DataFrame.join(usingColumns) could be wrong for outer join >

[jira] [Created] (SPARK-12402) Memory leak in broadcast hash join

Davies Liu created SPARK-12402: -- Summary: Memory leak in broadcast hash join Key: SPARK-12402 URL: https://issues.apache.org/jira/browse/SPARK-12402 Project: Spark Issue Type: Bug Comp

[jira] [Resolved] (SPARK-12395) Result of DataFrame.join(usingColumns) could be wrong for outer join

[ https://issues.apache.org/jira/browse/SPARK-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12395. Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull reque

[jira] [Updated] (SPARK-12402) Memory leak in pyspark

[ https://issues.apache.org/jira/browse/SPARK-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12402: --- Summary: Memory leak in pyspark (was: Memory leak in broadcast hash join) > Memory leak in pyspark >

[jira] [Updated] (SPARK-12402) Memory leak in pyspark

[ https://issues.apache.org/jira/browse/SPARK-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12402: --- Description: After run some SQL query in PySpark, the DataFrame are still referenced by py4j Gateway,

[jira] [Resolved] (SPARK-12054) Consider nullable in codegen

[ https://issues.apache.org/jira/browse/SPARK-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12054. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10333 [https://github.

[jira] [Assigned] (SPARK-12335) CentralMomentAgg should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12335: -- Assignee: Davies Liu (was: Apache Spark) > CentralMomentAgg should be nullable >

[jira] [Resolved] (SPARK-12335) CentralMomentAgg should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12335. Resolution: Fixed Fix Version/s: 2.0.0 https://github.com/apache/spark/pull/10333 > Central

[jira] [Assigned] (SPARK-12336) Outer join using multiple columns results in wrong nullability

[ https://issues.apache.org/jira/browse/SPARK-12336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12336: -- Assignee: Davies Liu (was: Cheng Lian) > Outer join using multiple columns results in wrong n

[jira] [Resolved] (SPARK-12336) Outer join using multiple columns results in wrong nullability

[ https://issues.apache.org/jira/browse/SPARK-12336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12336. Resolution: Fixed Fix Version/s: 2.0.0 https://github.com/apache/spark/pull/10333 > Outer j

[jira] [Assigned] (SPARK-12342) Corr (Pearson correlation) should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12342: -- Assignee: Davies Liu (was: Cheng Lian) > Corr (Pearson correlation) should be nullable >

[jira] [Assigned] (SPARK-12341) The "comment" field of DESCRIBE result set should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12341: -- Assignee: Davies Liu (was: Apache Spark) > The "comment" field of DESCRIBE result set should

[jira] [Resolved] (SPARK-12341) The "comment" field of DESCRIBE result set should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12341. Resolution: Fixed Fix Version/s: 2.0.0 https://github.com/apache/spark/pull/10333 > The "co

[jira] [Resolved] (SPARK-12342) Corr (Pearson correlation) should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12342. Resolution: Fixed Fix Version/s: 2.0.0 https://github.com/apache/spark/pull/10333 > Corr (P

[jira] [Resolved] (SPARK-12091) [PySpark] Removal of the JAVA-specific deserialized storage levels

[ https://issues.apache.org/jira/browse/SPARK-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12091. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10092 [https://github.

[jira] [Resolved] (SPARK-12388) Change default compressor to LZ4

[ https://issues.apache.org/jira/browse/SPARK-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12388. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10342 [https://github.

[jira] [Created] (SPARK-12472) OOM when sort a table and save as parquet

Davies Liu created SPARK-12472: -- Summary: OOM when sort a table and save as parquet Key: SPARK-12472 URL: https://issues.apache.org/jira/browse/SPARK-12472 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12472) OOM when sort a table and save as parquet

[ https://issues.apache.org/jira/browse/SPARK-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067413#comment-15067413 ] Davies Liu commented on SPARK-12472: could be workarounded by decreasing the memory u

[jira] [Resolved] (SPARK-12520) Python API dataframe join returns wrong results on outer join

[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12520. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10477 [https://github.

[jira] [Updated] (SPARK-12520) Python API dataframe join returns wrong results on outer join

[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12520: --- Fix Version/s: 1.6.0 > Python API dataframe join returns wrong results on outer join > --

[jira] [Updated] (SPARK-12520) Python API dataframe join returns wrong results on outer join

[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12520: --- Assignee: Xiao Li > Python API dataframe join returns wrong results on outer join > -

[jira] [Updated] (SPARK-12520) Python API dataframe join returns wrong results on outer join

[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12520: --- Fix Version/s: 1.5.3 > Python API dataframe join returns wrong results on outer join > --

[jira] [Created] (SPARK-12540) Support all TPCDS queries

Davies Liu created SPARK-12540: -- Summary: Support all TPCDS queries Key: SPARK-12540 URL: https://issues.apache.org/jira/browse/SPARK-12540 Project: Spark Issue Type: Epic Components:

[jira] [Created] (SPARK-12541) Support rollup/cube/povit in SQL query

Davies Liu created SPARK-12541: -- Summary: Support rollup/cube/povit in SQL query Key: SPARK-12541 URL: https://issues.apache.org/jira/browse/SPARK-12541 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-12542) Support intersect/except in SQL

Davies Liu created SPARK-12542: -- Summary: Support intersect/except in SQL Key: SPARK-12542 URL: https://issues.apache.org/jira/browse/SPARK-12542 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-12543) Support subquery in select/where/having

Davies Liu created SPARK-12543: -- Summary: Support subquery in select/where/having Key: SPARK-12543 URL: https://issues.apache.org/jira/browse/SPARK-12543 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-12545) Support exists condition

Davies Liu created SPARK-12545: -- Summary: Support exists condition Key: SPARK-12545 URL: https://issues.apache.org/jira/browse/SPARK-12545 Project: Spark Issue Type: New Feature Repo

[jira] [Created] (SPARK-12544) Support window functions in SQLContext

Davies Liu created SPARK-12544: -- Summary: Support window functions in SQLContext Key: SPARK-12544 URL: https://issues.apache.org/jira/browse/SPARK-12544 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12511: --- Assignee: Shixiong Zhu > streaming driver with checkpointing unable to finalize leading to OOM >

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Summary: Support union/intersect/except in Hive SQL (was: Support intersect/except in SQL) > Suppor

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Description: (was: Also `UNION ALL`in Hive SQL parser) > Support union/intersect/except in Hive S

[jira] [Commented] (SPARK-12542) Support union/intersect/except in Hive SQL

[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074368#comment-15074368 ] Davies Liu commented on SPARK-12542: [~viirya] [~xiaol] That's true, updated the titl

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Assignee: Xiao Li > Support union/intersect/except in Hive SQL >

[jira] [Commented] (SPARK-12544) Support window functions in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074369#comment-15074369 ] Davies Liu commented on SPARK-12544: [~hvanhovell] Right now, the SQL parser can't p

[jira] [Commented] (SPARK-12385) Push projection into Join

[ https://issues.apache.org/jira/browse/SPARK-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074383#comment-15074383 ] Davies Liu commented on SPARK-12385: [~smilegator] Thanks for working on this, could

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Summary: Support rollup/cube in SQL query (was: Support rollup/cube/povit in SQL query) > Support r

[jira] [Assigned] (SPARK-12541) Support rollup/cube in SQL query

[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12541: -- Assignee: Davies Liu > Support rollup/cube in SQL query > > >

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Description: We have DataFrame API for rollup/cube, but do not support them in SQL parser (both SQLCo

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Description: We have DataFrame API for rollup/cube, but do not support them in SQL parser (both SQLC

[jira] [Commented] (SPARK-11437) createDataFrame shouldn't .take() when provided schema

[ https://issues.apache.org/jira/browse/SPARK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074726#comment-15074726 ] Davies Liu commented on SPARK-11437: [~maver1ck] Maybe we should provide a API to ver

[jira] [Closed] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-11855. -- Resolution: Won't Fix > Catalyst breaks backwards compatibility in branch-1.6 > ---

[jira] [Assigned] (SPARK-12540) Support all TPCDS queries

[ https://issues.apache.org/jira/browse/SPARK-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12540: -- Assignee: Davies Liu > Support all TPCDS queries > - > >

[jira] [Commented] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

[ https://issues.apache.org/jira/browse/SPARK-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074729#comment-15074729 ] Davies Liu commented on SPARK-12291: Thanks, I will close this one. > Support Unsafe

[jira] [Closed] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

[ https://issues.apache.org/jira/browse/SPARK-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-12291. -- Resolution: Not A Problem > Support UnsafeRow in BroadcastLeftSemiJoinHash > --

[jira] [Created] (SPARK-12585) The numFields of UnsafeRow should not changed by pointTo()

Davies Liu created SPARK-12585: -- Summary: The numFields of UnsafeRow should not changed by pointTo() Key: SPARK-12585 URL: https://issues.apache.org/jira/browse/SPARK-12585 Project: Spark Issue

[jira] [Updated] (SPARK-12585) The numFields of UnsafeRow should not changed by pointTo()

[ https://issues.apache.org/jira/browse/SPARK-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12585: --- Description: Right now, numFields will be passed in by pointTo(), then bitSetWidthInBytes is calcula

[jira] [Assigned] (SPARK-12585) The numFields of UnsafeRow should not changed by pointTo()

[ https://issues.apache.org/jira/browse/SPARK-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12585: -- Assignee: Davies Liu > The numFields of UnsafeRow should not changed by pointTo() > --

[jira] [Updated] (SPARK-12300) Fix schema inferance on local collections

[ https://issues.apache.org/jira/browse/SPARK-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12300: --- Fix Version/s: (was: 1.6.0) 1.6.1 > Fix schema inferance on local collections

<    4   5   6   7   8   9   10   11   12   13   >