[jira] [Assigned] (SPARK-17317) Add package vignette to SparkR

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17317: Assignee: (was: Apache Spark) > Add package vignette to SparkR >

[jira] [Updated] (SPARK-17416) Add Dataset.groupByKey overload that takes a value selector function

2016-09-06 Thread Daniel Shields (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Shields updated SPARK-17416: --- Description: I propose that the following overload be added to Dataset[T]: def

[jira] [Created] (SPARK-17416) Add Dataset.groupByKey overload that takes a value selector function

2016-09-06 Thread Daniel Shields (JIRA)
Daniel Shields created SPARK-17416: -- Summary: Add Dataset.groupByKey overload that takes a value selector function Key: SPARK-17416 URL: https://issues.apache.org/jira/browse/SPARK-17416 Project:

[jira] [Assigned] (SPARK-17415) Better error message for driver-side broadcast join OOMs

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17415: Assignee: (was: Apache Spark) > Better error message for driver-side broadcast join

[jira] [Assigned] (SPARK-17415) Better error message for driver-side broadcast join OOMs

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17415: Assignee: Apache Spark > Better error message for driver-side broadcast join OOMs >

[jira] [Commented] (SPARK-17415) Better error message for driver-side broadcast join OOMs

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468210#comment-15468210 ] Apache Spark commented on SPARK-17415: -- User 'sameeragarwal' has created a pull request for this

[jira] [Created] (SPARK-17415) Better error message for driver-side broadcast join OOMs

2016-09-06 Thread Sameer Agarwal (JIRA)
Sameer Agarwal created SPARK-17415: -- Summary: Better error message for driver-side broadcast join OOMs Key: SPARK-17415 URL: https://issues.apache.org/jira/browse/SPARK-17415 Project: Spark

[jira] [Commented] (SPARK-17408) Flaky test: org.apache.spark.sql.hive.StatisticsSuite

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468186#comment-15468186 ] Apache Spark commented on SPARK-17408: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17408) Flaky test: org.apache.spark.sql.hive.StatisticsSuite

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17408: Assignee: (was: Apache Spark) > Flaky test: org.apache.spark.sql.hive.StatisticsSuite

[jira] [Assigned] (SPARK-17408) Flaky test: org.apache.spark.sql.hive.StatisticsSuite

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17408: Assignee: Apache Spark > Flaky test: org.apache.spark.sql.hive.StatisticsSuite >

[jira] [Resolved] (SPARK-16922) Query with Broadcast Hash join fails due to executor OOM in Spark 2.0

2016-09-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-16922. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Resolved] (SPARK-17211) Broadcast join produces incorrect results when compressed Oops differs between driver, executor

2016-09-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17211. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17396) Threads number keep increasing when query on external CSV partitioned table

2016-09-06 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467956#comment-15467956 ] Ryan Blue commented on SPARK-17396: --- I'll put together a patch for this with a shared executor service.

[jira] [Created] (SPARK-17414) Set type is not supported for creating data frames

2016-09-06 Thread Emre Colak (JIRA)
Emre Colak created SPARK-17414: -- Summary: Set type is not supported for creating data frames Key: SPARK-17414 URL: https://issues.apache.org/jira/browse/SPARK-17414 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17409) Query in CTAS is Optimized Twice

2016-09-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-17409: --- Assignee: Xiao Li > Query in CTAS is Optimized Twice > > >

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-06 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467792#comment-15467792 ] Josh Rosen commented on SPARK-17405: [~qifan], I believe that you may be able to work around the

[jira] [Resolved] (SPARK-17411) Cannot set fromOffsets in createDirectStream function

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17411. --- Resolution: Duplicate > Cannot set fromOffsets in createDirectStream function >

[jira] [Reopened] (SPARK-17411) Cannot set fromOffsets in createDirectStream function

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-17411: --- "Fixed" is not the right resolution -- Duplicate is more useful. > Cannot set fromOffsets in

[jira] [Closed] (SPARK-17411) Cannot set fromOffsets in createDirectStream function

2016-09-06 Thread Piotr Milanowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Milanowski closed SPARK-17411. Resolution: Fixed Duplicate of https://issues.apache.org/jira/browse/SPARK-16950 > Cannot

[jira] [Commented] (SPARK-3261) KMeans clusterer can return duplicate cluster centers

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467564#comment-15467564 ] Apache Spark commented on SPARK-3261: - User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-6235) Address various 2G limits

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467563#comment-15467563 ] Apache Spark commented on SPARK-6235: - User 'witgo' has created a pull request for this issue:

[jira] [Commented] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Carl Zmola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467544#comment-15467544 ] Carl Zmola commented on SPARK-17413: Our comments crossed paths. It doesn't work in scala 2.9.2. or

[jira] [Updated] (SPARK-17374) Improves the error message when fails to parse some json file lines in DataFrameReader

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17374: Assignee: Sean Zhong > Improves the error message when fails to parse some json file lines in >

[jira] [Comment Edited] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Carl Zmola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467511#comment-15467511 ] Carl Zmola edited comment on SPARK-17413 at 9/6/16 2:21 PM: I don't know. Is

[jira] [Resolved] (SPARK-17374) Improves the error message when fails to parse some json file lines in DataFrameReader

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17374. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14929

[jira] [Commented] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Carl Zmola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467525#comment-15467525 ] Carl Zmola commented on SPARK-17413: The problem exists upstream with the Scala shell. I will file a

[jira] [Commented] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467527#comment-15467527 ] Sean Owen commented on SPARK-17413: --- I was going to say the Scala shell though that seems to work (in

[jira] [Commented] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Carl Zmola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467511#comment-15467511 ] Carl Zmola commented on SPARK-17413: I don't know. Is there an upstream project that I can check

[jira] [Commented] (SPARK-17412) FsHistoryProviderSuite - FAILED

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467501#comment-15467501 ] Sean Owen commented on SPARK-17412: --- It's not failing in the Spark CI environment, which suggests the

[jira] [Commented] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467498#comment-15467498 ] Sean Owen commented on SPARK-17413: --- Is this a Spark issue though? > spark-shell loses gnu readline

[jira] [Created] (SPARK-17413) spark-shell loses gnu readline support after suspend and continue

2016-09-06 Thread Carl Zmola (JIRA)
Carl Zmola created SPARK-17413: -- Summary: spark-shell loses gnu readline support after suspend and continue Key: SPARK-17413 URL: https://issues.apache.org/jira/browse/SPARK-17413 Project: Spark

[jira] [Comment Edited] (SPARK-17412) FsHistoryProviderSuite - FAILED

2016-09-06 Thread Amita Chaudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467479#comment-15467479 ] Amita Chaudhary edited comment on SPARK-17412 at 9/6/16 2:04 PM: - yes, it

[jira] [Commented] (SPARK-17412) FsHistoryProviderSuite - FAILED

2016-09-06 Thread Amita Chaudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467479#comment-15467479 ] Amita Chaudhary commented on SPARK-17412: - yes, it is failing consistently for me, is there any

[jira] [Commented] (SPARK-17411) Cannot set fromOffsets in createDirectStream function

2016-09-06 Thread Piotr Milanowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467469#comment-15467469 ] Piotr Milanowski commented on SPARK-17411: -- I'll just add that I am using Python 3.5 and

[jira] [Commented] (SPARK-17412) FsHistoryProviderSuite - FAILED

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467470#comment-15467470 ] Sean Owen commented on SPARK-17412: --- Does this fail consistently? It seems to pass in master. It could

[jira] [Assigned] (SPARK-17306) QuantileSummaries doesn't compress

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17306: Assignee: Apache Spark > QuantileSummaries doesn't compress >

[jira] [Commented] (SPARK-17306) QuantileSummaries doesn't compress

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467466#comment-15467466 ] Apache Spark commented on SPARK-17306: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17306) QuantileSummaries doesn't compress

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17306: Assignee: (was: Apache Spark) > QuantileSummaries doesn't compress >

[jira] [Updated] (SPARK-17306) QuantileSummaries doesn't compress

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17306: -- Summary: QuantileSummaries doesn't compress (was: Memory leak in QuantileSummaries) >

[jira] [Created] (SPARK-17412) FsHistoryProviderSuite - FAILED

2016-09-06 Thread Amita Chaudhary (JIRA)
Amita Chaudhary created SPARK-17412: --- Summary: FsHistoryProviderSuite - FAILED Key: SPARK-17412 URL: https://issues.apache.org/jira/browse/SPARK-17412 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-17411) Cannot set fromOffsets in createDirectStream function

2016-09-06 Thread Piotr Milanowski (JIRA)
Piotr Milanowski created SPARK-17411: Summary: Cannot set fromOffsets in createDirectStream function Key: SPARK-17411 URL: https://issues.apache.org/jira/browse/SPARK-17411 Project: Spark

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467322#comment-15467322 ] Sean Owen commented on SPARK-17381: --- Yeah, I didn't mean disable particular types of stats for

[jira] [Updated] (SPARK-17356) A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17356: Fix Version/s: 1.6.3 > A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

[jira] [Commented] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

2016-09-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467247#comment-15467247 ] Steve Loughran commented on SPARK-17307: It's not yet in there. If you got the SPARK-7481 JIRA

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

2016-09-06 Thread Joao Duarte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467224#comment-15467224 ] Joao Duarte commented on SPARK-17381: - Oh, I see. I'll change the Issue type from Bug to Improvement

[jira] [Updated] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

2016-09-06 Thread Joao Duarte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joao Duarte updated SPARK-17381: Issue Type: Improvement (was: Bug) > Memory leak

[jira] [Resolved] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-11301. - Resolution: Fixed Fix Version/s: (was: 1.6.0) 1.6.2 > filter on

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467108#comment-15467108 ] Sean Owen commented on SPARK-17381: --- The issue is that it's maintaining min/max stats for columns,

[jira] [Commented] (SPARK-17381) Memory leak org.apache.spark.sql.execution.ui.SQLTaskMetrics

2016-09-06 Thread Joao Duarte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467102#comment-15467102 ] Joao Duarte commented on SPARK-17381: - Well, the application is stable after 24h+ (and running). If

[jira] [Resolved] (SPARK-8813) Combine files when there're many small files in table

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8813. -- Resolution: Duplicate Assignee: (was: Michael Armbrust) Fix Version/s: (was:

[jira] [Reopened] (SPARK-8813) Combine files when there're many small files in table

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-8813: -- > Combine files when there're many small files in table >

[jira] [Comment Edited] (SPARK-8813) Combine files when there're many small files in table

2016-09-06 Thread Harsh J (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467084#comment-15467084 ] Harsh J edited comment on SPARK-8813 at 9/6/16 10:37 AM: - Note that this was done

[jira] [Commented] (SPARK-8813) Combine files when there're many small files in table

2016-09-06 Thread Harsh J (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467084#comment-15467084 ] Harsh J commented on SPARK-8813: Note that this was done instead by SPARK-13664 and should me marked

[jira] [Commented] (SPARK-17356) A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467044#comment-15467044 ] Apache Spark commented on SPARK-17356: -- User 'clockfly' has created a pull request for this issue:

[jira] [Resolved] (SPARK-12844) Spark documentation should be more precise about the algebraic properties of functions in various transformations

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12844. --- Resolution: Duplicate > Spark documentation should be more precise about the algebraic properties of

[jira] [Commented] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466957#comment-15466957 ] Nick Pentreath commented on SPARK-17400: Could you explain further why you want to min-max scale

[jira] [Commented] (SPARK-12844) Spark documentation should be more precise about the algebraic properties of functions in various transformations

2016-09-06 Thread Jagadeesan A S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466916#comment-15466916 ] Jagadeesan A S commented on SPARK-12844: The algebraic properties have already been taken care by

[jira] [Commented] (SPARK-17400) MinMaxScaler.transform() outputs DenseVector by default, which causes poor performance

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466905#comment-15466905 ] Sean Owen commented on SPARK-17400: --- [~mlnick] is right -- scaling any sparse representation is going

[jira] [Updated] (SPARK-17397) Show example of what to do when awaitTermination() throws an Exception

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17397: -- Component/s: Documentation Issue Type: Improvement (was: Question) Summary: Show example

[jira] [Commented] (SPARK-17397) what to do when awaitTermination() throws?

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466898#comment-15466898 ] Sean Owen commented on SPARK-17397: --- I think it's reasonable to show the try { awaitTermination() }

[jira] [Issue Comment Deleted] (SPARK-12844) Spark documentation should be more precise about the algebraic properties of functions in various transformations

2016-09-06 Thread Jagadeesan A S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagadeesan A S updated SPARK-12844: --- Comment: was deleted (was: Started working on this.) > Spark documentation should be more

[jira] [Commented] (SPARK-17396) Threads number keep increasing when query on external CSV partitioned table

2016-09-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466879#comment-15466879 ] Sean Owen commented on SPARK-17396: --- Yeah [~rdblue] is right on the mark then. I agree, I wasn't clear

[jira] [Commented] (SPARK-5091) Hooks for PySpark tasks

2016-09-06 Thread Semet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466855#comment-15466855 ] Semet commented on SPARK-5091: -- It is a better option to use virtualenv and proper installation with pip,

[jira] [Assigned] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11301: Assignee: Wenchen Fan (was: Apache Spark) > filter on partitioned column is case

[jira] [Assigned] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11301: Assignee: Apache Spark (was: Wenchen Fan) > filter on partitioned column is case

[jira] [Resolved] (SPARK-17356) A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17356. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > A large Metadata filed

[jira] [Updated] (SPARK-17356) A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17356: Assignee: Sean Zhong > A large Metadata filed in Alias can cause OOM when calling TreeNode.toJSON

[jira] [Resolved] (SPARK-17369) MetastoreRelation toJSON throws exception

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17369. - Resolution: Fixed Fix Version/s: 2.0.1 > MetastoreRelation toJSON throws exception >

[jira] [Reopened] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2016-09-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-11301: --- Please see the followings.

[jira] [Assigned] (SPARK-17410) Move Hive-generated Stats Info to HiveClientImpl

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17410: Assignee: (was: Apache Spark) > Move Hive-generated Stats Info to HiveClientImpl >

[jira] [Commented] (SPARK-17284) Remove statistics-related table properties from SHOW CREATE TABLE

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466688#comment-15466688 ] Apache Spark commented on SPARK-17284: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17410) Move Hive-generated Stats Info to HiveClientImpl

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17410: Assignee: Apache Spark > Move Hive-generated Stats Info to HiveClientImpl >

[jira] [Created] (SPARK-17410) Move Hive-generated Stats Info to HiveClientImpl

2016-09-06 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17410: --- Summary: Move Hive-generated Stats Info to HiveClientImpl Key: SPARK-17410 URL: https://issues.apache.org/jira/browse/SPARK-17410 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17405) Simple aggregation query OOMing after SPARK-16525

2016-09-06 Thread Qifan Pu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466676#comment-15466676 ] Qifan Pu commented on SPARK-17405: -- [~joshrosen] Thanks for reporting. I haven't been able to reproduce

[jira] [Commented] (SPARK-11301) filter on partitioned column is case sensitive even the context is case insensitive

2016-09-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1546#comment-1546 ] Apache Spark commented on SPARK-11301: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-17307) Document what all access is needed on S3 bucket when trying to save a model

2016-09-06 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1540#comment-1540 ] Aseem Bansal commented on SPARK-17307: -- Not adding it there would be fine. But there needs to be

[jira] [Resolved] (SPARK-17361) file-based external table without path should not be created

2016-09-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17361. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14921

<    1   2