[jira] [Commented] (SPARK-19226) Report failure reason from Reporter Thread

[ https://issues.apache.org/jira/browse/SPARK-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862684#comment-15862684 ] Yuming Wang commented on SPARK-19226: - Try to increase ApplicationMaster's Java heap {{-- conf

[jira] [Assigned] (SPARK-19448) unify some duplication function in MetaStoreRelation

[ https://issues.apache.org/jira/browse/SPARK-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-19448: --- Assignee: Song Jun > unify some duplication function in MetaStoreRelation >

[jira] [Resolved] (SPARK-19448) unify some duplication function in MetaStoreRelation

[ https://issues.apache.org/jira/browse/SPARK-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19448. - Resolution: Fixed Fix Version/s: 2.2.0 > unify some duplication function in MetaStoreRelation >

[jira] [Commented] (SPARK-19532) [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true

[ https://issues.apache.org/jira/browse/SPARK-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862625#comment-15862625 ] StanZhai commented on SPARK-19532: -- We have been trying to upgrade our Spark from the releasing of Spark

[jira] [Updated] (SPARK-17897) not isnotnull is converted to the always false condition isnotnull && not isnotnull

[ https://issues.apache.org/jira/browse/SPARK-17897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17897: Fix Version/s: 2.0.3 > not isnotnull is converted to the always false condition isnotnull && not

[jira] [Commented] (SPARK-19557) Output parameters are not present in SQL Query Plan

[ https://issues.apache.org/jira/browse/SPARK-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862595#comment-15862595 ] Hyukjin Kwon commented on SPARK-19557: -- Could you describe "this information", input and output

[jira] [Commented] (SPARK-19354) Killed tasks are getting marked as FAILED

[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862588#comment-15862588 ] Zhan Zhang commented on SPARK-19354: This fix is actually critical. In production, we found that this

[jira] [Commented] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862562#comment-15862562 ] Wenchen Fan commented on SPARK-19477: - a simple workaround will be `ds.map(equality)`. > [SQL]

[jira] [Commented] (SPARK-19477) [SQL] Datasets created from a Dataframe with extra columns retain the extra columns

[ https://issues.apache.org/jira/browse/SPARK-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862560#comment-15862560 ] Don Drake commented on SPARK-19477: --- How does lazy apply here? If I read/create a dataframe with extra

[jira] [Commented] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862544#comment-15862544 ] Apache Spark commented on SPARK-19561: -- User 'JasonMWhite' has created a pull request for this

[jira] [Assigned] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19561: Assignee: Apache Spark > Pyspark Dataframes don't allow timestamps near epoch >

[jira] [Assigned] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19561: Assignee: (was: Apache Spark) > Pyspark Dataframes don't allow timestamps near epoch

[jira] [Created] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

Jason White created SPARK-19561: --- Summary: Pyspark Dataframes don't allow timestamps near epoch Key: SPARK-19561 URL: https://issues.apache.org/jira/browse/SPARK-19561 Project: Spark Issue

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862532#comment-15862532 ] Alex S commented on SPARK-19340: Looks like the error happens when spark tries to infers a schema from

[jira] [Commented] (SPARK-19557) Output parameters are not present in SQL Query Plan

[ https://issues.apache.org/jira/browse/SPARK-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862528#comment-15862528 ] Salil Surendran commented on SPARK-19557: - [~hyukjin.kwon] We don't have this information in the

[jira] [Commented] (SPARK-15615) Support for creating a dataframe from JSON in Dataset[String]

[ https://issues.apache.org/jira/browse/SPARK-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862496#comment-15862496 ] Apache Spark commented on SPARK-15615: -- User 'pjfanning' has created a pull request for this issue:

[jira] [Commented] (SPARK-15642) Metadata gets lost when selecting a field of a StructType

[ https://issues.apache.org/jira/browse/SPARK-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862467#comment-15862467 ] Christian Zommerfelds commented on SPARK-15642: --- Problem is still present in Spark

[jira] [Commented] (SPARK-9414) HiveContext:saveAsTable creates wrong partition for existing hive table(append mode)

[ https://issues.apache.org/jira/browse/SPARK-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862466#comment-15862466 ] Hyukjin Kwon commented on SPARK-9414: - [~xguo27], could you resolve this as {{Cannot Reproduce}} if

[jira] [Comment Edited] (SPARK-15642) Metadata gets lost when selecting a field of a StructType

[ https://issues.apache.org/jira/browse/SPARK-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862467#comment-15862467 ] Christian Zommerfelds edited comment on SPARK-15642 at 2/11/17 5:48 PM:

[jira] [Updated] (SPARK-15642) Metadata gets lost when selecting a field of a StructType

[ https://issues.apache.org/jira/browse/SPARK-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Zommerfelds updated SPARK-15642: -- Affects Version/s: 2.2.0 > Metadata gets lost when selecting a field of a

[jira] [Resolved] (SPARK-8081) Problems with Optimized Logical Plan Generation

[ https://issues.apache.org/jira/browse/SPARK-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8081. - Resolution: Invalid I hope there are no such JIRAs, like, copying and just pasting the production

[jira] [Commented] (SPARK-7101) Spark SQL should support java.sql.Time

[ https://issues.apache.org/jira/browse/SPARK-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862457#comment-15862457 ] Hyukjin Kwon commented on SPARK-7101: - Could anyone explain the use case of this type? > Spark SQL

[jira] [Commented] (SPARK-6678) select count(DISTINCT C_UID) from parquetdir may be can optimize

[ https://issues.apache.org/jira/browse/SPARK-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862455#comment-15862455 ] Hyukjin Kwon commented on SPARK-6678: - [~cnstar9988], do you mind if I ask your other spark code? I

[jira] [Commented] (SPARK-6628) ClassCastException occurs when executing sql statement "insert into" on hbase table

[ https://issues.apache.org/jira/browse/SPARK-6628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862454#comment-15862454 ] Hyukjin Kwon commented on SPARK-6628: - Hi all, then, is this not an Spark issue? > ClassCastException

[jira] [Commented] (SPARK-19532) [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true

[ https://issues.apache.org/jira/browse/SPARK-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862443#comment-15862443 ] Hyukjin Kwon commented on SPARK-19532: -- Is it a blocker BTW? > [Core]`DataStreamer for file`

[jira] [Commented] (SPARK-19557) Output parameters are not present in SQL Query Plan

[ https://issues.apache.org/jira/browse/SPARK-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862371#comment-15862371 ] Hyukjin Kwon commented on SPARK-19557: -- I mean, up to my understanding, we do have path in the query

[jira] [Commented] (SPARK-19557) Output parameters are not present in SQL Query Plan

[ https://issues.apache.org/jira/browse/SPARK-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862368#comment-15862368 ] Hyukjin Kwon commented on SPARK-19557: -- I am sorry that I don't follow the JIRA. Could I please ask