[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Jose Torres (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388399#comment-16388399 ] Jose Torres commented on SPARK-23325: - How hard would it be to just declare that InternalRow is

[jira] [Commented] (SPARK-23613) Different Analyzed logical plan data types for the same table in different queries

2018-03-06 Thread Ramandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388494#comment-16388494 ] Ramandeep Singh commented on SPARK-23613: - To add to it, the query works fine with subquery

[jira] [Commented] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388230#comment-16388230 ] Marcelo Vanzin commented on SPARK-23607: I think this is a nice trick to speed things up, even

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388322#comment-16388322 ] Wenchen Fan commented on SPARK-23325: - The problem is that, `Row` is a stable class Spark promises it

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388223#comment-16388223 ] Marcelo Vanzin commented on SPARK-18673: We can't close this because Spark is not using the

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388411#comment-16388411 ] Ryan Blue commented on SPARK-23325: --- I agree that we should declare \{{InternalRow}} stable. It is

[jira] [Created] (SPARK-23613) Different Analyzed logical plan data types for the same table in different queries

2018-03-06 Thread Ramandeep Singh (JIRA)
Ramandeep Singh created SPARK-23613: --- Summary: Different Analyzed logical plan data types for the same table in different queries Key: SPARK-23613 URL: https://issues.apache.org/jira/browse/SPARK-23613

[jira] [Updated] (SPARK-23614) Union produces incorrect results when caching is used

2018-03-06 Thread Morten Hornbech (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Morten Hornbech updated SPARK-23614: Description: We just upgraded from 2.2 to 2.3 and our test suite caught this error:

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388742#comment-16388742 ] Herman van Hovell commented on SPARK-23582: --- That is a good start! I am just wondering if

[jira] [Updated] (SPARK-23609) Test EnsureRequirements's test cases to eliminate ShuffleExchange while is not expected

2018-03-06 Thread caoxuewen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caoxuewen updated SPARK-23609: -- Summary: Test EnsureRequirements's test cases to eliminate ShuffleExchange while is not expected

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388520#comment-16388520 ] Wenchen Fan commented on SPARK-23325: - Making `InternalRow` stable is not only about stabilizing the

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388729#comment-16388729 ] Kazuaki Ishizaki commented on SPARK-23582: -- I see. Now, I have a prototype using old-school

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-03-06 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388757#comment-16388757 ] Darek commented on SPARK-18673: --- When running the pyspark tests using Hadoop 3.0.0 I am not getting the 

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-06 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388761#comment-16388761 ] Saisai Shao commented on SPARK-23534: - I don't think so. Spark uses its own fork hive version

[jira] [Created] (SPARK-23615) Add maxDF Parameter to Python CountVectorizer

2018-03-06 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-23615: Summary: Add maxDF Parameter to Python CountVectorizer Key: SPARK-23615 URL: https://issues.apache.org/jira/browse/SPARK-23615 Project: Spark Issue Type:

[jira] [Created] (SPARK-23616) Streaming self-join using SQL throws resolution exceptions

2018-03-06 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23616: - Summary: Streaming self-join using SQL throws resolution exceptions Key: SPARK-23616 URL: https://issues.apache.org/jira/browse/SPARK-23616 Project: Spark

[jira] [Created] (SPARK-23614) Union produces incorrect results when caching is used

2018-03-06 Thread Morten Hornbech (JIRA)
Morten Hornbech created SPARK-23614: --- Summary: Union produces incorrect results when caching is used Key: SPARK-23614 URL: https://issues.apache.org/jira/browse/SPARK-23614 Project: Spark

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388707#comment-16388707 ] Takeshi Yamamuro commented on SPARK-23595: -- ok, If you need help in other tickets, please let me

[jira] [Updated] (SPARK-23615) Add maxDF Parameter to Python CountVectorizer

2018-03-06 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-23615: - Component/s: ML > Add maxDF Parameter to Python CountVectorizer >

[jira] [Resolved] (SPARK-23616) Streaming self-join using SQL throws resolution exceptions

2018-03-06 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23616. --- Resolution: Duplicate This is the same underlying issue as SPARK-23406. However the error is

[jira] [Comment Edited] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-06 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388761#comment-16388761 ] Saisai Shao edited comment on SPARK-23534 at 3/7/18 12:37 AM: -- I don't think

[jira] [Assigned] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23582: Assignee: Kazuaki Ishizaki (was: Apache Spark) > Add interpreted execution to

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388792#comment-16388792 ] Apache Spark commented on SPARK-23582: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23582: Assignee: Apache Spark (was: Kazuaki Ishizaki) > Add interpreted execution to

[jira] [Commented] (SPARK-23406) Stream-stream self joins does not work

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388898#comment-16388898 ] Apache Spark commented on SPARK-23406: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23287) Spark scheduler does not remove initial executor if not one job submitted

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23287: Assignee: Apache Spark > Spark scheduler does not remove initial executor if not one job

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Description:   !2018-03-07_121010.png!   when the hive session closed, we should also cleanup 

[jira] [Commented] (SPARK-23584) Add interpreted execution to NewInstance expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389140#comment-16389140 ] Takeshi Yamamuro commented on SPARK-23584: -- I'm working on it. > Add interpreted execution to

[jira] [Updated] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-23617: Description: One can register a function using Scala: {{spark.udf.register("uuid",

[jira] [Updated] (SPARK-23618) docker-image-tool.sh Fails While Building Image

2018-03-06 Thread Ninad Ingole (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ninad Ingole updated SPARK-23618: - Description: I am trying to build kubernetes image for version 2.3.0, using  {code:java}

[jira] [Updated] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-23617: Description: One can register a function using Scala: spark.udf.register("uuid",

[jira] [Updated] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-23617: Description: One can register a function using Scala: {{spark.udf.register("uuid",

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2018-03-06 Thread Franck Tago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389150#comment-16389150 ] Franck Tago commented on SPARK-23519: - Any updates on this ? > Create View Commands Fails with The

[jira] [Commented] (SPARK-23287) Spark scheduler does not remove initial executor if not one job submitted

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1635#comment-1635 ] Apache Spark commented on SPARK-23287: -- User 'devaraj-kavali' has created a pull request for this

[jira] [Assigned] (SPARK-23287) Spark scheduler does not remove initial executor if not one job submitted

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23287: Assignee: (was: Apache Spark) > Spark scheduler does not remove initial executor if

[jira] [Updated] (SPARK-23495) Creating a json file using a dataframe Generates an issue

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23495: -- Target Version/s: (was: 2.1.0) Fix Version/s: (was: 2.1.0) > Creating a json file using a

[jira] [Updated] (SPARK-23607) Use HDFS extended attributes to store application summary to improve the Spark History Server performance

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23607: -- Target Version/s: (was: 2.4.0) Priority: Minor (was: Major) Fix Version/s:

[jira] [Resolved] (SPARK-23495) Creating a json file using a dataframe Generates an issue

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23495. --- Resolution: Invalid You've just listed some code and output and not described a problem. > Creating

[jira] [Updated] (SPARK-23499) Mesos Cluster Dispatcher should support priority queues to submit drivers

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23499: -- Affects Version/s: (was: 2.4.0) Target Version/s: (was: 2.2.1, 2.2.2, 2.3.0, 2.3.1)

[jira] [Resolved] (SPARK-21795) Broadcast hint ignored when dataframe is cached

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21795. --- Resolution: Duplicate > Broadcast hint ignored when dataframe is cached >

[jira] [Commented] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388954#comment-16388954 ] Hyukjin Kwon commented on SPARK-23617: -- Is this a duplicate of SPARK-19285? and does this work in

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Description:     when the hive session closed, we should also cleanup the .pipeout file.    

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Attachment: (was: 2018-03-01_202415.png) > Cleanup the .pipeout file when the Hive Session

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Attachment: 2018-03-07_121010.png > Cleanup the .pipeout file when the Hive Session closed >

[jira] [Resolved] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu resolved SPARK-23617. - Resolution: Duplicate Fix Version/s: 2.3.0 As commented by Hyukjin Kwon, the issue is duplicated

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389135#comment-16389135 ] Reynold Xin commented on SPARK-23325: - Yes perhaps we should do that. It is a lot more work than what

[jira] [Assigned] (SPARK-23593) Add interpreted execution for InitializeJavaBean expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23593: Assignee: Apache Spark > Add interpreted execution for InitializeJavaBean expression >

[jira] [Commented] (SPARK-23593) Add interpreted execution for InitializeJavaBean expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388938#comment-16388938 ] Apache Spark commented on SPARK-23593: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23593) Add interpreted execution for InitializeJavaBean expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23593: Assignee: (was: Apache Spark) > Add interpreted execution for InitializeJavaBean

[jira] [Assigned] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23595: Assignee: (was: Apache Spark) > Add interpreted execution for ValidateExternalType

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389058#comment-16389058 ] Apache Spark commented on SPARK-23595: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23595: Assignee: Apache Spark > Add interpreted execution for ValidateExternalType expression >

[jira] [Created] (SPARK-23618) docker-image-tool.sh Fails While Building Image

2018-03-06 Thread Ninad Ingole (JIRA)
Ninad Ingole created SPARK-23618: Summary: docker-image-tool.sh Fails While Building Image Key: SPARK-23618 URL: https://issues.apache.org/jira/browse/SPARK-23618 Project: Spark Issue Type:

[jira] [Created] (SPARK-23617) Register a Function without params with Spark SQL Java API

2018-03-06 Thread Paul Wu (JIRA)
Paul Wu created SPARK-23617: --- Summary: Register a Function without params with Spark SQL Java API Key: SPARK-23617 URL: https://issues.apache.org/jira/browse/SPARK-23617 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23266) Matrix Inversion on BlockMatrix

2018-03-06 Thread Chandan Misra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389061#comment-16389061 ] Chandan Misra commented on SPARK-23266: --- I have implemented matrix inversion using Spark version

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Attachment: 2018-03-07_121010.png > Cleanup the .pipeout file when the Hive Session closed >

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Attachment: (was: 2018-03-07_121010.png) > Cleanup the .pipeout file when the Hive Session

[jira] [Updated] (SPARK-23547) Cleanup the .pipeout file when the Hive Session closed

2018-03-06 Thread zuotingbing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-23547: Description:   !2018-03-07_121010.png! when the hive session closed, we should also cleanup the

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388114#comment-16388114 ] Takeshi Yamamuro commented on SPARK-23595: -- [~DylanGuedes] oh, I'm already working on it. But,

[jira] [Commented] (SPARK-23559) add epoch ID to data writer factory

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388122#comment-16388122 ] Apache Spark commented on SPARK-23559: -- User 'jose-torres' has created a pull request for this

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388136#comment-16388136 ] Dylan Guedes commented on SPARK-23595: -- [~maropu] I checked your progress, and looks like you are 

[jira] [Updated] (SPARK-23610) Cast of ArrayType of NullType to ArrayType of nullable material type does not work

2018-03-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-23610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Kießling updated SPARK-23610: - Description: Given a DataFrame that contains a column with _ArrayType of NullType_ casting this

[jira] [Commented] (SPARK-23537) Logistic Regression without standardization

2018-03-06 Thread Jordi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387587#comment-16387587 ] Jordi commented on SPARK-23537: --- [~Teng Peng] we don't need standardization for L-BFGS but it's recommended

[jira] [Created] (SPARK-23611) Extend ExpressionEvalHelper harness to also test failures

2018-03-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-23611: - Summary: Extend ExpressionEvalHelper harness to also test failures Key: SPARK-23611 URL: https://issues.apache.org/jira/browse/SPARK-23611 Project: Spark

[jira] [Commented] (SPARK-22450) Safely register class for mllib

2018-03-06 Thread Richard Wilkinson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387630#comment-16387630 ] Richard Wilkinson commented on SPARK-22450: --- Just as an FYI, the change to 

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-06 Thread imran shaik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387969#comment-16387969 ] imran shaik commented on SPARK-18492: - truetradescast schema root |-- Event_Time: long (nullable =

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

2018-03-06 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387963#comment-16387963 ] Thomas Graves commented on SPARK-22683: --- I left comments on the open PR already, lets move the

[jira] [Assigned] (SPARK-23591) Add interpreted execution for EncodeUsingSerializer expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23591: Assignee: (was: Apache Spark) > Add interpreted execution for EncodeUsingSerializer

[jira] [Commented] (SPARK-23591) Add interpreted execution for EncodeUsingSerializer expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387998#comment-16387998 ] Apache Spark commented on SPARK-23591: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23591) Add interpreted execution for EncodeUsingSerializer expression

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23591: Assignee: Apache Spark > Add interpreted execution for EncodeUsingSerializer expression >

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-03-06 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388014#comment-16388014 ] Darek commented on SPARK-18673: --- HIVE tickets are closed already, can we close this ticket? > Dataframes

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387864#comment-16387864 ] Herman van Hovell commented on SPARK-23595: --- [~DylanGuedes] feel free to pick this up. I think

[jira] [Commented] (SPARK-23581) Add an interpreted version of GenerateUnsafeProjection

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387872#comment-16387872 ] Apache Spark commented on SPARK-23581: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23601) Remove .md5 files from release

2018-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23601. --- Resolution: Fixed Fix Version/s: 2.4.0 2.3.1 Resolved by 

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

2018-03-06 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387938#comment-16387938 ] Thomas Graves commented on SPARK-22683: --- [~jcuquemelle] do you have time to update the PR,

[jira] [Commented] (SPARK-22683) DynamicAllocation wastes resources by allocating containers that will barely be used

2018-03-06 Thread Julien Cuquemelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387947#comment-16387947 ] Julien Cuquemelle commented on SPARK-22683: --- Yes, I have time. I was waiting for suggestions

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-06 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388017#comment-16388017 ] Darek commented on SPARK-23534: --- SPARK-18673 should be closed since HIVE-15016 and HIVE-18550 are closed.

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387853#comment-16387853 ] Dylan Guedes commented on SPARK-23595: -- Hi, I would like to help with this issue, but since I am a

[jira] [Commented] (SPARK-17495) Hive hash implementation

2018-03-06 Thread Xiaoju Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387873#comment-16387873 ] Xiaoju Wu commented on SPARK-17495: --- [~tejasp] I can see HiveHash merged but never used. Seems the

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

2018-03-06 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387846#comment-16387846 ] Thomas Graves commented on SPARK-16630: --- yes something along these lines is what I was thinking. we 

[jira] [Comment Edited] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387853#comment-16387853 ] Dylan Guedes edited comment on SPARK-23595 at 3/6/18 2:24 PM: -- Hi, I would

[jira] [Commented] (SPARK-21048) Add an option --merged-properties-file to distinguish the configuration loading behavior

2018-03-06 Thread Yaqub Alwan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388045#comment-16388045 ] Yaqub Alwan commented on SPARK-21048: - Commenting here because SPARK-21023 was closed and contrary to

[jira] [Created] (SPARK-23612) Specify formats for individual DateType and TimestampType columns in schemas

2018-03-06 Thread Patrick Young (JIRA)
Patrick Young created SPARK-23612: - Summary: Specify formats for individual DateType and TimestampType columns in schemas Key: SPARK-23612 URL: https://issues.apache.org/jira/browse/SPARK-23612

[jira] [Updated] (SPARK-23612) Specify formats for individual DateType and TimestampType columns in schemas

2018-03-06 Thread Patrick Young (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Young updated SPARK-23612: -- Description:

[jira] [Assigned] (SPARK-23591) Add interpreted execution for EncodeUsingSerializer expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23591: - Assignee: Marco Gaido > Add interpreted execution for EncodeUsingSerializer

[jira] [Resolved] (SPARK-23590) Add interpreted execution for CreateExternalRow expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23590. --- Resolution: Fixed Fix Version/s: 2.4.0 > Add interpreted execution for

[jira] [Commented] (SPARK-23346) Failed tasks reported as success if the failure reason is not ExceptionFailure

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388088#comment-16388088 ] Herman van Hovell commented on SPARK-23346: --- [~wuzhilon88] This is completely un-actionable.

[jira] [Commented] (SPARK-23592) Add interpreted execution for DecodeUsingSerializer expression

2018-03-06 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388075#comment-16388075 ] Marco Gaido commented on SPARK-23592: - I will submit a PR as soon as SPARK-23591 gets merged, thanks

[jira] [Assigned] (SPARK-23592) Add interpreted execution for DecodeUsingSerializer expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23592: - Assignee: Marco Gaido > Add interpreted execution for DecodeUsingSerializer

[jira] [Updated] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-23580: -- Labels: release-notes (was: ) > Interpreted mode fallback should be implemented for

[jira] [Created] (SPARK-23609) Test code does not conform to the test title

2018-03-06 Thread caoxuewen (JIRA)
caoxuewen created SPARK-23609: - Summary: Test code does not conform to the test title Key: SPARK-23609 URL: https://issues.apache.org/jira/browse/SPARK-23609 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-23609) Test code does not conform to the test title

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23609: Assignee: Apache Spark > Test code does not conform to the test title >

[jira] [Commented] (SPARK-23609) Test code does not conform to the test title

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387470#comment-16387470 ] Apache Spark commented on SPARK-23609: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23609) Test code does not conform to the test title

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23609: Assignee: (was: Apache Spark) > Test code does not conform to the test title >

[jira] [Comment Edited] (SPARK-20162) Reading data from MySQL - Cannot up cast from decimal(30,6) to decimal(38,18)

2018-03-06 Thread Caio Quirino da Silva (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383471#comment-16383471 ] Caio Quirino da Silva edited comment on SPARK-20162 at 3/6/18 11:53 AM:

[jira] [Resolved] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23594. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 >

[jira] [Commented] (SPARK-23590) Add interpreted execution for CreateExternalRow expression

2018-03-06 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387721#comment-16387721 ] Marco Gaido commented on SPARK-23590: - I am working on this > Add interpreted execution for

[jira] [Comment Edited] (SPARK-20162) Reading data from MySQL - Cannot up cast from decimal(30,6) to decimal(38,18)

2018-03-06 Thread Caio Quirino da Silva (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387661#comment-16387661 ] Caio Quirino da Silva edited comment on SPARK-20162 at 3/6/18 12:08 PM:

[jira] [Assigned] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23582: - Assignee: Kazuaki Ishizaki > Add interpreted execution to StaticInvoke

[jira] [Assigned] (SPARK-23611) Extend ExpressionEvalHelper harness to also test failures

2018-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23611: Assignee: Apache Spark > Extend ExpressionEvalHelper harness to also test failures >

  1   2   >