[jira] [Resolved] (SPARK-23384) When it has no incomplete(completed) applications found, the last updated time is not formatted and client local time zone is not show in history server web ui.

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23384. --- Resolution: Fixed Assignee: guoxiaolongzte Fix Version/s: 2.4.0

[jira] [Assigned] (SPARK-23318) FP-growth: WARN FPGrowth: Input data is not cached

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23318: - Assignee: Arseniy Tashoyan > FP-growth: WARN FPGrowth: Input data is not cached >

[jira] [Resolved] (SPARK-23318) FP-growth: WARN FPGrowth: Input data is not cached

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23318. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20578

[jira] [Created] (SPARK-23407) add a config to try to inline all mutable states during codegen

2018-02-13 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-23407: --- Summary: add a config to try to inline all mutable states during codegen Key: SPARK-23407 URL: https://issues.apache.org/jira/browse/SPARK-23407 Project: Spark

[jira] [Commented] (SPARK-23377) Bucketizer with multiple columns persistence bug

2018-02-13 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362182#comment-16362182 ] Nick Pentreath commented on SPARK-23377: Should this be a blocker for 2.3? I think so since it

[jira] [Commented] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362162#comment-16362162 ] Yuming Wang commented on SPARK-23405: - I think it's data skew, you should broadcast small table. >

[jira] [Comment Edited] (SPARK-23351) checkpoint corruption in long running application

2018-02-13 Thread David Ahern (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362156#comment-16362156 ] David Ahern edited comment on SPARK-23351 at 2/13/18 11:17 AM: --- hi, yes -

[jira] [Commented] (SPARK-23351) checkpoint corruption in long running application

2018-02-13 Thread David Ahern (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362156#comment-16362156 ] David Ahern commented on SPARK-23351: - hi, am on Cloudera... only 2.2.0 is available from them for

[jira] [Commented] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362126#comment-16362126 ] Sean Owen commented on SPARK-23405: --- There's just not enough info here to even establish something

[jira] [Resolved] (SPARK-23397) Scheduling delay causes Spark Streaming to miss batches.

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23397. --- Resolution: Not A Problem It's the part in "foreachRDD" that gets executed at each batch; one answer

[jira] [Resolved] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23403. --- Resolution: Not A Problem This isn't a Spark problem. You'd need to make sure your input was valid

[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Naresh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362088#comment-16362088 ] Naresh Kumar commented on SPARK-23403: -- I am sure that exception is occurred because of bad

[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Naresh Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362076#comment-16362076 ] Naresh Kumar commented on SPARK-23403: -- yes , csv has multiple missing fields , you can see the file

[jira] [Updated] (SPARK-23406) Stream-stream self joins does not work

2018-02-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23406: -- Summary: Stream-stream self joins does not work (was: Enable stream-stream self joins ) >

[jira] [Assigned] (SPARK-23406) Enable stream-stream self joins

2018-02-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23406: Assignee: Apache Spark (was: Tathagata Das) > Enable stream-stream self joins >

[jira] [Updated] (SPARK-23406) Enable stream-stream self joins

2018-02-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23406: -- Issue Type: Bug (was: Improvement) > Enable stream-stream self joins >

[jira] [Commented] (SPARK-23406) Enable stream-stream self joins

2018-02-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362068#comment-16362068 ] Apache Spark commented on SPARK-23406: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23406) Enable stream-stream self joins

2018-02-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23406: Assignee: Tathagata Das (was: Apache Spark) > Enable stream-stream self joins >

[jira] [Updated] (SPARK-23002) SparkUI inconsistent driver hostname compare with other executors

2018-02-13 Thread Ran Tao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Tao updated SPARK-23002: Description: As the picture shows, driver name is ip address and other executors are machine hostname.

[jira] [Commented] (SPARK-23227) Add user guide entry for collecting sub models for cross-validation classes

2018-02-13 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362044#comment-16362044 ] Weichen Xu commented on SPARK-23227: I am working on this, thanks! > Add user guide entry for

[jira] [Updated] (SPARK-23399) Register a task completion listener first for OrcColumnarBatchReader

2018-02-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23399: -- Summary: Register a task completion listener first for OrcColumnarBatchReader (was: Register

[jira] [Updated] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23405: -- Description: I run a sql: `select ls.cs_order_number from ls left semi join catalog_sales cs

[jira] [Updated] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23405: -- Description: I run a sql: `select ls.cs_order_number from ls left semi join catalog_sales cs

[jira] [Created] (SPARK-23406) Enable stream-stream self joins

2018-02-13 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23406: - Summary: Enable stream-stream self joins Key: SPARK-23406 URL: https://issues.apache.org/jira/browse/SPARK-23406 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23403) java.lang.ArrayIndexOutOfBoundsException: 10

2018-02-13 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362030#comment-16362030 ] Liang-Chi Hsieh commented on SPARK-23403: - Have you checked the content of the csv file? Is there

[jira] [Updated] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23405: -- Description: __I run a sql "" > The task will hang up when a small table left semi join a big

[jira] [Updated] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-23405: -- Attachment: taskhang up.png SQL.png > The task will hang up when a small table

[jira] [Created] (SPARK-23405) The task will hang up when a small table left semi join a big table

2018-02-13 Thread KaiXinXIaoLei (JIRA)
KaiXinXIaoLei created SPARK-23405: - Summary: The task will hang up when a small table left semi join a big table Key: SPARK-23405 URL: https://issues.apache.org/jira/browse/SPARK-23405 Project: Spark

[jira] [Comment Edited] (SPARK-20327) Add CLI support for YARN custom resources, like GPUs

2018-02-13 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360904#comment-16360904 ] Szilard Nemeth edited comment on SPARK-20327 at 2/13/18 9:08 AM: - Hey

[jira] [Commented] (SPARK-23388) Support for Parquet Binary DecimalType in VectorizedColumnReader

2018-02-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361990#comment-16361990 ] Wenchen Fan commented on SPARK-23388: - This is an interoperability problem: although Spark SQL always

[jira] [Commented] (SPARK-23299) __repr__ broken for Rows instantiated with *args

2018-02-13 Thread Shashwat Anand (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361966#comment-16361966 ] Shashwat Anand commented on SPARK-23299: [~hyukjin.kwon]   What do we do about this ? > __repr__

[jira] [Updated] (SPARK-23388) Support for Parquet Binary DecimalType in VectorizedColumnReader

2018-02-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-23388: Fix Version/s: (was: 2.3.0) 2.3.1 > Support for Parquet Binary DecimalType

<    1   2