[jira] [Resolved] (SPARK-18092) add type cast to avoid error "Column prediction must be of type DoubleType but was actually FloatType"

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18092. --- Resolution: Not A Problem >From the PR, it seems this isn't a problem now. > add type cast to

[jira] [Commented] (SPARK-26422) Unable to disable Hive support in SparkR when Hadoop version is unsupported

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725946#comment-16725946 ] ASF GitHub Bot commented on SPARK-26422: HyukjinKwon opened a new pull request #23356:

[jira] [Assigned] (SPARK-24687) When NoClassDefError thrown during task serialization will cause job hang

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24687: - Assignee: zhoukang > When NoClassDefError thrown during task serialization will cause job hang

[jira] [Resolved] (SPARK-24687) When NoClassDefError thrown during task serialization will cause job hang

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24687. --- Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 2.3.3

[jira] [Commented] (SPARK-18092) add type cast to avoid error "Column prediction must be of type DoubleType but was actually FloatType"

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725930#comment-16725930 ] ASF GitHub Bot commented on SPARK-18092: srowen closed pull request #15622: [SPARK-18092][ML]

[jira] [Commented] (SPARK-26324) Spark submit does not work with messos over ssl [Missing docs]

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725898#comment-16725898 ] ASF GitHub Bot commented on SPARK-26324: srowen closed pull request #23342: [SPARK-26324][DOCS]

[jira] [Updated] (SPARK-26324) Document Mesos SSL config

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26324: -- Priority: Minor (was: Major) Component/s: (was: Spark Submit) Mesos

[jira] [Created] (SPARK-26422) Unable to disable Hive support in SparkR when Hadoop version is unsupported

2018-12-20 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-26422: Summary: Unable to disable Hive support in SparkR when Hadoop version is unsupported Key: SPARK-26422 URL: https://issues.apache.org/jira/browse/SPARK-26422 Project:

[jira] [Created] (SPARK-26421) Spark2.4.0 integration hadoop3.1.1 causes hive sql not to use,just in idea local mode

2018-12-20 Thread thinktothings (JIRA)
thinktothings created SPARK-26421: - Summary: Spark2.4.0 integration hadoop3.1.1 causes hive sql not to use,just in idea local mode Key: SPARK-26421 URL: https://issues.apache.org/jira/browse/SPARK-26421

[jira] [Resolved] (SPARK-26324) Spark submit does not work with messos over ssl [Missing docs]

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26324. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23342

[jira] [Commented] (SPARK-24687) When NoClassDefError thrown during task serialization will cause job hang

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725892#comment-16725892 ] ASF GitHub Bot commented on SPARK-24687: srowen closed pull request #21664: [SPARK-24687][CORE]

[jira] [Assigned] (SPARK-26324) Spark submit does not work with messos over ssl [Missing docs]

2018-12-20 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26324: - Assignee: Jorge Machado > Spark submit does not work with messos over ssl [Missing docs] >

[jira] [Assigned] (SPARK-26422) Unable to disable Hive support in SparkR when Hadoop version is unsupported

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26422: Assignee: Apache Spark > Unable to disable Hive support in SparkR when Hadoop version is

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

2018-12-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725949#comment-16725949 ] Steve Loughran commented on SPARK-26359: bq. This seems to be related to the fact that we

[jira] [Resolved] (SPARK-26284) Spark History server object vs file storage behavior difference

2018-12-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-26284. Resolution: Invalid > Spark History server object vs file storage behavior difference >

[jira] [Commented] (SPARK-26284) Spark History server object vs file storage behavior difference

2018-12-20 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725958#comment-16725958 ] Steve Loughran commented on SPARK-26284: This is how object stores work. PUTs / multipart puts

[jira] [Commented] (SPARK-26392) Cancel pending allocate requests by taking locality preference into account

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726113#comment-16726113 ] ASF GitHub Bot commented on SPARK-26392: asfgit closed pull request #23344: [SPARK-26392][YARN]

[jira] [Commented] (SPARK-25970) Add Instrumentation to PrefixSpan

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726164#comment-16726164 ] ASF GitHub Bot commented on SPARK-25970: asfgit closed pull request #22971: [SPARK-25970][ML]

[jira] [Commented] (SPARK-21544) Test jar of some module should not install or deploy twice

2018-12-20 Thread Matt Foley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726156#comment-16726156 ] Matt Foley commented on SPARK-21544: The problem described in detail in the Description above,

[jira] [Commented] (SPARK-26410) Support per Pandas UDF configuration

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726155#comment-16726155 ] Xiangrui Meng commented on SPARK-26410: --- On the same cluster, there could be different workloads.

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames for the entire partition

2018-12-20 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725957#comment-16725957 ] Li Jin commented on SPARK-26412: So this is similar to the mapPartitions API in Scala but instead of

[jira] [Commented] (SPARK-26409) SQLConf should be serializable in test sessions

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726097#comment-16726097 ] ASF GitHub Bot commented on SPARK-26409: asfgit closed pull request #23352:

[jira] [Commented] (SPARK-26410) Support per Pandas UDF configuration

2018-12-20 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725948#comment-16725948 ] Li Jin commented on SPARK-26410: I am curious why would user want to configure maxRecordsPerBatch? As

[jira] [Resolved] (SPARK-26409) SQLConf should be serializable in test sessions

2018-12-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26409. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > SQLConf should be

[jira] [Assigned] (SPARK-26392) Cancel pending allocate requests by taking locality preference into account

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26392: -- Assignee: wuyi > Cancel pending allocate requests by taking locality preference into

[jira] [Assigned] (SPARK-25970) Add Instrumentation to PrefixSpan

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25970: - Assignee: zhengruifeng > Add Instrumentation to PrefixSpan >

[jira] [Resolved] (SPARK-25970) Add Instrumentation to PrefixSpan

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25970. --- Resolution: Done > Add Instrumentation to PrefixSpan > - >

[jira] [Assigned] (SPARK-26422) Unable to disable Hive support in SparkR when Hadoop version is unsupported

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26422: Assignee: (was: Apache Spark) > Unable to disable Hive support in SparkR when Hadoop

[jira] [Resolved] (SPARK-26392) Cancel pending allocate requests by taking locality preference into account

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26392. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23344

[jira] [Updated] (SPARK-26409) SQLConf should be serializable in test sessions

2018-12-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26409: -- Fix Version/s: 2.4.1 > SQLConf should be serializable in test sessions >

[jira] [Updated] (SPARK-26423) [K8s] Make sure that disconnected executors eventually get deleted

2018-12-20 Thread David Vogelbacher (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-26423: -- Description: If an executor disconnects we currently only disable it in the

[jira] [Assigned] (SPARK-26424) Use java.time API in timestamp/date functions

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26424: Assignee: (was: Apache Spark) > Use java.time API in timestamp/date functions >

[jira] [Commented] (SPARK-26424) Use java.time API in timestamp/date functions

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726270#comment-16726270 ] ASF GitHub Bot commented on SPARK-26424: MaxGekk opened a new pull request #23358:

[jira] [Commented] (SPARK-26416) Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala`

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726326#comment-16726326 ] ASF GitHub Bot commented on SPARK-26416: dbtsai opened a new pull request #23359: [SPARK-26416]

[jira] [Updated] (SPARK-26415) Mark StreamSinkProvider and StreamSourceProvider as stable

2018-12-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26415: -- Target Version/s: (was: 2.2.3, 2.3.3, 2.4.1, 2.5.0) > Mark StreamSinkProvider and

[jira] [Assigned] (SPARK-25064) Total Tasks in WebUI does not match Active+Failed+Complete Tasks

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25064: Assignee: (was: Apache Spark) > Total Tasks in WebUI does not match

[jira] [Created] (SPARK-26424) Use java.time API in timestamp/date function

2018-12-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26424: -- Summary: Use java.time API in timestamp/date function Key: SPARK-26424 URL: https://issues.apache.org/jira/browse/SPARK-26424 Project: Spark Issue Type:

[jira] [Closed] (SPARK-26415) Mark StreamSinkProvider and StreamSourceProvider as stable

2018-12-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26415. - > Mark StreamSinkProvider and StreamSourceProvider as stable >

[jira] [Created] (SPARK-26423) [K8s] Make sure that disconnected executors eventually get deleted

2018-12-20 Thread David Vogelbacher (JIRA)
David Vogelbacher created SPARK-26423: - Summary: [K8s] Make sure that disconnected executors eventually get deleted Key: SPARK-26423 URL: https://issues.apache.org/jira/browse/SPARK-26423

[jira] [Commented] (SPARK-25064) Total Tasks in WebUI does not match Active+Failed+Complete Tasks

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726281#comment-16726281 ] ASF GitHub Bot commented on SPARK-25064: vanzin closed pull request #22051: [SPARK-25064][WEBUI]

[jira] [Assigned] (SPARK-25064) Total Tasks in WebUI does not match Active+Failed+Complete Tasks

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25064: -- Assignee: (was: Marcelo Vanzin) > Total Tasks in WebUI does not match

[jira] [Assigned] (SPARK-25064) Total Tasks in WebUI does not match Active+Failed+Complete Tasks

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25064: Assignee: Apache Spark > Total Tasks in WebUI does not match Active+Failed+Complete

[jira] [Assigned] (SPARK-25064) Total Tasks in WebUI does not match Active+Failed+Complete Tasks

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25064: -- Assignee: Marcelo Vanzin > Total Tasks in WebUI does not match

[jira] [Assigned] (SPARK-26416) Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala`

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26416: Assignee: DB Tsai (was: Apache Spark) > Refactor `ColumnPruning` from `Optimizer.scala`

[jira] [Resolved] (SPARK-26415) Mark StreamSinkProvider and StreamSourceProvider as stable

2018-12-20 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26415. --- Resolution: Not A Problem According to the discussion, the API will be effectively `Stable`

[jira] [Resolved] (SPARK-26311) [YARN] New feature: custom log URL for stdout/stderr

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26311. Resolution: Duplicate Duping to the existing bug; no need to track this in two different

[jira] [Commented] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726287#comment-16726287 ] ASF GitHub Bot commented on SPARK-23155: vanzin closed pull request #20326:

[jira] [Commented] (SPARK-26415) Mark StreamSinkProvider and StreamSourceProvider as stable

2018-12-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726190#comment-16726190 ] ASF GitHub Bot commented on SPARK-26415: dongjoon-hyun closed pull request #23354:

[jira] [Updated] (SPARK-26424) Use java.time API in timestamp/date functions

2018-12-20 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-26424: --- Summary: Use java.time API in timestamp/date functions (was: Use java.time API in timestamp/date

[jira] [Assigned] (SPARK-26424) Use java.time API in timestamp/date functions

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26424: Assignee: Apache Spark > Use java.time API in timestamp/date functions >

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23155: Assignee: Marcelo Vanzin (was: Apache Spark) > YARN-aggregated executor/driver logs

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23155: -- Assignee: (was: Marcelo Vanzin) > YARN-aggregated executor/driver logs appear

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23155: -- Assignee: (was: Marcelo Vanzin) > YARN-aggregated executor/driver logs appear

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23155: -- Assignee: Marcelo Vanzin > YARN-aggregated executor/driver logs appear unavailable

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23155: -- Assignee: Marcelo Vanzin > YARN-aggregated executor/driver logs appear unavailable

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23155: Assignee: Apache Spark (was: Marcelo Vanzin) > YARN-aggregated executor/driver logs

[jira] [Assigned] (SPARK-26416) Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala`

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26416: Assignee: Apache Spark (was: DB Tsai) > Refactor `ColumnPruning` from `Optimizer.scala`

[jira] [Updated] (SPARK-26420) [kubernetes] driver service id is not random

2018-12-20 Thread Viktor Bogdanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Bogdanov updated SPARK-26420: Description: I faced with a very rare situation {code:java} 2018-12-20 04:34:46 WARN

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Attachment: image-2018-12-20-17-11-44-568.png > spark metric source > --- > >

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Description: Today I write a metric source to collect HikariCp metrics. My source code like this:

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Description: Today I write a metric source to collect HikariCp metrics. My source code like this:

[jira] [Commented] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Wang Shuo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725649#comment-16725649 ] Wang Shuo commented on SPARK-26418: --- We use YarnShuffleService as aux service of NodeManager in our

[jira] [Comment Edited] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Wang Shuo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725649#comment-16725649 ] Wang Shuo edited comment on SPARK-26418 at 12/20/18 8:38 AM: - We use

[jira] [Assigned] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26418: Assignee: (was: Apache Spark) > Only OpenBlocks without any ChunkFetch for one

[jira] [Assigned] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26418: Assignee: Apache Spark > Only OpenBlocks without any ChunkFetch for one stream will

[jira] [Commented] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725679#comment-16725679 ] Apache Spark commented on SPARK-26418: -- User 'wangshuo128' has created a pull request for this

[jira] [Commented] (SPARK-26405) OOM

2018-12-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725691#comment-16725691 ] Liang-Chi Hsieh commented on SPARK-26405: - I think the exception message is somehow clear:

[jira] [Created] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
Si Chen created SPARK-26419: --- Summary: spark metric source Key: SPARK-26419 URL: https://issues.apache.org/jira/browse/SPARK-26419 Project: Spark Issue Type: Bug Components: Spark Core

[jira] [Commented] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725677#comment-16725677 ] Apache Spark commented on SPARK-26418: -- User 'wangshuo128' has created a pull request for this

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Attachment: image-2018-12-20-17-07-48-020.png > spark metric source > --- > >

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Attachment: image-2018-12-20-17-07-40-920.png > spark metric source > --- > >

[jira] [Created] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

2018-12-20 Thread Wang Shuo (JIRA)
Wang Shuo created SPARK-26418: - Summary: Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService Key: SPARK-26418 URL: https://issues.apache.org/jira/browse/SPARK-26418

[jira] [Commented] (SPARK-26408) java.util.NoSuchElementException: None.get at scala.None$.get(Option.scala:347)

2018-12-20 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725683#comment-16725683 ] Liang-Chi Hsieh commented on SPARK-26408: - Looks like an issue at

[jira] [Commented] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

2018-12-20 Thread Kaspar Tint (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725685#comment-16725685 ] Kaspar Tint commented on SPARK-26396: - Yes, issue can be closed. I do think the documentation about

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Attachment: image-2018-12-20-17-14-35-157.png > spark metric source > --- > >

[jira] [Updated] (SPARK-26419) spark metric source

2018-12-20 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26419: Attachment: image-2018-12-20-17-05-42-245.png > spark metric source > --- > >

[jira] [Created] (SPARK-26420) [kubernetes] driver service id is not random

2018-12-20 Thread Viktor Bogdanov (JIRA)
Viktor Bogdanov created SPARK-26420: --- Summary: [kubernetes] driver service id is not random Key: SPARK-26420 URL: https://issues.apache.org/jira/browse/SPARK-26420 Project: Spark Issue