[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460559#comment-16460559 ] Anirudh Ramanathan commented on SPARK-24135: +1 to detecting all pod error states and doing

[jira] [Commented] (SPARK-24146) spark.ml parity for sequential pattern mining - PrefixSpan: Python API

2018-05-02 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460804#comment-16460804 ] Weichen Xu commented on SPARK-24146: I will create PR soon. :) > spark.ml parity for sequential

[jira] [Updated] (SPARK-24146) spark.ml parity for sequential pattern mining - PrefixSpan: Python API

2018-05-02 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-24146: --- Issue Type: Sub-task (was: New Feature) Parent: SPARK-14501 > spark.ml parity for

[jira] [Updated] (SPARK-24147) .count() reports wrong size of dataframe when filtering dataframe on corrupt record field

2018-05-02 Thread Rich Smith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Smith updated SPARK-24147: --- Description: Spark reports the wrong size of dataframe using .count() after filtering on a

[jira] [Commented] (SPARK-24067) Backport SPARK-17147 to 2.3 (Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction))

2018-05-02 Thread Joachim Hereth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460870#comment-16460870 ] Joachim Hereth commented on SPARK-24067: It would be great if this fix could go into 2.3.1. Any

[jira] [Updated] (SPARK-24147) .count() reports wrong size of dataframe when filtering dataframe on corrupt record field

2018-05-02 Thread Rich Smith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Smith updated SPARK-24147: --- Summary: .count() reports wrong size of dataframe when filtering dataframe on corrupt record field

[jira] [Created] (SPARK-24147) .count() reports wrong size of dataframe when filtering dataframe on

2018-05-02 Thread Rich Smith (JIRA)
Rich Smith created SPARK-24147: -- Summary: .count() reports wrong size of dataframe when filtering dataframe on Key: SPARK-24147 URL: https://issues.apache.org/jira/browse/SPARK-24147 Project: Spark

[jira] [Assigned] (SPARK-24120) Show `Jobs` page when `jobId` is missing

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24120: Assignee: (was: Apache Spark) > Show `Jobs` page when `jobId` is missing >

[jira] [Commented] (SPARK-24120) Show `Jobs` page when `jobId` is missing

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460721#comment-16460721 ] Apache Spark commented on SPARK-24120: -- User 'jongyoul' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24120) Show `Jobs` page when `jobId` is missing

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24120: Assignee: Apache Spark > Show `Jobs` page when `jobId` is missing >

[jira] [Commented] (SPARK-23775) Flaky test: DataFrameRangeSuite

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460802#comment-16460802 ] Apache Spark commented on SPARK-23775: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Created] (SPARK-24145) spark.ml parity for sequential pattern mining - PrefixSpan: Python API

2018-05-02 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-24145: -- Summary: spark.ml parity for sequential pattern mining - PrefixSpan: Python API Key: SPARK-24145 URL: https://issues.apache.org/jira/browse/SPARK-24145 Project: Spark

[jira] [Created] (SPARK-24146) spark.ml parity for sequential pattern mining - PrefixSpan: Python API

2018-05-02 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-24146: -- Summary: spark.ml parity for sequential pattern mining - PrefixSpan: Python API Key: SPARK-24146 URL: https://issues.apache.org/jira/browse/SPARK-24146 Project: Spark

[jira] [Commented] (SPARK-23180) RFormulaModel should have labels member

2018-05-02 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460967#comment-16460967 ] Teng Peng commented on SPARK-23180: --- Can you give me an example for 1. the current workaround 2. the

[jira] [Commented] (SPARK-22943) OneHotEncoder supports manual specification of categorySizes

2018-05-02 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461006#comment-16461006 ] Teng Peng commented on SPARK-22943: --- This issue looks quiet interesting, but can you be more specific

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461024#comment-16461024 ] Matt Cheah commented on SPARK-24135: I think we should not count these towards job failures, and that

[jira] [Created] (SPARK-24148) Adding Ability to Specify SQL Type of Empty Arrays

2018-05-02 Thread Marek Novotny (JIRA)
Marek Novotny created SPARK-24148: - Summary: Adding Ability to Specify SQL Type of Empty Arrays Key: SPARK-24148 URL: https://issues.apache.org/jira/browse/SPARK-24148 Project: Spark Issue

[jira] [Commented] (SPARK-24147) .count() reports wrong size of dataframe when filtering dataframe on corrupt record field

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461062#comment-16461062 ] Hyukjin Kwon commented on SPARK-24147: -- I think it's a duplicate of SPARK-21610. > .count() reports

[jira] [Assigned] (SPARK-24148) Adding Ability to Specify SQL Type of Empty Arrays

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24148: Assignee: Apache Spark > Adding Ability to Specify SQL Type of Empty Arrays >

[jira] [Assigned] (SPARK-24148) Adding Ability to Specify SQL Type of Empty Arrays

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24148: Assignee: (was: Apache Spark) > Adding Ability to Specify SQL Type of Empty Arrays >

[jira] [Commented] (SPARK-24148) Adding Ability to Specify SQL Type of Empty Arrays

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461064#comment-16461064 ] Apache Spark commented on SPARK-24148: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24149: Assignee: (was: Apache Spark) > Automatic namespaces discovery in HDFS federation >

[jira] [Assigned] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24149: Assignee: Apache Spark > Automatic namespaces discovery in HDFS federation >

[jira] [Commented] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461078#comment-16461078 ] Apache Spark commented on SPARK-24149: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461188#comment-16461188 ] Matt Cheah commented on SPARK-24135: > Restarting seems like it would eventually be limited by the

[jira] [Created] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread James Thompson (JIRA)
James Thompson created SPARK-24151: -- Summary: CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled Key: SPARK-24151 URL:

[jira] [Assigned] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24151: Assignee: (was: Apache Spark) > CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved

[jira] [Assigned] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24151: Assignee: Apache Spark > CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column

[jira] [Commented] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461237#comment-16461237 ] Apache Spark commented on SPARK-24151: -- User 'jamesthomp' has created a pull request for this issue:

[jira] [Created] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-02 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24149: --- Summary: Automatic namespaces discovery in HDFS federation Key: SPARK-24149 URL: https://issues.apache.org/jira/browse/SPARK-24149 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-24107) ChunkedByteBuffer.writeFully method has not reset the limit value

2018-05-02 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24107. - Resolution: Fixed Assignee: wangjinhai Fix Version/s: 2.3.1 >

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition between the method checkLogs and cleanLogs.

[jira] [Comment Edited] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461188#comment-16461188 ] Matt Cheah edited comment on SPARK-24135 at 5/2/18 3:37 PM: {quote}Restarting

[jira] [Comment Edited] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460047#comment-16460047 ] Matt Cheah edited comment on SPARK-24135 at 5/2/18 3:37 PM: {quote}But I'm

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461154#comment-16461154 ] Erik Erlandson commented on SPARK-24135: IIRC the dynamic allocation heuristic was to avoid

[jira] [Created] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
William Montaz created SPARK-24150: -- Summary: Race condition in FsHistoryProvider Key: SPARK-24150 URL: https://issues.apache.org/jira/browse/SPARK-24150 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22918) sbt test (spark - local) fail after upgrading to 2.2.1 with: java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine",

2018-05-02 Thread Sam Garrett (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461250#comment-16461250 ] Sam Garrett commented on SPARK-22918: - +1 same issue > sbt test (spark - local) fail after upgrading

[jira] [Updated] (SPARK-23971) Should not leak Spark sessions across test suites

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23971: Fix Version/s: 2.3.1 > Should not leak Spark sessions across test suites >

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Priority: Major (was: Minor) > Race condition in FsHistoryProvider >

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition in checkLogs method between threads of

[jira] [Updated] (SPARK-23971) Should not leak Spark sessions across test suites

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23971: Component/s: Tests > Should not leak Spark sessions across test suites >

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

2018-05-02 Thread Evan McClain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461502#comment-16461502 ] Evan McClain commented on SPARK-4502: - The workaround I've been using is to explicitly pass in the

[jira] [Created] (SPARK-24154) AccumulatorV2 loses type information during serialization

2018-05-02 Thread Sergey Zhemzhitsky (JIRA)
Sergey Zhemzhitsky created SPARK-24154: -- Summary: AccumulatorV2 loses type information during serialization Key: SPARK-24154 URL: https://issues.apache.org/jira/browse/SPARK-24154 Project: Spark

[jira] [Resolved] (SPARK-24013) ApproximatePercentile grinds to a halt on sorted input.

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24013. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > ApproximatePercentile

[jira] [Updated] (SPARK-24154) AccumulatorV2 loses type information during serialization

2018-05-02 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-24154: --- Description: AccumulatorV2 loses type information during serialization. It happens

[jira] [Resolved] (SPARK-24133) Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24133. - Resolution: Fixed Assignee: Ala Luszczak Fix Version/s: 2.4.0 > Reading Parquet files

[jira] [Updated] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

2018-05-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24097: -- Shepherd: Joseph K. Bradley > Instruments improvements - RandomForest and

[jira] [Assigned] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

2018-05-02 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-24097: - Assignee: Weichen Xu > Instruments improvements - RandomForest and

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition in checkLogs method between threads of

[jira] [Created] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-24152: - Summary: Flaky Test: SparkR Key: SPARK-24152 URL: https://issues.apache.org/jira/browse/SPARK-24152 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition in checkLogs method between threads of

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition in checkLogs method between threads of

[jira] [Updated] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24152: -- Description: PR builder fails with the following SparkR error with unknown reason. The

[jira] [Updated] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24152: -- Description: PR builder and master branch test fails with the following SparkR error with

[jira] [Updated] (SPARK-24150) Race condition in FsHistoryProvider

2018-05-02 Thread William Montaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-24150: --- Description: There exist a race condition in checkLogs method between threads of

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461328#comment-16461328 ] Dongjoon Hyun commented on SPARK-24152: --- cc [~shivaram], [~felixcheung], [~yanboliang] > Flaky

[jira] [Updated] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24152: -- Description: PR builder and master branch test fails with the following SparkR error with

[jira] [Updated] (SPARK-23489) Flaky Test: HiveExternalCatalogVersionsSuite

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Description: I saw this error in an unrelated PR. It seems to me a bad configuration in the

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461324#comment-16461324 ] Erik Erlandson commented on SPARK-24135: > In the case of the executor failing to start at all,

[jira] [Updated] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24152: -- Description: PR builder fails with the following SparkR error with unknown reason. The

[jira] [Created] (SPARK-24153) Flaky Test: DirectKafkaStreamSuite

2018-05-02 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-24153: - Summary: Flaky Test: DirectKafkaStreamSuite Key: SPARK-24153 URL: https://issues.apache.org/jira/browse/SPARK-24153 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461920#comment-16461920 ] Shivaram Venkataraman commented on SPARK-24152: --- Unfortunately I dont have time to look at

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461929#comment-16461929 ] Hyukjin Kwon commented on SPARK-24152: -- >From Liang-Chi's comment and given previous discussion and

[jira] [Comment Edited] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461929#comment-16461929 ] Hyukjin Kwon edited comment on SPARK-24152 at 5/3/18 4:39 AM: -- >From

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461931#comment-16461931 ] Shivaram Venkataraman commented on SPARK-24152: --- If this is blocking all PRs I think its

[jira] [Comment Edited] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461929#comment-16461929 ] Hyukjin Kwon edited comment on SPARK-24152 at 5/3/18 4:40 AM: -- >From

[jira] [Comment Edited] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461932#comment-16461932 ] Hyukjin Kwon edited comment on SPARK-24152 at 5/3/18 4:44 AM: -- For the past

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461932#comment-16461932 ] Hyukjin Kwon commented on SPARK-24152: -- For the past issue, it was fixed within only a couple of

[jira] [Created] (SPARK-24166) InMemoryTableScanExec should not access SQLConf at executor side

2018-05-02 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-24166: --- Summary: InMemoryTableScanExec should not access SQLConf at executor side Key: SPARK-24166 URL: https://issues.apache.org/jira/browse/SPARK-24166 Project: Spark

[jira] [Assigned] (SPARK-24166) InMemoryTableScanExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24166: Assignee: Wenchen Fan (was: Apache Spark) > InMemoryTableScanExec should not access

[jira] [Commented] (SPARK-24166) InMemoryTableScanExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461935#comment-16461935 ] Apache Spark commented on SPARK-24166: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24166) InMemoryTableScanExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24166: Assignee: Apache Spark (was: Wenchen Fan) > InMemoryTableScanExec should not access

[jira] [Created] (SPARK-24167) ParquetFilters should not access SQLConf at executor side

2018-05-02 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-24167: --- Summary: ParquetFilters should not access SQLConf at executor side Key: SPARK-24167 URL: https://issues.apache.org/jira/browse/SPARK-24167 Project: Spark

[jira] [Assigned] (SPARK-24167) ParquetFilters should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24167: Assignee: Apache Spark (was: Wenchen Fan) > ParquetFilters should not access SQLConf at

[jira] [Commented] (SPARK-24167) ParquetFilters should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461949#comment-16461949 ] Apache Spark commented on SPARK-24167: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24167) ParquetFilters should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24167: Assignee: Wenchen Fan (was: Apache Spark) > ParquetFilters should not access SQLConf at

[jira] [Created] (SPARK-24168) WindowExec should not access SQLConf at executor side

2018-05-02 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-24168: --- Summary: WindowExec should not access SQLConf at executor side Key: SPARK-24168 URL: https://issues.apache.org/jira/browse/SPARK-24168 Project: Spark Issue

[jira] [Assigned] (SPARK-24168) WindowExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24168: Assignee: Wenchen Fan (was: Apache Spark) > WindowExec should not access SQLConf at

[jira] [Assigned] (SPARK-24168) WindowExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24168: Assignee: Apache Spark (was: Wenchen Fan) > WindowExec should not access SQLConf at

[jira] [Commented] (SPARK-24168) WindowExec should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461951#comment-16461951 ] Apache Spark commented on SPARK-24168: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24151: -- Affects Version/s: 2.2.1 > CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column

[jira] [Created] (SPARK-24169) JsonToStructs should not access SQLConf at executor side

2018-05-02 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-24169: --- Summary: JsonToStructs should not access SQLConf at executor side Key: SPARK-24169 URL: https://issues.apache.org/jira/browse/SPARK-24169 Project: Spark Issue

[jira] [Commented] (SPARK-24169) JsonToStructs should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461954#comment-16461954 ] Apache Spark commented on SPARK-24169: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24169) JsonToStructs should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24169: Assignee: Wenchen Fan (was: Apache Spark) > JsonToStructs should not access SQLConf at

[jira] [Assigned] (SPARK-24169) JsonToStructs should not access SQLConf at executor side

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24169: Assignee: Apache Spark (was: Wenchen Fan) > JsonToStructs should not access SQLConf at

[jira] [Commented] (SPARK-24151) CURRENT_DATE, CURRENT_TIMESTAMP incorrectly resolved as column names when caseSensitive is enabled

2018-05-02 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461957#comment-16461957 ] Dongjoon Hyun commented on SPARK-24151: --- Thank you for reporting and fixing this, [~jamesthomp]. I

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461962#comment-16461962 ] Liang-Chi Hsieh commented on SPARK-24152: - CRAN sysadmin replied me it should be fixed now. I

[jira] [Assigned] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24152: Assignee: Liang-Chi Hsieh > Flaky Test: SparkR > -- > >

[jira] [Commented] (SPARK-24152) Flaky Test: SparkR

2018-05-02 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461965#comment-16461965 ] Hyukjin Kwon commented on SPARK-24152: -- Thanks [~viirya], I retriggered one build. Will resolve this

[jira] [Resolved] (SPARK-24123) Fix a flaky test `DateTimeUtilsSuite.monthsBetween`

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24123. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.4.0 > Fix a flaky test

[jira] [Resolved] (SPARK-23923) High-order function: cardinality(x) → bigint

2018-05-02 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23923. - Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.4.0 > High-order function:

[jira] [Created] (SPARK-24155) Instrument improvement for clustering

2018-05-02 Thread Lu Wang (JIRA)
Lu Wang created SPARK-24155: --- Summary: Instrument improvement for clustering Key: SPARK-24155 URL: https://issues.apache.org/jira/browse/SPARK-24155 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-24155) Instrument improvement for clustering

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24155: Assignee: Apache Spark > Instrument improvement for clustering >

[jira] [Commented] (SPARK-24155) Instrument improvement for clustering

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461640#comment-16461640 ] Apache Spark commented on SPARK-24155: -- User 'ludatabricks' has created a pull request for this

[jira] [Assigned] (SPARK-24155) Instrument improvement for clustering

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24155: Assignee: (was: Apache Spark) > Instrument improvement for clustering >

[jira] [Created] (SPARK-24159) Enable no-data micro batches for streaming mapGroupswithState

2018-05-02 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-24159: - Summary: Enable no-data micro batches for streaming mapGroupswithState Key: SPARK-24159 URL: https://issues.apache.org/jira/browse/SPARK-24159 Project: Spark

[jira] [Updated] (SPARK-24132) Instrumentation improvement for classification

2018-05-02 Thread Lu Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Wang updated SPARK-24132: Summary: Instrumentation improvement for classification (was: Instruments improvement for classification)

[jira] [Created] (SPARK-24160) ShuffleBlockFetcherIterator should fail if it receives zero-size blocks

2018-05-02 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-24160: -- Summary: ShuffleBlockFetcherIterator should fail if it receives zero-size blocks Key: SPARK-24160 URL: https://issues.apache.org/jira/browse/SPARK-24160 Project: Spark

[jira] [Updated] (SPARK-24155) Instrumentation improvement for clustering

2018-05-02 Thread Lu Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Wang updated SPARK-24155: Summary: Instrumentation improvement for clustering (was: Instrument improvement for clustering) >

[jira] [Assigned] (SPARK-24160) ShuffleBlockFetcherIterator should fail if it receives zero-size blocks

2018-05-02 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24160: Assignee: Apache Spark (was: Josh Rosen) > ShuffleBlockFetcherIterator should fail if it

  1   2   >