[jira] [Commented] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416835#comment-16416835 ] Kazuaki Ishizaki commented on SPARK-23801: -- While we know it is time-consuming work to narrow

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2018-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416821#comment-16416821 ] Wenchen Fan commented on SPARK-22618: - Looks like we can apply the same fix to `Broadcast.unpersist`.

[jira] [Commented] (SPARK-23291) SparkR : substr : In SparkR dataframe , starting and ending position arguments in "substr" is giving wrong result when the position is greater than 1

2018-03-27 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416763#comment-16416763 ] Wenchen Fan commented on SPARK-23291: - shall we backport this bug fix to 2.3? > SparkR : substr : In

[jira] [Resolved] (SPARK-23699) PySpark should raise same Error when Arrow fallback is disabled

2018-03-27 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23699. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20839

[jira] [Assigned] (SPARK-23699) PySpark should raise same Error when Arrow fallback is disabled

2018-03-27 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-23699: Assignee: Bryan Cutler > PySpark should raise same Error when Arrow fallback is disabled

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2018-03-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416309#comment-16416309 ] Thomas Graves commented on SPARK-22618: --- thanks for fixing this, hitting it now in spark 2.2, I

[jira] [Resolved] (SPARK-23096) Migrate rate source to v2

2018-03-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23096. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 20688

[jira] [Assigned] (SPARK-23096) Migrate rate source to v2

2018-03-27 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23096: - Assignee: Saisai Shao > Migrate rate source to v2 > - > >

[jira] [Commented] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416239#comment-16416239 ] Apache Spark commented on SPARK-19964: -- User 'vanzin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23804) Flaky test: SparkSubmitSuite "repositories"

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23804. Resolution: Duplicate Same root cause as SPARK-19964. > Flaky test: SparkSubmitSuite

[jira] [Assigned] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19964: Assignee: (was: Apache Spark) > Flaky test: SparkSubmitSuite "includes jars passed in

[jira] [Assigned] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19964: Assignee: Apache Spark > Flaky test: SparkSubmitSuite "includes jars passed in through

[jira] [Commented] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416141#comment-16416141 ] Marcelo Vanzin commented on SPARK-19964: http://dl.bintray.com doesn't seem to be loading right

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

2018-03-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416138#comment-16416138 ] Ted Yu commented on SPARK-23714: [~tdas]: What do you think ? Thanks > Add metrics for cached

[jira] [Commented] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416091#comment-16416091 ] Marcelo Vanzin commented on SPARK-19964: I ran into this failure in our internal jenkins also...

[jira] [Updated] (SPARK-9209) Using executor allocation, a executor is removed but it exists in ExecutorsPage of the web ui

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9209: - Due Date: (was: 21/Jul/15) > Using executor allocation, a executor is removed but it exists in >

[jira] [Updated] (SPARK-23694) The staging directory should under hive.exec.stagingdir if we set hive.exec.stagingdir but not under the table directory

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23694: -- Target Version/s: (was: 2.3.0) > The staging directory should under hive.exec.stagingdir if we set

[jira] [Updated] (SPARK-22797) Add multiple column support to PySpark Bucketizer

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22797: -- Target Version/s: (was: 2.3.0) > Add multiple column support to PySpark Bucketizer >

[jira] [Updated] (SPARK-10473) EventLog will loss message in the long-running security application

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10473: -- Due Date: (was: 7/Sep/15) > EventLog will loss message in the long-running security application >

[jira] [Updated] (SPARK-23425) load data for hdfs file path with wild card usage is not working properly

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23425: -- Due Date: (was: 15/Feb/18) > load data for hdfs file path with wild card usage is not working

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-03-27 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416038#comment-16416038 ] holdenk commented on SPARK-22809: - This _should_ be resolved by SPARK-23169 but I'll double check when

[jira] [Updated] (SPARK-19964) Flaky test: SparkSubmitSuite "includes jars passed in through --packages"

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-19964: --- Summary: Flaky test: SparkSubmitSuite "includes jars passed in through --packages" (was:

[jira] [Created] (SPARK-23804) Flaky test: SparkSubmitSuite "repositories"

2018-03-27 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-23804: -- Summary: Flaky test: SparkSubmitSuite "repositories" Key: SPARK-23804 URL: https://issues.apache.org/jira/browse/SPARK-23804 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19964) Flaky test: SparkSubmitSuite fails due to Timeout

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416033#comment-16416033 ] Marcelo Vanzin commented on SPARK-19964: Still flaky:

[jira] [Updated] (SPARK-23195) Hint of cached data is lost

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23195: -- Target Version/s: 2.3.1 (was: 2.3.0) > Hint of cached data is lost > --- > >

[jira] [Updated] (SPARK-22809) pyspark is sensitive to imports with dots

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22809: -- Target Version/s: 2.3.1, 2.4.0 (was: 2.3.0, 2.4.0) > pyspark is sensitive to imports with dots >

[jira] [Updated] (SPARK-23292) python tests related to pandas are skipped with python 2

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23292: -- Target Version/s: (was: 2.3.0) > python tests related to pandas are skipped with python 2 >

[jira] [Updated] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23309: -- Target Version/s: (was: 2.3.0) > Spark 2.3 cached query performance 20-30% worse then spark 2.2 >

[jira] [Updated] (SPARK-17859) persist should not impede with spark's ability to perform a broadcast join.

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17859: -- Fix Version/s: (was: 2.2.1) (was: 2.0.2) > persist should not impede with

[jira] [Updated] (SPARK-23625) spark sql long-running mission will be dead

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23625: -- Target Version/s: (was: 1.6.2) Fix Version/s: (was: 1.6.2) > spark sql long-running

[jira] [Updated] (SPARK-23657) Document InternalRow and expose it as a stable interface

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23657: -- Fix Version/s: (was: 2.4.0) > Document InternalRow and expose it as a stable interface >

[jira] [Updated] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

2018-03-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22839: -- Fix Version/s: (was: 2.4.0) > Refactor Kubernetes code for configuring driver/executor pods to use

[jira] [Commented] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415997#comment-16415997 ] Apache Spark commented on SPARK-23803: -- User 'sabanas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23803: Assignee: Apache Spark > Support bucket pruning to optimize filtering on a bucketed

[jira] [Assigned] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23803: Assignee: (was: Apache Spark) > Support bucket pruning to optimize filtering on a

[jira] [Comment Edited] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2018-03-27 Thread imran shaik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415985#comment-16415985 ] imran shaik edited comment on SPARK-5594 at 3/27/18 5:48 PM: - [~NeoYE] Thank

[jira] [Commented] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2018-03-27 Thread imran shaik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415985#comment-16415985 ] imran shaik commented on SPARK-5594: [~NeoYE] Thank you for your awesome comment!! It worked!!

[jira] [Updated] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Asher Saban (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asher Saban updated SPARK-23803: Target Version/s: (was: 2.3.1) > Support bucket pruning to optimize filtering on a bucketed

[jira] [Updated] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Asher Saban (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asher Saban updated SPARK-23803: Fix Version/s: (was: 2.3.1) > Support bucket pruning to optimize filtering on a bucketed

[jira] [Created] (SPARK-23803) Support bucket pruning to optimize filtering on a bucketed column

2018-03-27 Thread Asher Saban (JIRA)
Asher Saban created SPARK-23803: --- Summary: Support bucket pruning to optimize filtering on a bucketed column Key: SPARK-23803 URL: https://issues.apache.org/jira/browse/SPARK-23803 Project: Spark

[jira] [Commented] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415911#comment-16415911 ] Marcelo Vanzin commented on SPARK-23801: [~joshrosen] looks like the code you added to find

[jira] [Updated] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

2018-03-27 Thread Joshua Howard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Howard updated SPARK-23784: -- Description: I have code

[jira] [Updated] (SPARK-23784) Cannot use custom Aggregator with groupBy/agg

2018-03-27 Thread Joshua Howard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Howard updated SPARK-23784: -- Description: {{I have code

[jira] [Commented] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415832#comment-16415832 ] Apache Spark commented on SPARK-23802: -- User 'robert3005' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23802: Assignee: (was: Apache Spark) > PropagateEmptyRelation can leave query plan in

[jira] [Comment Edited] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2018-03-27 Thread Volodymyr Glushak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415825#comment-16415825 ] Volodymyr Glushak edited comment on SPARK-15276 at 3/27/18 3:55 PM:

[jira] [Assigned] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23802: Assignee: Apache Spark > PropagateEmptyRelation can leave query plan in unresolved state

[jira] [Created] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

2018-03-27 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-23802: - Summary: PropagateEmptyRelation can leave query plan in unresolved state Key: SPARK-23802 URL: https://issues.apache.org/jira/browse/SPARK-23802 Project:

[jira] [Comment Edited] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2018-03-27 Thread Volodymyr Glushak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415825#comment-16415825 ] Volodymyr Glushak edited comment on SPARK-15276 at 3/27/18 3:51 PM:

[jira] [Commented] (SPARK-15276) CREATE TABLE with LOCATION should imply EXTERNAL

2018-03-27 Thread Volodymyr Glushak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415825#comment-16415825 ] Volodymyr Glushak commented on SPARK-15276: --- [~andrewor14], HIVE is not adding EXTERNAL

[jira] [Updated] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Kleyn updated SPARK-23801: - Description: After upgrading to Spark v2.3.0 from Spark v2.1.1, we are seeing consistent

[jira] [Updated] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Kleyn updated SPARK-23801: - Environment: Mesos coarse grained executor 18 * r3.4xlarge (16 core boxes) with 105G of executor

[jira] [Updated] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Kleyn updated SPARK-23801: - Environment: 18 * r3.4xlarge (16 core boxes) with 105G of executor memory > Consistent SIGSEGV

[jira] [Created] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Nathan Kleyn (JIRA)
Nathan Kleyn created SPARK-23801: Summary: Consistent SIGSEGV after upgrading to Spark v2.3.0 Key: SPARK-23801 URL: https://issues.apache.org/jira/browse/SPARK-23801 Project: Spark Issue

[jira] [Updated] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Kleyn updated SPARK-23801: - Attachment: spark-executor-failure.coredump.log > Consistent SIGSEGV after upgrading to Spark

[jira] [Updated] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Michael Shtelma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shtelma updated SPARK-23799: Description: Spark 2.2.1 and 2.3.0 can produce NumberFormatException (see below) during

[jira] [Updated] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Michael Shtelma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shtelma updated SPARK-23799: Description: Spark 2.2.1 and 2.3.0 can produce NumberFormatException (see below) during

[jira] [Updated] (SPARK-23800) Support partial function and callable object with pandas UDF

2018-03-27 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-23800: --- Description: Per discussion here:  [https://github.com/apache/spark/pull/20900#issuecomment-376195597]   >

[jira] [Created] (SPARK-23800) Support partial function and callable object with pandas UDF

2018-03-27 Thread Li Jin (JIRA)
Li Jin created SPARK-23800: -- Summary: Support partial function and callable object with pandas UDF Key: SPARK-23800 URL: https://issues.apache.org/jira/browse/SPARK-23800 Project: Spark Issue

[jira] [Updated] (SPARK-23800) Support partial function and callable object with pandas UDF

2018-03-27 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-23800: --- Environment: (was: Per discussion here: 

[jira] [Commented] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415652#comment-16415652 ] Apache Spark commented on SPARK-23799: -- User 'mshtelma' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23799: Assignee: (was: Apache Spark) > [CBO] FilterEstimation.evaluateInSet produces

[jira] [Assigned] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23799: Assignee: Apache Spark > [CBO] FilterEstimation.evaluateInSet produces devision by zero

[jira] [Created] (SPARK-23799) [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics

2018-03-27 Thread Michael Shtelma (JIRA)
Michael Shtelma created SPARK-23799: --- Summary: [CBO] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics Key: SPARK-23799 URL:

[jira] [Commented] (SPARK-6162) Handle missing values in GBM

2018-03-27 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415600#comment-16415600 ] Barry Becker commented on SPARK-6162: - If we all agree that is is something that would be very nice to

[jira] [Created] (SPARK-23798) The CreateArray and ConcatArray should return the default array type when no children provided

2018-03-27 Thread Marek Novotny (JIRA)
Marek Novotny created SPARK-23798: - Summary: The CreateArray and ConcatArray should return the default array type when no children provided Key: SPARK-23798 URL: https://issues.apache.org/jira/browse/SPARK-23798

[jira] [Resolved] (SPARK-23794) UUID() should be stateful

2018-03-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23794. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.4.0 >

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

2018-03-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415320#comment-16415320 ] Steve Loughran commented on SPARK-22513: bq. So I guess at the summary level Sean was correct

[jira] [Commented] (SPARK-23504) Flaky test: RateSourceV2Suite.basic microbatch execution

2018-03-27 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415313#comment-16415313 ] Marek Novotny commented on SPARK-23504: --- Experienced the [same

[jira] [Assigned] (SPARK-23794) UUID() should be stateful

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23794: Assignee: (was: Apache Spark) > UUID() should be stateful > -

[jira] [Commented] (SPARK-23794) UUID() should be stateful

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415295#comment-16415295 ] Apache Spark commented on SPARK-23794: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23794) UUID() should be stateful

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23794: Assignee: Apache Spark > UUID() should be stateful > - > >

[jira] [Reopened] (SPARK-22968) java.lang.IllegalStateException: No current assignment for partition kssh-2

2018-03-27 Thread Jepson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jepson reopened SPARK-22968: This issue is reappear. > java.lang.IllegalStateException: No current assignment for partition kssh-2 >

[jira] [Comment Edited] (SPARK-23191) Workers registration failes in case of network drop

2018-03-27 Thread Sujith Jay Nair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415194#comment-16415194 ] Sujith Jay Nair edited comment on SPARK-23191 at 3/27/18 8:10 AM: -- This

[jira] [Commented] (SPARK-23191) Workers registration failes in case of network drop

2018-03-27 Thread Sujith Jay Nair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415194#comment-16415194 ] Sujith Jay Nair commented on SPARK-23191: - This is a known race condition, in which the

[jira] [Commented] (SPARK-23500) Filters on named_structs could be pushed into scans

2018-03-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415141#comment-16415141 ] Apache Spark commented on SPARK-23500: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Updated] (SPARK-23766) Not able to execute multiple queries in spark structured streaming

2018-03-27 Thread Apeksha Agnihotri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apeksha Agnihotri updated SPARK-23766: -- Component/s: (was: Spark Core) Structured Streaming > Not able to

[jira] [Commented] (SPARK-23739) Spark structured streaming long running problem

2018-03-27 Thread Davide Isoardi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415093#comment-16415093 ] Davide Isoardi commented on SPARK-23739: We know that kafka client 0.8 has not this class, but