[jira] [Resolved] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26019. -- Resolution: Cannot Reproduce Reopen this if this issue is persistent in Apache master, or if

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692778#comment-16692778 ] Hyukjin Kwon commented on SPARK-26019: -- Also, I followed the guide: {quote} For issues that can’t

[jira] [Comment Edited] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692773#comment-16692773 ] Hyukjin Kwon edited comment on SPARK-26019 at 11/20/18 7:29 AM:

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692773#comment-16692773 ] Hyukjin Kwon commented on SPARK-26019: -- [~Tagar], can you reproduce this in Apache master? >

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692769#comment-16692769 ] Hyukjin Kwon commented on SPARK-26019: -- I can't reproduce the issue in my local and yarn cluster.

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692722#comment-16692722 ] Liang-Chi Hsieh commented on SPARK-26019: - {{TCPServer}} begins to process requests only after

[jira] [Created] (SPARK-26126) Should put scala-library deps into root pom instead of spark-tags module

2018-11-19 Thread liupengcheng (JIRA)
liupengcheng created SPARK-26126: Summary: Should put scala-library deps into root pom instead of spark-tags module Key: SPARK-26126 URL: https://issues.apache.org/jira/browse/SPARK-26126 Project:

[jira] [Reopened] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov reopened SPARK-26019: --- > pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" > in

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692716#comment-16692716 ] Ruslan Dautkhanov commented on SPARK-26019: --- [~viirya] exception stack reads that error

[jira] [Created] (SPARK-26125) Delegation Token seems not appropriately stored on secrets of Kubernetes/Kerberized HDFS

2018-11-19 Thread Kei Kori (JIRA)
Kei Kori created SPARK-26125: Summary: Delegation Token seems not appropriately stored on secrets of Kubernetes/Kerberized HDFS Key: SPARK-26125 URL: https://issues.apache.org/jira/browse/SPARK-26125

[jira] [Updated] (SPARK-26125) Delegation Token seems not appropriately stored on secrets of Kubernetes/Kerberized HDFS

2018-11-19 Thread Kei Kori (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kei Kori updated SPARK-26125: - Attachment: spark-submit-stern.log > Delegation Token seems not appropriately stored on secrets of >

[jira] [Commented] (SPARK-25959) Difference in featureImportances results on computed vs saved models

2018-11-19 Thread Dagang Wei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692567#comment-16692567 ] Dagang Wei commented on SPARK-25959: Can we backport the fix "[SPARK-25959][ML] GBTClassifier picks

[jira] [Commented] (SPARK-26077) Reserved SQL words are not escaped by JDBC writer for table name

2018-11-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692546#comment-16692546 ] Takeshi Yamamuro commented on SPARK-26077: -- Thanks for your reporting! It seems the master

[jira] [Assigned] (SPARK-26124) Update plugins, including MiMa

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26124: Assignee: Apache Spark (was: Sean Owen) > Update plugins, including MiMa >

[jira] [Assigned] (SPARK-26124) Update plugins, including MiMa

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26124: Assignee: Sean Owen (was: Apache Spark) > Update plugins, including MiMa >

[jira] [Commented] (SPARK-26124) Update plugins, including MiMa

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692539#comment-16692539 ] Apache Spark commented on SPARK-26124: -- User 'srowen' has created a pull request for this issue:

[jira] [Created] (SPARK-26124) Update plugins, including MiMa

2018-11-19 Thread Sean Owen (JIRA)
Sean Owen created SPARK-26124: - Summary: Update plugins, including MiMa Key: SPARK-26124 URL: https://issues.apache.org/jira/browse/SPARK-26124 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-26107) Extend ReplaceNullWithFalseInPredicate to support higher-order functions: ArrayExists, ArrayFilter, MapFilter

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26107. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23079

[jira] [Assigned] (SPARK-26107) Extend ReplaceNullWithFalseInPredicate to support higher-order functions: ArrayExists, ArrayFilter, MapFilter

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26107: --- Assignee: Kris Mok > Extend ReplaceNullWithFalseInPredicate to support higher-order

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692504#comment-16692504 ] Liang-Chi Hsieh commented on SPARK-26019: - Isn't the server beginning to handle requests after

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692498#comment-16692498 ] Hyukjin Kwon commented on SPARK-26019: -- Please reopen if this is still being reproduced. Or if

[jira] [Updated] (SPARK-25919) Date value corrupts when tables are "ParquetHiveSerDe" formatted and target table is Partitioned

2018-11-19 Thread Pawan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pawan updated SPARK-25919: -- Affects Version/s: 2.1.0 > Date value corrupts when tables are "ParquetHiveSerDe" formatted and target >

[jira] [Resolved] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26019. -- Resolution: Cannot Reproduce > pyspark/accumulators.py: "TypeError: object of type 'NoneType'

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692495#comment-16692495 ] Hyukjin Kwon commented on SPARK-26019: -- I don't understand how the swapping two lines solve this

[jira] [Created] (SPARK-26123) Make all MLlib/ML Models public

2018-11-19 Thread Nicholas Resnick (JIRA)
Nicholas Resnick created SPARK-26123: Summary: Make all MLlib/ML Models public Key: SPARK-26123 URL: https://issues.apache.org/jira/browse/SPARK-26123 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692382#comment-16692382 ] Apache Spark commented on SPARK-26094: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26094: Assignee: Apache Spark (was: Imran Rashid) > Streaming WAL should create parent dirs >

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692380#comment-16692380 ] Apache Spark commented on SPARK-26094: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26094: Assignee: Imran Rashid (was: Apache Spark) > Streaming WAL should create parent dirs >

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692377#comment-16692377 ] Ruslan Dautkhanov commented on SPARK-26019: --- [~irashid] thanks a lot for looking at this ! It

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692355#comment-16692355 ] Imran Rashid commented on SPARK-26019: -- [~hyukjin.kwon] you think maybe these two lines need to be

[jira] [Assigned] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26122: Assignee: (was: Apache Spark) > Support encoding for multiLine in CSV datasource >

[jira] [Commented] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692306#comment-16692306 ] Apache Spark commented on SPARK-26122: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692305#comment-16692305 ] Apache Spark commented on SPARK-26122: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26122: Assignee: Apache Spark > Support encoding for multiLine in CSV datasource >

[jira] [Created] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26122: -- Summary: Support encoding for multiLine in CSV datasource Key: SPARK-26122 URL: https://issues.apache.org/jira/browse/SPARK-26122 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26118: Assignee: Apache Spark > Make Jetty's requestHeaderSize configurable in Spark >

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692248#comment-16692248 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692246#comment-16692246 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Assigned] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26118: Assignee: (was: Apache Spark) > Make Jetty's requestHeaderSize configurable in Spark

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692237#comment-16692237 ] Ruslan Dautkhanov commented on SPARK-26019: --- cc [~lucacanali] > pyspark/accumulators.py:

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692233#comment-16692233 ] Ruslan Dautkhanov commented on SPARK-26019: --- Sorry, nope it was broken by this change -

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692226#comment-16692226 ] Ruslan Dautkhanov commented on SPARK-26019: --- Might be broken by

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692219#comment-16692219 ] Ruslan Dautkhanov commented on SPARK-26019: --- [~hyukjin.kwon] today I reproduced this first

[jira] [Reopened] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov reopened SPARK-26019: --- Reproduced myself > pyspark/accumulators.py: "TypeError: object of type 'NoneType'

[jira] [Created] (SPARK-26121) [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id)

2018-11-19 Thread Anastasios Zouzias (JIRA)
Anastasios Zouzias created SPARK-26121: -- Summary: [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id) Key: SPARK-26121 URL:

[jira] [Assigned] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26120: Assignee: Shixiong Zhu (was: Apache Spark) > Fix a streaming query leak in Structured

[jira] [Created] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-26120: Summary: Fix a streaming query leak in Structured Streaming R tests Key: SPARK-26120 URL: https://issues.apache.org/jira/browse/SPARK-26120 Project: Spark

[jira] [Commented] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692178#comment-16692178 ] Apache Spark commented on SPARK-26120: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26119: Assignee: (was: Apache Spark) > Task metrics summary in the stage page should

[jira] [Commented] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692176#comment-16692176 ] Apache Spark commented on SPARK-26120: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26120: Assignee: Apache Spark (was: Shixiong Zhu) > Fix a streaming query leak in Structured

[jira] [Assigned] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26119: Assignee: Apache Spark > Task metrics summary in the stage page should contain only

[jira] [Commented] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692175#comment-16692175 ] Apache Spark commented on SPARK-26119: -- User 'shahidki31' has created a pull request for this

[jira] [Updated] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26120: - Priority: Minor (was: Major) > Fix a streaming query leak in Structured Streaming R tests >

[jira] [Updated] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26120: - Component/s: Structured Streaming SparkR > Fix a streaming query leak in

[jira] [Commented] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692093#comment-16692093 ] shahid commented on SPARK-26119: Thanks. I am working on it > Task metrics summary in the stage page

[jira] [Created] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-26119: Summary: Task metrics summary in the stage page should contain only successful tasks metrics Key: SPARK-26119 URL:

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692018#comment-16692018 ] Attila Zsolt Piros commented on SPARK-26118: I am working on a PR. > Make Jetty's

[jira] [Updated] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-26118: --- Affects Version/s: (was: 2.3.2) (was: 2.4.0)

[jira] [Created] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
Attila Zsolt Piros created SPARK-26118: -- Summary: Make Jetty's requestHeaderSize configurable in Spark Key: SPARK-26118 URL: https://issues.apache.org/jira/browse/SPARK-26118 Project: Spark

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-19 Thread Long Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691965#comment-16691965 ] Long Cao commented on SPARK-26026: -- [~srowen] thanks for the expedient resolution! It's a small thing

[jira] [Commented] (SPARK-23886) update query.status

2018-11-19 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691881#comment-16691881 ] Gabor Somogyi commented on SPARK-23886: --- Thanks [~efim.poberezkin]! May I ask you then to close

[jira] [Resolved] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26090. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23065

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691828#comment-16691828 ] Sunil Rangwani commented on SPARK-14492: [~ottomata] you are using a CDH version I believe. I

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691811#comment-16691811 ] Sunil Rangwani commented on SPARK-14492: [~srowen] Please reopen this. It is not about

[jira] [Updated] (SPARK-25528) data source V2 API refactoring (batch read)

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25528: Summary: data source V2 API refactoring (batch read) (was: data source V2 read side API

[jira] [Commented] (SPARK-25528) data source V2 API refactoring (batch read)

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691799#comment-16691799 ] Apache Spark commented on SPARK-25528: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-26071) disallow map as map key

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26071. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23045

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691787#comment-16691787 ] Andrew Otto commented on SPARK-14492: - I found my issue: we were loading some Hive 1.1.0 jars

[jira] [Resolved] (SPARK-26112) Update since versions of new built-in functions.

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26112. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23082

[jira] [Assigned] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26024: --- Assignee: Julien Peloton > Dataset API: repartitionByRange(...) has inconsistent behaviour

[jira] [Resolved] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26024. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23025

[jira] [Assigned] (SPARK-26112) Update since versions of new built-in functions.

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26112: --- Assignee: Takuya Ueshin > Update since versions of new built-in functions. >

[jira] [Assigned] (SPARK-26068) ChunkedByteBufferInputStream is truncated by empty chunk

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26068: --- Assignee: Liu, Linhong > ChunkedByteBufferInputStream is truncated by empty chunk >

[jira] [Resolved] (SPARK-26068) ChunkedByteBufferInputStream is truncated by empty chunk

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26068. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23040

[jira] [Commented] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691725#comment-16691725 ] Apache Spark commented on SPARK-26117: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26043) Make SparkHadoopUtil private to Spark

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26043: - Assignee: Sean Owen > Make SparkHadoopUtil private to Spark >

[jira] [Resolved] (SPARK-26043) Make SparkHadoopUtil private to Spark

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26043. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23066

[jira] [Resolved] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26026. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23069

[jira] [Assigned] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26117: Assignee: (was: Apache Spark) > use SparkOutOfMemoryError instead of

[jira] [Assigned] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26117: Assignee: Apache Spark > use SparkOutOfMemoryError instead of OutOfMemoryError when

[jira] [Created] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread caoxuewen (JIRA)
caoxuewen created SPARK-26117: - Summary: use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception Key: SPARK-26117 URL: https://issues.apache.org/jira/browse/SPARK-26117 Project: Spark

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Description: Trying to use _coalesce_ after shuffle-oriented transformations leads

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Summary: Spark SQL - Sort when writing partitioned parquet leads to OOM errors (was:

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {code:java}partitionBy{code}, it

[jira] [Created] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
Pierre Lienhart created SPARK-26116: --- Summary: Spark SQL - Sort when writing partitioned parquet lead to OOM errors Key: SPARK-26116 URL: https://issues.apache.org/jira/browse/SPARK-26116 Project:

[jira] [Resolved] (SPARK-26115) Fix deprecated warnings related to scala 2.12 in core module

2018-11-19 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-26115. Resolution: Duplicate Duplicated to SPARK-2609 > Fix deprecated warnings related to

[jira] [Created] (SPARK-26115) Fix deprecated warnings related to scala 2.12 in core module

2018-11-19 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-26115: -- Summary: Fix deprecated warnings related to scala 2.12 in core module Key: SPARK-26115 URL: https://issues.apache.org/jira/browse/SPARK-26115 Project: Spark

[jira] [Assigned] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26114: Assignee: (was: Apache Spark) > Memory leak of PartitionedPairBuffer when coalescing

  1   2   >