[jira] [Created] (SPARK-31242) Clone SparkSession should respect spark.sql.legacy.sessionInitWithConfigDefaults

2020-03-24 Thread wuyi (Jira)
wuyi created SPARK-31242: Summary: Clone SparkSession should respect spark.sql.legacy.sessionInitWithConfigDefaults Key: SPARK-31242 URL: https://issues.apache.org/jira/browse/SPARK-31242 Project: Spark

[jira] [Resolved] (SPARK-31207) Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks

2020-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31207. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27972

[jira] [Resolved] (SPARK-31239) Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds

2020-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31239. -- Target Version/s: 3.1.0 Resolution: Fixed Fixed in

[jira] [Created] (SPARK-31241) Support Hive on DataSourceV2

2020-03-24 Thread Jackey Lee (Jira)
Jackey Lee created SPARK-31241: -- Summary: Support Hive on DataSourceV2 Key: SPARK-31241 URL: https://issues.apache.org/jira/browse/SPARK-31241 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-31240) Constant fold deterministic Scala UDFs with foldable arguments

2020-03-24 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31240. -- Resolution: Duplicate > Constant fold deterministic Scala UDFs with foldable

[jira] [Created] (SPARK-31240) Constant fold deterministic Scala UDFs with foldable arguments

2020-03-24 Thread Kris Mok (Jira)
Kris Mok created SPARK-31240: Summary: Constant fold deterministic Scala UDFs with foldable arguments Key: SPARK-31240 URL: https://issues.apache.org/jira/browse/SPARK-31240 Project: Spark

[jira] [Created] (SPARK-31239) Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds

2020-03-24 Thread Xingbo Jiang (Jira)
Xingbo Jiang created SPARK-31239: Summary: Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds Key: SPARK-31239 URL: https://issues.apache.org/jira/browse/SPARK-31239

[jira] [Commented] (SPARK-31238) Incompatible ORC dates with Spark 2.4

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066210#comment-17066210 ] Dongjoon Hyun commented on SPARK-31238: --- I confirmed the issue, too. > Incompatible ORC dates

[jira] [Updated] (SPARK-31236) Spark error while consuming data from Kinesis direct end point

2020-03-24 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31236: Priority: Critical (was: Blocker) > Spark error while consuming data from Kinesis direct end point >

[jira] [Commented] (SPARK-31081) Make display of stageId/stageAttemptId/taskId of sql metrics toggleable

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066188#comment-17066188 ] Dongjoon Hyun commented on SPARK-31081: --- Hi, [~Gengliang.Wang]. Please set the `Fix Version`. >

[jira] [Updated] (SPARK-31081) Make display of stageId/stageAttemptId/taskId of sql metrics toggleable

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31081: -- Fix Version/s: 3.0.0 > Make display of stageId/stageAttemptId/taskId of sql metrics

[jira] [Commented] (SPARK-31238) Incompatible ORC dates with Spark 2.4

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066179#comment-17066179 ] Dongjoon Hyun commented on SPARK-31238: --- This issue inherits the `Priority` of the parent issue,

[jira] [Updated] (SPARK-31238) Incompatible ORC dates with Spark 2.4

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31238: -- Priority: Blocker (was: Major) > Incompatible ORC dates with Spark 2.4 >

[jira] [Updated] (SPARK-31238) Incompatible ORC dates with Spark 2.4

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31238: -- Target Version/s: 3.0.0 > Incompatible ORC dates with Spark 2.4 >

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066177#comment-17066177 ] Dongjoon Hyun commented on SPARK-30951: --- Thank you so much, [~bersprockets]! > Potential data

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-03-24 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066170#comment-17066170 ] Bruce Robbins commented on SPARK-30951: --- I added a subtask for ORC. The issue is only for date

[jira] [Created] (SPARK-31238) Incompatible ORC dates with Spark 2.4

2020-03-24 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-31238: - Summary: Incompatible ORC dates with Spark 2.4 Key: SPARK-31238 URL: https://issues.apache.org/jira/browse/SPARK-31238 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-31081) Make display of stageId/stageAttemptId/taskId of sql metrics toggleable

2020-03-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31081: --- Summary: Make display of stageId/stageAttemptId/taskId of sql metrics toggleable (was:

[jira] [Resolved] (SPARK-31081) Make the display of stageId/stageAttemptId/taskId of sql metrics configurable in UI

2020-03-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31081. Assignee: Kousuke Saruta Resolution: Fixed The issue is resolved in

[jira] [Updated] (SPARK-31081) Make the display of stageId/stageAttemptId/taskId of sql metrics configurable in UI

2020-03-24 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31081: --- Affects Version/s: (was: 3.1.0) 3.0.0 > Make the display of

[jira] [Updated] (SPARK-30494) Duplicates cached RDD when create or replace an existing view

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30494: -- Fix Version/s: 2.4.6 > Duplicates cached RDD when create or replace an existing view >

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-03-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066159#comment-17066159 ] Dongjoon Hyun commented on SPARK-30951: --- Hi, [~bersprockets]. Could you file a new JIRA about

[jira] [Created] (SPARK-31237) Replace 3-letter time zones by zone offsets

2020-03-24 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31237: -- Summary: Replace 3-letter time zones by zone offsets Key: SPARK-31237 URL: https://issues.apache.org/jira/browse/SPARK-31237 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-03-24 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066153#comment-17066153 ] Bruce Robbins commented on SPARK-30951: --- [~cloud_fan] {quote} For ORC, it follows the Java

[jira] [Commented] (SPARK-31209) Not compatible with new version of scalatest (3.1.0 and above)

2020-03-24 Thread Timothy Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066090#comment-17066090 ] Timothy Zhang commented on SPARK-31209: --- Sure, I'll try to work on it soon.  > Not compatible

[jira] [Resolved] (SPARK-31161) Refactor the on-click timeline action in streagming-page.js

2020-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31161. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27921

[jira] [Assigned] (SPARK-30127) UDF should work for case class like Dataset operations

2020-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30127: --- Assignee: wuyi > UDF should work for case class like Dataset operations >

[jira] [Resolved] (SPARK-30127) UDF should work for case class like Dataset operations

2020-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30127. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27937

[jira] [Created] (SPARK-31236) Spark error while consuming data from Kinesis direct end point

2020-03-24 Thread Thukarama Prabhu (Jira)
Thukarama Prabhu created SPARK-31236: Summary: Spark error while consuming data from Kinesis direct end point Key: SPARK-31236 URL: https://issues.apache.org/jira/browse/SPARK-31236 Project:

[jira] [Resolved] (SPARK-31221) Rebase all dates/timestamps in conversion in Java types

2020-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31221. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27980

[jira] [Created] (SPARK-31235) Separates different categories of applications

2020-03-24 Thread wangzhun (Jira)
wangzhun created SPARK-31235: Summary: Separates different categories of applications Key: SPARK-31235 URL: https://issues.apache.org/jira/browse/SPARK-31235 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-31221) Rebase all dates/timestamps in conversion in Java types

2020-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31221: --- Assignee: Maxim Gekk > Rebase all dates/timestamps in conversion in Java types >

[jira] [Created] (SPARK-31234) ResetCommand should not wipe out all configs

2020-03-24 Thread Kent Yao (Jira)
Kent Yao created SPARK-31234: Summary: ResetCommand should not wipe out all configs Key: SPARK-31234 URL: https://issues.apache.org/jira/browse/SPARK-31234 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-31233) TimeoutException contains null remoteAddr

2020-03-24 Thread Yi Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Huang updated SPARK-31233: - Description: Application log:  {code:java} Failed to process batch

[jira] [Updated] (SPARK-31233) TimeoutException contains null remoteAddr

2020-03-24 Thread Yi Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Huang updated SPARK-31233: - Description: Application log:  {code:java} Failed to process batch

[jira] [Updated] (SPARK-31233) TimeoutException contains null remoteAddr

2020-03-24 Thread Yi Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Huang updated SPARK-31233: - Description: Application log:  {code:java} Failed to process batch

[jira] [Created] (SPARK-31233) TimeoutException contains null remoteAddr

2020-03-24 Thread Yi Huang (Jira)
Yi Huang created SPARK-31233: Summary: TimeoutException contains null remoteAddr Key: SPARK-31233 URL: https://issues.apache.org/jira/browse/SPARK-31233 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31232) Specify formats of `spark.sql.session.timeZone`

2020-03-24 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31232: -- Summary: Specify formats of `spark.sql.session.timeZone` Key: SPARK-31232 URL: https://issues.apache.org/jira/browse/SPARK-31232 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31231) Support setuptools 46.1.0+ in PySpark packaging

2020-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31231: - Priority: Blocker (was: Critical) > Support setuptools 46.1.0+ in PySpark packaging >

[jira] [Created] (SPARK-31231) Support setuptools 46.1.0+ in PySpark packaging

2020-03-24 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-31231: Summary: Support setuptools 46.1.0+ in PySpark packaging Key: SPARK-31231 URL: https://issues.apache.org/jira/browse/SPARK-31231 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-31220) repartition obeys spark.sql.adaptive.coalescePartitions.initialPartitionNum when spark.sql.adaptive.enabled

2020-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31220: Comment: was deleted (was: I'm working on.) > repartition obeys