[jira] [Created] (SPARK-32585) Support scala enumeration in ScalaReflection

2020-08-10 Thread ulysses you (Jira)
ulysses you created SPARK-32585: --- Summary: Support scala enumeration in ScalaReflection Key: SPARK-32585 URL: https://issues.apache.org/jira/browse/SPARK-32585 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175237#comment-17175237 ] Apache Spark commented on SPARK-32584: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32584: Assignee: Apache Spark > Exclude _images and _sources that are generated by Sphinx in

[jira] [Assigned] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32584: Assignee: (was: Apache Spark) > Exclude _images and _sources that are generated by

[jira] [Updated] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32584: - Description: For _images directory, after SPARK-31851, now we added some images to use within

[jira] [Updated] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32584: - Attachment: Screen Shot 2020-08-11 at 1.52.46 PM.png > Exclude _images and _sources that are

[jira] [Created] (SPARK-32584) Exclude _images and _sources that are generated by Sphinx in Jekyll build

2020-08-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32584: Summary: Exclude _images and _sources that are generated by Sphinx in Jekyll build Key: SPARK-32584 URL: https://issues.apache.org/jira/browse/SPARK-32584 Project:

[jira] [Commented] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Ayrat Sadreev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175219#comment-17175219 ] Ayrat Sadreev commented on SPARK-32580: --- If you select values from one of the arrays, it works

[jira] [Resolved] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32583. -- Resolution: Invalid > PySpark Structured Streaming Testing Support >

[jira] [Reopened] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-32583: -- > PySpark Structured Streaming Testing Support > > >

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175212#comment-17175212 ] Hyukjin Kwon commented on SPARK-32583: -- [~FelixKJose], let's interact in the mailing list or

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175205#comment-17175205 ] Rohit Mishra commented on SPARK-32583: -- cc [~hyukjin.kwon].  > PySpark Structured Streaming

[jira] [Commented] (SPARK-32582) Spark SQL Infer Schema Performance

2020-08-10 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175203#comment-17175203 ] Lantao Jin commented on SPARK-32582: Maybe we could offer a new interface to break out in one

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175202#comment-17175202 ] Felix Kizhakkel Jose commented on SPARK-32583: -- [~rohitmishr1484] It would have been nice

[jira] [Resolved] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra resolved SPARK-32583. -- Resolution: Not A Problem > PySpark Structured Streaming Testing Support >

[jira] [Commented] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175200#comment-17175200 ] Rohit Mishra commented on SPARK-32583: -- [~FelixKJose], Questions are usually answered using either

[jira] [Commented] (SPARK-32582) Spark SQL Infer Schema Performance

2020-08-10 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175197#comment-17175197 ] Lantao Jin commented on SPARK-32582: I see. The implementation of {{inferSchema}} method depends on

[jira] [Assigned] (SPARK-32216) Remove redundant ProjectExec

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32216: --- Assignee: Allison Wang > Remove redundant ProjectExec > > >

[jira] [Resolved] (SPARK-32216) Remove redundant ProjectExec

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32216. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29031

[jira] [Assigned] (SPARK-32400) Test coverage of HiveScripTransformationExec

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32400: Assignee: (was: Apache Spark) > Test coverage of HiveScripTransformationExec >

[jira] [Assigned] (SPARK-32400) Test coverage of HiveScripTransformationExec

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32400: Assignee: Apache Spark > Test coverage of HiveScripTransformationExec >

[jira] [Commented] (SPARK-32400) Test coverage of HiveScripTransformationExec

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175167#comment-17175167 ] Apache Spark commented on SPARK-32400: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-32582) Spark SQL Infer Schema Performance

2020-08-10 Thread Jarred Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175165#comment-17175165 ] Jarred Li commented on SPARK-32582: --- The performance I mentioned here is not the read file, but "LIST"

[jira] [Updated] (SPARK-32543) Remove arrow::as_tibble usage in SparkR

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32543: - Fix Version/s: 3.0.1 > Remove arrow::as_tibble usage in SparkR >

[jira] [Commented] (SPARK-32582) Spark SQL Infer Schema Performance

2020-08-10 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175143#comment-17175143 ] Lantao Jin commented on SPARK-32582: {code} files.toIterator.map(file => readSchema(file.getPath,

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2020-08-10 Thread Tianchen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175117#comment-17175117 ] Tianchen Zhang commented on SPARK-25299: Hi [~mcheah], by following the history of this remote

[jira] [Created] (SPARK-32583) PySpark Structured Streaming Testing Support

2020-08-10 Thread Felix Kizhakkel Jose (Jira)
Felix Kizhakkel Jose created SPARK-32583: Summary: PySpark Structured Streaming Testing Support Key: SPARK-32583 URL: https://issues.apache.org/jira/browse/SPARK-32583 Project: Spark

[jira] [Updated] (SPARK-32528) The analyze method should make sure the plan is analyzed

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32528: -- Fix Version/s: 3.0.1 > The analyze method should make sure the plan is analyzed >

[jira] [Commented] (SPARK-32571) yarnClient.killApplication(appId) is never called

2020-08-10 Thread A Tester (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175096#comment-17175096 ] A Tester commented on SPARK-32571: -- [~viirya] , if the application is a spark-streaming application, I

[jira] [Updated] (SPARK-32577) Fix the config value for shuffled hash join in test in-joins.sql

2020-08-10 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32577: - Parent: SPARK-32461 Issue Type: Sub-task (was: Improvement) > Fix the config value for

[jira] [Commented] (SPARK-32530) SPIP: Kotlin support for Apache Spark

2020-08-10 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175052#comment-17175052 ] Sean R. Owen commented on SPARK-32530: -- I speak for myself, but, I think the cost is viewed as very

[jira] [Commented] (SPARK-32530) SPIP: Kotlin support for Apache Spark

2020-08-10 Thread Pasha Finkeshteyn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175010#comment-17175010 ] Pasha Finkeshteyn commented on SPARK-32530: --- Hi [~srowen], we agree with you that it’s good to

[jira] [Commented] (SPARK-32528) The analyze method should make sure the plan is analyzed

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174989#comment-17174989 ] Apache Spark commented on SPARK-32528: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32528) The analyze method should make sure the plan is analyzed

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174988#comment-17174988 ] Apache Spark commented on SPARK-32528: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32469) ApplyColumnarRulesAndInsertTransitions should be idempotent

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32469. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29273

[jira] [Resolved] (SPARK-32403) SCRIP TRANSFORM Extract common method from process row to avoid repeated judgement

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32403. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29199

[jira] [Assigned] (SPARK-32403) SCRIP TRANSFORM Extract common method from process row to avoid repeated judgement

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32403: --- Assignee: angerszhu > SCRIP TRANSFORM Extract common method from process row to avoid

[jira] [Updated] (SPARK-32409) Document the dependency between spark.metrics.staticSources.enabled and JVMSource registration

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32409: -- Issue Type: Documentation (was: Bug) > Document the dependency between

[jira] [Assigned] (SPARK-32409) Document the dependency between spark.metrics.staticSources.enabled and JVMSource registration

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32409: - Assignee: Luca Canali > Document the dependency between

[jira] [Resolved] (SPARK-32409) Document the dependency between spark.metrics.staticSources.enabled and JVMSource registration

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32409. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Resolved] (SPARK-32235) Kubernetes Configuration to set Service Account to Executors

2020-08-10 Thread Pedro Rossi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Rossi resolved SPARK-32235. - Fix Version/s: 3.1.0 Resolution: Fixed Issue was solved by SPARK-30122, but the same was

[jira] [Commented] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174361#comment-17174361 ] Takeshi Yamamuro commented on SPARK-32580: -- I made the given query simpler; {code} scala>

[jira] [Comment Edited] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174361#comment-17174361 ] Takeshi Yamamuro edited comment on SPARK-32580 at 8/10/20, 2:38 PM:

[jira] [Assigned] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32517: - Assignee: Dongjoon Hyun > Add StorageLevel.DISK_ONLY_3 >

[jira] [Resolved] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32517. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29331

[jira] [Created] (SPARK-32582) Spark SQL Infer Schema Performance

2020-08-10 Thread Jarred Li (Jira)
Jarred Li created SPARK-32582: - Summary: Spark SQL Infer Schema Performance Key: SPARK-32582 URL: https://issues.apache.org/jira/browse/SPARK-32582 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-32554) Remove the words "experimental" in the k8s document

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32554: -- Description: To make users understood more correctly about the current development status of

[jira] [Updated] (SPARK-32554) Remove the words "experimental" in the k8s document

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32554: -- Summary: Remove the words "experimental" in the k8s document (was: Update the k8s document

[jira] [Assigned] (SPARK-32554) Update the k8s document according to the current development status

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32554: - Assignee: Takeshi Yamamuro > Update the k8s document according to the current

[jira] [Resolved] (SPARK-32554) Update the k8s document according to the current development status

2020-08-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32554. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29368

[jira] [Updated] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Ayrat Sadreev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayrat Sadreev updated SPARK-32580: -- Attachment: ExplodeTest.java > Issue accessing a column values after 'explode' function >

[jira] [Updated] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Ayrat Sadreev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayrat Sadreev updated SPARK-32580: -- Attachment: data.json > Issue accessing a column values after 'explode' function >

[jira] [Comment Edited] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174303#comment-17174303 ] Takeshi Yamamuro edited comment on SPARK-32580 at 8/10/20, 1:26 PM:

[jira] [Commented] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174303#comment-17174303 ] Takeshi Yamamuro commented on SPARK-32580: -- Please do not use "Blocker" in the priority and it

[jira] [Updated] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32580: - Component/s: (was: Spark Core) > Issue accessing a column values after 'explode'

[jira] [Updated] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32580: - Priority: Major (was: Blocker) > Issue accessing a column values after 'explode'

[jira] [Assigned] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32581: Assignee: Apache Spark > update duration property for live ui application list and

[jira] [Commented] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174283#comment-17174283 ] Apache Spark commented on SPARK-32581: -- User 'zhli1142015' has created a pull request for this

[jira] [Assigned] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32581: Assignee: (was: Apache Spark) > update duration property for live ui application

[jira] [Commented] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174286#comment-17174286 ] Apache Spark commented on SPARK-32581: -- User 'zhli1142015' has created a pull request for this

[jira] [Updated] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-32581: Attachment: updatedapiJPG.JPG oldapi.JPG > update duration property for live ui

[jira] [Created] (SPARK-32581) update duration property for live ui application list and application apis

2020-08-10 Thread Zhen Li (Jira)
Zhen Li created SPARK-32581: --- Summary: update duration property for live ui application list and application apis Key: SPARK-32581 URL: https://issues.apache.org/jira/browse/SPARK-32581 Project: Spark

[jira] [Comment Edited] (SPARK-29767) Core dump happening on executors while doing simple union of Data Frames

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174263#comment-17174263 ] Takeshi Yamamuro edited comment on SPARK-29767 at 8/10/20, 11:48 AM: -

[jira] [Commented] (SPARK-29767) Core dump happening on executors while doing simple union of Data Frames

2020-08-10 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174263#comment-17174263 ] Takeshi Yamamuro commented on SPARK-29767: -- I checked the given query and I think your union

[jira] [Created] (SPARK-32580) Issue accessing a column values after 'explode' function

2020-08-10 Thread Ayrat Sadreev (Jira)
Ayrat Sadreev created SPARK-32580: - Summary: Issue accessing a column values after 'explode' function Key: SPARK-32580 URL: https://issues.apache.org/jira/browse/SPARK-32580 Project: Spark

[jira] [Updated] (SPARK-32290) NotInSubquery SingleColumn Optimize

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-32290: Priority: Major (was: Minor) > NotInSubquery SingleColumn Optimize >

[jira] [Updated] (SPARK-32409) Document the dependency between spark.metrics.staticSources.enabled and JVMSource registration

2020-08-10 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-32409: Summary: Document the dependency between spark.metrics.staticSources.enabled and JVMSource

[jira] [Updated] (SPARK-32573) Anti Join Improvement with EmptyHashedRelation and EmptyHashedRelationWithAllNullKeys

2020-08-10 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-32573: Description: In SPARK-32290, we introduced several new types of HashedRelation *

[jira] [Commented] (SPARK-32576) Support PostgreSQL `bpchar` type and array of char type

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174152#comment-17174152 ] Apache Spark commented on SPARK-32576: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-32576) Support PostgreSQL `bpchar` type and array of char type

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174151#comment-17174151 ] Apache Spark commented on SPARK-32576: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-32579) Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174129#comment-17174129 ] Apache Spark commented on SPARK-32579: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-32579) Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174128#comment-17174128 ] Apache Spark commented on SPARK-32579: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32579) Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32579: Assignee: Apache Spark > Implement JDBCScan/ScanBuilder/WriteBuilder >

[jira] [Assigned] (SPARK-32579) Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32579: Assignee: (was: Apache Spark) > Implement JDBCScan/ScanBuilder/WriteBuilder >

[jira] [Created] (SPARK-32579) Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-10 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-32579: -- Summary: Implement JDBCScan/ScanBuilder/WriteBuilder Key: SPARK-32579 URL: https://issues.apache.org/jira/browse/SPARK-32579 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-10 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174113#comment-17174113 ] Apache Spark commented on SPARK-32518: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32191) Migration Guide

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32191. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29385

[jira] [Assigned] (SPARK-32191) Migration Guide

2020-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32191: Assignee: L. C. Hsieh > Migration Guide > --- > > Key:

[jira] [Updated] (SPARK-32578) PageRank not sending the correct values in Pergel sendMessage

2020-08-10 Thread Shay Elbaz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Elbaz updated SPARK-32578: --- Description: The core sendMessage method is incorrect: {code:java} def sendMessage(edge:

[jira] [Created] (SPARK-32578) PageRank not sending the correct values in Pergel sendMessage

2020-08-10 Thread Shay Elbaz (Jira)
Shay Elbaz created SPARK-32578: -- Summary: PageRank not sending the correct values in Pergel sendMessage Key: SPARK-32578 URL: https://issues.apache.org/jira/browse/SPARK-32578 Project: Spark

[jira] [Assigned] (SPARK-32337) Show initial plan in AQE plan tree string

2020-08-10 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32337: --- Assignee: Allison Wang > Show initial plan in AQE plan tree string >