[jira] [Created] (SPARK-30632) to_timestamp() doesn't work with certain timezones

2020-01-24 Thread Anton Daitche (Jira)
Anton Daitche created SPARK-30632: - Summary: to_timestamp() doesn't work with certain timezones Key: SPARK-30632 URL: https://issues.apache.org/jira/browse/SPARK-30632 Project: Spark Issue

[jira] [Updated] (SPARK-30633) Codegen fails when xxHash seed is not an integer

2020-01-24 Thread Patrick Cording (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Cording updated SPARK-30633: Description: If the seed for xxHash is not an integer the generated code does not

[jira] [Comment Edited] (SPARK-30421) Dropped columns still available for filtering

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022850#comment-17022850 ] Dongjoon Hyun edited comment on SPARK-30421 at 1/24/20 10:44 AM: - While

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022850#comment-17022850 ] Dongjoon Hyun commented on SPARK-30421: --- While rethinking about this, the original column's index

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-24 Thread Tobias Hermann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022900#comment-17022900 ] Tobias Hermann commented on SPARK-30421: [~dongjoon] I'm glad we are aligned now. :) For future

[jira] [Commented] (SPARK-30617) Is there any possible that spark no longer restrict enumerate types of spark.sql.catalogImplementation

2020-01-24 Thread weiwenda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022869#comment-17022869 ] weiwenda commented on SPARK-30617: -- [~dongjoon] Thanks for your advise. I will write Fix versions /

[jira] [Updated] (SPARK-30634) Delta Merge and Arbitrary Stateful Processing in Structured streaming (foreachBatch)

2020-01-24 Thread Yurii Oleynikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yurii Oleynikov updated SPARK-30634: Description: Hi , I have an application that makes Arbitrary Stateful Processing in

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022835#comment-17022835 ] Dongjoon Hyun commented on SPARK-30421: --- Nope. Your example is different. I illustrated what I

[jira] [Commented] (SPARK-30635) Document PARTITIONED BY Clause of CREATE statement in SQL Reference

2020-01-24 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022966#comment-17022966 ] jobit mathew commented on SPARK-30635: -- I will work on this > Document PARTITIONED BY Clause of

[jira] [Created] (SPARK-30635) Document PARTITIONED BY Clause of CREATE statement in SQL Reference

2020-01-24 Thread jobit mathew (Jira)
jobit mathew created SPARK-30635: Summary: Document PARTITIONED BY Clause of CREATE statement in SQL Reference Key: SPARK-30635 URL: https://issues.apache.org/jira/browse/SPARK-30635 Project: Spark

[jira] [Updated] (SPARK-30634) Delta Merge and Arbitrary Stateful Processing in Structured streaming (foreachBatch)

2020-01-24 Thread Yurii Oleynikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yurii Oleynikov updated SPARK-30634: Attachment: Capture1.PNG > Delta Merge and Arbitrary Stateful Processing in Structured

[jira] [Created] (SPARK-30634) Delta Merge and Arbitrary Stateful Processing in Structured streaming (foreachBatch)

2020-01-24 Thread Yurii Oleynikov (Jira)
Yurii Oleynikov created SPARK-30634: --- Summary: Delta Merge and Arbitrary Stateful Processing in Structured streaming (foreachBatch) Key: SPARK-30634 URL: https://issues.apache.org/jira/browse/SPARK-30634

[jira] [Created] (SPARK-30633) Codegen fails when xxHash seed is not an integer

2020-01-24 Thread Patrick Cording (Jira)
Patrick Cording created SPARK-30633: --- Summary: Codegen fails when xxHash seed is not an integer Key: SPARK-30633 URL: https://issues.apache.org/jira/browse/SPARK-30633 Project: Spark Issue

[jira] [Commented] (SPARK-30218) Columns used in inequality conditions for joins not resolved correctly in case of common lineage

2020-01-24 Thread Rahul Kumar Challapalli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022969#comment-17022969 ] Rahul Kumar Challapalli commented on SPARK-30218: - [~dongjoon] I am not sure but I was

[jira] [Updated] (SPARK-30637) upgrade testthat on jenkins workers to 2.0.0

2020-01-24 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-30637: Parent: SPARK-23435 Issue Type: Sub-task (was: Task) > upgrade testthat on jenkins

[jira] [Created] (SPARK-30637) upgrade testthat on jenkins workers to 2.0.0

2020-01-24 Thread Shane Knapp (Jira)
Shane Knapp created SPARK-30637: --- Summary: upgrade testthat on jenkins workers to 2.0.0 Key: SPARK-30637 URL: https://issues.apache.org/jira/browse/SPARK-30637 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-30637) upgrade testthat on jenkins workers to 2.0.0

2020-01-24 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023202#comment-17023202 ] Shane Knapp commented on SPARK-30637: - ok, i was able to easily uninstall 1.0.2 and reinstall 2.0.0

[jira] [Assigned] (SPARK-29924) Document Arrow requirement in JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29924: - Assignee: Dongjoon Hyun > Document Arrow requirement in JDK9+ >

[jira] [Resolved] (SPARK-29924) Document Arrow requirement in JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29924. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27356

[jira] [Updated] (SPARK-30626) Add SPARK_APPLICATION_ID into driver pod env

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30626: -- Summary: Add SPARK_APPLICATION_ID into driver pod env (was: [K8S] Spark driver pod doesn't

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2020-01-24 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023250#comment-17023250 ] Shane Knapp commented on SPARK-28900: - FYI:  i will be OOO next week (mon-thur) with VERY limited

[jira] [Updated] (SPARK-29924) Document Arrow requirement in JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29924: -- Description: At least, we need to mention `io.netty.tryReflectionSetAccessible=true` is

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023224#comment-17023224 ] Dongjoon Hyun commented on SPARK-28900: --- Hi, All. Can we restart this before `branch-3.0` cut

[jira] [Updated] (SPARK-30626) Add SPARK_APPLICATION_ID into driver pod env

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30626: -- Affects Version/s: (was: 2.4.4) > Add SPARK_APPLICATION_ID into driver pod env >

[jira] [Resolved] (SPARK-30630) Deprecate numTrees in GBT at 2.4.5 and remove it at 3.0.0

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30630. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-30630) Deprecate numTrees in GBT

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30630: - Assignee: Huaxin Gao > Deprecate numTrees in GBT > - > >

[jira] [Updated] (SPARK-30630) Deprecate numTrees in GBT at 2.4.5 and remove it at 3.0.0

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30630: -- Summary: Deprecate numTrees in GBT at 2.4.5 and remove it at 3.0.0 (was: Deprecate numTrees

[jira] [Created] (SPARK-30639) Upgrade Jersey to 2.30

2020-01-24 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30639: - Summary: Upgrade Jersey to 2.30 Key: SPARK-30639 URL: https://issues.apache.org/jira/browse/SPARK-30639 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28704: -- Target Version/s: 3.1.0 > Test backward compatibility on JDK9+ once we have a version

[jira] [Updated] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28704: -- Labels: 3.1.0 (was: ) > Test backward compatibility on JDK9+ once we have a version supports

[jira] [Updated] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28704: -- Parent: (was: SPARK-29194) Issue Type: Test (was: Sub-task) > Test backward

[jira] [Updated] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28704: -- Labels: (was: 3.1.0) > Test backward compatibility on JDK9+ once we have a version supports

[jira] [Commented] (SPARK-30632) to_timestamp() doesn't work with certain timezones

2020-01-24 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023263#comment-17023263 ] Maxim Gekk commented on SPARK-30632: Spark 2.4 and earlier versions use SimpleDateFormat to parse

[jira] [Created] (SPARK-30638) add resources as parameter to the PluginContext

2020-01-24 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30638: - Summary: add resources as parameter to the PluginContext Key: SPARK-30638 URL: https://issues.apache.org/jira/browse/SPARK-30638 Project: Spark Issue

[jira] [Updated] (SPARK-25382) Remove ImageSchema.readImages in 3.0

2020-01-24 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25382: - Docs Text: In Spark 3.0.0, the deprecated ImageSchema class and its readImages methods have

[jira] [Updated] (SPARK-25382) Remove ImageSchema.readImages in 3.0

2020-01-24 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25382: - Docs Text: In Spark 3.0.0, the deprecated ImageSchema class and its readImages methods have

[jira] [Commented] (SPARK-23435) R tests should support latest testthat

2020-01-24 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023194#comment-17023194 ] Shane Knapp commented on SPARK-23435: - since we can't have different R environments for different

[jira] [Resolved] (SPARK-30626) [K8S] Spark driver pod doesn't have SPARK_APPLICATION_ID env

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30626. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27347

[jira] [Assigned] (SPARK-30626) [K8S] Spark driver pod doesn't have SPARK_APPLICATION_ID env

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30626: - Assignee: Jiaxin Shan > [K8S] Spark driver pod doesn't have SPARK_APPLICATION_ID env >

[jira] [Comment Edited] (SPARK-30632) to_timestamp() doesn't work with certain timezones

2020-01-24 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023263#comment-17023263 ] Maxim Gekk edited comment on SPARK-30632 at 1/24/20 9:12 PM: - Spark 2.4 and

[jira] [Updated] (SPARK-30639) Upgrade Jersey to 2.30

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30639: -- Parent: SPARK-29194 Issue Type: Sub-task (was: Improvement) > Upgrade Jersey to 2.30

[jira] [Assigned] (SPARK-30636) Unable to add packages on spark-packages.org

2020-01-24 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30636: --- Assignee: Burak Yavuz > Unable to add packages on spark-packages.org >

[jira] [Updated] (SPARK-30636) Unable to add packages on spark-packages.org

2020-01-24 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30636: Affects Version/s: (was: 3.0.0) 2.4.4 > Unable to add packages on

[jira] [Commented] (SPARK-30612) can't resolve qualified column name with v2 tables

2020-01-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023036#comment-17023036 ] Wenchen Fan commented on SPARK-30612: - I think the example from [~brkyvz] is right. The column name

[jira] [Created] (SPARK-30636) Unable to add packages on spark-packages.org

2020-01-24 Thread Xiao Li (Jira)
Xiao Li created SPARK-30636: --- Summary: Unable to add packages on spark-packages.org Key: SPARK-30636 URL: https://issues.apache.org/jira/browse/SPARK-30636 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-30218) Columns used in inequality conditions for joins not resolved correctly in case of common lineage

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023118#comment-17023118 ] Dongjoon Hyun edited comment on SPARK-30218 at 1/24/20 5:10 PM: No, this

[jira] [Commented] (SPARK-30218) Columns used in inequality conditions for joins not resolved correctly in case of common lineage

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023118#comment-17023118 ] Dongjoon Hyun commented on SPARK-30218: --- No, this is fixed. The following is the same case. User

[jira] [Comment Edited] (SPARK-30218) Columns used in inequality conditions for joins not resolved correctly in case of common lineage

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023118#comment-17023118 ] Dongjoon Hyun edited comment on SPARK-30218 at 1/24/20 5:10 PM: No, this

[jira] [Resolved] (SPARK-30617) Is there any possible that spark no longer restrict enumerate types of spark.sql.catalogImplementation

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30617. --- Resolution: Duplicate > Is there any possible that spark no longer restrict enumerate types

[jira] [Reopened] (SPARK-30617) Is there any possible that spark no longer restrict enumerate types of spark.sql.catalogImplementation

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-30617: --- > Is there any possible that spark no longer restrict enumerate types of >

[jira] [Closed] (SPARK-30617) Is there any possible that spark no longer restrict enumerate types of spark.sql.catalogImplementation

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-30617. - > Is there any possible that spark no longer restrict enumerate types of >

[jira] [Resolved] (SPARK-29721) Spark SQL reads unnecessary nested fields from Parquet after using explode

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29721. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26978

[jira] [Assigned] (SPARK-29721) Spark SQL reads unnecessary nested fields from Parquet after using explode

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29721: - Assignee: L. C. Hsieh > Spark SQL reads unnecessary nested fields from Parquet after

[jira] [Updated] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25330: -- Fix Version/s: 2.3.2 > Permission issue after upgrade hadoop version to 2.7.7 >

[jira] [Assigned] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25330: - Assignee: Yuming Wang > Permission issue after upgrade hadoop version to 2.7.7 >

[jira] [Commented] (SPARK-28921) Spark jobs failing on latest versions of Kubernetes (1.15.3, 1.14.6, 1,13.10, 1.12.10, 1.11.10)

2020-01-24 Thread Mathew Wicks (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023439#comment-17023439 ] Mathew Wicks commented on SPARK-28921: -- [~dongjoon], it's just very bad practice to not update all

[jira] [Commented] (SPARK-19248) Regex_replace works in 1.6 but not in 2.0

2020-01-24 Thread Jeff Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023019#comment-17023019 ] Jeff Evans commented on SPARK-19248: I'm not a Spark maintainer, so can't answer definitively.

[jira] [Updated] (SPARK-30636) Unable to add packages on spark-packages.org

2020-01-24 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30636: Priority: Critical (was: Blocker) > Unable to add packages on spark-packages.org >

[jira] [Resolved] (SPARK-30617) Is there any possible that spark no longer restrict enumerate types of spark.sql.catalogImplementation

2020-01-24 Thread weiwenda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiwenda resolved SPARK-30617. -- Resolution: Pending Closed > Is there any possible that spark no longer restrict enumerate types of

[jira] [Updated] (SPARK-29721) Spark SQL reads unnecessary nested fields after using explode

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29721: -- Summary: Spark SQL reads unnecessary nested fields after using explode (was: Spark SQL reads

[jira] [Resolved] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25330. --- Resolution: Fixed > Permission issue after upgrade hadoop version to 2.7.7 >

[jira] [Updated] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25330: -- Fix Version/s: 2.4.0 > Permission issue after upgrade hadoop version to 2.7.7 >

[jira] [Updated] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-27117: Summary: current_date/current_timestamp should be reserved keywords in ansi parser mode (was:

[jira] [Commented] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023317#comment-17023317 ] Reynold Xin commented on SPARK-27117: - I changed the title to make it more clear to end users what's

[jira] [Commented] (SPARK-29189) Add an option to ignore block locations when listing file

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023340#comment-17023340 ] Reynold Xin commented on SPARK-29189: - This is great, but how would users know when to set this?

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2020-01-24 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023354#comment-17023354 ] Dongjoon Hyun commented on SPARK-28900: --- Thank you for update. > Test Pyspark, SparkR on JDK 11

[jira] [Created] (SPARK-30640) Prevent unnessary copies of data in Arrow to Pandas conversion with Timestamps

2020-01-24 Thread Bryan Cutler (Jira)
Bryan Cutler created SPARK-30640: Summary: Prevent unnessary copies of data in Arrow to Pandas conversion with Timestamps Key: SPARK-30640 URL: https://issues.apache.org/jira/browse/SPARK-30640