[jira] [Commented] (SPARK-31517) SparkR::orderBy with multiple columns descending produces error

2020-04-27 Thread Michael Chirico (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094172#comment-17094172 ] Michael Chirico commented on SPARK-31517: - > Error in ns[[i]] : subscript out of bounds This

[jira] [Commented] (SPARK-31583) grouping_id calculation should be improved

2020-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094153#comment-17094153 ] Wenchen Fan commented on SPARK-31583: - cc [~maropu] > grouping_id calculation should be improved >

[jira] [Resolved] (SPARK-31589) Use `r-lib/actions/setup-r` in GitHub Action

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31589. -- Fix Version/s: 3.0.0 Assignee: Dongjoon Hyun Resolution: Fixed Fixed in

[jira] [Updated] (SPARK-31589) Use `r-lib/actions/setup-r` in GitHub Action

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31589: -- Description: `r-lib/actions/setup-r` is more stabler and maintained 3rd party action. I made

[jira] [Updated] (SPARK-31589) Use `r-lib/actions/setup-r` in GitHub Action

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31589: -- Issue Type: Bug (was: Improvement) > Use `r-lib/actions/setup-r` in GitHub Action >

[jira] [Resolved] (SPARK-31587) R installation in Github Actions is being failed

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31587. -- Resolution: Duplicate > R installation in Github Actions is being failed >

[jira] [Updated] (SPARK-31590) The filter used by Metadata-only queries should not have Unevaluable

2020-04-27 Thread dzcxzl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-31590: --- Description: When using SPARK-23877, some sql execution errors. code: {code:scala} sql("set

[jira] [Created] (SPARK-31590) The filter used by Metadata-only queries should not have Unevaluable

2020-04-27 Thread dzcxzl (Jira)
dzcxzl created SPARK-31590: -- Summary: The filter used by Metadata-only queries should not have Unevaluable Key: SPARK-31590 URL: https://issues.apache.org/jira/browse/SPARK-31590 Project: Spark

[jira] [Created] (SPARK-31589) Use `r-lib/actions/setup-r` in GitHub Action

2020-04-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31589: - Summary: Use `r-lib/actions/setup-r` in GitHub Action Key: SPARK-31589 URL: https://issues.apache.org/jira/browse/SPARK-31589 Project: Spark Issue Type:

[jira] [Created] (SPARK-31588) merge small files may need more common setting

2020-04-27 Thread philipse (Jira)
philipse created SPARK-31588: Summary: merge small files may need more common setting Key: SPARK-31588 URL: https://issues.apache.org/jira/browse/SPARK-31588 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31587) R installation in Github Actions is being failed

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31587: - Summary: R installation in Github Actions is being failed (was: Fixes the repository for

[jira] [Created] (SPARK-31587) Fixes the repository for downloading R in Github Actions

2020-04-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-31587: Summary: Fixes the repository for downloading R in Github Actions Key: SPARK-31587 URL: https://issues.apache.org/jira/browse/SPARK-31587 Project: Spark

[jira] [Updated] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31585: Description: Z-ordering is a technique that allows you to map multidimensional data to a single

[jira] [Updated] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31585: Attachment: lexicalorder-AUCT_END_DT.png > Support Z-order curve > - > >

[jira] [Created] (SPARK-31586) Replace expression TimeSub(l, r) with TimeAdd(l -r)

2020-04-27 Thread Kent Yao (Jira)
Kent Yao created SPARK-31586: Summary: Replace expression TimeSub(l, r) with TimeAdd(l -r) Key: SPARK-31586 URL: https://issues.apache.org/jira/browse/SPARK-31586 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31585: Attachment: zorder-AUCT_END_DT.png > Support Z-order curve > - > >

[jira] [Updated] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31585: Attachment: lexicalorder.png > Support Z-order curve > - > >

[jira] [Updated] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31585: Attachment: zorder.png > Support Z-order curve > - > > Key:

[jira] [Updated] (SPARK-31581) paste0 is always better than paste(sep="")

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31581: - Fix Version/s: (was: 2.4.6) > paste0 is always better than paste(sep="") >

[jira] [Created] (SPARK-31585) Support Z-order curve

2020-04-27 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-31585: --- Summary: Support Z-order curve Key: SPARK-31585 URL: https://issues.apache.org/jira/browse/SPARK-31585 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-31578) Internal checkSchemaInArrow is inefficient

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31578: Assignee: Michael Chirico > Internal checkSchemaInArrow is inefficient >

[jira] [Resolved] (SPARK-31578) Internal checkSchemaInArrow is inefficient

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31578. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28372

[jira] [Updated] (SPARK-31578) Internal checkSchemaInArrow is inefficient

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31578: - Affects Version/s: (was: 2.4.5) 3.0.0 > Internal checkSchemaInArrow

[jira] [Resolved] (SPARK-31572) Improve task logs at executor side

2020-04-27 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-31572. -- Resolution: Won't Fix we shall use debug level to improve logs. > Improve task logs at executor side >

[jira] [Resolved] (SPARK-31581) paste0 is always better than paste(sep="")

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31581. -- Fix Version/s: 3.0.0 2.4.6 Assignee: Michael Chirico

[jira] [Commented] (SPARK-31581) paste0 is always better than paste(sep="")

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094086#comment-17094086 ] Hyukjin Kwon commented on SPARK-31581: -- Fixed in https://github.com/apache/spark/pull/28374 >

[jira] [Resolved] (SPARK-31580) Upgrade Apache ORC to 1.5.10

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31580. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28373

[jira] [Assigned] (SPARK-31580) Upgrade Apache ORC to 1.5.10

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31580: - Assignee: Dongjoon Hyun > Upgrade Apache ORC to 1.5.10 >

[jira] [Commented] (SPARK-31538) Backport SPARK-25338 Ensure to call super.beforeAll() and super.afterAll() in test cases

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094081#comment-17094081 ] Hyukjin Kwon commented on SPARK-31538: -- Okay I understood the intention, but Spark 3.0.0 is still a

[jira] [Updated] (SPARK-31584) NullPointerException when parsing event log with InMemoryStore

2020-04-27 Thread Baohe Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baohe Zhang updated SPARK-31584: Attachment: errorstack.txt > NullPointerException when parsing event log with InMemoryStore >

[jira] [Created] (SPARK-31584) NullPointerException when parsing event log with InMemoryStore

2020-04-27 Thread Baohe Zhang (Jira)
Baohe Zhang created SPARK-31584: --- Summary: NullPointerException when parsing event log with InMemoryStore Key: SPARK-31584 URL: https://issues.apache.org/jira/browse/SPARK-31584 Project: Spark

[jira] [Commented] (SPARK-31525) Inconsistent result of df.head(1) and df.head()

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094028#comment-17094028 ] Holden Karau commented on SPARK-31525: -- I agree it's inconsistent, also the docs are a little

[jira] [Resolved] (SPARK-31577) fix various problems when check name conflicts of CTE relations

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31577. --- Fix Version/s: 3.0.0 Resolution: Fixed This is resolved via

[jira] [Resolved] (SPARK-31536) Backport SPARK-25407 Allow nested access for non-existent field for Parquet file when nested pruning is enabled

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31536. -- Resolution: Won't Fix > Backport SPARK-25407 Allow nested access for non-existent field

[jira] [Commented] (SPARK-31536) Backport SPARK-25407 Allow nested access for non-existent field for Parquet file when nested pruning is enabled

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094024#comment-17094024 ] Holden Karau commented on SPARK-31536: -- If the code path is already disabled by default then yeah

[jira] [Resolved] (SPARK-31537) Backport SPARK-25559 Remove the unsupported predicates in Parquet when possible

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31537. -- Resolution: Won't Fix per [~hyukjin.kwon] > Backport SPARK-25559 Remove the unsupported

[jira] [Commented] (SPARK-31538) Backport SPARK-25338 Ensure to call super.beforeAll() and super.afterAll() in test cases

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094022#comment-17094022 ] Holden Karau commented on SPARK-31538: -- [~hyukjin.kwon]if the Jira is already closed affecting the

[jira] [Commented] (SPARK-31545) Backport SPARK-27676 InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094020#comment-17094020 ] Holden Karau commented on SPARK-31545: -- Sounds reasonable, I'll close this as won't fix. >

[jira] [Resolved] (SPARK-31545) Backport SPARK-27676 InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles

2020-04-27 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31545. -- Resolution: Won't Fix > Backport SPARK-27676 InMemoryFileIndex should respect >

[jira] [Created] (SPARK-31583) grouping_id calculation should be improved

2020-04-27 Thread Costas Piliotis (Jira)
Costas Piliotis created SPARK-31583: --- Summary: grouping_id calculation should be improved Key: SPARK-31583 URL: https://issues.apache.org/jira/browse/SPARK-31583 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-27340) Alias on TimeWIndow expression may cause watermark metadata lost

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27340. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28326

[jira] [Assigned] (SPARK-27340) Alias on TimeWIndow expression may cause watermark metadata lost

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27340: - Assignee: Yuanjian Li > Alias on TimeWIndow expression may cause watermark metadata

[jira] [Created] (SPARK-31582) Be able to not populate Hadoop classpath

2020-04-27 Thread DB Tsai (Jira)
DB Tsai created SPARK-31582: --- Summary: Be able to not populate Hadoop classpath Key: SPARK-31582 URL: https://issues.apache.org/jira/browse/SPARK-31582 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-26631) Issue while reading Parquet data from Hadoop Archive files (.har)

2020-04-27 Thread Tien Dat (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093781#comment-17093781 ] Tien Dat commented on SPARK-26631: -- Dear experts, this also happens on the Spark 2.4.0. Could you

[jira] [Updated] (SPARK-31580) Upgrade Apache ORC to 1.5.10

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31580: -- Issue Type: Bug (was: Improvement) > Upgrade Apache ORC to 1.5.10 >

[jira] [Updated] (SPARK-31580) Upgrade Apache ORC to 1.5.10

2020-04-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31580: -- Affects Version/s: (was: 3.1.0) 3.0.0 > Upgrade Apache ORC to

[jira] [Created] (SPARK-31581) paste0 is always better than paste(sep="")

2020-04-27 Thread Michael Chirico (Jira)
Michael Chirico created SPARK-31581: --- Summary: paste0 is always better than paste(sep="") Key: SPARK-31581 URL: https://issues.apache.org/jira/browse/SPARK-31581 Project: Spark Issue Type:

[jira] [Created] (SPARK-31580) Upgrade Apache ORC to 1.5.10

2020-04-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31580: - Summary: Upgrade Apache ORC to 1.5.10 Key: SPARK-31580 URL: https://issues.apache.org/jira/browse/SPARK-31580 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-31579) Replace floorDiv by / in localRebaseGregorianToJulianDays()

2020-04-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31579: -- Summary: Replace floorDiv by / in localRebaseGregorianToJulianDays() Key: SPARK-31579 URL: https://issues.apache.org/jira/browse/SPARK-31579 Project: Spark

[jira] [Created] (SPARK-31578) Internal checkSchemaInArrow is inefficient

2020-04-27 Thread Michael Chirico (Jira)
Michael Chirico created SPARK-31578: --- Summary: Internal checkSchemaInArrow is inefficient Key: SPARK-31578 URL: https://issues.apache.org/jira/browse/SPARK-31578 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-31569) Add links to subsections in SQL Reference main page

2020-04-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31569. -- Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed Resolved by

[jira] [Created] (SPARK-31577) fix various problems when check name conflicts of CTE relations

2020-04-27 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31577: --- Summary: fix various problems when check name conflicts of CTE relations Key: SPARK-31577 URL: https://issues.apache.org/jira/browse/SPARK-31577 Project: Spark

[jira] [Updated] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuzhang updated SPARK-31576: - Description: I'm trying to fetch back data in Spark SQL using a JDBC connection to Hive.

[jira] [Updated] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuzhang updated SPARK-31576: - Description: I'm trying to fetch back data in Spark SQL using a JDBC connection to Hive.

[jira] [Updated] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuzhang updated SPARK-31576: - Description: I'm trying to fetch back data in Spark SQL using a JDBC connection to Hive.

[jira] [Updated] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuzhang updated SPARK-31576: - Description: I'm trying to fetch back data in Spark SQL using a JDBC connection to Hive.

[jira] [Updated] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuzhang updated SPARK-31576: - Description: I'm trying to fetch back data in Spark SQL using a JDBC connection to Hive.

[jira] [Created] (SPARK-31576) Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED

2020-04-27 Thread liuzhang (Jira)
liuzhang created SPARK-31576: Summary: Unable to return Hive data into Spark via Hive JDBC driver Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED Key: SPARK-31576

[jira] [Comment Edited] (SPARK-31527) date add/subtract interval only allow those day precision in ansi mode

2020-04-27 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093462#comment-17093462 ] Kent Yao edited comment on SPARK-31527 at 4/27/20, 12:44 PM: - Added a

[jira] [Commented] (SPARK-31527) date add/subtract interval only allow those day precision in ansi mode

2020-04-27 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093462#comment-17093462 ] Kent Yao commented on SPARK-31527: -- Add a followup PR for benchmark 

[jira] [Created] (SPARK-31575) Synchronise global JVM security configuration modification

2020-04-27 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31575: - Summary: Synchronise global JVM security configuration modification Key: SPARK-31575 URL: https://issues.apache.org/jira/browse/SPARK-31575 Project: Spark

[jira] [Resolved] (SPARK-31529) Remove extra whitespaces in the formatted explain

2020-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31529. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28315

[jira] [Assigned] (SPARK-31529) Remove extra whitespaces in the formatted explain

2020-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31529: --- Assignee: wuyi > Remove extra whitespaces in the formatted explain >

[jira] [Issue Comment Deleted] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2020-04-27 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-14850: Comment: was deleted (was: /*code is for spark 1.6.1*/ object Example{ def main (args:Array[String]){ val

[jira] [Issue Comment Deleted] (SPARK-14850) VectorUDT/MatrixUDT should take primitive arrays without boxing

2020-04-27 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-14850: Comment: was deleted (was: I try to run your code on spark1.6.1 but i found that "toDF" cannot be used in this

[jira] [Updated] (SPARK-17604) Support purging aged file entry for FileStreamSource metadata log

2020-04-27 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-17604: - Affects Version/s: 3.1.0 Labels: (was: bulk-closed) Priority:

[jira] [Assigned] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31550: Assignee: Kent Yao > nondeterministic configurations with general meanings in sql

[jira] [Reopened] (SPARK-17604) Support purging aged file entry for FileStreamSource metadata log

2020-04-27 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reopened SPARK-17604: -- Reopening this, as end user reported this in user mailing list recently.

[jira] [Resolved] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31550. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28322

[jira] [Created] (SPARK-31574) Schema evolution in spark while using the storage format as parquet

2020-04-27 Thread sharad Gupta (Jira)
sharad Gupta created SPARK-31574: Summary: Schema evolution in spark while using the storage format as parquet Key: SPARK-31574 URL: https://issues.apache.org/jira/browse/SPARK-31574 Project: Spark

[jira] [Resolved] (SPARK-31568) R: gapply documentation could be clearer about what the func argument is

2020-04-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31568. -- Fix Version/s: 3.0.0 2.4.6 Assignee: Michael Chirico

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-04-27 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093151#comment-17093151 ] Gabor Somogyi commented on SPARK-12312: --- Not sure what you mean renewal. Renewal of what? * When

[jira] [Created] (SPARK-31573) Use fixed=TRUE where possible for internal efficiency

2020-04-27 Thread Michael Chirico (Jira)
Michael Chirico created SPARK-31573: --- Summary: Use fixed=TRUE where possible for internal efficiency Key: SPARK-31573 URL: https://issues.apache.org/jira/browse/SPARK-31573 Project: Spark

[jira] [Commented] (SPARK-31448) Difference in Storage Levels used in cache() and persist() for pyspark dataframes

2020-04-27 Thread Abhishek Dixit (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093079#comment-17093079 ] Abhishek Dixit commented on SPARK-31448: Any update on this? > Difference in Storage Levels

[jira] [Created] (SPARK-31572) Improve task logs at executor side

2020-04-27 Thread wuyi (Jira)
wuyi created SPARK-31572: Summary: Improve task logs at executor side Key: SPARK-31572 URL: https://issues.apache.org/jira/browse/SPARK-31572 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24194) HadoopFsRelation cannot overwrite a path that is also being read from

2020-04-27 Thread philipse (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093055#comment-17093055 ] philipse commented on SPARK-24194: -- Hi  is the issue closed ? can i try it in product env?   Thanks

[jira] [Created] (SPARK-31571) don't use stop(paste to build R errors

2020-04-27 Thread Michael Chirico (Jira)
Michael Chirico created SPARK-31571: --- Summary: don't use stop(paste to build R errors Key: SPARK-31571 URL: https://issues.apache.org/jira/browse/SPARK-31571 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-31485) Barrier stage can hang if only partial tasks launched

2020-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31485: --- Assignee: wuyi > Barrier stage can hang if only partial tasks launched >

[jira] [Resolved] (SPARK-31485) Barrier stage can hang if only partial tasks launched

2020-04-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31485. - Fix Version/s: 2.4.6 Resolution: Fixed Issue resolved by pull request 28357