[jira] [Updated] (SPARK-30886) Warn two-parameter TRIM/LTRIM/RTRIM functions

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30886: -- Issue Type: Task (was: Bug) > Warn two-parameter TRIM/LTRIM/RTRIM functions >

[jira] [Created] (SPARK-30891) Arrange version info of history

2020-02-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30891: -- Summary: Arrange version info of history Key: SPARK-30891 URL: https://issues.apache.org/jira/browse/SPARK-30891 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-30890) Arrange version info of history

2020-02-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30890: -- Summary: Arrange version info of history Key: SPARK-30890 URL: https://issues.apache.org/jira/browse/SPARK-30890 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-30886) Warn two-parameter TRIM/LTRIM/RTRIM functions

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040729#comment-17040729 ] Dongjoon Hyun commented on SPARK-30886: --- Thanks to SPARK-28126, we can give a directional warning

[jira] [Created] (SPARK-30889) Arrange version info of worker

2020-02-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30889: -- Summary: Arrange version info of worker Key: SPARK-30889 URL: https://issues.apache.org/jira/browse/SPARK-30889 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-30888) Arrange version info of network

2020-02-19 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30888: --- Description: spark/core/src/main/scala/org/apache/spark/internal/config/Network.scala > Arrange

[jira] [Updated] (SPARK-30886) Warn two-parameter TRIM/LTRIM/RTRIM functions

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30886: -- Affects Version/s: (was: 2.4.5) (was: 2.3.4) > Warn

[jira] [Created] (SPARK-30888) Arrange version info of network

2020-02-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30888: -- Summary: Arrange version info of network Key: SPARK-30888 URL: https://issues.apache.org/jira/browse/SPARK-30888 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-30887) Arrange version info of deploy

2020-02-19 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-30887: -- Summary: Arrange version info of deploy Key: SPARK-30887 URL: https://issues.apache.org/jira/browse/SPARK-30887 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-30886) Warn two-parameter TRIM/LTRIM/RTRIM functions

2020-02-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30886: - Summary: Warn two-parameter TRIM/LTRIM/RTRIM functions Key: SPARK-30886 URL: https://issues.apache.org/jira/browse/SPARK-30886 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30886) Warn two-parameter TRIM/LTRIM/RTRIM functions

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30886: -- Description: Apache Spark community decided to keep the existing esoteric two-parameter use

[jira] [Updated] (SPARK-30885) V1 table name should be fully qualified if catalog name is provided

2020-02-19 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-30885: -- Summary: V1 table name should be fully qualified if catalog name is provided (was: V1 table name

[jira] [Updated] (SPARK-30885) V1 table name should be fully qualified

2020-02-19 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-30885: -- Description: For the following example, {code:java} sql("CREATE TABLE t USING json AS SELECT 1 AS

[jira] [Created] (SPARK-30885) V1 table name should be fully qualified

2020-02-19 Thread Terry Kim (Jira)
Terry Kim created SPARK-30885: - Summary: V1 table name should be fully qualified Key: SPARK-30885 URL: https://issues.apache.org/jira/browse/SPARK-30885 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-30884) Upgrade to Py4J 0.10.9

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30884: - Assignee: Dongjoon Hyun > Upgrade to Py4J 0.10.9 > -- > >

[jira] [Updated] (SPARK-30884) Upgrade to Py4J 0.10.9

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30884: -- Affects Version/s: (was: 2.4.5) 3.0.0 > Upgrade to Py4J 0.10.9 >

[jira] [Updated] (SPARK-30884) Upgrade to Py4J 0.10.9

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30884: -- Affects Version/s: (was: 3.1.0) 2.4.5 > Upgrade to Py4J 0.10.9 >

[jira] [Created] (SPARK-30884) Upgrade to Py4J 0.10.9

2020-02-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30884: - Summary: Upgrade to Py4J 0.10.9 Key: SPARK-30884 URL: https://issues.apache.org/jira/browse/SPARK-30884 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30876) Optimizer cannot infer from inferred constraints with join

2020-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30876: Description: How to reproduce this issue: {code:sql} create table t1(a int, b int, c int); create

[jira] [Updated] (SPARK-30876) Optimizer cannot infer from inferred constraints with join

2020-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30876: Description: How to reproduce this issue: {code:sql} create table t1(a int, b int, c int); create

[jira] [Updated] (SPARK-30883) Tests that use setWritable,setReadable and setExecutable should be cancel when user is root

2020-02-19 Thread deshanxiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-30883: --- Environment: The java api *setWritable,setReadable and setExecutable* dosen't work well when the

[jira] [Created] (SPARK-30883) Tests that use setWritable,setReadable and setExecutable should be cancel when user is root

2020-02-19 Thread deshanxiao (Jira)
deshanxiao created SPARK-30883: -- Summary: Tests that use setWritable,setReadable and setExecutable should be cancel when user is root Key: SPARK-30883 URL: https://issues.apache.org/jira/browse/SPARK-30883

[jira] [Comment Edited] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-02-19 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039884#comment-17039884 ] Saurabh Chawla edited comment on SPARK-30873 at 2/20/20 3:43 AM: - We

[jira] [Resolved] (SPARK-30856) SQLContext retains reference to unusable instance after SparkContext restarted

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30856. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27610

[jira] [Updated] (SPARK-30860) Different behavior between rolling and non-rolling event log

2020-02-19 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-30860: - Priority: Major (was: Minor) > Different behavior between rolling and non-rolling event log >

[jira] [Resolved] (SPARK-28990) SparkSQL invalid call to toAttribute on unresolved object, tree: *

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28990. -- Resolution: Cannot Reproduce > SparkSQL invalid call to toAttribute on unresolved object,

[jira] [Resolved] (SPARK-30332) When running sql query with limit catalyst throw StackOverFlow exception

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30332. -- Resolution: Cannot Reproduce I am resolving this. I can't reproduce and take a look for the

[jira] [Commented] (SPARK-30332) When running sql query with limit catalyst throw StackOverFlow exception

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040560#comment-17040560 ] Hyukjin Kwon commented on SPARK-30332: -- Where is error message, and how do you create the tables?

[jira] [Commented] (SPARK-30868) Throw Exception if runHive(sql) failed

2020-02-19 Thread Jackey Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040557#comment-17040557 ] Jackey Lee commented on SPARK-30868: [~Ankitraj] No, I think we should just throw Exception once

[jira] [Commented] (SPARK-30837) spark-master-test-k8s is broken

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040555#comment-17040555 ] Hyukjin Kwon commented on SPARK-30837: -- Thanks! I see green light :-). resolving it. >

[jira] [Resolved] (SPARK-30837) spark-master-test-k8s is broken

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30837. -- Resolution: Fixed > spark-master-test-k8s is broken > --- > >

[jira] [Updated] (SPARK-30876) Optimizer cannot infer from inferred constraints with join

2020-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30876: Summary: Optimizer cannot infer from inferred constraints with join (was: Optimizer cannot infer

[jira] [Updated] (SPARK-30882) Inaccurate results with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Summary: Inaccurate results with higher precision ApproximatePercentile results

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Description: Results of ApproximatePercentile should have better accuracy with

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Description: Results of ApproximatePercentile should have better accuracy with

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Description: Results of ApproximatePercentile should have better accuracy with

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Description: Results of ApproximatePercentile should have better accuracy with

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Description: Results of ApproximatePercentile should have better accuracy with

[jira] [Updated] (SPARK-30882) Inaccurate ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Summary: Inaccurate ApproximatePercentile results (was: ApproximatePercentile

[jira] [Updated] (SPARK-30882) Inaccurate results even with higher precision ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandini malempati updated SPARK-30882: -- Summary: Inaccurate results even with higher precision ApproximatePercentile results

[jira] [Created] (SPARK-30882) ApproximatePercentile results

2020-02-19 Thread Nandini malempati (Jira)
Nandini malempati created SPARK-30882: - Summary: ApproximatePercentile results Key: SPARK-30882 URL: https://issues.apache.org/jira/browse/SPARK-30882 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-30802) Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite

2020-02-19 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30802. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27555

[jira] [Updated] (SPARK-30802) Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite

2020-02-19 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30802: - Priority: Minor (was: Major) > Use Summarizer instead of MultivariateOnlineSummarizer in

[jira] [Assigned] (SPARK-30802) Use Summarizer instead of MultivariateOnlineSummarizer in Aggregator test suite

2020-02-19 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30802: Assignee: Huaxin Gao > Use Summarizer instead of MultivariateOnlineSummarizer in

[jira] [Updated] (SPARK-30881) Revise the doc of spark.sql.sources.parallelPartitionDiscovery.threshold

2020-02-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-30881: --- Priority: Minor (was: Major) > Revise the doc of

[jira] [Created] (SPARK-30881) Revise the doc of spark.sql.sources.parallelPartitionDiscovery.threshold

2020-02-19 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-30881: -- Summary: Revise the doc of spark.sql.sources.parallelPartitionDiscovery.threshold Key: SPARK-30881 URL: https://issues.apache.org/jira/browse/SPARK-30881

[jira] [Commented] (SPARK-21065) Spark Streaming concurrentJobs + StreamingJobProgressListener conflict

2020-02-19 Thread Artur Sukhenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040456#comment-17040456 ] Artur Sukhenko commented on SPARK-21065: [~zsxwing] Is `spark.streaming.concurrentJobs` still

[jira] [Updated] (SPARK-30879) Refine doc-building workflow

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30879: -- Affects Version/s: (was: 3.0.0) 3.1.0 > Refine doc-building

[jira] [Updated] (SPARK-30880) Delete Sphinx Makefile cruft

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30880: -- Description: (was: There are a few rough edges in the workflow for building docs that

[jira] [Updated] (SPARK-30880) Delete Sphinx Makefile cruft

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30880: -- Affects Version/s: (was: 3.0.0) 3.1.0 > Delete Sphinx Makefile

[jira] [Created] (SPARK-30880) Delete Sphinx Makefile cruft

2020-02-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30880: - Summary: Delete Sphinx Makefile cruft Key: SPARK-30880 URL: https://issues.apache.org/jira/browse/SPARK-30880 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-30731) Update deprecated Mkdocs option

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30731. --- Fix Version/s: 2.4.6 Assignee: Nicholas Chammas Resolution: Fixed This is

[jira] [Updated] (SPARK-30731) Update deprecated Mkdocs option

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30731: -- Issue Type: Bug (was: Improvement) > Update deprecated Mkdocs option >

[jira] [Updated] (SPARK-30731) Update deprecated Mkdocs option

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30731: -- Description: (was: There are a few rough edges in the workflow for building docs that

[jira] [Updated] (SPARK-30731) Update deprecated Mkdocs option

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30731: -- Priority: Trivial (was: Minor) > Update deprecated Mkdocs option >

[jira] [Created] (SPARK-30879) Refine doc-building workflow

2020-02-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30879: - Summary: Refine doc-building workflow Key: SPARK-30879 URL: https://issues.apache.org/jira/browse/SPARK-30879 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30731) Update deprecated Mkdocs option

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30731: -- Summary: Update deprecated Mkdocs option (was: Refine doc-building workflow) > Update

[jira] [Commented] (SPARK-30811) CTE that refers to non-existent table with same name causes StackOverflowError

2020-02-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040299#comment-17040299 ] Dongjoon Hyun commented on SPARK-30811: --- The test cases are landed to `master/branch-3.0` to

[jira] [Resolved] (SPARK-29908) Support partitioning for DataSource V2 tables in DataFrameWriter.save

2020-02-19 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-29908. - Resolution: Duplicate > Support partitioning for DataSource V2 tables in DataFrameWriter.save >

[jira] [Created] (SPARK-30878) improve the CREATE TABLE document

2020-02-19 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-30878: --- Summary: improve the CREATE TABLE document Key: SPARK-30878 URL: https://issues.apache.org/jira/browse/SPARK-30878 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-30877) Make Aggregate higher order function with Spark Aggregate Expressions

2020-02-19 Thread Jira
Herman van Hövell created SPARK-30877: - Summary: Make Aggregate higher order function with Spark Aggregate Expressions Key: SPARK-30877 URL: https://issues.apache.org/jira/browse/SPARK-30877

[jira] [Created] (SPARK-30876) Optimizer cannot infer more constraint

2020-02-19 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-30876: --- Summary: Optimizer cannot infer more constraint Key: SPARK-30876 URL: https://issues.apache.org/jira/browse/SPARK-30876 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24497) ANSI SQL: Recursive query

2020-02-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040121#comment-17040121 ] Peter Toth commented on SPARK-24497: Thanks [~dmateusp] for your +1. That PR is mine and I will

[jira] [Commented] (SPARK-30875) Revisit the decision of writing parquet TIMESTAMP_MICROS by default

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040042#comment-17040042 ] Wenchen Fan commented on SPARK-30875: - cc [~maxgekk] [~hyukjin.kwon] > Revisit the decision of

[jira] [Updated] (SPARK-30875) Revisit the decision of writing parquet TIMESTAMP_MICROS by default

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30875: Description: In Spark 3.0, we write out timestamp values as parquet TIMESTAMP_MICROS by default,

[jira] [Updated] (SPARK-30875) Revisit the decision of writing parquet TIMESTAMP_MICROS by default

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30875: Description: In Spark 3.0, we write out timestamp values as parquet > Revisit the decision of

[jira] [Created] (SPARK-30875) Revisit the decision of writing parquet TIMESTAMP_MICROS by default

2020-02-19 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-30875: --- Summary: Revisit the decision of writing parquet TIMESTAMP_MICROS by default Key: SPARK-30875 URL: https://issues.apache.org/jira/browse/SPARK-30875 Project: Spark

[jira] [Created] (SPARK-30874) Support Postgres Kerberos login in JDBC connector

2020-02-19 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-30874: - Summary: Support Postgres Kerberos login in JDBC connector Key: SPARK-30874 URL: https://issues.apache.org/jira/browse/SPARK-30874 Project: Spark Issue

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-02-19 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039975#comment-17039975 ] Gabor Somogyi commented on SPARK-12312: --- After deep analysis it has turned out each database type

[jira] [Resolved] (SPARK-30763) Fix java.lang.IndexOutOfBoundsException No group 1 for regexp_extract

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30763. - Fix Version/s: 2.4.6 3.0.0 Assignee: jiaan.geng Resolution:

[jira] [Resolved] (SPARK-30786) Block replication is not retried on other BlockManagers when it fails on 1 of the peers

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30786. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27539

[jira] [Assigned] (SPARK-30786) Block replication is not retried on other BlockManagers when it fails on 1 of the peers

2020-02-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30786: --- Assignee: Prakhar Jain > Block replication is not retried on other BlockManagers when it

[jira] [Comment Edited] (SPARK-26346) Upgrade parquet to 1.11.1

2020-02-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039904#comment-17039904 ] Ismaël Mejía edited comment on SPARK-26346 at 2/19/20 10:54 AM: It seems

[jira] [Commented] (SPARK-26346) Upgrade parquet to 1.11.1

2020-02-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039904#comment-17039904 ] Ismaël Mejía commented on SPARK-26346: -- It seems this issue is blocked because Spark is having

[jira] [Commented] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-02-19 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039884#comment-17039884 ] Saurabh Chawla commented on SPARK-30873: We have raised the WIP PR for this. cc [~holdenkarau]

[jira] [Updated] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-02-19 Thread Saurabh Chawla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saurabh Chawla updated SPARK-30873: --- Description: In many public cloud environments, the node loss (in case of AWS SpotLoss,Spot

[jira] [Created] (SPARK-30873) Handling Node Decommissioning for Yarn cluster manger in Spark

2020-02-19 Thread Saurabh Chawla (Jira)
Saurabh Chawla created SPARK-30873: -- Summary: Handling Node Decommissioning for Yarn cluster manger in Spark Key: SPARK-30873 URL: https://issues.apache.org/jira/browse/SPARK-30873 Project: Spark

[jira] [Commented] (SPARK-26346) Upgrade parquet to 1.11.1

2020-02-19 Thread Gabor Szadovszky (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039871#comment-17039871 ] Gabor Szadovszky commented on SPARK-26346: -- [~h-vetinari], Parquet 1.11.1 was initiated to

[jira] [Updated] (SPARK-30872) Constraints inferred from inferred attributes

2020-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30872: Affects Version/s: (was: 3.0.0) 3.1.0 > Constraints inferred from

[jira] [Updated] (SPARK-30872) Constraints inferred from inferred attributes

2020-02-19 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-30872: Description: {code:scala} scala> spark.range(20).selectExpr("id as a", "id as b", "id as

[jira] [Resolved] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30349. -- Resolution: Cannot Reproduce Resolving this due to no feedback from the author. > The result

[jira] [Commented] (SPARK-29908) Support partitioning for DataSource V2 tables in DataFrameWriter.save

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039784#comment-17039784 ] Hyukjin Kwon commented on SPARK-29908: -- [~brkyvz], the PRs were closed. Do we still need this? can

[jira] [Updated] (SPARK-29699) Different answers in nested aggregates with window functions

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29699: - Priority: Critical (was: Blocker) > Different answers in nested aggregates with window

[jira] [Updated] (SPARK-29699) Different answers in nested aggregates with window functions

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29699: - Target Version/s: (was: 3.0.0) > Different answers in nested aggregates with window functions

[jira] [Updated] (SPARK-29699) Different answers in nested aggregates with window functions

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29699: - Labels: (was: correctness) > Different answers in nested aggregates with window functions >

[jira] [Commented] (SPARK-29699) Different answers in nested aggregates with window functions

2020-02-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039783#comment-17039783 ] Hyukjin Kwon commented on SPARK-29699: -- I lowered to Critical+ for now. > Different answers in