[jira] [Resolved] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29604. --- Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed This is

[jira] [Created] (SPARK-29653) Fix MICROS_PER_MONTH in IntervalUtils

2019-10-30 Thread Kent Yao (Jira)
Kent Yao created SPARK-29653: Summary: Fix MICROS_PER_MONTH in IntervalUtils Key: SPARK-29653 URL: https://issues.apache.org/jira/browse/SPARK-29653 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-29636) Can't parse '11:00 BST' or '2000-10-19 10:23:54+01' signatures to timestamp

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962865#comment-16962865 ] Aman Omer commented on SPARK-29636: --- [~DylanGuedes] Query 1-   {code:java} select cast ('2000-10-19

[jira] [Commented] (SPARK-29621) Querying internal corrupt record column should not be allowed in filter operation

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962734#comment-16962734 ] Hyukjin Kwon commented on SPARK-29621: -- the problem here is {{count()}} since internally it only

[jira] [Created] (SPARK-29649) Stop task set if FileAlreadyExistsException was thrown when writing to output file

2019-10-30 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-29649: --- Summary: Stop task set if FileAlreadyExistsException was thrown when writing to output file Key: SPARK-29649 URL: https://issues.apache.org/jira/browse/SPARK-29649

[jira] [Commented] (SPARK-29371) Support interval field values with fractional parts

2019-10-30 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962798#comment-16962798 ] Kent Yao commented on SPARK-29371: -- {code:java} postgres=# select interval '1.41 years 2.51 months 2.21

[jira] [Resolved] (SPARK-29110) Add window.sql - Part 4

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29110. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26238

[jira] [Assigned] (SPARK-29110) Add window.sql - Part 4

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29110: Assignee: Dylan Guedes > Add window.sql - Part 4 > --- > >

[jira] [Created] (SPARK-29651) Incorrect parsing of interval seconds fraction

2019-10-30 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29651: -- Summary: Incorrect parsing of interval seconds fraction Key: SPARK-29651 URL: https://issues.apache.org/jira/browse/SPARK-29651 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-29371) Support interval field values with fractional parts

2019-10-30 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962797#comment-16962797 ] Kent Yao commented on SPARK-29371: -- I am working on this > Support interval field values with

[jira] [Created] (SPARK-29657) Iterator spill supporting radix sort with null prefix

2019-10-30 Thread dzcxzl (Jira)
dzcxzl created SPARK-29657: -- Summary: Iterator spill supporting radix sort with null prefix Key: SPARK-29657 URL: https://issues.apache.org/jira/browse/SPARK-29657 Project: Spark Issue Type:

[jira] [Created] (SPARK-29656) ML algs expose aggregationDepth

2019-10-30 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-29656: Summary: ML algs expose aggregationDepth Key: SPARK-29656 URL: https://issues.apache.org/jira/browse/SPARK-29656 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-29650) Discard a NULL constant in LIMIT

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29650: Summary: Discard a NULL constant in LIMIT Key: SPARK-29650 URL: https://issues.apache.org/jira/browse/SPARK-29650 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29634) spark-sql can't query hive table values with schema Char by equality.

2019-10-30 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962791#comment-16962791 ] Zhaoyang Qin commented on SPARK-29634: -- I didn't find any similar issues. Is the question

[jira] [Created] (SPARK-29652) AlterTable/DescribeTable should not list the table as a child

2019-10-30 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29652: --- Summary: AlterTable/DescribeTable should not list the table as a child Key: SPARK-29652 URL: https://issues.apache.org/jira/browse/SPARK-29652 Project: Spark

[jira] [Assigned] (SPARK-29647) Use Python 3.7 in GitHub Action to recover lint-python

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29647: Assignee: Dongjoon Hyun > Use Python 3.7 in GitHub Action to recover lint-python >

[jira] [Resolved] (SPARK-29647) Use Python 3.7 in GitHub Action to recover lint-python

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29647. -- Fix Version/s: 2.4.5 Resolution: Fixed Issue resolved by pull request 26308

[jira] [Created] (SPARK-29648) Add limit.sql

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29648: Summary: Add limit.sql Key: SPARK-29648 URL: https://issues.apache.org/jira/browse/SPARK-29648 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-29654) Add configuration to allow disabling registration of static sources to the metrics system

2019-10-30 Thread Luca Canali (Jira)
Luca Canali created SPARK-29654: --- Summary: Add configuration to allow disabling registration of static sources to the metrics system Key: SPARK-29654 URL: https://issues.apache.org/jira/browse/SPARK-29654

[jira] [Commented] (SPARK-29655) Prefer bucket join if adaptive execution is enabled and maxNumPostShufflePartitions != bucket number

2019-10-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962893#comment-16962893 ] Yuming Wang commented on SPARK-29655: - cc [~Jk_Self] > Prefer bucket join if adaptive execution is

[jira] [Commented] (SPARK-28632) Cannot read from a partitioned table by Int

2019-10-30 Thread Sunny Shukla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962724#comment-16962724 ] Sunny Shukla commented on SPARK-28632: -- The same works for Spark 2.2.x (tested this on Spark 2.2.1)

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962851#comment-16962851 ] Dongjoon Hyun commented on SPARK-29604: --- This lands at `branch-2.4` via

[jira] [Updated] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29604: -- Fix Version/s: 2.4.5 > SessionState is initialized with isolated classloader for Hive if >

[jira] [Comment Edited] (SPARK-29636) Can't parse '11:00 BST' or '2000-10-19 10:23:54+01' signatures to timestamp

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962865#comment-16962865 ] Aman Omer edited comment on SPARK-29636 at 10/30/19 9:51 AM: - [~DylanGuedes]

[jira] [Issue Comment Deleted] (SPARK-29179) Aggregated Metrics by Executor table has refresh issue

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Omer updated SPARK-29179: -- Comment: was deleted (was: Working on this.) > Aggregated Metrics by Executor table has refresh

[jira] [Updated] (SPARK-29634) spark-sql can't query hive table values with schema Char by equality.

2019-10-30 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhaoyang Qin updated SPARK-29634: - Description: spark-sql can't query hive table values  with char schema by equality. When i use

[jira] [Commented] (SPARK-24035) SQL syntax for Pivot

2019-10-30 Thread Xuedong Luan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962813#comment-16962813 ] Xuedong Luan commented on SPARK-24035: -- Hi [~smilegator]  I found new added keyword 'ANY' in this

[jira] [Comment Edited] (SPARK-29591) Support data insertion in a different order if you wish or even omit some columns in spark sql also like postgresql

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962830#comment-16962830 ] Aman Omer edited comment on SPARK-29591 at 10/30/19 9:12 AM: - [~Ankitraj]

[jira] [Commented] (SPARK-29591) Support data insertion in a different order if you wish or even omit some columns in spark sql also like postgresql

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962830#comment-16962830 ] Aman Omer commented on SPARK-29591: --- [~Ankitraj] Do refer 

[jira] [Created] (SPARK-29655) Prefer bucket join if adaptive execution is enabled and maxNumPostShufflePartitions != bucket number

2019-10-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29655: --- Summary: Prefer bucket join if adaptive execution is enabled and maxNumPostShufflePartitions != bucket number Key: SPARK-29655 URL:

[jira] [Commented] (SPARK-29621) Querying internal corrupt record column should not be allowed in filter operation

2019-10-30 Thread Suchintak Patnaik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962741#comment-16962741 ] Suchintak Patnaik commented on SPARK-29621: --- [~hyukjin.kwon] count() returns the row count, it

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962859#comment-16962859 ] Dongjoon Hyun commented on SPARK-29604: --- BTW, [~kabhwan]. Could you check the old version (at

[jira] [Created] (SPARK-29661) Support cascaded syntax in CREATE SCHEMA

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29661: Summary: Support cascaded syntax in CREATE SCHEMA Key: SPARK-29661 URL: https://issues.apache.org/jira/browse/SPARK-29661 Project: Spark Issue Type:

[jira] [Updated] (SPARK-29640) [K8S] Intermittent "java.net.UnknownHostException: kubernetes.default.svc" in Spark driver

2019-10-30 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-29640: --- Description: We are running into intermittent DNS issues where the Spark driver fails to resolve

[jira] [Updated] (SPARK-29640) [K8S] Intermittent "java.net.UnknownHostException: kubernetes.default.svc" in Spark driver

2019-10-30 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-29640: --- Description: We are running into intermittent DNS issues where the Spark driver fails to resolve

[jira] [Created] (SPARK-29660) Dropping columns and changing column names/types are prohibited in VIEW definition

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29660: Summary: Dropping columns and changing column names/types are prohibited in VIEW definition Key: SPARK-29660 URL: https://issues.apache.org/jira/browse/SPARK-29660

[jira] [Updated] (SPARK-29653) Fix MICROS_PER_MONTH in IntervalUtils

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29653: -- Priority: Minor (was: Major) > Fix MICROS_PER_MONTH in IntervalUtils >

[jira] [Resolved] (SPARK-29653) Fix MICROS_PER_MONTH in IntervalUtils

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29653. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26321

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963000#comment-16963000 ] Jungtaek Lim commented on SPARK-29604: -- I think it doesn't apply to branch-2.3 as the root issue is

[jira] [Created] (SPARK-29662) Cannot have circular references in bean class, but got the circular reference of class class io.cdap.cdap.api.data.schema.Schema

2019-10-30 Thread romain (Jira)
romain created SPARK-29662: -- Summary: Cannot have circular references in bean class, but got the circular reference of class class io.cdap.cdap.api.data.schema.Schema Key: SPARK-29662 URL:

[jira] [Assigned] (SPARK-29653) Fix MICROS_PER_MONTH in IntervalUtils

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29653: - Assignee: Kent Yao > Fix MICROS_PER_MONTH in IntervalUtils >

[jira] [Created] (SPARK-29658) Support geometric types

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29658: Summary: Support geometric types Key: SPARK-29658 URL: https://issues.apache.org/jira/browse/SPARK-29658 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-29659) Support COMMENT ON syntax

2019-10-30 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-29659: Summary: Support COMMENT ON syntax Key: SPARK-29659 URL: https://issues.apache.org/jira/browse/SPARK-29659 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29640) [K8S] Make it possible to set DNS option to use TCP instead of UDP

2019-10-30 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963059#comment-16963059 ] Andy Grove commented on SPARK-29640: Installing node local dns cache daemon might be another

[jira] [Updated] (SPARK-29667) implicitly convert mismatched datatypes on right side of "IN" operator

2019-10-30 Thread Cheng Lian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-29667: --- Environment: (was: spark-2.4.3-bin-dbr-5.5-snapshot-9833d0f) > implicitly convert mismatched

[jira] [Commented] (SPARK-29660) Dropping columns and changing column names/types are prohibited in VIEW definition

2019-10-30 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963186#comment-16963186 ] Aman Omer commented on SPARK-29660: --- Checking this one  > Dropping columns and changing column

[jira] [Created] (SPARK-29666) Release script fail to publish release under dry run mode

2019-10-30 Thread Xingbo Jiang (Jira)
Xingbo Jiang created SPARK-29666: Summary: Release script fail to publish release under dry run mode Key: SPARK-29666 URL: https://issues.apache.org/jira/browse/SPARK-29666 Project: Spark

[jira] [Updated] (SPARK-29666) Release script fail to publish release under dry run mode

2019-10-30 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29666: - Description: `release-build.sh` fail to publish release under dry run mode with the following

[jira] [Created] (SPARK-29668) Deprecate Python prior to version 3.6

2019-10-30 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29668: - Summary: Deprecate Python prior to version 3.6 Key: SPARK-29668 URL: https://issues.apache.org/jira/browse/SPARK-29668 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-29462) The data type of "array()" should be array

2019-10-30 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29462. -- Resolution: Not A Problem > The data type of "array()" should be array >

[jira] [Assigned] (SPARK-29120) Add create_view.sql

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29120: - Assignee: Takeshi Yamamuro > Add create_view.sql > --- > >

[jira] [Resolved] (SPARK-29120) Add create_view.sql

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29120. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26290

[jira] [Created] (SPARK-29664) Column.getItem behavior is not consistent with Scala version

2019-10-30 Thread Terry Kim (Jira)
Terry Kim created SPARK-29664: - Summary: Column.getItem behavior is not consistent with Scala version Key: SPARK-29664 URL: https://issues.apache.org/jira/browse/SPARK-29664 Project: Spark

[jira] [Commented] (SPARK-29640) [K8S] Intermittent "java.net.UnknownHostException: kubernetes.default.svc" in Spark driver

2019-10-30 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963175#comment-16963175 ] Andy Grove commented on SPARK-29640: A hacky workaround is to wait for DNS to resolve before

[jira] [Commented] (SPARK-29662) Cannot have circular references in bean class, but got the circular reference of class class io.cdap.cdap.api.data.schema.Schema

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963198#comment-16963198 ] Dongjoon Hyun commented on SPARK-29662: --- Hi, [~coudray]. What is `CDAP`? Since Apache Spark 2.3.x

[jira] [Comment Edited] (SPARK-29662) Cannot have circular references in bean class, but got the circular reference of class class io.cdap.cdap.api.data.schema.Schema

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963198#comment-16963198 ] Dongjoon Hyun edited comment on SPARK-29662 at 10/30/19 4:35 PM: - Hi,

[jira] [Commented] (SPARK-29667) implicitly convert mismatched datatypes on right side of "IN" operator

2019-10-30 Thread Cheng Lian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963305#comment-16963305 ] Cheng Lian commented on SPARK-29667: Reproduced this with the following snippet: {code}

[jira] [Created] (SPARK-29663) Support sum with interval type values

2019-10-30 Thread Kent Yao (Jira)
Kent Yao created SPARK-29663: Summary: Support sum with interval type values Key: SPARK-29663 URL: https://issues.apache.org/jira/browse/SPARK-29663 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-29665) refine the TableProvider interface

2019-10-30 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29665: --- Summary: refine the TableProvider interface Key: SPARK-29665 URL: https://issues.apache.org/jira/browse/SPARK-29665 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963264#comment-16963264 ] Dongjoon Hyun commented on SPARK-29604: --- Thank you so much for confirming, [~kabhwan]! The newly

[jira] [Created] (SPARK-29667) implicitly convert mismatched datatypes on right side of "IN" operator

2019-10-30 Thread Jessie Lin (Jira)
Jessie Lin created SPARK-29667: -- Summary: implicitly convert mismatched datatypes on right side of "IN" operator Key: SPARK-29667 URL: https://issues.apache.org/jira/browse/SPARK-29667 Project: Spark

[jira] [Updated] (SPARK-29667) implicitly convert mismatched datatypes on right side of "IN" operator

2019-10-30 Thread Jessie Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jessie Lin updated SPARK-29667: --- Priority: Minor (was: Major) > implicitly convert mismatched datatypes on right side of "IN"

[jira] [Updated] (SPARK-29668) Deprecate Python 3 prior to version 3.6

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29668: -- Summary: Deprecate Python 3 prior to version 3.6 (was: Deprecate Python prior to version

[jira] [Resolved] (SPARK-29523) SHOW COLUMNS should look up catalog/table like v2 commands

2019-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29523. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26182

[jira] [Assigned] (SPARK-29523) SHOW COLUMNS should look up catalog/table like v2 commands

2019-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29523: --- Assignee: Pablo Langa Blanco > SHOW COLUMNS should look up catalog/table like v2 commands

[jira] [Commented] (SPARK-29664) Column.getItem behavior is not consistent with Scala version

2019-10-30 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963222#comment-16963222 ] Terry Kim commented on SPARK-29664: --- [~hyukjin.kwon], is there a reason why getItem's behavior is not

[jira] [Commented] (SPARK-29676) ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands

2019-10-30 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963537#comment-16963537 ] Huaxin Gao commented on SPARK-29676: I am working on this > ALTER TABLE (RENAME PARTITION) should

[jira] [Created] (SPARK-29676) ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands

2019-10-30 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-29676: -- Summary: ALTER TABLE (RENAME PARTITION) should look up catalog/table like v2 commands Key: SPARK-29676 URL: https://issues.apache.org/jira/browse/SPARK-29676 Project:

[jira] [Assigned] (SPARK-29651) Incorrect parsing of interval seconds fraction

2019-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29651: --- Assignee: Maxim Gekk > Incorrect parsing of interval seconds fraction >

[jira] [Assigned] (SPARK-29126) Add usage guide for cogroup Pandas UDF

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29126: Assignee: Hyukjin Kwon > Add usage guide for cogroup Pandas UDF >

[jira] [Assigned] (SPARK-29126) Add usage guide for cogroup Pandas UDF

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29126: Assignee: Chris Martin (was: Hyukjin Kwon) > Add usage guide for cogroup Pandas UDF >

[jira] [Commented] (SPARK-29660) Dropping columns and changing column names/types are prohibited in VIEW definition

2019-10-30 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963533#comment-16963533 ] Takeshi Yamamuro commented on SPARK-29660: -- I'm not sure that we need to do something about

[jira] [Created] (SPARK-29675) Add exception when isolationLevel is Illegal

2019-10-30 Thread ulysses you (Jira)
ulysses you created SPARK-29675: --- Summary: Add exception when isolationLevel is Illegal Key: SPARK-29675 URL: https://issues.apache.org/jira/browse/SPARK-29675 Project: Spark Issue Type:

[jira] [Created] (SPARK-29677) Upgrade Kinesis Client

2019-10-30 Thread Eric S Meisel (Jira)
Eric S Meisel created SPARK-29677: - Summary: Upgrade Kinesis Client Key: SPARK-29677 URL: https://issues.apache.org/jira/browse/SPARK-29677 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-29277) DataSourceV2: Add early filter and projection pushdown

2019-10-30 Thread Ryan Blue (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue resolved SPARK-29277. --- Fix Version/s: 3.0.0 Resolution: Fixed Fixed by #25955. > DataSourceV2: Add early filter

[jira] [Commented] (SPARK-29671) Change format of interval string

2019-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963553#comment-16963553 ] Wenchen Fan commented on SPARK-29671: - I think it's good enough to display the second field with

[jira] [Resolved] (SPARK-29651) Incorrect parsing of interval seconds fraction

2019-10-30 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29651. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26313

[jira] [Resolved] (SPARK-29126) Add usage guide for cogroup Pandas UDF

2019-10-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29126. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26110

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

2019-10-30 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963505#comment-16963505 ] Shane Knapp commented on SPARK-29106: - first pass @ the python tests:

[jira] [Created] (SPARK-29674) Update dropwizard metrics to 4.1.1 for JDK 9+ support

2019-10-30 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29674: Summary: Update dropwizard metrics to 4.1.1 for JDK 9+ support Key: SPARK-29674 URL: https://issues.apache.org/jira/browse/SPARK-29674 Project: Spark Issue

[jira] [Updated] (SPARK-29643) ALTER TABLE (DROP PARTITION) should look up catalog/table like v2 commands

2019-10-30 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-29643: --- Environment: (was: ALTER TABLE (DROP PARTITION) should look up catalog/table like v2 commands)

[jira] [Updated] (SPARK-29643) ALTER TABLE (DROP PARTITION) should look up catalog/table like v2 commands

2019-10-30 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-29643: --- Description: ALTER TABLE (DROP PARTITION) should look up catalog/table like v2 commands > ALTER

[jira] [Created] (SPARK-29669) Refactor IntervalUtils.fromDayTimeString()

2019-10-30 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29669: -- Summary: Refactor IntervalUtils.fromDayTimeString() Key: SPARK-29669 URL: https://issues.apache.org/jira/browse/SPARK-29669 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29671) Change format of interval string

2019-10-30 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963375#comment-16963375 ] Maxim Gekk commented on SPARK-29671: For example, PostgreSQL displays intervals like: {code} maxim=#

[jira] [Created] (SPARK-29673) upgrade jenkins pypy to PyPy3.6 v7.2.0

2019-10-30 Thread Shane Knapp (Jira)
Shane Knapp created SPARK-29673: --- Summary: upgrade jenkins pypy to PyPy3.6 v7.2.0 Key: SPARK-29673 URL: https://issues.apache.org/jira/browse/SPARK-29673 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29604) SessionState is initialized with isolated classloader for Hive if spark.sql.hive.metastore.jars is being set

2019-10-30 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963476#comment-16963476 ] Jungtaek Lim commented on SPARK-29604: -- [~dongjoon] Do we have any annotation/trait to "isolate"

[jira] [Commented] (SPARK-29671) Change format of interval string

2019-10-30 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963373#comment-16963373 ] Maxim Gekk commented on SPARK-29671: [~cloud_fan][~dongjoon] Let's discuss here how to improve the

[jira] [Created] (SPARK-29671) Change format of interval string

2019-10-30 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29671: -- Summary: Change format of interval string Key: SPARK-29671 URL: https://issues.apache.org/jira/browse/SPARK-29671 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27705) kubernetes integration test break on osx when test PVTestsSuite

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27705: -- Component/s: Tests > kubernetes integration test break on osx when test PVTestsSuite >

[jira] [Resolved] (SPARK-27705) kubernetes integration test break on osx when test PVTestsSuite

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27705. --- Resolution: Fixed As [~Andrew HUALI] reported originally, minikube should be started with

[jira] [Resolved] (SPARK-27705) kubernetes integration test break on osx when test PVTestsSuite

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27705. --- Resolution: Invalid > kubernetes integration test break on osx when test PVTestsSuite >

[jira] [Reopened] (SPARK-27705) kubernetes integration test break on osx when test PVTestsSuite

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-27705: --- > kubernetes integration test break on osx when test PVTestsSuite >

[jira] [Updated] (SPARK-27884) Deprecate Python 2 and Python 3 prior to 3.6 support in Spark 3.0

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27884: -- Summary: Deprecate Python 2 and Python 3 prior to 3.6 support in Spark 3.0 (was: Deprecate

[jira] [Updated] (SPARK-29668) Deprecate Python 3 prior to version 3.6

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29668: -- Parent: SPARK-27884 Issue Type: Sub-task (was: Task) > Deprecate Python 3 prior to

[jira] [Created] (SPARK-29670) Make executor's bindAddress configurable

2019-10-30 Thread Nishchal Venkataramana (Jira)
Nishchal Venkataramana created SPARK-29670: -- Summary: Make executor's bindAddress configurable Key: SPARK-29670 URL: https://issues.apache.org/jira/browse/SPARK-29670 Project: Spark

[jira] [Created] (SPARK-29672) remove python2 test from python/run-tests.py

2019-10-30 Thread Shane Knapp (Jira)
Shane Knapp created SPARK-29672: --- Summary: remove python2 test from python/run-tests.py Key: SPARK-29672 URL: https://issues.apache.org/jira/browse/SPARK-29672 Project: Spark Issue Type:

[jira] [Updated] (SPARK-29670) Make executor's bindAddress configurable

2019-10-30 Thread Nishchal Venkataramana (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishchal Venkataramana updated SPARK-29670: --- Affects Version/s: (was: 2.1.1) 2.0.2

[jira] [Updated] (SPARK-29670) Make executor's bindAddress configurable

2019-10-30 Thread Nishchal Venkataramana (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishchal Venkataramana updated SPARK-29670: --- Labels: (was: bulk-closed) > Make executor's bindAddress configurable >

[jira] [Assigned] (SPARK-29646) Allow pyspark version name format `${versionNumber}-preview` in release script

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29646: - Assignee: Xingbo Jiang > Allow pyspark version name format `${versionNumber}-preview`

[jira] [Resolved] (SPARK-29646) Allow pyspark version name format `${versionNumber}-preview` in release script

2019-10-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29646. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26306

  1   2   >