[jira] [Created] (SPARK-46714) Overwrite partitions with custom location should reset partition locations

2024-01-14 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-46714: --- Summary: Overwrite partitions with custom location should reset partition locations Key: SPARK-46714 URL: https://issues.apache.org/jira/browse/SPARK-46714 Project:

[jira] [Resolved] (SPARK-41816) Spark ThriftServer should not close file system when log out

2023-01-07 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-41816. - Resolution: Invalid > Spark ThriftServer should not close file system when log out >

[jira] [Updated] (SPARK-41816) Spark ThriftServer should not close file system when log out

2023-01-02 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-41816: Description: Currently when enable impersonation, Spark Thriftserver will close filesystem

[jira] [Updated] (SPARK-41816) Spark ThriftServer should not close file system when log out

2023-01-02 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-41816: Description: Currently when enabled impersonation, Spark Thriftserver will close filesystem

[jira] [Created] (SPARK-41816) Spark ThriftServer should not close file system when log out

2023-01-02 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-41816: --- Summary: Spark ThriftServer should not close file system when log out Key: SPARK-41816 URL: https://issues.apache.org/jira/browse/SPARK-41816 Project: Spark

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

2021-06-28 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370993#comment-17370993 ] Adrian Wang commented on SPARK-26764: - [~zshao] Thanks for the interest. We created an open-source

[jira] [Commented] (SPARK-30130) Hardcoded numeric values in common table expressions which utilize GROUP BY are interpreted as ordinal positions

2021-05-16 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345730#comment-17345730 ] Adrian Wang commented on SPARK-30130: - I also met this on 2.4.7, and this has been fix on

[jira] [Created] (SPARK-35238) Add JindoFS SDK in cloud integration documents

2021-04-27 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-35238: --- Summary: Add JindoFS SDK in cloud integration documents Key: SPARK-35238 URL: https://issues.apache.org/jira/browse/SPARK-35238 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31595) Spark sql cli should allow unescaped quote mark in quoted string

2020-04-28 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17094980#comment-17094980 ] Adrian Wang commented on SPARK-31595: - [~Ankitraj] Thanks, I have already created a pull request on

[jira] [Created] (SPARK-31595) Spark sql cli should allow unescaped quote mark in quoted string

2020-04-28 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-31595: --- Summary: Spark sql cli should allow unescaped quote mark in quoted string Key: SPARK-31595 URL: https://issues.apache.org/jira/browse/SPARK-31595 Project: Spark

[jira] [Updated] (SPARK-29177) Zombie tasks prevents executor from releasing when task exceeds maxResultSize

2019-09-19 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-29177: Description: When we fetch results from executors and found the total size has exceeded the

[jira] [Created] (SPARK-29177) Zombie tasks prevents executor from releasing when task exceeds maxResultSize

2019-09-19 Thread Adrian Wang (Jira)
Adrian Wang created SPARK-29177: --- Summary: Zombie tasks prevents executor from releasing when task exceeds maxResultSize Key: SPARK-29177 URL: https://issues.apache.org/jira/browse/SPARK-29177 Project:

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2019-09-16 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931057#comment-16931057 ] Adrian Wang commented on SPARK-13446: - or you can just apply the patch from SPARK-27349 and

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2019-09-16 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931055#comment-16931055 ] Adrian Wang commented on SPARK-13446: - [~jpbordi][~headcra6] I am using mysql as hive metastore

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2019-09-16 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930298#comment-16930298 ] Adrian Wang commented on SPARK-13446: - [~elgalu][~toopt4][~headcra6][~jpbordi][~F7753] The reference

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

2019-09-11 Thread Adrian Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928100#comment-16928100 ] Adrian Wang commented on SPARK-29038: - This seems duplicates with our proposal of SPARK-26764 . We

[jira] [Created] (SPARK-27279) Reuse subquery should compare child node only

2019-03-25 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-27279: --- Summary: Reuse subquery should compare child node only Key: SPARK-27279 URL: https://issues.apache.org/jira/browse/SPARK-27279 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-22601) Data load is getting displayed successful on providing non existing hdfs file path

2019-03-22 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-22601: Fix Version/s: (was: 2.2.1) 2.2.2 > Data load is getting displayed

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

2019-02-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779170#comment-16779170 ] Adrian Wang commented on SPARK-26764: - Hi [~Tagar] , the idea has something common with materialized

[jira] [Created] (SPARK-26764) [SPIP] Spark Relational Cache

2019-01-29 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-26764: --- Summary: [SPIP] Spark Relational Cache Key: SPARK-26764 URL: https://issues.apache.org/jira/browse/SPARK-26764 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-26764) [SPIP] Spark Relational Cache

2019-01-29 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-26764: Attachment: Relational+Cache+SPIP.pdf > [SPIP] Spark Relational Cache >

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700836#comment-16700836 ] Adrian Wang commented on SPARK-26155: - [~Jk_Self] can you also test this on Spark 2.4? > Spark SQL

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700253#comment-16700253 ] Adrian Wang commented on SPARK-26155: - [~viirya] , thanks for your reply. [~Jk_Self] initially found

[jira] [Created] (SPARK-26181) the `hasMinMaxStats` method of `ColumnStatsMap` is not correct

2018-11-26 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-26181: --- Summary: the `hasMinMaxStats` method of `ColumnStatsMap` is not correct Key: SPARK-26181 URL: https://issues.apache.org/jira/browse/SPARK-26181 Project: Spark

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-11-23 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696541#comment-16696541 ] Adrian Wang commented on SPARK-26155: - It seems the performance downgrade is related to CPU cache,

[jira] [Closed] (SPARK-14631) "drop database cascade" needs to unregister functions for HiveExternalCatalog

2016-12-12 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang closed SPARK-14631. --- Resolution: Not A Problem > "drop database cascade" needs to unregister functions for

[jira] [Created] (SPARK-17427) function SIZE should return -1 when parameter is null

2016-09-06 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-17427: --- Summary: function SIZE should return -1 when parameter is null Key: SPARK-17427 URL: https://issues.apache.org/jira/browse/SPARK-17427 Project: Spark Issue

[jira] [Commented] (SPARK-4003) Add {Big Decimal, Timestamp, Date} types to Java SqlContext

2016-07-21 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388922#comment-15388922 ] Adrian Wang commented on SPARK-4003: DataTypes.TimestampType is not using java.sql.Timestamp

[jira] [Commented] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-12 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374241#comment-15374241 ] Adrian Wang commented on SPARK-16515: - The problem is spark did not find the right record writer from

[jira] [Resolved] (SPARK-15397) 'locate' UDF got different result with boundary value case compared to Hive engine

2016-06-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-15397. - Resolution: Fixed > 'locate' UDF got different result with boundary value case compared to Hive

[jira] [Commented] (SPARK-14126) [Table related commands] Truncate table

2016-04-18 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247161#comment-15247161 ] Adrian Wang commented on SPARK-14126: - Yes, still working. > [Table related commands] Truncate table

[jira] [Created] (SPARK-14631) "drop database cascade" needs to unregister functions for HiveExternalCatalog

2016-04-14 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-14631: --- Summary: "drop database cascade" needs to unregister functions for HiveExternalCatalog Key: SPARK-14631 URL: https://issues.apache.org/jira/browse/SPARK-14631 Project:

[jira] [Commented] (SPARK-14126) [Table related commands] Truncate table

2016-04-11 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236353#comment-15236353 ] Adrian Wang commented on SPARK-14126: - I'm working on this. > [Table related commands] Truncate

[jira] [Updated] (SPARK-14021) Support custom context derived from HiveContext for SparkSQLEnv

2016-03-19 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-14021: Description: This is to create a custom context for command bin/spark-sql and

[jira] [Created] (SPARK-14021) Support custom context derived from HiveContext for SparkSQLEnv

2016-03-19 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-14021: --- Summary: Support custom context derived from HiveContext for SparkSQLEnv Key: SPARK-14021 URL: https://issues.apache.org/jira/browse/SPARK-14021 Project: Spark

[jira] [Commented] (SPARK-13819) using a tegexp_replace in a gropu by clause raises a nullpointerexception

2016-03-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194902#comment-15194902 ] Adrian Wang commented on SPARK-13819: - I'll take a look at this. > using a tegexp_replace in a gropu

[jira] [Commented] (SPARK-13837) SQL Context function to_date() returns wrong date

2016-03-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194892#comment-15194892 ] Adrian Wang commented on SPARK-13837: - Which timezone are your system in? > SQL Context function

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186674#comment-15186674 ] Adrian Wang commented on SPARK-13393: - That's the case we should throw exceptions. > Column mismatch

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186673#comment-15186673 ] Adrian Wang commented on SPARK-13393: - See my updated comment. That's not reasonable. > Column

[jira] [Comment Edited] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186660#comment-15186660 ] Adrian Wang edited comment on SPARK-13393 at 3/9/16 7:31 AM: - How do you

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186661#comment-15186661 ] Adrian Wang commented on SPARK-13393: - How do you resolve it? Both sides are `df`, so we can resolve

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186660#comment-15186660 ] Adrian Wang commented on SPARK-13393: - How do you resolve it? Both sides are `df`, so we can resolve

[jira] [Issue Comment Deleted] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-13393: Comment: was deleted (was: How do you resolve it? Both sides are `df`, so we can resolve df("key")

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186652#comment-15186652 ] Adrian Wang commented on SPARK-13393: - In your example, df1("name") and df2("name") is exactly the

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186642#comment-15186642 ] Adrian Wang commented on SPARK-13393: - This is another issue; here we are talking about `varadha` and

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186605#comment-15186605 ] Adrian Wang commented on SPARK-13393: - So that's the reason I have to introduce the layer of

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-08 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186580#comment-15186580 ] Adrian Wang commented on SPARK-13393: - Hi [~srinathsmn] In this `errorDF`, both `df('id')` and

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2016-03-02 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177356#comment-15177356 ] Adrian Wang commented on SPARK-13446: - That's not enough. We still need some code change. > Spark

[jira] [Commented] (SPARK-13393) Column mismatch issue in left_outer join using Spark DataFrame

2016-03-01 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175034#comment-15175034 ] Adrian Wang commented on SPARK-13393: - [~srinathsmn] I have identified the issue, and working on

[jira] [Commented] (SPARK-13446) Spark need to support reading data from HIve 2.0.0 metastore

2016-02-23 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158516#comment-15158516 ] Adrian Wang commented on SPARK-13446: - Hive 2.0 use HIVE_ZOOKEEPER_SESSION_TIMEOUT instead of

[jira] [Commented] (SPARK-12930) NullPointerException running hive query with array dereference in select and where clause

2016-02-22 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156600#comment-15156600 ] Adrian Wang commented on SPARK-12930: - Could you try SPARK-13056? > NullPointerException running

[jira] [Commented] (SPARK-13301) PySpark Dataframe return wrong results with custom UDF

2016-02-18 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151913#comment-15151913 ] Adrian Wang commented on SPARK-13301: - Hi Simone, I tried your code using master branch and the

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148196#comment-15148196 ] Adrian Wang commented on SPARK-13283: - So the problem here is that "from" is a reserved word in

[jira] [Commented] (SPARK-13283) Spark doesn't escape column names when creating table on JDBC

2016-02-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148171#comment-15148171 ] Adrian Wang commented on SPARK-13283: - See comments from SPARK-13297, this have been fixed in master

[jira] [Commented] (SPARK-12985) Spark Hive thrift server big decimal data issue

2016-02-03 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130013#comment-15130013 ] Adrian Wang commented on SPARK-12985: - I think this is a problem of Simba. JDBC never require a

[jira] [Created] (SPARK-13056) Map column would throw NPE if value is null

2016-01-27 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-13056: --- Summary: Map column would throw NPE if value is null Key: SPARK-13056 URL: https://issues.apache.org/jira/browse/SPARK-13056 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-12828) support natural join

2016-01-14 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-12828: --- Summary: support natural join Key: SPARK-12828 URL: https://issues.apache.org/jira/browse/SPARK-12828 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-11983) remove all unused codegen fallback traits

2015-11-25 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11983: --- Summary: remove all unused codegen fallback traits Key: SPARK-11983 URL: https://issues.apache.org/jira/browse/SPARK-11983 Project: Spark Issue Type:

[jira] [Updated] (SPARK-11983) remove all unused codegen fallback traits

2015-11-25 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-11983: Description: We use trait `CodegenFallback` to generate default code gen code, and if we have

[jira] [Commented] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026093#comment-15026093 ] Adrian Wang commented on SPARK-11972: - SPARK-11624 would resolve this, too. That's because we created

[jira] [Created] (SPARK-11916) Expression TRIM/LTRIM/RTRIM to support specific trim word

2015-11-22 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11916: --- Summary: Expression TRIM/LTRIM/RTRIM to support specific trim word Key: SPARK-11916 URL: https://issues.apache.org/jira/browse/SPARK-11916 Project: Spark

[jira] [Created] (SPARK-11624) Spark SQL CLI will set sessionstate twice

2015-11-09 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11624: --- Summary: Spark SQL CLI will set sessionstate twice Key: SPARK-11624 URL: https://issues.apache.org/jira/browse/SPARK-11624 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-11624) Spark SQL CLI will set sessionstate twice

2015-11-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-11624: Description: spark-sql> !echo "test"; Exception in thread "main" java.lang.ClassCastException:

[jira] [Created] (SPARK-11591) flush spark-sql command line history to history file

2015-11-09 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11591: --- Summary: flush spark-sql command line history to history file Key: SPARK-11591 URL: https://issues.apache.org/jira/browse/SPARK-11591 Project: Spark Issue

[jira] [Created] (SPARK-11592) flush spark-sql command line history to history file

2015-11-09 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11592: --- Summary: flush spark-sql command line history to history file Key: SPARK-11592 URL: https://issues.apache.org/jira/browse/SPARK-11592 Project: Spark Issue

[jira] [Closed] (SPARK-11591) flush spark-sql command line history to history file

2015-11-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang closed SPARK-11591. --- Resolution: Fixed > flush spark-sql command line history to history file >

[jira] [Created] (SPARK-11396) datetime function: to_unix_timestamp

2015-10-28 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11396: --- Summary: datetime function: to_unix_timestamp Key: SPARK-11396 URL: https://issues.apache.org/jira/browse/SPARK-11396 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-11312) Cannot drop temporary function

2015-10-26 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-11312: --- Summary: Cannot drop temporary function Key: SPARK-11312 URL: https://issues.apache.org/jira/browse/SPARK-11312 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-11312) Cannot drop temporary function

2015-10-26 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-11312. - Resolution: Duplicate > Cannot drop temporary function > -- > >

[jira] [Commented] (SPARK-10507) reject temporal expressions such as timestamp - timestamp at parse time

2015-10-12 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952747#comment-14952747 ] Adrian Wang commented on SPARK-10507: - It seems this bug has been fixed long long ago. I just checked

[jira] [Created] (SPARK-10463) remove PromotePrecision during optimization

2015-09-06 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-10463: --- Summary: remove PromotePrecision during optimization Key: SPARK-10463 URL: https://issues.apache.org/jira/browse/SPARK-10463 Project: Spark Issue Type:

[jira] [Commented] (SPARK-8360) Streaming DataFrames

2015-08-25 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712377#comment-14712377 ] Adrian Wang commented on SPARK-8360:

[jira] [Updated] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-21 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-10130: Priority: Blocker (was: Major) type coercion for IF should have children resolved first

[jira] [Updated] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-21 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-10130: Fix Version/s: (was: 1.5.0) type coercion for IF should have children resolved first

[jira] [Updated] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-21 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-10130: Target Version/s: 1.5.0 type coercion for IF should have children resolved first

[jira] [Created] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-10130: --- Summary: type coercion for IF should have children resolved first Key: SPARK-10130 URL: https://issues.apache.org/jira/browse/SPARK-10130 Project: Spark Issue

[jira] [Updated] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-10130: Fix Version/s: 1.5.0 type coercion for IF should have children resolved first

[jira] [Created] (SPARK-10083) CaseWhen should support type coercion of DecimalType and FractionalType

2015-08-18 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-10083: --- Summary: CaseWhen should support type coercion of DecimalType and FractionalType Key: SPARK-10083 URL: https://issues.apache.org/jira/browse/SPARK-10083 Project: Spark

[jira] [Commented] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643761#comment-14643761 ] Adrian Wang commented on SPARK-9374: [~chenghao][~cloud_fan][~jameszhouyi]

[jira] [Commented] (SPARK-9196) DatetimeExpressionsSuite: function current_timestamp is flaky

2015-07-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633733#comment-14633733 ] Adrian Wang commented on SPARK-9196: Thanks, I'll fix it asap

[jira] [Commented] (SPARK-9196) DatetimeExpressionsSuite: function current_timestamp is flaky

2015-07-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633740#comment-14633740 ] Adrian Wang commented on SPARK-9196: if this is very often, we can ignore this test

[jira] [Commented] (SPARK-9196) DatetimeExpressionsSuite: function current_timestamp is flaky

2015-07-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634420#comment-14634420 ] Adrian Wang commented on SPARK-9196: [~davies] I got 2 solutions for this problem: 1.

[jira] [Commented] (SPARK-9196) DatetimeExpressionsSuite: function current_timestamp is flaky

2015-07-20 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634452#comment-14634452 ] Adrian Wang commented on SPARK-9196: [~marmbrus]we have test for that case. The flaky

[jira] [Commented] (SPARK-9051) SortMergeCompatibilitySuite is flaky

2015-07-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629104#comment-14629104 ] Adrian Wang commented on SPARK-9051: This might have something to do with SPARK-9027 ,

[jira] [Commented] (SPARK-9051) SortMergeCompatibilitySuite is flaky

2015-07-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629105#comment-14629105 ] Adrian Wang commented on SPARK-9051: Just find that Michael has revert it and

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616301#comment-14616301 ] Adrian Wang commented on SPARK-8864: just provide the precise of current design for

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616299#comment-14616299 ] Adrian Wang commented on SPARK-8864: no, that's not enough. Date/time function and

[jira] [Comment Edited] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616293#comment-14616293 ] Adrian Wang edited comment on SPARK-8864 at 7/7/15 7:34 AM:

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616268#comment-14616268 ] Adrian Wang commented on SPARK-8864: Thanks for the design. Two comments: 1. If a

[jira] [Commented] (SPARK-8864) Date/time function and data type design

2015-07-07 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616293#comment-14616293 ] Adrian Wang commented on SPARK-8864: Then we are using a Long for us. Long can be up

[jira] [Resolved] (SPARK-5215) concat support in sqlcontext

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-5215. Resolution: Duplicate concat support in sqlcontext

[jira] [Commented] (SPARK-8174) date/time function: unix_timestamp

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578513#comment-14578513 ] Adrian Wang commented on SPARK-8174: I'll deal with this. date/time function:

[jira] [Commented] (SPARK-8182) date/time function: minute

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578523#comment-14578523 ] Adrian Wang commented on SPARK-8182: I'll deal with this. date/time function: minute

[jira] [Commented] (SPARK-8183) date/time function: second

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578524#comment-14578524 ] Adrian Wang commented on SPARK-8183: I'll deal with this. date/time function: second

[jira] [Commented] (SPARK-8184) date/time function: weekofyear

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578525#comment-14578525 ] Adrian Wang commented on SPARK-8184: I'll deal with this. date/time function:

[jira] [Commented] (SPARK-8193) date/time function: current_timestamp

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578601#comment-14578601 ] Adrian Wang commented on SPARK-8193: I'll deal with this. date/time function:

[jira] [Commented] (SPARK-8159) Improve SQL/DataFrame expression coverage

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578509#comment-14578509 ] Adrian Wang commented on SPARK-8159: Are we missing xpath functions? Improve

[jira] [Issue Comment Deleted] (SPARK-8159) Improve SQL/DataFrame expression coverage

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-8159: --- Comment: was deleted (was: Are we missing xpath functions?) Improve SQL/DataFrame expression

[jira] [Commented] (SPARK-8159) Improve SQL/DataFrame expression coverage

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578508#comment-14578508 ] Adrian Wang commented on SPARK-8159: Are we missing xpath functions? Improve

[jira] [Commented] (SPARK-8177) date/time function: year

2015-06-09 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578518#comment-14578518 ] Adrian Wang commented on SPARK-8177: I'll deal with this. date/time function: year

  1   2   3   >