[jira] [Updated] (SPARK-46354) Add restriction parameters to dynamic partitions for writing datasource

2023-12-10 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hao updated SPARK-46354: Priority: Trivial (was: Major) > Add restriction parameters to dynamic partitions for writing datasource >

[jira] [Created] (SPARK-46354) Add restriction parameters to dynamic partitions for writing datasource

2023-12-10 Thread hao (Jira)
hao created SPARK-46354: --- Summary: Add restriction parameters to dynamic partitions for writing datasource Key: SPARK-46354 URL: https://issues.apache.org/jira/browse/SPARK-46354 Project: Spark Issue

[jira] [Created] (SPARK-44483) When using Spark to read the hive table, the number of file partitions cannot be set using Spark's configuration settings

2023-07-19 Thread hao (Jira)
hao created SPARK-44483: --- Summary: When using Spark to read the hive table, the number of file partitions cannot be set using Spark's configuration settings Key: SPARK-44483 URL:

[jira] [Commented] (SPARK-42714) Sparksql temporary file conflict

2023-03-07 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697760#comment-17697760 ] hao commented on SPARK-42714: - This problem will cause the task to throw the problem of deleting the current

[jira] [Created] (SPARK-42714) Sparksql temporary file conflict

2023-03-07 Thread hao (Jira)
hao created SPARK-42714: --- Summary: Sparksql temporary file conflict Key: SPARK-42714 URL: https://issues.apache.org/jira/browse/SPARK-42714 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-41241) Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL

2023-02-01 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiliang hao updated SPARK-41241: - Description: -- Hive > create table table_test(id int); > alter table table_test change

[jira] [Commented] (SPARK-41241) Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL

2022-12-04 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643141#comment-17643141 ] weiliang hao commented on SPARK-41241: -- [~xkrogen] The problem is that Spark modifies the Hive

[jira] [Commented] (SPARK-41241) Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL

2022-11-23 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637848#comment-17637848 ] weiliang hao commented on SPARK-41241: -- I will fix it > Use Hive and Spark SQL to modify table

[jira] [Created] (SPARK-41241) Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL

2022-11-23 Thread weiliang hao (Jira)
weiliang hao created SPARK-41241: Summary: Use Hive and Spark SQL to modify table field comment, the modified results of Hive cannot be queried using Spark SQL Key: SPARK-41241 URL:

[jira] [Commented] (SPARK-41160) An error is reported when submitting a task to the yarn that enabled the timeline service

2022-11-16 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634788#comment-17634788 ] weiliang hao commented on SPARK-41160: -- I will try to fix it. Depends on jersey-client:1.19 when

[jira] [Created] (SPARK-41160) An error is reported when submitting a task to the yarn that enabled the timeline service

2022-11-16 Thread weiliang hao (Jira)
weiliang hao created SPARK-41160: Summary: An error is reported when submitting a task to the yarn that enabled the timeline service Key: SPARK-41160 URL: https://issues.apache.org/jira/browse/SPARK-41160

[jira] [Commented] (SPARK-39396) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-06 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550772#comment-17550772 ] weiliang hao commented on SPARK-39396: -- When a user with DN (cn=user, ou=people, dc=example,

[jira] [Commented] (SPARK-39396) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-06 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550751#comment-17550751 ] weiliang hao commented on SPARK-39396: -- I will try to solve this problem > Spark Thriftserver

[jira] [Created] (SPARK-39396) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-06 Thread weiliang hao (Jira)
weiliang hao created SPARK-39396: Summary: Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials Key: SPARK-39396 URL:

[jira] [Created] (SPARK-39395) Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials

2022-06-06 Thread weiliang hao (Jira)
weiliang hao created SPARK-39395: Summary: Spark Thriftserver enabled LDAP,Error using beeline connection: error code 49 - invalid credentials Key: SPARK-39395 URL:

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

2022-06-03 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17547810#comment-17547810 ] weiliang hao commented on SPARK-32570: -- cc [~dongjoon]  [~hyukjin.kwon]  > Thriftserver LDAP

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

2022-05-31 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544209#comment-17544209 ] weiliang hao commented on SPARK-32570: -- Can this JIRA be assigned to me? I will try to solve this

[jira] (SPARK-32570) Thriftserver LDAP failed

2022-05-31 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32570 ] weiliang hao deleted comment on SPARK-32570: -- was (Author: haoweiliang): Can this JIRA be assigned to me? I will try to solve this problem > Thriftserver LDAP failed >

[jira] [Commented] (SPARK-32570) Thriftserver LDAP failed

2022-05-31 Thread weiliang hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544208#comment-17544208 ] weiliang hao commented on SPARK-32570: -- Can this JIRA be assigned to me? I will try to solve this

[jira] [Updated] (SPARK-37274) When the value of this parameter is greater than the maximum value of int type, the value will be thrown out of bounds. The document description of this parameter should

2021-11-10 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hao updated SPARK-37274: Summary: When the value of this parameter is greater than the maximum value of int type, the value will be thrown

[jira] [Updated] (SPARK-37274) When the value of this parameter is greater than the maximum value of int type, the value will be thrown out of bounds. The document description of this parameter should

2021-11-10 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hao updated SPARK-37274: Description: When the value of this parameter is greater than the maximum value of int type, the value will be

[jira] [Created] (SPARK-37274) These parameters should be of type long, not int

2021-11-10 Thread hao (Jira)
hao created SPARK-37274: --- Summary: These parameters should be of type long, not int Key: SPARK-37274 URL: https://issues.apache.org/jira/browse/SPARK-37274 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

2021-02-08 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281560#comment-17281560 ] hao commented on SPARK-34406: - Yes, sir. I'm using the Yarn cluster mode. What I mean here is that when the

[jira] [Created] (SPARK-34406) When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure

2021-02-08 Thread hao (Jira)
hao created SPARK-34406: --- Summary: When we submit spark core tasks frequently, the submitted nodes will have a lot of resource pressure Key: SPARK-34406 URL: https://issues.apache.org/jira/browse/SPARK-34406

[jira] [Commented] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-05 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259391#comment-17259391 ] hao commented on SPARK-33982: - [~hyukjin.kwon] 小哥,能看懂中文的呀!是的这是同一个问题>.< > Sparksql does not support when the

[jira] [Created] (SPARK-34006) [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document

2021-01-04 Thread hao (Jira)
hao created SPARK-34006: --- Summary: [spark.sql.hive.convertMetastoreOrc]This parameter can solve orc format table insert overwrite read table, it should be stated in the document Key: SPARK-34006 URL:

[jira] [Comment Edited] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258052#comment-17258052 ] hao edited comment on SPARK-33982 at 1/4/21, 11:19 AM: --- 我认为sparksql应该得到支持insert

[jira] [Commented] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258052#comment-17258052 ] hao commented on SPARK-33982: - 我认为sparksql应该得到支持 > Sparksql does not support when the inserted table is a

[jira] [Created] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
hao created SPARK-33982: --- Summary: Sparksql does not support when the inserted table is a read table Key: SPARK-33982 URL: https://issues.apache.org/jira/browse/SPARK-33982 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-31 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451914#comment-15451914 ] Cheng Hao commented on SPARK-17299: --- Or come after SPARK-14878 ? > TRIM/LTRIM/RTRIM strips characters

[jira] [Commented] (SPARK-17299) TRIM/LTRIM/RTRIM strips characters other than spaces

2016-08-31 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451810#comment-15451810 ] Cheng Hao commented on SPARK-17299: --- Yes, that's my bad, I thought it should be the same behavior of

[jira] [Commented] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405686#comment-15405686 ] GUAN Hao commented on SPARK-16869: -- I've update the outer joining part (which was missing) to the code

[jira] [Comment Edited] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405669#comment-15405669 ] GUAN Hao edited comment on SPARK-16869 at 8/3/16 10:17 AM: --- {{i = colaesce(b.i,

[jira] [Updated] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GUAN Hao updated SPARK-16869: - Description: I have to DataFrames, both contain a column named *i* which are derived from a same

[jira] [Comment Edited] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405669#comment-15405669 ] GUAN Hao edited comment on SPARK-16869 at 8/3/16 10:16 AM: --- {{i = colaesce(b.i,

[jira] [Comment Edited] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405669#comment-15405669 ] GUAN Hao edited comment on SPARK-16869 at 8/3/16 10:14 AM: --- Oh, {{i =

[jira] [Commented] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405669#comment-15405669 ] GUAN Hao commented on SPARK-16869: -- Oh, {{ i = colaesce(b.i, c.i) }} is actually just a comment. Please

[jira] [Created] (SPARK-16869) Wrong projection when join on columns with the same name which are derived from the same dataframe

2016-08-03 Thread GUAN Hao (JIRA)
GUAN Hao created SPARK-16869: Summary: Wrong projection when join on columns with the same name which are derived from the same dataframe Key: SPARK-16869 URL: https://issues.apache.org/jira/browse/SPARK-16869

[jira] [Updated] (SPARK-16214) fix the denominator of SparkPi

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Description: As the iteration number is n - 1, the denominator would also be n, was: As the

[jira] [Updated] (SPARK-16214) fix the denominator of SparkPi

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Description: As the iteration number is n - 1, then the denominator would also be n, was: As the

[jira] [Updated] (SPARK-16214) fix the denominator of SparkPi

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Summary: fix the denominator of SparkPi (was: make SparkPi iteration number correct) > fix the

[jira] [Updated] (SPARK-16214) make SparkPi iteration number correct

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Attachment: (was: SPARK-16214.patch) > make SparkPi iteration number correct >

[jira] [Updated] (SPARK-16214) make SparkPi iteration number correct

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Description: As the denominator is n, the iteration number should also be n was: As the denominator

[jira] [Updated] (SPARK-16214) make SparkPi iteration number correct

2016-06-26 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Description: As the denominator is n, the iteration number should also be n A pull request is

[jira] [Updated] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Attachment: (was: SPARK-16214.patch) > make SparkPi iteration time correct >

[jira] [Updated] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Description: As the denominator is n, the iteration time should also be n was: As the denominator is

[jira] [Updated] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Attachment: SPARK-16214.patch > make SparkPi iteration time correct >

[jira] [Resolved] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao resolved SPARK-16214. -- Resolution: Resolved > make SparkPi iteration time correct > --- > >

[jira] [Updated] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Attachment: SPARK-16214.patch > make SparkPi iteration time correct >

[jira] [Updated] (SPARK-16214) make SparkPi iteration time correct

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Summary: make SparkPi iteration time correct (was: optimize SparkPi) > make SparkPi iteration time

[jira] [Updated] (SPARK-16214) optimize SparkPi

2016-06-25 Thread Yang Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Hao updated SPARK-16214: - Summary: optimize SparkPi (was: calculate pi is not correct) > optimize SparkPi > > >

[jira] [Created] (SPARK-16214) calculate pi is not correct

2016-06-25 Thread Yang Hao (JIRA)
Yang Hao created SPARK-16214: Summary: calculate pi is not correct Key: SPARK-16214 URL: https://issues.apache.org/jira/browse/SPARK-16214 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-15859) Optimize the Partition Pruning with Disjunction

2016-06-09 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-15859: - Summary: Optimize the Partition Pruning with Disjunction Key: SPARK-15859 URL: https://issues.apache.org/jira/browse/SPARK-15859 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-07 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318654#comment-15318654 ] Cheng Hao commented on SPARK-15730: --- [~jameszhouyi], can you please verify this fixing? > [Spark SQL]

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300072#comment-15300072 ] Cheng Hao commented on SPARK-15034: --- [~yhuai], but it probably not respect the `hive-site.xml`, and

[jira] [Comment Edited] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2016-05-09 Thread Xin Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276109#comment-15276109 ] Xin Hao edited comment on SPARK-4452 at 5/9/16 8:36 AM: Since this is an old issue

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

2016-05-09 Thread Xin Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276109#comment-15276109 ] Xin Hao commented on SPARK-4452: Since this is an old issue which impact Spark since 1.1.0, can the patch

[jira] [Commented] (SPARK-13894) SQLContext.range should return Dataset[Long]

2016-03-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195274#comment-15195274 ] Cheng Hao commented on SPARK-13894: --- The existing functions "SQLContext.range()" returns the underlying

[jira] [Commented] (SPARK-13326) Dataset in spark 2.0.0-SNAPSHOT missing columns

2016-03-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15195022#comment-15195022 ] Cheng Hao commented on SPARK-13326: --- Can not reproduce it anymore, can you try it again? > Dataset in

[jira] [Commented] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior of Spark application is difficult to un

2016-02-02 Thread Ji Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127911#comment-15127911 ] Ji Hao commented on SPARK-13133: Sean Owen, I think you should consider this issue, the error log may be

[jira] [Issue Comment Deleted] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior of Spark application is dif

2016-02-02 Thread Ji Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ji Hao updated SPARK-13133: --- Comment: was deleted (was: Sean Owen, I think you should consider this issue, the error log may be more

[jira] [Commented] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior of Spark application is difficult to un

2016-02-02 Thread Ji Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127912#comment-15127912 ] Ji Hao commented on SPARK-13133: Sean Owen, I think you should consider this issue, the error log may be

[jira] [Updated] (SPARK-12610) Add Anti Join Operators

2016-01-03 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-12610: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-4226 > Add Anti Join Operators >

[jira] [Created] (SPARK-12610) Add Anti Join Operators

2016-01-03 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-12610: - Summary: Add Anti Join Operators Key: SPARK-12610 URL: https://issues.apache.org/jira/browse/SPARK-12610 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-12196) Store blocks in different speed storage devices by hierarchy way

2015-12-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072634#comment-15072634 ] Cheng Hao commented on SPARK-12196: --- Thank you wei wu to support this feature! However, we're trying

[jira] [Updated] (SPARK-8360) Streaming DataFrames

2015-12-02 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-8360: - Attachment: StreamingDataFrameProposal.pdf This is a proposal for streaming dataframes that we were

[jira] [Comment Edited] (SPARK-8360) Streaming DataFrames

2015-12-02 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao edited comment on SPARK-8360 at 12/2/15 12:14 PM: Remove the google

[jira] [Comment Edited] (SPARK-8360) Streaming DataFrames

2015-12-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao edited comment on SPARK-8360 at 12/2/15 6:19 AM: --- Add some thoughts on

[jira] [Commented] (SPARK-8360) Streaming DataFrames

2015-12-01 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035335#comment-15035335 ] Cheng Hao commented on SPARK-8360: -- Add some thoughts on StreamingSQL.

[jira] [Created] (SPARK-12064) Make the SqlParser as trait for better integrated with extensions

2015-11-30 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-12064: - Summary: Make the SqlParser as trait for better integrated with extensions Key: SPARK-12064 URL: https://issues.apache.org/jira/browse/SPARK-12064 Project: Spark

[jira] [Resolved] (SPARK-12064) Make the SqlParser as trait for better integrated with extensions

2015-11-30 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao resolved SPARK-12064. --- Resolution: Won't Fix DBX has plan to remove the SqlParser in 2.0. > Make the SqlParser as trait

[jira] [Commented] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-11-11 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001423#comment-15001423 ] Cheng Hao commented on SPARK-10865: --- 1.5.2 is released, I am not sure whether part of it now or not. >

[jira] [Commented] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-11-11 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001422#comment-15001422 ] Cheng Hao commented on SPARK-10865: --- We actually follow the criteria of Hive, and actually I tested it

[jira] [Created] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11512: - Summary: Bucket Join Key: SPARK-11512 URL: https://issues.apache.org/jira/browse/SPARK-11512 Project: Spark Issue Type: Sub-task Components: SQL

[jira] [Commented] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990867#comment-14990867 ] Cheng Hao commented on SPARK-11512: --- Oh, yes, but SPARK-5292 is only about to support the Hive bucket,

[jira] [Commented] (SPARK-11512) Bucket Join

2015-11-04 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990868#comment-14990868 ] Cheng Hao commented on SPARK-11512: --- We need to support the "bucket" for DataSource API. > Bucket Join

[jira] [Commented] (SPARK-10371) Optimize sequential projections

2015-10-29 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981650#comment-14981650 ] Cheng Hao commented on SPARK-10371: --- Eliminating the common sub expression within the projection? >

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979646#comment-14979646 ] Cheng Hao commented on SPARK-11330: --- [~saif.a.ellafi] I've checked that with 1.5.0 and it's confirmed

[jira] [Comment Edited] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979699#comment-14979699 ] Cheng Hao edited comment on SPARK-11330 at 10/29/15 2:48 AM: - OK, seems it's

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-28 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979699#comment-14979699 ] Cheng Hao commented on SPARK-11330: --- OK, seems it's solved in

[jira] [Comment Edited] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977600#comment-14977600 ] Cheng Hao edited comment on SPARK-11330 at 10/28/15 2:28 AM: - Hi,

[jira] [Commented] (SPARK-11330) Filter operation on StringType after groupBy PERSISTED brings no results

2015-10-27 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977600#comment-14977600 ] Cheng Hao commented on SPARK-11330: --- Hi, [~saif.a.ellafi], I've tried the code like below: {code} case

[jira] [Updated] (SPARK-9735) Auto infer partition schema of HadoopFsRelation should should respected the user specified one

2015-10-20 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-9735: - Description: This code is copied from the hadoopFsRelationSuite.scala {code} partitionedTestDF = (for {

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2015-10-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958524#comment-14958524 ] Cheng Hao commented on SPARK-4226: -- [~nadenf] Actually I am working on it right now, and the first PR is

[jira] [Created] (SPARK-11076) Decimal Support for Ceil/Floor

2015-10-12 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11076: - Summary: Decimal Support for Ceil/Floor Key: SPARK-11076 URL: https://issues.apache.org/jira/browse/SPARK-11076 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-11041) Add (NOT) IN / EXISTS support for predicates

2015-10-09 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao closed SPARK-11041. - Resolution: Duplicate > Add (NOT) IN / EXISTS support for predicates >

[jira] [Created] (SPARK-11041) Add (NOT) IN / EXISTS support for predicates

2015-10-09 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-11041: - Summary: Add (NOT) IN / EXISTS support for predicates Key: SPARK-11041 URL: https://issues.apache.org/jira/browse/SPARK-11041 Project: Spark Issue Type:

[jira] [Updated] (SPARK-10992) Partial Aggregation Support for Hive UDAF

2015-10-07 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10992: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-4366 > Partial Aggregation Support for

[jira] [Created] (SPARK-10831) Spark SQL Configuration missing in the doc

2015-09-25 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10831: - Summary: Spark SQL Configuration missing in the doc Key: SPARK-10831 URL: https://issues.apache.org/jira/browse/SPARK-10831 Project: Spark Issue Type:

[jira] [Created] (SPARK-10829) Scan DataSource with predicate expression combine partition key and attributes doesn't work

2015-09-24 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-10829: - Summary: Scan DataSource with predicate expression combine partition key and attributes doesn't work Key: SPARK-10829 URL: https://issues.apache.org/jira/browse/SPARK-10829

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-23 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14904778#comment-14904778 ] Cheng Hao commented on SPARK-10733: --- [~jameszhouyi] Can you please patch the

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-17 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802912#comment-14802912 ] Cheng Hao commented on SPARK-10474: --- The root reason for this failure, is because of the

[jira] [Comment Edited] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-17 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802912#comment-14802912 ] Cheng Hao edited comment on SPARK-10474 at 9/17/15 1:48 PM: The root reason

[jira] [Commented] (SPARK-10606) Cube/Rollup/GrpSet doesn't create the correct plan when group by is on something other than an AttributeReference

2015-09-16 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791499#comment-14791499 ] Cheng Hao commented on SPARK-10606: --- [~rhbutani] Which version are you using, actually I've fixed the

[jira] [Commented] (SPARK-4226) SparkSQL - Add support for subqueries in predicates

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746642#comment-14746642 ] Cheng Hao commented on SPARK-4226: -- Thank you [~brooks], you're right! I meant it will makes more

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744969#comment-14744969 ] Cheng Hao commented on SPARK-10474: --- The root causes for the exception is the executor don't have

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744966#comment-14744966 ] Cheng Hao commented on SPARK-10466: --- [~naliazheli] It's an irrelevant issue, you'd better to subscribe

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745008#comment-14745008 ] Cheng Hao commented on SPARK-10474: --- But from the current implementation, we'd better not to throw

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744967#comment-14744967 ] Cheng Hao commented on SPARK-10466: --- [~naliazheli] It's an irrelevant issue, you'd better to subscribe

[jira] [Issue Comment Deleted] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

2015-09-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated SPARK-10466: -- Comment: was deleted (was: [~naliazheli] It's an irrelevant issue, you'd better to subscribe the

  1   2   3   4   5   >