[jira] [Commented] (SPARK-37800) TreeNode.argString incorrectly formats arguments of type Set[_]

2022-01-01 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17467526#comment-17467526 ] Simeon Simeonov commented on SPARK-37800: - [~hyukjin.kwon] Done:

[jira] [Created] (SPARK-37800) TreeNode.argString incorrectly formats arguments of type Set[_]

2022-01-01 Thread Simeon Simeonov (Jira)
Simeon Simeonov created SPARK-37800: --- Summary: TreeNode.argString incorrectly formats arguments of type Set[_] Key: SPARK-37800 URL: https://issues.apache.org/jira/browse/SPARK-37800 Project: Spark

[jira] [Comment Edited] (SPARK-35097) Add column name to SparkUpgradeException about ancient datetime

2021-04-18 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324643#comment-17324643 ] Simeon Simeonov edited comment on SPARK-35097 at 4/19/21, 1:58 AM: ---

[jira] [Commented] (SPARK-35097) Add column name to SparkUpgradeException about ancient datetime

2021-04-18 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324643#comment-17324643 ] Simeon Simeonov commented on SPARK-35097: - [~maxgekk] thanks for creating this issue; it came

[jira] [Commented] (SPARK-27790) Support ANSI SQL INTERVAL types

2021-03-18 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304526#comment-17304526 ] Simeon Simeonov commented on SPARK-27790: - Maxim, this is good stuff.  Does ANSI SQL allow

[jira] [Commented] (SPARK-32630) Reduce user confusion and subtle bugs by optionally preventing date & timestamp comparison

2020-08-16 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178624#comment-17178624 ] Simeon Simeonov commented on SPARK-32630: - [~rxin] fyi, one of the subtle issues that add

[jira] [Created] (SPARK-32630) Reduce user confusion and subtle bugs by optionally preventing date & timestamp comparison

2020-08-16 Thread Simeon Simeonov (Jira)
Simeon Simeonov created SPARK-32630: --- Summary: Reduce user confusion and subtle bugs by optionally preventing date & timestamp comparison Key: SPARK-32630 URL: https://issues.apache.org/jira/browse/SPARK-32630

[jira] [Comment Edited] (SPARK-30127) UDF should work for case class like Dataset operations

2019-12-11 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993750#comment-16993750 ] Simeon Simeonov edited comment on SPARK-30127 at 12/11/19 6:01 PM: ---

[jira] [Comment Edited] (SPARK-30127) UDF should work for case class like Dataset operations

2019-12-11 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993750#comment-16993750 ] Simeon Simeonov edited comment on SPARK-30127 at 12/11/19 5:42 PM: ---

[jira] [Commented] (SPARK-30127) UDF should work for case class like Dataset operations

2019-12-11 Thread Simeon Simeonov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993750#comment-16993750 ] Simeon Simeonov commented on SPARK-30127: - The ability to transform one or more columns with

[jira] [Commented] (SPARK-26696) Dataset encoder should be publicly accessible

2019-01-22 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749518#comment-16749518 ] Simeon Simeonov commented on SPARK-26696: - [PR with

[jira] [Created] (SPARK-26696) Dataset encoder should be publicly accessible

2019-01-22 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-26696: --- Summary: Dataset encoder should be publicly accessible Key: SPARK-26696 URL: https://issues.apache.org/jira/browse/SPARK-26696 Project: Spark Issue

[jira] [Commented] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

2018-11-17 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690751#comment-16690751 ] Simeon Simeonov commented on SPARK-26084: - [~hvanhovell] done 

[jira] [Created] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

2018-11-15 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-26084: --- Summary: AggregateExpression.references fails on unresolved expression trees Key: SPARK-26084 URL: https://issues.apache.org/jira/browse/SPARK-26084 Project:

[jira] [Commented] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

2018-11-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688760#comment-16688760 ] Simeon Simeonov commented on SPARK-26084: - /cc [~maropu] [~hvanhovell] who worked on the PR that

[jira] [Updated] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-25769: Description: {{UnresolvedAttribute.sql()}} output is incorrectly escaped for nested

[jira] [Updated] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-25769: Description:  This issue affects dynamic SQL generation that relies on {{sql()}}.

[jira] [Created] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-10-18 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-25769: --- Summary: UnresolvedAttribute.sql() incorrectly escapes nested columns Key: SPARK-25769 URL: https://issues.apache.org/jira/browse/SPARK-25769 Project: Spark

[jira] [Updated] (SPARK-16483) Unifying struct fields and columns

2018-07-22 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-16483: Description: This issue comes as a result of an exchange with Michael Armbrust outside of

[jira] [Updated] (SPARK-16483) Unifying struct fields and columns

2018-07-22 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-16483: Affects Version/s: 2.3.1 Description: This issue comes as a result of an

[jira] [Commented] (SPARK-24269) Infer nullability rather than declaring all columns as nullable

2018-05-22 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484071#comment-16484071 ] Simeon Simeonov commented on SPARK-24269: - There are many reasons why correct nullability

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

2018-01-25 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340515#comment-16340515 ] Simeon Simeonov commented on SPARK-4502: +1 [~holdenk] this should be a big boost for any Spark

[jira] [Commented] (SPARK-18727) Support schema evolution as new files are inserted into table

2017-04-27 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987668#comment-15987668 ] Simeon Simeonov commented on SPARK-18727: - [~xwu0226] The merged PR handles the use case of new

[jira] [Commented] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

2017-02-23 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881490#comment-15881490 ] Simeon Simeonov commented on SPARK-19716: - This is an important issue because it prevent schema

[jira] [Commented] (SPARK-18727) Support schema evolution as new files are inserted into table

2017-01-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816538#comment-15816538 ] Simeon Simeonov commented on SPARK-18727: - [~xwu0226] A common use case is adding a new field in

[jira] [Created] (SPARK-16954) UDFs should allow output type to be specified in terms of the input type

2016-08-08 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-16954: --- Summary: UDFs should allow output type to be specified in terms of the input type Key: SPARK-16954 URL: https://issues.apache.org/jira/browse/SPARK-16954

[jira] [Created] (SPARK-16483) Unifying struct fields and columns

2016-07-11 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-16483: --- Summary: Unifying struct fields and columns Key: SPARK-16483 URL: https://issues.apache.org/jira/browse/SPARK-16483 Project: Spark Issue Type: New

[jira] [Created] (SPARK-16210) DataFrame.drop(colName) fails if another column has a period in its name

2016-06-25 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-16210: --- Summary: DataFrame.drop(colName) fails if another column has a period in its name Key: SPARK-16210 URL: https://issues.apache.org/jira/browse/SPARK-16210

[jira] [Commented] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-16 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335419#comment-15335419 ] Simeon Simeonov commented on SPARK-14048: - I can confirm that this workaround works. >

[jira] [Comment Edited] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-16 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334456#comment-15334456 ] Simeon Simeonov edited comment on SPARK-14048 at 6/16/16 7:02 PM: --

[jira] [Commented] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-16 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334456#comment-15334456 ] Simeon Simeonov commented on SPARK-14048: - [~clockfly] The code executes with no error on the

[jira] [Commented] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333076#comment-15333076 ] Simeon Simeonov commented on SPARK-14048: - Yes, I get the exact same failure with 1.6.1. >

[jira] [Comment Edited] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333076#comment-15333076 ] Simeon Simeonov edited comment on SPARK-14048 at 6/16/16 4:46 AM: -- Yes,

[jira] [Commented] (SPARK-13207) _SUCCESS should not break partition discovery

2016-06-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325718#comment-15325718 ] Simeon Simeonov commented on SPARK-13207: - [~yhuai] The PR associated with that ticket explicitly

[jira] [Commented] (SPARK-13207) _SUCCESS should not break partition discovery

2016-05-28 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305703#comment-15305703 ] Simeon Simeonov commented on SPARK-13207: - [~yhuai] I see the same problem with common metadata

[jira] [Comment Edited] (SPARK-10574) HashingTF should use MurmurHash3

2016-04-19 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248627#comment-15248627 ] Simeon Simeonov edited comment on SPARK-10574 at 4/19/16 9:05 PM: --

[jira] [Commented] (SPARK-10574) HashingTF should use MurmurHash3

2016-04-19 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248627#comment-15248627 ] Simeon Simeonov commented on SPARK-10574: - [~josephkb] I agree that it would be an improvement.

[jira] [Created] (SPARK-14443) parse_url() does not escape query parameters

2016-04-06 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-14443: --- Summary: parse_url() does not escape query parameters Key: SPARK-14443 URL: https://issues.apache.org/jira/browse/SPARK-14443 Project: Spark Issue

[jira] [Comment Edited] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-03-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210213#comment-15210213 ] Simeon Simeonov edited comment on SPARK-14048 at 3/24/16 1:31 PM: -- You

[jira] [Updated] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-03-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-14048: Attachment: bug_structs_with_backticks.html You can see the problem on a 1.6.0 cluster in

[jira] [Created] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-03-21 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-14048: --- Summary: Aggregation operations on structs fail when the structs have fields with special characters Key: SPARK-14048 URL: https://issues.apache.org/jira/browse/SPARK-14048

[jira] [Commented] (SPARK-13445) Seleting "data" with window function does not work unless aliased (using PARTITION BY)

2016-02-25 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168527#comment-15168527 ] Simeon Simeonov commented on SPARK-13445: - I can reproduce it consistently in our 1.6.0

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-25 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115558#comment-15115558 ] Simeon Simeonov commented on SPARK-12890: - [~viirya] If schema merging is the cause of the

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114829#comment-15114829 ] Simeon Simeonov commented on SPARK-12890: - Thanks for the clarification, [~hyukjin.kwon]. Still,

[jira] [Comment Edited] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106124#comment-15106124 ] Simeon Simeonov edited comment on SPARK-12890 at 1/19/16 2:07 AM: -- I've

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106124#comment-15106124 ] Simeon Simeonov commented on SPARK-12890: - I've experienced this issue with a multi-level

[jira] [Commented] (SPARK-7301) Issue with duplicated fields in interpreted json schemas

2015-12-18 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065021#comment-15065021 ] Simeon Simeonov commented on SPARK-7301: It would be nice if using backticks to refer to the

[jira] [Created] (SPARK-12022) spark-shell cannot run on master created with spark-ec2

2015-11-26 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-12022: --- Summary: spark-shell cannot run on master created with spark-ec2 Key: SPARK-12022 URL: https://issues.apache.org/jira/browse/SPARK-12022 Project: Spark

[jira] [Closed] (SPARK-11770) Spark SQL field resolution error in GROUP BY HAVING clause

2015-11-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov closed SPARK-11770. --- Resolution: Cannot Reproduce [~smilegator] I cannot reproduce the problem under v1.5.2.

[jira] [Comment Edited] (SPARK-11770) Spark SQL field resolution error in GROUP BY HAVING clause

2015-11-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026065#comment-15026065 ] Simeon Simeonov edited comment on SPARK-11770 at 11/25/15 2:53 AM: ---

[jira] [Reopened] (SPARK-11770) Spark SQL field resolution error in GROUP BY HAVING clause

2015-11-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov reopened SPARK-11770: - [~smilegator] Actually, your code runs fine in 1.5.2. My code generates the exact same

[jira] [Issue Comment Deleted] (SPARK-11770) Spark SQL field resolution error in GROUP BY HAVING clause

2015-11-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-11770: Comment: was deleted (was: [~smilegator] I cannot reproduce the problem under v1.5.2.

[jira] [Commented] (SPARK-10574) HashingTF should use MurmurHash3

2015-11-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026082#comment-15026082 ] Simeon Simeonov commented on SPARK-10574: - [~josephkb] +1 on (1)-(3) and sorry about the delay in

[jira] [Created] (SPARK-11770) Spark SQL field resolution error in GROUP BY HAVING clause

2015-11-16 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-11770: --- Summary: Spark SQL field resolution error in GROUP BY HAVING clause Key: SPARK-11770 URL: https://issues.apache.org/jira/browse/SPARK-11770 Project: Spark

[jira] [Created] (SPARK-11522) input_file_name() returns "" for external tables

2015-11-04 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-11522: --- Summary: input_file_name() returns "" for external tables Key: SPARK-11522 URL: https://issues.apache.org/jira/browse/SPARK-11522 Project: Spark Issue

[jira] [Created] (SPARK-11523) spark_partition_id() considered invalid function

2015-11-04 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-11523: --- Summary: spark_partition_id() considered invalid function Key: SPARK-11523 URL: https://issues.apache.org/jira/browse/SPARK-11523 Project: Spark Issue

[jira] [Commented] (SPARK-10217) Spark SQL cannot handle ordering directive in ORDER BY clauses with expressions

2015-10-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959197#comment-14959197 ] Simeon Simeonov commented on SPARK-10217: - Well, that would suggest the issue is fixed. :) >

[jira] [Closed] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov closed SPARK-9344. -- Resolution: Cannot Reproduce > Spark SQL documentation does not clarify INSERT INTO behavior >

[jira] [Commented] (SPARK-10574) HashingTF should use MurmurHash3

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939876#comment-14939876 ] Simeon Simeonov commented on SPARK-10574: - [~josephkb] any thoughts on the above? > HashingTF

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939871#comment-14939871 ] Simeon Simeonov commented on SPARK-9344: [~joshrosen] you are absolutely right. I was rushing to

[jira] [Commented] (SPARK-9762) ALTER TABLE cannot find column

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939883#comment-14939883 ] Simeon Simeonov commented on SPARK-9762: [~yhuai] any thoughts on this? > ALTER TABLE cannot find

[jira] [Commented] (SPARK-9762) ALTER TABLE cannot find column

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940757#comment-14940757 ] Simeon Simeonov commented on SPARK-9762: [~yhuai] Refreshing is not the issue here. The issue is

[jira] [Commented] (SPARK-9762) ALTER TABLE cannot find column

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940776#comment-14940776 ] Simeon Simeonov commented on SPARK-9762: [~yhuai] the Hive compatibility section of the

[jira] [Commented] (SPARK-9761) Inconsistent metadata handling with ALTER TABLE

2015-10-01 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940778#comment-14940778 ] Simeon Simeonov commented on SPARK-9761: [~yhuai] What about this one? The problem survives as

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938814#comment-14938814 ] Simeon Simeonov commented on SPARK-9344: /cc [~joshrosen] [~andrewor14] > Spark SQL documentation

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938825#comment-14938825 ] Simeon Simeonov commented on SPARK-9344: [~joshrosen] when I logged the bug I was using

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938806#comment-14938806 ] Simeon Simeonov commented on SPARK-9344: Are you suggesting to fix the docs or the code? > Spark

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938918#comment-14938918 ] Simeon Simeonov commented on SPARK-9344: [~joshrosen] Here is the reproducible test case you can

[jira] [Created] (SPARK-10724) SQL's floor() returns DOUBLE

2015-09-20 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10724: --- Summary: SQL's floor() returns DOUBLE Key: SPARK-10724 URL: https://issues.apache.org/jira/browse/SPARK-10724 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-10722) Uncaught exception: RDDBlockId not found in driver-heartbeater

2015-09-20 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10722: --- Summary: Uncaught exception: RDDBlockId not found in driver-heartbeater Key: SPARK-10722 URL: https://issues.apache.org/jira/browse/SPARK-10722 Project: Spark

[jira] [Commented] (SPARK-10574) HashingTF should use MurmurHash3

2015-09-14 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743922#comment-14743922 ] Simeon Simeonov commented on SPARK-10574: - [~josephkb] this makes sense. There are a few

[jira] [Comment Edited] (SPARK-10574) HashingTF should use MurmurHash3

2015-09-14 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743922#comment-14743922 ] Simeon Simeonov edited comment on SPARK-10574 at 9/14/15 5:55 PM: --

[jira] [Commented] (SPARK-8345) Add an SQL node as a feature transformer

2015-09-11 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741618#comment-14741618 ] Simeon Simeonov commented on SPARK-8345: This would be very nice, especially as it can leverage

[jira] [Updated] (SPARK-10574) HashingTF should use MurmurHash3

2015-09-11 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-10574: Description: {{HashingTF}} uses the Scala native hashing {{##}} implementation. There are

[jira] [Created] (SPARK-10574) HashingTF should use MurmurHash3

2015-09-11 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10574: --- Summary: HashingTF should use MurmurHash3 Key: SPARK-10574 URL: https://issues.apache.org/jira/browse/SPARK-10574 Project: Spark Issue Type:

[jira] [Created] (SPARK-10476) Add common RDD API methods to standard Scala collections

2015-09-07 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10476: --- Summary: Add common RDD API methods to standard Scala collections Key: SPARK-10476 URL: https://issues.apache.org/jira/browse/SPARK-10476 Project: Spark

[jira] [Updated] (SPARK-10476) Add common RDD operations on standard Scala collections

2015-09-07 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-10476: Summary: Add common RDD operations on standard Scala collections (was: Add common RDD API

[jira] [Updated] (SPARK-10476) Enable common RDD operations on standard Scala collections

2015-09-07 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-10476: Summary: Enable common RDD operations on standard Scala collections (was: Add common RDD

[jira] [Updated] (SPARK-10186) Add support for more postgres column types

2015-09-07 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-10186: Description: The specific observations below are based on Postgres 9.4 tables accessed via

[jira] [Commented] (SPARK-10476) Enable common RDD operations on standard Scala collections

2015-09-07 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733988#comment-14733988 ] Simeon Simeonov commented on SPARK-10476: - No need for the abstraction to leak as all

[jira] [Commented] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-08-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709646#comment-14709646 ] Simeon Simeonov commented on SPARK-7869: This is also a problem for {{json}}

[jira] [Created] (SPARK-10186) Inconsistent handling of complex column types in external databases

2015-08-24 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10186: --- Summary: Inconsistent handling of complex column types in external databases Key: SPARK-10186 URL: https://issues.apache.org/jira/browse/SPARK-10186 Project:

[jira] [Created] (SPARK-10217) Spark SQL cannot handle ordering directive in ORDER BY clauses with expressions

2015-08-24 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-10217: --- Summary: Spark SQL cannot handle ordering directive in ORDER BY clauses with expressions Key: SPARK-10217 URL: https://issues.apache.org/jira/browse/SPARK-10217

[jira] [Updated] (SPARK-10217) Spark SQL cannot handle ordering directive in ORDER BY clauses with expressions

2015-08-24 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-10217: Description: Spark SQL supports expressions in ORDER BY clauses, e.g., {code} scala

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-12 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693512#comment-14693512 ] Simeon Simeonov edited comment on SPARK-9813 at 8/12/15 1:59 PM:

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-12 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693512#comment-14693512 ] Simeon Simeonov edited comment on SPARK-9813 at 8/12/15 2:00 PM:

[jira] [Commented] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-12 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693512#comment-14693512 ] Simeon Simeonov commented on SPARK-9813: [~hvanhovell] that's a great, tight

[jira] [Commented] (SPARK-9345) Failure to cleanup on exceptions causes persistent I/O problems later on

2015-08-12 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693537#comment-14693537 ] Simeon Simeonov commented on SPARK-9345: [~marmbrus] Yes, Michael: {{kill -9}} is

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-11 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681326#comment-14681326 ] Simeon Simeonov edited comment on SPARK-9813 at 8/11/15 6:47 AM:

[jira] [Commented] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-11 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681326#comment-14681326 ] Simeon Simeonov commented on SPARK-9813: [~hvanhovell] Oracle requires the number

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-11 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681326#comment-14681326 ] Simeon Simeonov edited comment on SPARK-9813 at 8/11/15 6:46 AM:

[jira] [Created] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-9813: -- Summary: Incorrect UNION ALL behavior Key: SPARK-9813 URL: https://issues.apache.org/jira/browse/SPARK-9813 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-9813: --- Description: According to the [Hive Language

[jira] [Commented] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

2015-08-08 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663233#comment-14663233 ] Simeon Simeonov commented on SPARK-9625: [~sowen] I can reproduce this problem at

[jira] [Created] (SPARK-9761) Inconsistent metadata handling with ALTER TABLE

2015-08-08 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-9761: -- Summary: Inconsistent metadata handling with ALTER TABLE Key: SPARK-9761 URL: https://issues.apache.org/jira/browse/SPARK-9761 Project: Spark Issue

[jira] [Created] (SPARK-9762) ALTER TABLE cannot find column

2015-08-08 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-9762: -- Summary: ALTER TABLE cannot find column Key: SPARK-9762 URL: https://issues.apache.org/jira/browse/SPARK-9762 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-9762) ALTER TABLE cannot find column

2015-08-08 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-9762: --- Description: {{ALTER TABLE tbl CHANGE}} cannot find a column that {{DESCRIBE COLUMN}} lists.

[jira] [Created] (SPARK-9764) Spark SQL uses table metadata inconsistently

2015-08-08 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-9764: -- Summary: Spark SQL uses table metadata inconsistently Key: SPARK-9764 URL: https://issues.apache.org/jira/browse/SPARK-9764 Project: Spark Issue Type:

[jira] [Closed] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

2015-08-04 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov closed SPARK-9625. -- Resolution: Won't Fix SparkILoop creates sql context continuously, thousands of times

[jira] [Commented] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

2015-08-04 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654806#comment-14654806 ] Simeon Simeonov commented on SPARK-9625: This is a spark shell-specific issue

  1   2   >