[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907841#comment-14907841 ] Pete Robbins commented on SPARK-6873: - I no longer see these errors in my 1.5 branch Java 8 build. Did someone fix them, remove the tests or is it just chance? > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650480#comment-14650480 ] Sean Owen commented on SPARK-6873: -- [~rxin] [~lian cheng] It's still a problem. Yes I'm sure it's just a test issue, not a problem with the code, but ideally the test must not rely on the ordering. Right now tests don't actually pass in Java 8 because of things like ... {code} - show_create_table_serde *** FAILED *** Results do not match for show_create_table_serde: == Parsed Logical Plan == HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1 == Analyzed Logical Plan == result: string HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1 == Optimized Logical Plan == HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1 == Physical Plan == ExecutedCommand (HiveNativeCommand SHOW CREATE TABLE tmp_showcrt1) Code Generation: true == RDD == result !== HIVE - 13 row(s) == == CATALYST - 13 row(s) == CREATE EXTERNAL TABLE `tmp_showcrt1`( CREATE EXTERNAL TABLE `tmp_showcrt1`( `key` string, `key` string, `value` boolean)`value` boolean) ROW FORMAT SERDEROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe' 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe' STORED BY STORED BY 'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler' 'org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler' WITH SERDEPROPERTIES ( WITH SERDEPROPERTIES ( ! 'serialization.format'='$', 'field.delim'=',', ! 'field.delim'=',') 'serialization.format'='$') LOCATIONLOCATION 'tmp_showcrt1' 'tmp_showcrt1' TBLPROPERTIES ( TBLPROPERTIES ( (HiveComparisonTest.scala:397) {code} I build with {{-Pyarn -Phive}} from master. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650268#comment-14650268 ] Cheng Lian commented on SPARK-6873: --- It's not important. Internally, Hive just traverses a hash map and dumps everything in it. So the order is decided by the implementation of the hash map. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647322#comment-14647322 ] Pete Robbins commented on SPARK-6873: - We've been trying to get a clean build/test using Java 8 and we see these errors so I think this is still a problem. It looks like the Catalyst output changes from Java 7 to Java 8. Is the ordering supposed to be defined for this or is the ordering really unimportant? > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633002#comment-14633002 ] Reynold Xin commented on SPARK-6873: [~srowen] Sean - is this still a problem? If not, can you close the ticket? Thanks. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526218#comment-14526218 ] Cheng Lian commented on SPARK-6873: --- The UDF synonyms lines are easy to be filtered out. Table properties lines can be fixed by sorting the results. The SerDe lines can be problematic, maybe we can ignore this case for now. If we plan to add Java 8 Jenkins builder, I can try to find some time to work on this after 1.4 release. Otherwise it's not that harmful for now. The time consuming part is regenerating all golden files since we need to change golden file generation and comparison strategies. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526019#comment-14526019 ] Sean Owen commented on SPARK-6873: -- Sorry for the late reply, missed this. Yes, HashMap ordering did change in Java 8. Hm, so it sounds like the test can't pass on both Java 7 and 8 at the same time then, since the golden output would fail to match one or the other? or are you saying there's a way to ignore this part of the output? Otherwise we can track this for a later version like Spark 2.x where we may require Java 8. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Assignee: Cheng Lian >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494351#comment-14494351 ] Cheng Lian commented on SPARK-6873: --- I had once hit this recently, but unfortunately forgot to create a JIRA for this. The reason is that, when printing function synonymns and table/serde properties, Hive code uses a hash map to store strings to be printed. My guess is that Java 8 uses a different hash map implementation and thus causes different iteration order. To fix this, we can ignore corresponding output lines and regenerate all golden answer files. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6873) Some Hive-Catalyst comparison tests fail due to unimportant order of some printed elements
[ https://issues.apache.org/jira/browse/SPARK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14491567#comment-14491567 ] Sean Owen commented on SPARK-6873: -- CC [~lian cheng] [~marmbrus] as I bet this would be fairly easy to diagnose for someone close to the query planner / catalyst bits. > Some Hive-Catalyst comparison tests fail due to unimportant order of some > printed elements > -- > > Key: SPARK-6873 > URL: https://issues.apache.org/jira/browse/SPARK-6873 > Project: Spark > Issue Type: Bug > Components: SQL, Tests >Affects Versions: 1.3.1 >Reporter: Sean Owen >Priority: Minor > > As I mentioned, I've been seeing 4 test failures in Hive tests for a while, > and actually it still affects master. I think it's a superficial problem that > only turns up when running on Java 8, but still, would probably be an easy > fix and good to fix. > Specifically, here are four tests and the bit that fails the comparison, > below. I tried to diagnose this but had trouble even finding where some of > this occurs, like the list of synonyms? > {code} > - show_tblproperties *** FAILED *** > Results do not match for show_tblproperties: > ... > !== HIVE - 2 row(s) == == CATALYST - 2 row(s) == > !tmptruebar bar value > !barbar value tmp true (HiveComparisonTest.scala:391) > {code} > {code} > - show_create_table_serde *** FAILED *** > Results do not match for show_create_table_serde: > ... >WITH SERDEPROPERTIES ( WITH > SERDEPROPERTIES ( > ! 'serialization.format'='$', > 'field.delim'=',', > ! 'field.delim'=',') > 'serialization.format'='$') > {code} > {code} > - udf_std *** FAILED *** > Results do not match for udf_std: > ... > !== HIVE - 2 row(s) == == CATALYST > - 2 row(s) == >std(x) - Returns the standard deviation of a set of numbers std(x) - > Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stddev Synonyms: > stddev, stddev_pop (HiveComparisonTest.scala:391) > {code} > {code} > - udf_stddev *** FAILED *** > Results do not match for udf_stddev: > ... > !== HIVE - 2 row(s) ==== > CATALYST - 2 row(s) == >stddev(x) - Returns the standard deviation of a set of numbers stddev(x) > - Returns the standard deviation of a set of numbers > !Synonyms: stddev_pop, stdSynonyms: > std, stddev_pop (HiveComparisonTest.scala:391) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org