[jira] [Commented] (SPARK-23613) Different Analyzed logical plan data types for the same table in different queries
[ https://issues.apache.org/jira/browse/SPARK-23613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392298#comment-16392298 ] Hyukjin Kwon commented on SPARK-23613: -- Let's avoid to set a blocker which is usually reserved for committers. Mind if I ask a self-contained reproducer? I would like to check it > Different Analyzed logical plan data types for the same table in different > queries > -- > > Key: SPARK-23613 > URL: https://issues.apache.org/jira/browse/SPARK-23613 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 > Environment: Spark 2.2.0 > Hive: 2 >Reporter: Ramandeep Singh >Priority: Major > Labels: SparkSQL > > Hi, > The column datatypes are correctly analyzed for simple select query. Note > that the problematic column is not selected anywhere in the complicated > scenario. > Let's say Select * from a; > Now let's say there is a query involving temporary view on another table and > its join with this table. > Let's call that table b (temporary view on a dataframe); > select * from jq ( select a.col1, b.col2 from a,b where a.col3=b=col3) > Fails with exception on some column not part of the projection in the join > query > Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot up > cast `a`.col5 from from decimal(8,0) to col5#1234: decimal(6,2) as it may > truncate. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-23613) Different Analyzed logical plan data types for the same table in different queries
[ https://issues.apache.org/jira/browse/SPARK-23613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388494#comment-16388494 ] Ramandeep Singh commented on SPARK-23613: - To add to it, the query works fine with subquery factoring. with b1 as (select b.* from b) select * from jq ( select a.col1, b.col2 from a,b1 where a.col3=b1.col3) > Different Analyzed logical plan data types for the same table in different > queries > -- > > Key: SPARK-23613 > URL: https://issues.apache.org/jira/browse/SPARK-23613 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 > Environment: Spark 2.2.0 > Hive: 2 >Reporter: Ramandeep Singh >Priority: Blocker > Labels: SparkSQL > > Hi, > The column datatypes are correctly analyzed for simple select query. Note > that the problematic column is not selected anywhere in the complicated > scenario. > Let's say Select * from a; > Now let's say there is a query involving temporary view on another table and > its join with this table. > Let's call that table b (temporary view on a dataframe); > select * from jq ( select a.col1, b.col2 from a,b where a.col3=b=col3) > Fails with exception on some column not part of the projection in the join > query > Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot up > cast `a`.col5 from from decimal(8,0) to col5#1234: decimal(6,2) as it may > truncate. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
