[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761086#comment-16761086 ] Sujith commented on SPARK-26821: Yeah with spaces it will work fine, will try to check the behavior in couple of other systems also. > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760897#comment-16760897 ] Sean Owen commented on SPARK-26821: --- Can you find the result when filtering for "ds " with spaces? I find that result odd from MySQL but unless there's harder evidence that this is correct behavior (e.g. ANSI SQL) or behavior we want to match (Hive) I don't think this is incorrect. > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760063#comment-16760063 ] Sujith commented on SPARK-26821: bit tricky to handle this scenario eventhough. > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760061#comment-16760061 ] Sujith commented on SPARK-26821: yes sean, but same i tested with MYSQL its giving me a result. not sure how they are handling internally. > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760058#comment-16760058 ] Sean Owen commented on SPARK-26821: --- I don't know a lot about this, but assuming the padding behavior is standard SQL, this doesn't seem like a bug. Inserting "ds" into char(5) results in "ds ", and searching for "ds" returns no rows. Right? > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759381#comment-16759381 ] Sujith commented on SPARK-26821: As per the initial analysis, this phenomenon is happening because the actual char data type length is 5 where as we are trying to insert a data with length 2, since its a char data type the system will pad the remaining part of the array block with 'space'. now when we try to apply a filter, the system will try to compare the predicate value with the actual table data which contains the space char like 'ds' == 'ds ' which leads to wrong result. I am trying to analyze more on this issue please let me know for any suggestions or guidance. thanks > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26821) filters not working with char datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759382#comment-16759382 ] Sujith commented on SPARK-26821: cc [~dongjoon] > filters not working with char datatype when querying against hive table > --- > > Key: SPARK-26821 > URL: https://issues.apache.org/jira/browse/SPARK-26821 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Sujith >Priority: Major > > creates a table with a char type field, While inserting data to char data > type column, if the data string length is less than the specified datatype > length, spark2x will not process filter query properly leading to incorrect > result . > 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name > char(5)); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (0.894 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj > values(232,'ds'); > +--+-+ > |Result| > +--+-+ > +--+-+ > No rows selected (1.815 seconds) > 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds'; > +--+--++-- > |id|name| > +--+--++-- > +--+--++-- > > The above query will not give any result. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
