[ https://issues.apache.org/jira/browse/SPARK-42450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vadim updated SPARK-42450: -------------------------- Description: dataset.where()/filter() omit string quotes if where IN clause has more than 10 operands. With datasourceV1 works as expected. Attached files: java-code.txt, stacktrace.txt - Spark verison 3.3.0 - Scala version 2.12 - DatasourceV2 - Postgres - Postrgres JDBC Driver: 42+ - Java8 *Expected query:* SELECT "flight_id", "flight_no" FROM "bookings"."flights" WHERE ( "flight_no"IN ( '55', '826', '845', '799', '561', '39', '385', '549', '576', '15', '857', '248', '324', '569', '267' ) ) *actual query:* SELECT "flight_id", "flight_no", FROM "bookings"."flights" WHERE ( "flight_no"IN ( 55, 826, 845, 799, 561, 39, 385, 549, 576, 15, 857, 248, 324, 569, 267 ) ) was: dataset.where()/filter() omit string quotes if where IN clause has more than 10 operands. With datasourceV1 works as expected. - Spark verison 3.3.0 - Scala version 2.12 - DatasourceV2 - Postgres - Postrgres JDBC Driver: 42+ - Java8 *Expected query:* SELECT "flight_id", "flight_no" FROM "bookings"."flights" WHERE ( "flight_no"IN ( '55', '826', '845', '799', '561', '39', '385', '549', '576', '15', '857', '248', '324', '569', '267' ) ) *actual query:* SELECT "flight_id", "flight_no", FROM "bookings"."flights" WHERE ( "flight_no"IN ( 55, 826, 845, 799, 561, 39, 385, 549, 576, 15, 857, 248, 324, 569, 267 ) ) > dataset.where() omit quotes if where IN clause has more than 10 operands > ------------------------------------------------------------------------ > > Key: SPARK-42450 > URL: https://issues.apache.org/jira/browse/SPARK-42450 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.3.0 > Reporter: Vadim > Priority: Major > Fix For: 3.3.2, 3.4.0 > > Attachments: java-code.txt, stacktrace.txt > > > dataset.where()/filter() omit string quotes if where IN clause has more than > 10 operands. With datasourceV1 works as expected. > Attached files: java-code.txt, stacktrace.txt > - Spark verison 3.3.0 > - Scala version 2.12 > - DatasourceV2 > - Postgres > - Postrgres JDBC Driver: 42+ > - Java8 > *Expected query:* > SELECT > "flight_id", > "flight_no" > FROM > "bookings"."flights" > WHERE > ( > "flight_no"IN ( > '55', > '826', > '845', > '799', > '561', > '39', > '385', > '549', > '576', > '15', > '857', > '248', > '324', > '569', > '267' > ) > ) > *actual query:* > SELECT > "flight_id", > "flight_no", > FROM > "bookings"."flights" > WHERE > ( > "flight_no"IN ( > 55, > 826, > 845, > 799, > 561, > 39, > 385, > 549, > 576, > 15, > 857, > 248, > 324, > 569, > 267 > ) > ) > > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org