Marina Krasilnikova created SPARK-46077: -------------------------------------------
Summary: Error in postgresql when pushing down filter by timestamp_ntz field Key: SPARK-46077 URL: https://issues.apache.org/jira/browse/SPARK-46077 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.5.0 Reporter: Marina Krasilnikova code to reproduce: SparkSession sparkSession = SparkSession .builder() .appName("test-app") .master("local[*]") .config("spark.sql.timestampType", "TIMESTAMP_NTZ") .getOrCreate(); String url = "..."; String catalogPropPrefix = "spark.sql.catalog.myc"; sparkSession.conf().set(catalogPropPrefix, JDBCTableCatalog.class.getName()); sparkSession.conf().set(catalogPropPrefix + ".url", url); Map<String, String> options = new HashMap<>(); options.put("driver", "org.postgresql.Driver"); // options.put("pushDownPredicate", "false"); it works fine if this line is uncommented Dataset<Row> dataset = sparkSession.read() .options(options) .table("myc.demo.`My table`"); dataset.createOrReplaceTempView("view1"); String sql = "select * from view1 where `my date` = '2021-04-01 00:00:00'"; Dataset<Row> result = sparkSession.sql(sql); result.show(); result.printSchema(); Field `my date` is of type timestamp. This code results in org.postgresql.util.PSQLException syntax error , because resulting sql lacks straight quotes in filter condition. (Something like this "my date" = 2021-04-01T00:00) -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org