RE: How to add MaxDOP option in spark mssql JDBC

2024-04-24 Thread Appel, Kevin
You might be able to leverage the prepareQuery option, that is at https://spark.apache.org/docs/3.5.1/sql-data-sources-jdbc.html#data-source-option ... this was introduced in Spark 3.4.0 to handle temp table query and CTE query against MSSQL server since what you send in is not actually what

RE: determine week of month from date in spark3

2022-02-11 Thread Appel, Kevin
-03-30| 2| | 4|2014-03-31| 3| | 5|2015-03-07| 7| | 6|2015-03-08| 1| | 7|2015-03-30| 2| | 8|2015-03-31| 3| +---+--++ From: Appel, Kevin Sent: Friday, February 11, 2022 2:35 PM To: user@spark.apache.org; 'Sean Owen

determine week of month from date in spark3

2022-02-11 Thread Appel, Kevin
Previously in Spark2 we could use the spark function date_format with the "W" flag and it will provide back the week of month of that date. In Spark3 when trying this there is an error back: * org.apache.spark.SparkUpgradeException: You may get a different result due to the upgrading