Hi, we are trying to migrate some of the data lake pipelines to run in SPARK 3.x, where as the dependent pipelines using those tables will be still running in SPARK 2.4.x for sometime to come.
Does anyone know of any issues that can happen: 1. when reading Parquet files written in 3.1.x in SPARK 2.4 2. when in the data lake some partitions have parquet files written in SPARK 2.4.x and some are in SPARK 3.1.x. Please note that there are no changes in schema, but later on we might end up adding or removing some columns. I will be really grateful for your kind help on this. Regards, Gourav Sengupta
