Hello, We are upgrading spark from 2.4.7 to 3.0.1. we use spark sql (hive) to checkpoint data frames (intermediate data). DF write is very slow in 3.0.1 compared to 2.4.7. Have read the release notes and there were no major changes except managed tables and adaptive scheduling. We are not using adaptive scheduling and going with default config. We made changes to handle managed tables by adding explicit paths during writes and delete.
Do you have any suggestions to debug and fix the slowness problem ? Thanks,
