Thanks for checking it, Jacek. 2020년 2월 16일 (일) 오후 7:23, Jacek Laskowski <ja...@japila.pl>님이 작성:
> Hi, > > Never mind. Found this [1]: > > > This config is deprecated and it will be removed in 3.0.0. > > And so it has :) Thanks and sorry for the trouble. > > [1] > https://github.com/apache/spark/blob/830a4ec59b86253f18eb7dfd6ed0bbe0d7920e5b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L1306-L1307 > > Pozdrawiam, > Jacek Laskowski > ---- > https://about.me/JacekLaskowski > "The Internals Of" Online Books <https://books.japila.pl/> > Follow me on https://twitter.com/jaceklaskowski > > <https://twitter.com/jaceklaskowski> > > > On Sat, Feb 15, 2020 at 7:44 PM Jacek Laskowski <ja...@japila.pl> wrote: > >> Hi, >> >> Just noticed that >> http://spark.apache.org/docs/latest/sql-migration-guide-upgrade.html (Spark >> 2.4.5) has formatting issues in "Upgrading from Spark SQL 2.4.3 to 2.4.4" >> [1] which got fixed in master [2]. That's OK. >> >> What made me wonder was the other change to the section "Upgrading from >> Spark SQL 2.4 to 2.4.5" [3] that had the following item included: >> >> "Starting from 2.4.5, SQL configurations are effective also when a >> Dataset is converted to an RDD and its plan is executed due to action on >> the derived RDD. The previous behavior can be restored setting >> spark.sql.legacy.rdd.applyConf to false: in this case, SQL configurations >> are ignored for operations performed on a RDD derived from a Dataset." >> >> Why was this removed in master [4]? It was mentioned in "Notable changes" >> of Spark Release 2.4.5 [5]. >> >> [1] >> http://spark.apache.org/docs/latest/sql-migration-guide-upgrade.html#upgrading-from-spark-sql-243-to-244 >> [2] >> https://github.com/apache/spark/blob/master/docs/sql-migration-guide.md#upgrading-from-spark-sql-243-to-244 >> [3] >> http://spark.apache.org/docs/latest/sql-migration-guide-upgrade.html#upgrading-from-spark-sql-24-to-245 >> [4] >> https://github.com/apache/spark/blob/master/docs/sql-migration-guide.md#upgrading-from-spark-sql-244-to-245 >> [5] http://spark.apache.org/releases/spark-release-2-4-5.html >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://about.me/JacekLaskowski >> "The Internals Of" Online Books <https://books.japila.pl/> >> Follow me on https://twitter.com/jaceklaskowski >> >> <https://twitter.com/jaceklaskowski> >> >
