I've created a ticket and submitted https://github.com/apache/spark/pull/48476 to fix it. However, I've a more general question on storeAssignmentPolicy for DSv2 in the comment. Please kindly review and reply to the PR.
Thanks, Manu On Mon, Oct 14, 2024 at 8:58 PM Wenchen Fan <cloud0...@gmail.com> wrote: > Yea looks like a bug, the SQL and DataFrame APIs should be consistent. > Please create a JIRA ticket, thanks! > > On Mon, Oct 14, 2024 at 3:29 PM Manu Zhang <owenzhang1...@gmail.com> > wrote: > >> Hi community, >> >> With `spark.sql.storeAssignmentPolicy=LEGACY` in Spark 3.5, it's not >> allowed to write to DSv2 with insert SQL. However, this can be worked >> around with DataFrame API, i.e., `df.writeTo($dsv2Table).append()` >> >> Is this expected? >> >> Thanks, >> Manu >> >>
