Thank you for all the reports! The vote has failed. I plan to cut RC4 in two days.
@Dipayan Dev <dev.dipaya...@gmail.com> I quickly skimmed through the corresponding ticket, and it doesn't seem to be a regression introduced in 3.5. Additionally, someone is asking if this is the same issue as SPARK-35279. @Yuming Wang <yumw...@apache.org> I will check the signature for RC4 @Jungtaek Lim <kabhwan.opensou...@gmail.com> I will follow-up with you regarding SPARK-45045 <https://issues.apache.org/jira/browse/SPARK-45045> @Wenchen Fan <cloud0...@gmail.com> Agree, we should include the correctness fix in 3.5 Jungtaek Lim <kabhwan.opensou...@gmail.com> 于2023年8月31日周四 23:45写道： > My apologies, I have to add another ticket for a blocker, SPARK-45045 > <https://issues.apache.org/jira/browse/SPARK-45045>. That said, I'm -1 > (non-binding). > > SPARK-43183 <https://issues.apache.org/jira/browse/SPARK-43183> made a > behavioral change regarding the StreamingQueryListener as well as > StreamingQuery API as a side-effect, while the intention was more about > introducing the change in the former one. I just got some reports that the > behavioral change for StreamingQuery API broke various tests in 3rd party > data sources. To help 3rd party ecosystems to adopt 3.5 without hassle, I'd > like to see this be fixed in 3.5.0. > > There is no fix yet but I'm working on it. I'll give an update here. Maybe > we could lower down priority and let the release go with describing this > as a "known issue", if I couldn't make progress in a couple of days. I'm > sorry about that. > > Thanks, > Jungtaek Lim > > On Fri, Sep 1, 2023 at 12:12 PM Wenchen Fan <cloud0...@gmail.com> wrote: > >> Sorry for the last-minute bug report, but we found a regression in 3.5: >> the SQL INSERT command without a column list fills missing columns with >> NULL while Spark 3.4 does not allow it. According to the SQL standard, this >> shouldn't be allowed and thus a regression in 3.5. >> >> The fix has been merged but one day after the RC3 cut: >> https://github.com/apache/spark/pull/42393 . I'm -1 and let's include >> this fix in 3.5. >> >> Thanks, >> Wenchen >> >> On Thu, Aug 31, 2023 at 9:09 PM Ian Manning <ian.a.mann...@gmail.com> >> wrote: >> >>> +1 (non-binding) >>> >>> Using Spark Core, Spark SQL, Structured Streaming. >>> >>> On Tue, Aug 29, 2023 at 8:12 PM Yuanjian Li <xyliyuanj...@gmail.com> >>> wrote: >>> >>>> Please vote on releasing the following candidate(RC3) as Apache Spark >>>> version 3.5.0. >>>> >>>> The vote is open until 11:59pm Pacific time Aug 31st and passes if a >>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes. >>>> >>>> [ ] +1 Release this package as Apache Spark 3.5.0 >>>> >>>> [ ] -1 Do not release this package because ... >>>> >>>> To learn more about Apache Spark, please see http://spark.apache.org/ >>>> >>>> The tag to be voted on is v3.5.0-rc3 (commit >>>> 9f137aa4dc43398aafa0c3e035ed3174182d7d6c): >>>> >>>> https://github.com/apache/spark/tree/v3.5.0-rc3 >>>> >>>> The release files, including signatures, digests, etc. can be found at: >>>> >>>> https://dist.apache.org/repos/dist/dev/spark/v3.5.0-rc3-bin/ >>>> >>>> Signatures used for Spark RCs can be found in this file: >>>> >>>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>> >>>> The staging repository for this release can be found at: >>>> >>>> https://repository.apache.org/content/repositories/orgapachespark-1447 >>>> >>>> The documentation corresponding to this release can be found at: >>>> >>>> https://dist.apache.org/repos/dist/dev/spark/v3.5.0-rc3-docs/ >>>> >>>> The list of bug fixes going into 3.5.0 can be found at the following >>>> URL: >>>> >>>> https://issues.apache.org/jira/projects/SPARK/versions/12352848 >>>> >>>> This release is using the release script of the tag v3.5.0-rc3. >>>> >>>> >>>> FAQ >>>> >>>> ========================= >>>> >>>> How can I help test this release? >>>> >>>> ========================= >>>> >>>> If you are a Spark user, you can help us test this release by taking >>>> >>>> an existing Spark workload and running on this release candidate, then >>>> >>>> reporting any regressions. >>>> >>>> If you're working in PySpark you can set up a virtual env and install >>>> >>>> the current RC and see if anything important breaks, in the Java/Scala >>>> >>>> you can add the staging repository to your projects resolvers and test >>>> >>>> with the RC (make sure to clean up the artifact cache before/after so >>>> >>>> you don't end up building with an out of date RC going forward). >>>> >>>> =========================================== >>>> >>>> What should happen to JIRA tickets still targeting 3.5.0? >>>> >>>> =========================================== >>>> >>>> The current list of open tickets targeted at 3.5.0 can be found at: >>>> >>>> https://issues.apache.org/jira/projects/SPARK and search for "Target >>>> Version/s" = 3.5.0 >>>> >>>> Committers should look at those and triage. Extremely important bug >>>> >>>> fixes, documentation, and API tweaks that impact compatibility should >>>> >>>> be worked on immediately. Everything else please retarget to an >>>> >>>> appropriate release. >>>> >>>> ================== >>>> >>>> But my bug isn't fixed? >>>> >>>> ================== >>>> >>>> In order to make timely releases, we will typically not hold the >>>> >>>> release unless the bug in question is a regression from the previous >>>> >>>> release. That being said, if there is something which is a regression >>>> >>>> that has not been correctly targeted please ping me or a committer to >>>> >>>> help target the issue. >>>> >>>> Thanks, >>>> >>>> Yuanjian Li >>>> >>>
