parthchandra commented on issue #646: URL: https://github.com/apache/datafusion-comet/issues/646#issuecomment-2221023876
There are a couple of considerations here - 1) What version of Spark users are likely to be on (and therefore likely to want to use Comet with)? 2) What are the currently available versions of Spark? For the first question, users could be on any of the earlier versions of Spark like 3.2 or 3.3 and we have had support for these versions, but because we have not done a release users will have no way to take advantage of the work we did to support those versions. Even if we do not do a release, if we tag the source to identify the last point at which we had support for Spark 3.2 or Spark 3.3, users could potentially build their own Comet and try with their current version of Spark. Once we have tagged the source, we can then drop the support. For the second question the current versions of Spark available are 3.4.3, and 3.5.1 (https://spark.apache.org/downloads.html) so it seems we could safely drop support for 3.3 as soon as we are sure that 3.5 support is robust. On Wed, Jul 10, 2024 at 8:53 AM KAZUYUKI TANIMURA ***@***.***> wrote: > +1 To be accurate, only experimental support is provided for Spark 3.5.1 > and 4.0.0 as of now though > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/datafusion-comet/issues/646#issuecomment-2220891918>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABR2A4AP4TZE7LJZIWODH2LZLVKBHAVCNFSM6AAAAABKTQQIK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQHA4TCOJRHA> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
