I think this was all discussed at length. While Spark of course doesn't collect usage stats, we do have evidence that was discussed: low commit activity and traffic to this section of the docs. The argument that it could impact someone doesn't go anywhere - of course it will impact _someone_. How many? and I don't think anyone offered a use case on the list. I don't understand why being around since 2014 (and being superseded by another package) makes it _harder_ to deprecate.
Yes, this thread resulted in people saying they would start working on it, which is great (though, why not before this if it's somehow widely used). They may do it and the work will be available forever for users. I don't see any new ground here. On Tue, Nov 12, 2024 at 10:47 PM Ángel <angel.alvarez.pas...@gmail.com> wrote: > When you deprecate something, the message you're sending out is: "This > feature is no longer supported, maintained, and recommended for production > use." The problem is that nobody knows how many Spark programs currently > rely on GraphX/Graphframes in production and the impact that decisssion > could have to some people/companies. The way I see it, you can’t simply > deprecate an API that has been available around since 2014 (10 years) with > just a brief poll + light discussion over a couple of weeks. It’s > mind-blowing to me, but I understand you're the ones with experience in > open-source here. > > On the other hand, the only reasons I've read for deprecating GraphX were > about unfixed bugs and its lack of maintenance—and that's exactly what > we're aiming to address in this 100+ message discussion and through the > hackathon that Russell has organized. > > El mié, 13 nov 2024 a las 4:13, Sean Owen (<sro...@gmail.com>) escribió: > >> I think people are still reading "deprecated" as "removed". It 100% does >> not mean that. >> Wouldn't it be more likely that 'old' things are deprecated than new? >> What is light about this 100+ message discussion? I myself did not see >> any strong arguments against deprecation, which seemed to amount to "maybe >> someone is interested in it that hasn't been for the last few years", so it >> seemed clear this was the right step. >> What impact analysis have you seen conducted that would have addressed >> these? >> >> Just trying to understand the objection or thinking here >> >> >> On Tue, Nov 12, 2024 at 8:48 PM Ángel <angel.alvarez.pas...@gmail.com> >> wrote: >> >>> I thought that too ... until I read the message from Matei Zaharia: >>> >>> "Votes to deprecate both SparkR and GraphX have passed. These >>> components will officially be deprecated in Spark 4." >>> >>> Didn't know in open source you could deprecate things that have been >>> there years so lightly without carrying out any impact analysis and in the >>> middle of an active (and interesting, btw) discussion. >>> >>>>
