I support deprecating GraphX because: - GraphFrames supersedes it, really - No maintainers and no reason to believe there will be - we can take the last 5+ years as thorough evidence - Low (but not trivial) docs hits compared to other modules: https://analytics.apache.org/index.php?module=CoreHome&action=index&date=yesterday&period=day&idSite=40#?period=year&date=2024-09-29&idSite=40&category=General_Actions&subcategory=General_Pages - If it *exists* in 4.x then it has to live as long as 4.x does, and that's already a super long time (4+ years?); deprecating is just a step to removing it in 5.x. (Well, we *can* take a decision to remove it in some 4.x version if it's really a problem, but deprecating well in advance is a prerequisite.
There is one problem: deprecated in favor of what? GraphFrames. But, GraphFrames uses GraphX :) But it is likewise in a similar bucket. *Maintained* but no active development; not sure about usage. So I think this is kind of "deprecated without replacement". But we're only talking about deprecating here, which I think more accurately communicates its state to users than not doing so. On Mon, Sep 30, 2024 at 12:20 PM Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi, > > These are my Views: > > 1. Deprecation Consideration: I lean towards the idea of officially > deprecating GraphX, given the lack of active development and community > engagement over the past few years as you alluded. This would set clear > expectations for users about its future and encourage them to explore > alternatives that are actively maintained. > > 2. User Input: It would be prudent to gather feedback from those currently > utilizing GraphX. Their insights could help us understand whether they find > the functionality sufficient as-is or if they have specific needs that > remain unaddressed. > > 3. Search for Maintainers: While I believe deprecation is a prudent step, > I also think we should issue a call for new maintainers before making any > final decisions. If there are individuals or teams willing to invest in > GraphX, it may still have a place in our ecosystem. > > Ultimately, I feel that we should prioritize the health of the Spark > ecosystem and ensure that we are investing resources into actively > maintained components. > > HTH > > Mich Talebzadeh > > Architect | Data Engineer | Data Science | Financial Crime > PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College > London <https://en.wikipedia.org/wiki/Imperial_College_London> > > London, United Kingdom > > > > *Disclaimer:* The information provided is correct to the best of my > knowledge but of course cannot be guaranteed . It is essential to note > that, as with any advice, quote "one test result is worth one-thousand > expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von > Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". > > > On Sun, 29 Sept 2024 at 21:39, Holden Karau <holden.ka...@gmail.com> > wrote: > >> Since we're getting close to cutting a 4.0 branch I'd like to float the >> idea of officially deprecating Graph X. What that would mean (to me) is we >> would update the docs to indicate that Graph X is deprecated and it's APIs >> may be removed at anytime in the future. >> >> Alternatively, we could mark it as "unmaintained and in search of >> maintainers" with a note that if no maintainers are found, we may remove it >> in a future minor version. >> >> Looking at the source graph X, I don't see any meaningful active >> development going back over three years*. There is even a thread on user@ >> from 2017 asking if graph X is maintained anymore, with no response from >> the developers. >> >> Now I'm open to the idea that GraphX is stable and "works as is" and >> simply doesn't require modifications but given the user thread I'm a little >> concerned here about bringing this API with us into Spark 4 if we don't >> have anyone signed up to maintain it. >> >> * Excluding globally applied changes >> -- >> Twitter: https://twitter.com/holdenkarau >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >> <https://www.fighthealthinsurance.com/?q=hk_email> >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> Pronouns: she/her >> >
