Personally, I would promote Best Practices given fair bit of opinions either way.
- Share best practices for using GraphX effectively, including tips for optimizing performance and avoiding common pitfalls. - Encourage the use of alternative libraries when appropriate, highlighting their advantages and how they can complement GraphX. I concur that it is important to avoid labelling GraphX as deprecated without providing clear guidance and alternatives. Instead, perhaps we should focus on documenting its current status, encouraging contributions, and promoting best practices for graph processing in Spark. I saw someone created some documents HTH Mich Talebzadeh, *Disclaimer:* The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". On Fri, 4 Oct 2024 at 23:09, Mark Hamstra <markhams...@gmail.com> wrote: > No, I wouldn't encourage anyone to base a new production deployment on > GraphX, but neither would I encourage new production deployments based > on the RDD API without deep study and understanding of the > implications and limitations. What I would be most comfortable with is > documenting the current status and shortcomings of GraphX, along with > encouraging contributions to remedy that situation. I'm not sure what > the best (or even an effective) way of accomplishing that is, but I'm > pretty sure it's not just labeling GraphX as deprecated. > > On Fri, Oct 4, 2024 at 3:00 PM Holden Karau <holden.ka...@gmail.com> > wrote: > > > > Personally I think people should not depend on it — there’s literally no > one working on it, and not being up front about that I think draws > everything else into question. > > > > Would anyone here feel comfortable using GraphX for a new production > deployment today? > > > > > > Twitter: https://twitter.com/holdenkarau > > Fight Health Insurance: https://www.fighthealthinsurance.com/ > > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > > Pronouns: she/her > > > > > > On Fri, Oct 4, 2024 at 2:56 PM Mark Hamstra <markhams...@gmail.com> > wrote: > >> > >> I'm -1(*) because, while it technically means "might be removed in the > >> future", I think developers and users are more prone to interpret > >> something being marked as deprecated as "very likely will be removed > >> in the future, so don't depend on this or waste your time contributing > >> to its further development." I don't think the latter is what we want > >> just because something hasn't been updated meaningfully in a while. > >> There have been How To articles for GraphX and Graph Frames posted in > >> the not too distant past, and the Google Search trend shows a pretty > >> steady level of interest, not a decline to zero, so I don't think that > >> it is accurate to declare that there is no use or interest in GraphX. > >> > >> Unless retaining GraphX is imposing significant costs on continuing > >> Spark development, I can't support deprecating GraphX. I can support > >> encouraging GraphX and Graph Frames development through something like > >> a To Do list or document of "What we'd like to see in the way of > >> further development of Spark's graph processing capabilities" -- i.e., > >> things that encourage and support new contributions to address any > >> shortcomings in Spark's graph processing, not things that discourage > >> contributions and use in the way that I believe simply declaring > >> GraphX to be deprecated would. > >> > >> > >> On Sun, Sep 29, 2024 at 11:04 AM Holden Karau <holden.ka...@gmail.com> > wrote: > >> > > >> > Since we're getting close to cutting a 4.0 branch I'd like to float > the idea of officially deprecating Graph X. What that would mean (to me) is > we would update the docs to indicate that Graph X is deprecated and it's > APIs may be removed at anytime in the future. > >> > > >> > Alternatively, we could mark it as "unmaintained and in search of > maintainers" with a note that if no maintainers are found, we may remove it > in a future minor version. > >> > > >> > Looking at the source graph X, I don't see any meaningful active > development going back over three years*. There is even a thread on user@ > from 2017 asking if graph X is maintained anymore, with no response from > the developers. > >> > > >> > Now I'm open to the idea that GraphX is stable and "works as is" and > simply doesn't require modifications but given the user thread I'm a little > concerned here about bringing this API with us into Spark 4 if we don't > have anyone signed up to maintain it. > >> > > >> > * Excluding globally applied changes > >> > -- > >> > Twitter: https://twitter.com/holdenkarau > >> > Fight Health Insurance: https://www.fighthealthinsurance.com/ > >> > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > >> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >> > Pronouns: she/her > >> > >> --------------------------------------------------------------------- > >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >> > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
