GitHub user SYaoJun added a comment to the discussion: Proposal: Introduce Vortex Columnar Format Support in GraphAr
Hi @SemyonSinchenko, Thank you so much for your insightful feedback. Your perspectives are truly thought-provoking. Actually, I’m a bit confused about your comment: "hard for clients to support the standard if we have too many of different underlying formats". Let me clarify my understanding: GAR provides high-level APIs (e.g., for vertices and edges) that are completely transparent to users. Additionally, we already support CSV, JSON, ORC, and Parquet formats. For the new Vortex format, we could follow the ORC implementation approach and use macros to separate the original code logic. You suggested adopting an "extension" mechanism, which I fully agree is a sound and appropriate approach. However, our current codebase does not yet support extension plugins, and I’m also unsure about how to implement this mechanism effectively. To move forward, I used AI to generate a minimal viable product (MVP) of Vortex under the cpp/ directory: https://github.com/SYaoJun/incubator-graphar/commit/78a32baf155e662f22b405b3280a7d9378559108. Would you mind taking a look at it? This implementation aligns nearly perfectly with my initial vision for Vortex. Thank you again for your guidance! Best regards, Yaojun At 2026-03-01 04:01:52, "Sem" ***@***.***> wrote: Hi @SYaoJun ! My only concern is it may become very hard for clients to support the standard if we have too many of different underlying formats. Are we considering something like "extension" mechanics? As I remember, a lot of formats supported in tools like DuckDB or Apache Spark through the extensions API, instead of adding it to the core. I mean, for example, the official Java SDK of the Vortex, the vortex-jni is 136.9 Mb JAR file... Having 5-7 such a connectors will lead to the ~1Gb of dependencies only. What do you think about any of: extensions API GraphAr "contrib" Maybe something else? ? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> GitHub link: https://github.com/apache/incubator-graphar/discussions/887#discussioncomment-15974316 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
