Thank you for the information and links provided. While I understand the
application of JSON in GraphScope Flex and its advantages when integrated with
GraphAr, considering our specific use case, I still think that YAML might be a
more suitable choice for us. Here are the primary reasons:
1. YAML's format is more human-readable and easier to edit, which is a
significant advantage in scenarios where we frequently need to view or modify
configuration files. For example, to define a subgraph from an existing graph.
2. YAML often provides a more concise representation of the same data.
3. YAML natively supports comments and extensions, making it more flexible.
Therefore, we initially favored YAML over JSON. I hope we can further discuss
this topic to find the solution that best fits our project requirements.
------------------------------------------------------------------
发件人：Weibin Zeng <wei...@apache.org>
发送时间：2024年5月9日(星期四) 18:52
收件人：dev<dev@graphar.apache.org>
主 题：Re: [DISCUSS][format] Using an Interface Definition Language to define
GraphAr format
Sorry, miss the link
> GraphScope Flex now use json as communication format for graph schema and
> check with rest API[1]
[1]
https://github.com/alibaba/GraphScope/tree/main/python/graphscope/flex/rest/models
<https://github.com/alibaba/GraphScope/tree/main/python/graphscope/flex/rest/models
>
On 2024/05/09 10:49:24 Weibin Zeng wrote:
> JSONs is ok for me. And GraphScope Flex now use json as communication format
> for graph schema and check with rest API[1], I think switching to JSON is
> good for GraphAr. Since GraphAr has been integrated into GraphScope.
>
> On 2024/05/09 08:50:45 Sem wrote:
> > I made a small research about that and it seems to me that classes,
> > generated from protobuf are not serializable into another formats like
> > yaml/json.
> >
> > There is a 3d party project: https://github.com/krzko/proto2yaml
> > <https://github.com/krzko/proto2yaml > that
> > provide such utility, but it does not look well maintained.
> >
> > I see that there is an utility, provided by google. It allows
> > conversion to JSON and from JSON in Java/Python (most probably, cpp
> > too):
> > 1.
> > https://cloud.google.com/java/docs/reference/protobuf/latest/com.google.protobuf.util.JsonFormat
> >
> > <https://cloud.google.com/java/docs/reference/protobuf/latest/com.google.protobuf.util.JsonFormat
> > >
> > 2.
> > https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html
> >
> > <https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html
> > >
> >
> > But not to/from YAML.
> >
> > For me that question is important, because we need not only generate
> > the code but resolve the question about serialization/deserialization.
> >
> >
> > What do you think about using proto (binary messages) for underlaying
> > communication format in the code and JSONs for human-readable
> > representation on disk? Because it looks like only with switching from
> > YAML to JSON we achieve all the benefits of using protobuf.
> >
> > With JSON I see it like we just call once `fromJson` (via google
> > protobuf util) to read the data and create proto classes from JSON info
> > files and work with them. At the end we call `toJson` again to
> > serialize messages back.
> >
> > To achieve the same with YAML we need to use 3d-party and not well
> > maintained library or support our own serialization/deserialization of
> > proto messages (classes) to/from YAML for three languages (Python,
> > Java, Cpp).
> >
> > On Thu, 2024-05-09 at 15:27 +0800, weibin.zen wrote:
> > > Hi, everyone,
> > >
> > > I would like to propose that we should considering using an Interface
> > > Definition Language(IDL) like Protobuf[1] for GraphAr format
> > > definition.
> > > Currently we use YAML to describe schema and metadata of graph, and
> > > data storage with common format like CSV/Parquet. YAML
> > > provide human-readable ability but it can not provide much
> > > validation, version-controlled. And various programming languages
> > > need
> > > to parse them and check the validation by themself.
> > >
> > > Using IDL to describe format would bring benefits like:
> > >
> > > • provide a clear, standardized, language-agnostic format definition
> > > that can be version-controlled, shared by libraries and make the
> > > format consistent between implementations.
> > > • The validation by protobuf can be directly use by our validation of
> > > the schema, no need to let the libraries to implement the validation.
> > > • Cross-language support, libraries can use the generated structure
> > > as graph info directly.
> > >
> > >
> > > This proposal is not replace the YAML with Protobuf. We still use
> > > YAML as the final schema&metadata file for user readable, but with
> > > IDL to maintaining a
> > > robust and precis schema definition. It's kind a hybrid strategy to
> > > accommondates both human and machine needs.
> > >
> > > But Using IDL do bring some disadvantages, Sem has list some in the
> > > comment of pr[2]:
> > >
> > > • the generated code is huge and unreadable.
> > > • the generated code may need to store in git.
> > > • debugging is very hard.
> > >
> > >
> > > Since this would be a huge change, and I want to hear the thoughts
> > > about the proposal from you.
> > >
> > >
> > > [1] https://protobuf.dev/ <https://protobuf.dev/ >
> > > [2] https://github.com/apache/incubator-graphar/pull/475
> > > <https://github.com/apache/incubator-graphar/pull/475 >
> > >
> > > Best
> > > weibin.zen
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@graphar.apache.org
> > For additional commands, e-mail: dev-h...@graphar.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@graphar.apache.org
> For additional commands, e-mail: dev-h...@graphar.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@graphar.apache.org
For additional commands, e-mail: dev-h...@graphar.apache.org
