Hi,
Can you give a more detailed description about code/value in Response
definition?
RecordResponse {
Integer code;
Long value;
Enum errorCode; // if code != 200, please tell us what happened:
// 1、 recordSql syntax error
// 2、 internal error, dispatcher self is crashed
// 3、 external error, target engine is crashed when dispatcher
call,etc
Exception ex; // error detail info
}
Thx
Eugene
On Tue, Oct 18, 2022 at 10:19 PM William Guo <gu...@apache.org> wrote:
> Hi Warden,
>
> I think type long is not sufficient for metrics,if it is a number, type
> double should be more appropriate.
>
> Thanks,
> William
>
> On Mon, Oct 17, 2022 at 10:55 PM Warden Wang <wangd95...@gmail.com> wrote:
>
> > Hi Jianhua
> > For metrics, resp as long type is enough.
> >
> > Thanks
> > Warden
> >
> > Eugene Law <liu...@apache.org> 於 2022年10月13日 週四 晚上9:16寫道：
> >
> > > +1
> > >
> > > Let's clarify this. If the Griffin engine would be integrated into an
> > > external system, we need to consider message protocol, such as protobuf
> > or
> > > thrift.
> > >
> > > On the other hand, if the Griffin engine would connect various data
> > > engines, we only need to outline the data transfer structure.
> > >
> > > Thx
> > >
> > > On Thu, Oct 13, 2022 at 2:31 PM William Guo <gu...@apache.org> wrote:
> > >
> > > > hi,
> > > >
> > > > My opinion is,
> > > >
> > > > for external, to integrate with apache griffin,
> > > > we will keep it open on the protocol, since we need to consider
> > > integration
> > > > efforts.
> > > >
> > > > but for internal components,
> > > > say, to connect with different query engines(hive, spark, flink), we
> > > prefer
> > > > JDBC, which is very typical to connect to different engines as a hub.
> > > >
> > > > What do you think?
> > > >
> > > > Thanks,
> > > > William
> > > >
> > > > On Thu, Oct 13, 2022 at 12:16 PM Edgar Joya <euj...@gmail.com>
> wrote:
> > > >
> > > > > Will you be using proto files for grpc for the new architecture?
> > > > > https://grpc.io/
> > > > >
> > > > > On Wed, Oct 12, 2022 at 10:08 PM William Guo <gu...@apache.org>
> > wrote:
> > > > >
> > > > > > Hi jianhua,
> > > > > >
> > > > > > We cannot see the architecture diagram in wiki
> > > > > >
> > > https://cwiki.apache.org/confluence/display/GRIFFIN/Components+Diagram
> > > > > > But we can see the sequence diagram there.
> > > > > >
> > > > > > Could you rephrase it ?
> > > > > >
> > > > > > And as Eugence said, we need to make it clear which protocol we
> are
> > > > using
> > > > > > between different components.
> > > > > >
> > > > > > Thanks,
> > > > > > William
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 12, 2022 at 9:57 PM Eugene Law <liu...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Could you point out what kind of protocol definition schema you
> > > used
> > > > in
> > > > > > > interface description? I think we should know the accurate
> > fields'
> > > > > range,
> > > > > > > type in different language.
> > > > > > >
> > > > > > > Thx
> > > > > > >
> > > > > > > On Wed, Oct 12, 2022 at 7:48 PM William Guo <gu...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > Thanks jianhua.
> > > > > > > >
> > > > > > > > Could you draw the sequence diagram in wiki
> > > > > > > >
> > > > > > > >
> > > > >
> > https://cwiki.apache.org/confluence/display/GRIFFIN/Components+Diagram
> > > > > > > >
> > > > > > > > We cannot see your attachments in email.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > William
> > > > > > > >
> > > > > > > > On Wed, Oct 12, 2022 at 5:58 PM jianhua guo <
> > guojhk...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > The core module and dispatcher's architecture is below:
> > > > > > > > > [image: image.png]
> > > > > > > > > 1. core module will generate the DQ sql according to the
> > > > different
> > > > > > > > > engine‘s syntax features，and wrapped in the submit request.
> > > > > > > > > 2. dispatcher provided the restful api to accept the
> request
> > > from
> > > > > the
> > > > > > > > core
> > > > > > > > > server.
> > > > > > > > > 3. after received the request info, dispatcher will submit
> > the
> > > > > query
> > > > > > to
> > > > > > > > > the specified engine(presto, spark, hive, flink)
> > > > > > > > > 4. dispatcher need provided a method to the core module,
> > named
> > > > > > > "execute"
> > > > > > > > > is nice, and in the method, it will continue to get job
> > status
> > > > from
> > > > > > > > > dispatcher,when the job is finished, then fetch the result
> > from
> > > > > > > > dispatcher,
> > > > > > > > > then return the response to the core module.
> > > > > > > > > The below is the sequence diagram.
> > > > > > > > > [image: image.png]
> > > > > > > > > [image: griffinDispatcher1.png]
> > > > > > > > > [image: image.png]
> > > > > > > > > Thanks for you review.
> > > > > > > > >
> > > > > > > > > On Wed, Oct 12, 2022 at 5:09 PM jianhua guo <
> > > guojhk...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Most looks good to me. Just one confusion, the
> > > "RecordResponse"
> > > > > only
> > > > > > > > >> return long value result?
> > > > > > > > >>
> > > > > > > > >> On Wed, Oct 12, 2022 at 1:22 PM William Guo <
> > gu...@apache.org
> > > >
> > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >>> hi all,
> > > > > > > > >>>
> > > > > > > > >>> The following message from Warden, we can discuss
> > interaction
> > > > > > between
> > > > > > > > >>> core
> > > > > > > > >>> and dispatcher module here.
> > > > > > > > >>> ====================
> > > > > > > > >>> I have drafted the communication messages between core
> and
> > > > > > dispatcher
> > > > > > > > as
> > > > > > > > >>> below,
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> // submit sql
> > > > > > > > >>> execute(RecordRequest) return RecordResponse;
> > > > > > > > >>>
> > > > > > > > >>> RecordRequest {
> > > > > > > > >>> String recordSql;
> > > > > > > > >>> Enum engine; // Spark,Hive,Presto,etc.
> > > > > > > > >>> String owner;
> > > > > > > > >>> Integer maxRetryCount;
> > > > > > > > >>> }
> > > > > > > > >>>
> > > > > > > > >>> RecordResponse {
> > > > > > > > >>> Integer code;
> > > > > > > > >>> Long value;
> > > > > > > > >>> Enum errorCode; // if code != 200, please tell us what
> > > > happened:
> > > > > > > > >>> // 1、 recordSql syntax error
> > > > > > > > >>> // 2、 internal error, dispatcher self is crashed
> > > > > > > > >>> // 3、 external error, target engine is crashed
> when
> > > > > > > dispatcher
> > > > > > > > >>> call,etc
> > > > > > > > >>> Exception ex; // error detail info
> > > > > > > > >>> }
> > > > > > > > >>>
> > > > > > > > >>> // validate sql syntax
> > > > > > > > >>> validateSQL(CheckRequest) return CheckResponse;
> > > > > > > > >>>
> > > > > > > > >>> ValidateSQLRequest {
> > > > > > > > >>> String recordSql;
> > > > > > > > >>> Enum engine; // Spark,Hive,Presto,etc.
> > > > > > > > >>> }
> > > > > > > > >>>
> > > > > > > > >>> ValidateSQLResponse {
> > > > > > > > >>> Integer code;
> > > > > > > > >>> Enum errorCode; // if code != 200, please tell us what
> > > > happened:
> > > > > > > > >>> // 1、 recordSql syntax error
> > > > > > > > >>> // 2、 internal error, dispatcher self is crashed
> > > > > > > > >>> // 3、 external error, target engine is crashed
> when
> > > > > > > dispatcher
> > > > > > > > >>> call,etc
> > > > > > > > >>> Exception ex; // error detail info
> > > > > > > > >>> }
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> Could you please review it? Give us your feedback.
> > > > > > > > >>>
> > > > > > > > >>> ==================
> > > > > > > > >>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Edgar Joya
> > > > > 980.259.0683
> > > > > euj...@gmail.com
> > > > > @eujc21
> > > > >
> > > >
> > >
> >
>
