I'll admit that I don't understand why protocol buffers is being equated
with the transport only.
The biggest reason I think Avatica can benefit from using software like
protobuf is that it makes handling a shift in the API substantially
easier. For example, what happens when a new field is added to a
Request? What if you receive a Request that doesn't have a field that
you expected it to (old client)? This is the problem that I'm trying to
solve. Regardless of whether this is coming in over the Java API or an
HTTP connection, the version of the Request or Response (and the actual
attributes that it contains) are near guaranteed to change.
I don't really care what the bytes look like going over the wire. That's
just a side-effect to which my only concern is to meet any desires for
readability that may exist.
Does that make sense?
Julian Hyde wrote:
The conventional way of implementing a driver uses an end-to-end
design: "just use the MySQL driver and have a MySQL server on the
other end and it will just work".
With Avatica I'm trying to introduce a layered approach (inspired by
the OSI model) where we specify what happens at several levels. We
specify a wire format, and we specify the Java API.
An application doesn't have to to use all of the levels. Some
applications might want to use the Java API but plug in a different
transport. Other applications might want to build a C or Python API on
top of the wire format.
My goal is to ensure that the driver stack can be re-used for
different database engines, and from different languages, and that a
driver in local mode is substantially the same as a driver in remote
mode.
Notice I said "a wire format". I don't mind if there are other wire
formats. But I would be concerned if one transport muscled in and left
its mark up and down the whole stack.
On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser<[email protected]> wrote:
Ok, that's where the confusion is coming in :). I envision protocol buffer
objects replacing the POJOs in both the wire api and the Meta API.
Trying to do both seems without gain for me (again, aside from preserving
backwards compatibility with existing releases). If that requires a
different Meta interface+impl (or a translation layer), that's fine too --
just extra work (and a slight performance hit on whichever is treated as the
reference object).
Does that make sense?
Julian Hyde wrote:
Yes, the POJOs are needed. The service layer (which for a particular
client may or may not be backed by RPC) consists of methods that take
complex arguments and return complex results. Those arguments and
results are expressed as POJOs.
One example:
Frame fetch(StatementHandle h, List<TypedValue> parameterValues, long
offset,
int fetchMaxRowCount);
class Frame {
public final long offset;
public final boolean done;
public final Iterable<Object> rows;
}
public class TypedValue {
public final ColumnMetaData.Rep type;
public final Object value;
}
The POJOs are definitely needed for the Java API Meta and appear in
some form by whatever the wire protocol is.
Julian
On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<[email protected]> wrote:
Pulling this out to email to avoid cluttering JIRA. I feel like I might
not
be on the same page.
I see CALCITE-839 and CALCITE-840 being one in the same change, or at
least
the root cause being solved by it.
Julian, I'm getting the impression that you envision protocol buffer
encoding being just another option for encoding requests and responses.
My
opinion is that using protocol buffers to define these requests and
responses completely invalidates the need to support these POJOs. These
objects should be usable cross-language, so aside from support the
releases
of Calcite which shipped only these POJOs, I don't see a need to maintain
them.
I am admittedly hedging my bet that the PB devs will release a new
version
that has the advertised JSON-esque serialization format (instead of just
a
binary format). If this ultimately falls through, POJOs that just wrap
the
PB classes could also be done.
I just wanted to make sure I'm not dancing by myself and that we're all
in
agreement on a general direction.
-------- Original Message --------
Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
POJO classes in Meta
Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
From: Julian Hyde (JIRA)<[email protected]>
Reply-To: [email protected]
To: [email protected]
[
https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
]
Julian Hyde commented on CALCITE-839:
-------------------------------------
Well then, I've assigned CALCITE-840 to you.
Remove Jackson annotations from POJO classes in Meta
----------------------------------------------------
Key: CALCITE-839
URL: https://issues.apache.org/jira/browse/CALCITE-839
Project: Calcite
Issue Type: Bug
Components: avatica
Reporter: Julian Hyde
Assignee: Julian Hyde
The Meta interface contains several POJO classes that represent RPC
requests or responses. Currently a few of those classes have Jackson
annotations such as @JsonCreator, @JsonProperty to help Jackson
serialize
the POJO to JSON and de-serialize from JSON to the object.
As [~ndimiduk] pointed out in
http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=gvkgd+bkj4+ejmuo6ivhs+okgskg1vwdazcy-zijyy...@mail.gmail.com%3E
these annotations are a "code smell" and should be removed. It makes it
look
as if Jackson is the only possible transport, which is not the case. We
can
continue to use Jackson as a transport, just specify the mappings
elsewhere,
not as annotations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
