Re: Curl kerberized QueryServer using protobuf type
Thanks Josh!
Finally, I switched to JSON serialization as a simple workaround.
--
Best regards,
R.C
From: Josh Elser
Sent: 02 July 2019 22:55
To: user@phoenix.apache.org
Subject: Re: Curl kerberized QueryServer using protobuf type
Hey Reid,
Protobuf is a binary format -- this is error'ing out because you're
sending it plain-text.
You're going to have quite a hard time constructing messages in bash
alone. There are lots of language bindings[1]. You should be able to
pick any of these to help encode/decode messages (if you want to use
cURL as your "transport").
IMO, Avatica's protocol is too complex (by necessity of implementing all
of the JDBC API) to just throw some hand-constructed JSON at. I think
the better solution would be to think about some simpler API that
exposes just the bare bones if you want something developer focused.
[1] https://developers.google.com/protocol-buffers/docs/reference/overview
On 7/1/19 5:53 AM, Reid Chan wrote:
> Hi team and other users,
>
> Following is the script used for connecting to QS,
> {code}
> #!/usr/bin/env bash
>
> set -u
>
> AVATICA="hostname:8765"
> echo $AVATICA
> CONNECTION_ID="conn-$(whoami)-$(date +%s)"
>
> echo "Open connection"
> openConnectionReq="message OpenConnectionRequest {string connection_id =
> $CONNECTION_ID;}"
> curl -i --negotiate -u : -w "\n" "$AVATICA" -H "Content-Type:
> application/protobuf" --data "$openConnectionReq"
> {code}
>
> But it ended with:
> org.apache.calcite.avatica.proto.Responses$ErrorResponse�
> �rg.apache.calcite.avatica.com.google.protobuf.InvalidProtocolBufferException$InvalidWireTypeException:
> Protocol message tag had invalid wire type.
> at
> org.apache.calcite.avatica.com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:111)
> at
> org.apache.calcite.avatica.com.google.protobuf.CodedInputStream$ArrayDecoder.skipField(CodedInputStream.java:591)
> at
> org.apache.calcite.avatica.proto.Common$WireMessage.(Common.java:12544)
> at
> org.apache.calcite.avatica.proto.Common$WireMessage.(Common.java:12511)
> at
> org.apache.calcite.avatica.proto.Common$WireMessage$1.parsePartialFrom(Common.java:13054)
> at
> org.apache.calcite.avatica.proto.Common$WireMessage$1.parsePartialFrom(Common.java:13049)
> at
> org.apache.calcite.avatica.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:91)
> at
> org.apache.calcite.avatica.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:96)
> at
> org.apache.calcite.avatica.com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.calcite.avatica.com.google.protobuf.GeneratedMessageV3.parseWithIOException(GeneratedMessageV3.java:311)
> at
> org.apache.calcite.avatica.proto.Common$WireMessage.parseFrom(Common.java:12757)
> at
> org.apache.calcite.avatica.remote.ProtobufTranslationImpl.parseRequest(ProtobufTranslationImpl.java:410)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.decode(ProtobufHandler.java:51)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.decode(ProtobufHandler.java:31)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:93)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler$2.call(AvaticaProtobufHandler.java:123)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler$2.call(AvaticaProtobufHandler.java:121)
> at
> org.apache.phoenix.queryserver.server.QueryServer$PhoenixDoAsCallback$1.run(QueryServer.java:500)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
> at
> org.apache.phoenix.queryserver.server.QueryServer$PhoenixDoAsCallback.doAsRemoteUser(QueryServer.java:497)
> at
> org.apache.calcite.avatica.server.HttpServer$Builder$1.doAsRemoteUser(HttpServer.java:884)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:120)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
>
Questions about ZK Load Balancer
Hi community, Recently, i'm trying to apply the ZK-based Load Balancer on production env. But it looks like a half-done feature, i couldn't find how a query server client get a registered QS from LB in client side codebase. There's one method: LoadBalancer#getSingleServiceLocation, supposed to be called from client side, is dead codes and never invoked. Highly appreciate any code pointer or suggestion or advice. -- Best regards, R.C
Re: Questions about ZK Load Balancer
Yeah, that's correct. I think I requested some documentation to be added by the original author to clarify that it's not end-to-end usable, but I don't think it ever happened. The "load balancer" isn't anything more than service advertisement, IIRC. IMO, the write-up I made here[1] is going to give you something more usable out of the box. If you have the time to invest in fixing this up, let's chat. We can make this story better. [1] https://community.hortonworks.com/articles/9377/deploying-the-phoenix-query-server-in-production-e.html On 7/9/19 6:01 AM, Reid Chan wrote: Hi community, Recently, i'm trying to apply the ZK-based Load Balancer on production env. But it looks like a half-done feature, i couldn't find how a query server client get a registered QS from LB in client side codebase. There's one method: LoadBalancer#getSingleServiceLocation, supposed to be called from client side, is dead codes and never invoked. Highly appreciate any code pointer or suggestion or advice. -- Best regards, R.C
