[grpc-io] grpc-go balancer behavior when there are no backends

[aamit via grpc.io]

Hi,

I had a question about grpc-go's behavior when you use a custom balancer 
and there are no available backends available. My understanding is with 
FailFast callopt, the rpc should fail immediately rather then consuming the 
entire RPC deadline budget. However, that does not seem to be the case. 
I've modified the helloworld to use the manual 
balancer https://gist.github.com/aamitdb/7df4bc5c6b8c0bf3b667023f3b779ebf 
and despite FailFast, the program terminates with `2018/05/07 11:07:03 
could not greet: rpc error: code = DeadlineExceeded desc = context deadline 
exceeded`. 

This seems different than the behavior of the deprecated resolver package. 
The cause seems to be the RPC is waiting 
at https://github.com/grpc/grpc-go/blame/master/picker_wrapper.go#L170 but 
most of the situations blockingCh are written to are only in response to 
subconn state changing (but we have no subconns). 

Am I missing something about how to use the balancer APIs? Or is the 
expectation that we always at least call NewAddresses with an empty slice 
(which seems to resolve it has somewhere in the stack it tries to attempt a 
connection and fails fast). Attached a log of a run when I pass an empty 
slice and get the desired FailFast behavior. 

```

$ GRPC_GO_LOG_SEVERITY_LEVEL=INFO GRPC_GO_LOG_VERBOSITY_LEVEL=1000 go run 
greeter_client/main.go 

INFO: 2018/05/07 11:31:44 parsed scheme: "blatux149k5z"

INFO: 2018/05/07 11:31:44 ccResolverWrapper: sending new addresses to cc: 
[{ 0  }]

INFO: 2018/05/07 11:31:44 base.baseBalancer: got new resolved addresses:  [{ 
0  }]

INFO: 2018/05/07 11:31:44 base.baseBalancer: handle SubConn state change: 
0xc420172070, CONNECTING

WARNING: 2018/05/07 11:31:44 grpc: addrConn.createTransport failed to 
connect to { 0  }. Err :connection error: desc = "transport: Error 
while dialing dial tcp: missing address". Reconnecting...

INFO: 2018/05/07 11:31:44 base.baseBalancer: handle SubConn state change: 
0xc420172070, TRANSIENT_FAILURE

2018/05/07 11:31:44 could not greet: rpc error: code = Unavailable desc = 
all SubConns are in TransientFailure, latest connection error: connection 
error: desc = "transport: Error while dialing dial tcp: missing address"

exit status 1

```

Thanks,
Ashwin

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/a4dbc62d-e314-44c0-af72-dd5bcda6e7ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[grpc-io] Building grpcio-python without recompiling core files

[aamit via grpc.io]

I was wondering if it was reasonable to have a env variable that affected 
the `CORE_C_FILES` that is passed into Extension in setup.py (basically 
something like SKIP_COMPILING_CORE). I’d like to build grpcio python from 
source and link against grpc_core compiled separately.  My understanding of 
disttools is that when we pass in the gprc_core files as srcs 
(https://github.com/grpc/grpc/blob/master/setup.py#L220), it will compile 
and prefer to link those, even if we make the precompiled libraries 
available on library search path.

Right now I’ve accomplished this just by editing the setup.py locally. I 
think it might be useful for upstream/others as well as this allows me to 
build grpc python with :grpc as a dependency that handles compiling core, 
rather then recompiling it when building wheel using python setup.py 
build_ext. Unfortunately I’m using some in-house bazel rules to expose a 
bazel built grpcio-python that aren’t open source..

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/e0e42c37-9b0c-4804-abf2-bf75a0a40855%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[grpc-io] grpc.max_metadata_size

[aamit via grpc.io]

Hi,

Had 2 questions regarding using "grpc.max_metadata_size"

1. It was not clear to me that `max_metadata_size` takes into account size 
of "detail" part of response. Can we allow setting "max_metadata_size" to 
-1 to represent unlimited as with `max_send_message_length` and 
`max_receive_message_length`? 

The exception that gets raised (at least from python) when you try and 
return with details larger then `max_metadata_size` is 
`(StatusCode.RESOURCE_EXHAUSTED, to-be-sent trailing metadata size exceeds 
peer limit` which imo is somewhat confusing as I didn't consider "details" 
to be trailing metadata, and it does not show up under the response call 
trailing metadata.

2. Is the equivalent of "max_metadata_size" exposed anywhere in grpc-go? As 
far as I can tell Java does expose it through netty params 
`maxHeaderListSize`.

Thanks,
Ashwin

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/31385f13-7406-4af5-b124-f511098074f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[grpc-io] Status of Experimental Interceptor APIs in Go/Python

[aamit via grpc.io]

Hi,

My understanding is APIs that are marked as experimental can be removed 
without notice. I was wondering what the process to promote experimental 
APIs to stable is and where I can follow along on that. 

Both the go client side interceptors (which seem mostly unchanged for 1.5 
year) and python interceptor (much more recent, only added in December) are 
still marked as experimental.

Thanks,
Ashwin

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/c1d463e1-3b59-49e3-a472-71049fea5b7a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [grpc-io] [python] Subclassing RpcError

[aamit via grpc.io]

Hey Nathan,

Thanks for taking the time for the detailed answer. I'm still not entirely 
convinced by your points. To respond to your points and try and clarify 
what I wrote:

> (When you talk about a hierarchy of Exception classes, this is about 
making something more convenient than it is today, right? Not making 
possible something that today is impossible?)
Yes. As such, it's not a blocker for me that things are the way they are 
and I don't want to bikeshed my opinions too heavily here if you disagree 
with the points I raise below.

> It seems code-bloaty to have one exception class per code! We believe 
that classes that have no behavioral differences (for example: ever see a 
subclass that contains no method definitions but rather only a constructor 
that calls its superclass' constructor?) are a code smell. Exception 
classes are a bit of a special case because they're mostly behaviorless 
anyway, but they're not so special that different data values deserve 
different class definitions.
Normally I'd agree, but because of how exception handling works in Python 
(as well as other languages like Java), I think it is not uncommon to have 
behaviorless exception class e.g. 
https://github.com/requests/requests/blob/master/requests/exceptions.py. 
https://github.com/celery/celery/blob/master/celery/exceptions.py (picking 
2 random popular open source packages). Of course, not every package does 
this.

> Fun fact: the set of status codes is (at least in theory) not fixed. 
Additional codes are supposed to be able to be added one day without 
breaking working code. This may or may not ever happen, but it was a 
concern at the time we were designing the gRPC Python API.
What I meant to convey was that from the client perspective, we already 
treat any code that is not known by the client as unknown. It seems that is 
intended behavior. I brought that up to show that if we had a class per 
code,  the client behavior of falling back to "UnknownGrpcError" class is 
already established.

> except (, , ,) as my_exception:
Yes - if you had a few codes you want to handle the same way you could 
either specify the ones you care about or catch the base exception class 
and treat it as we currently do. The way to handle it now would be I think.
```
catch RpcError:
  if we care about code handle
  else raise
```

> grpc.RpcError isn't opaque; it's one of the few concrete classes exposed 
in the gRPC Python API. It defines no behavior, but it's there and it's an 
Exception subclass.
When I say it's opaque I mean that it could represent almost anything that 
went wrong with gRPC, and that it is has no defined behavior. I think this 
gets to your mention of how rather then having an inheritance chain for the 
behavior, the actual behavior is from an adjacent class from multiple 
inheritance. I do understand how treating a subclass as a base class is 
common and often preferred, but I'd argue that this multiple inheritance 
setup is different.

This gets into where it's strictly my opinion that it would be more clear 
if the behavior that I see most people expect on RpcError was part of its 
defined behavior. I understand that right now the docs make this guarantee, 
but from a perspective of statically typing this code it would be easier to 
make it clear that RpcError has methods I can call. I'd prefer not to rely 
on messages in the docs which can fall out of date.  Otherwise I think code 
that is like:
```
try:
  make grpc call
except RpcError as e:
  switch on e.code
```
which I see recommended as how to handle gRPC errros, should be written as 
below to be more defensive as:
```
try:
  make grpc call
except RpcError as e:
  if instanceof(e, Call):
switch on e.code
```

If the convenience of this is not worth the cost of committing to these 
concrete classes I understand, but in my opinion having an exception chain 
that inherits from a base class like `RpcCallError(RpcError,Call` makes 
sense for python and doesn't make the API surface more fragile.

Thanks,

On Tuesday, February 27, 2018 at 10:30:04 AM UTC-8, Nathaniel Manista wrote:
>
> On Mon, Feb 26, 2018 at 11:16 AM, aamit via grpc.io <
> grp...@googlegroups.com > wrote:
>
>> I had some questions about the api design of RpcError.
>>
>
> You're not alone; it's a part of the gRPC Python API that's somewhat 
> different than how APIs are often shaped in Python (we knew this all along) 
> and with which users have had some confusion (more than we expected!).
>
> From what I can tell in docs I should catch RpcError and switch on code 
>> (since it's also implicitly a `Call`).
>>
>
> Not all grpc.RpcError instances are also grpc.Call instances, but many 
> are. Speaking just for those that are: sure, switch on the value of the 
> code if that's what you want to do, but you don't have to do so.
>
> A few things frustrate me about this.
>>
>

[grpc-io] [python] Subclassing RpcError

[aamit via grpc.io]

Hi,

I had some questions about the api design of RpcError. From what I can tell 
in docs I should catch RpcError and switch on code (since it's also 
implicitly a `Call`). A few things frustrate me about this.

   1. There are certain codes I'd like to treat one way and certain codes 
   I'd like to treat another way. It seems slightly more pythonic to have one 
   exception class per code, especially given the number of codes is fixed and 
   the python client will treat any code that is not one of the known status 
   codes as "unknown". This would let me catch the specific codes I care about 
   by `excepting` only the classes that represent those codes, rather than 
   catching all RpcError, switching, and reraising otherwise.
   2. The api of `_Rendezvous` is pretty large, and we're told to count on 
   behavior that is implicitly there as a result of it being a `Call`, rather 
   that an api where we can catch an `GrpcException` that has a base api of 
   methods that are available and make sense to call on the exception. I'm not 
   sure which methods out of Call/Future make sense to call when I catch the 
   RpcError (that is actually a _Rendezvous). Right now I use `code` and 
   `details` and the `metadata`. I think changing this would also help toward 
   the goal of "Support static type-checking of both gRPC Python itself and of 
   code that uses gRPC Python" mentioned in the gsoc list. 
    

In my opinion, it makes more sense that instead of the opaque RpcError, 
grpc python exposed a exception class per code that derived from a base 
class that actually had a meaningful set of methods to call (rather than 
inheriting most of the future methods that don't make sense for me to call 
on the error). I was wondering if I'm missing something about idiomatically 
handling errors on the client, and what went into the current design of the 
exception api.

Thanks,
Ashwin

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/8c49645c-6d43-43ef-98b6-aaff7cff0b7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.