Re: [grpc-io] [GSoC] Assistance with gRPC Python Ideas

['Nathaniel Manista' via grpc.io]

On Sat, Mar 10, 2018 at 11:07 AM, Sword Chen  wrote:

> I am Sword Chen, a postgraduate student at Mid Sweden University.
>

Pleased to meet you!

I have used the gPRC for some time.
>

Cool! How long? With what programming languages?

So I would love to contribute to the project as part of GSoC. I have seen
> the given ideas, and I am quite interested in the "Support static
> type-checking of both gRPC Python" , but I can't find the relative open
> issue for this idea.  so I did some prework and I put them here.
>
> I have tried the Type Checking Tool Mypy, when I tested some file using
> the grpc module and I got this error " Cannot find module named 'grpc' "
> Then I went through with the Mypy document, Mypy can't do static
> type-checking of both gRPC Python itself and of the code that uses gRPC
> because Mypy can't find a stub for a grpc library module. and here is the
> information about Creating Stubs For Python Modules https://github.com/
> python/mypy/wiki/Creating-Stubs-For-Python-Modules. So what I need to do
> is to create the stubs for the grpc to help Mypy to detect the code. Am I
> working in the right direction now? It would be really helpful if someone
> can give me some guidance, so I can have some confidence to go on.
>

This sounds like it's going in the right direction - something that I think
we sort of expect is that there will be a certain incrementality to static
typing of gRPC Python, and having to add stubs describing the types of
lower-level systems so that the upper-level systems can have type
annotations added and checked is not a surprise. mypy and pytype are both
young enough, and gRPC Python is weird enough, that it is certainly not our
expectation that we can just throw either tool at our codebase and it will
figure types out without any further assistance. :-)

Thanks for reaching out!
-Nathaniel

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnASN%2BOYnJbmUTTq9ipFm7BxPzoPUrM8dpKbi0URXtgkUDA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

[grpc-io] Re: Non-blocking single-threaded streaming C++ server

[Todd Defilippi]

Thanks!

I have been messing around with using AsyncNext.  My long-term plan is to 
have some timer that calls back and checks if there is anything in the 
completion queue via AsyncNext.  For now, I've just been trying to get some 
basics to work.  Below I have the code I'm using.  (Ignore the sleeping for 
now, that's just so it loops a little slower for debugging purposes.)  The 
issue is, while I do get AsyncNext returning with ok=true and a valid tag, 
I do not get anything when I try to read from the stream.  Am I not using 
the stream correctly?

std::unique_ptr service2;
service2.reset(new gRPCMdtDialout::AsyncService());

ServerBuilder builder;
builder.AddListeningPort(addr, grpc::InsecureServerCredentials());
builder.RegisterService(service2.get());
std::unique_ptr cq = 
builder.AddCompletionQueue();
std::unique_ptr server(builder.BuildAndStart());

ServerContext ctx_;
ServerAsyncReaderWriter stream_(_);
service2->RequestMdtDialout(_, _, cq.get(), cq.get(), this);

while (true) {
void *tag = NULL;
bool ok = false;
bool ret = cq->AsyncNext(, , gpr_time_0(GPR_CLOCK_REALTIME));
if (ret == 1 && ok) {
MdtDialoutArgs args;
stream_.Read(, tag);
cout << " read " << args.ByteSizeLong() << " " << args.reqid() 
<< endl;
}
sleep(5);
}

Thanks,
Todd

On Monday, March 12, 2018 at 12:14:46 PM UTC-7, Yang Gao wrote:
>
> Hi Todd,
>
> Sorry for the late reply.
>
> You are right that Next will block until next event is coming out. 
> For your purpose, you may want to use AsyncNext with a deadline for a 
> server with async service.
> The deadline should be the next time point that you want to break out to 
> do something else on the server.
> Note grpc library needs to use the thread donated via Next or AsyncNext to 
> do some background work and thus only AsyncNext infrequently with very 
> short deadline may not be a good idea.
> Also, regarding the single-threadedness, the current grpc implementation 
> creates internal threads to do background work such as timer handling and 
> others. As a result, you will not have a truly single-threaded server even 
> if you only use one thread for the server.
> This may not be the final form as we intended to support threading model 
> fully controlled by the user. However, there is no timeline for those 
> background threads to be removed at the moment.
>
> On Thursday, February 22, 2018 at 11:01:37 AM UTC-8, Todd Defilippi wrote:
>>
>> I am trying to write a streaming server as part of implementing a gRPC 
>> dial-out collector for Cisco's model-driven telemetry (
>> https://github.com/cisco/bigmuddy-network-telemetry-proto/blob/master/proto_archive/mdt_grpc_dialout/mdt_grpc_dialout.proto
>> ).
>>
>> Is there a way to create the gRPC server with C++ that does not require 
>> blocking?  The basic synchronous examples all require blocking, which will 
>> not work, so I have been looking at the various asynchronous examples.  I 
>> haven't been able to get those to work but I'm not sure I'm doing it 
>> correctly.
>>
>> It seems like I would want to in some way use the CompletionQueues to 
>> hold messages that come in and process them when I can.  Is there a reason 
>> I would want to use Next() versus AsyncNext() (it seems the former blocks 
>> and the latter does not but I'm not sure)?  How does the combination of 
>> RegisterService() and AddCompletionQueue() work?
>>
>> Thanks,
>> Todd
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/9a3c9ab2-d4a3-4c98-aec2-600112218124%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [grpc-io] Assistance with GSoC 2018

['Nathaniel Manista' via grpc.io]

On Mon, Feb 26, 2018 at 11:24 AM, Naresh R  wrote:

> This is Naresh, final year CSE undergrad from IIT Kharagpur, India. I've
> been looking to work with grpc for GSoC this year.
>

Pleased to meet you!

I have been playing around with the code for the past 3 days, and I've
> added a few questions here: https://github.com/grpc/grpc/issues/13117
> (which seemed like one of the trivial issues I can work on to get a hang
> around the process of contributing to the repo). Can someone please guide
> me a little on this?
>

My apologies for the delay and I see that other team members have
responded. Thank you for the pull request
!
-Nathaniel

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnARscbdUBsA4BKnusJV%3D%2BRyf1P_waf6mecd3UdcLxyiarw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [grpc-io] [GSoC 2018] Enable building of gRPC Python with Bazel.

['Nathaniel Manista' via grpc.io]

On Fri, Feb 16, 2018 at 12:56 PM, Harsh Vardhan  wrote:

> I'm interested in working on enabling the building of gRPC python with
> Bazel. But, I'm unclear on what the goal of the project is.
>

So are we! But that's kind of the point - it's our most ambitious, most
open-ended, least clear project idea. We wanted at least one thing on our
ideas page that challenges students to figure out the project boundaries
for themselves. :-)

Last time I checked, the Bazel build and tests were failing on my pull
> request.
>

How long ago was that? I *think* the tests are currently healthy... but
it's on the more dynamic and actively-changing parts of the project, so
it's "exciting".

After crawling through different links in the github issues, I found that
> Bazel python rules don't support building native libraries yet. I found
> some Bazel + Python design notes floating around and some related issues.
> Should we wait for Bazel to complete their work or should we extend the
> rules_protobuf or write our own Bazel rules?
>

The work currently being done in Bazel for Python
 makes this
question's answer a bit of a moving target, but since Bazel's also an open
source project it's hard to forecast that rules written for gRPC Python
would stay exclusive to gRPC Python rather than getting somehow slurped
into Bazel. gRPC Python expressly wants to *avoid* being so special a case
that it needs its own rules in the long term.

What's the current status of the build of gRPC python with Bazel?
>

Close to zero - which means it's a wide-open possibility for a student
project to define various intermediate statuses, goals, and so forth.

Good to hear from you again!
-Nathaniel

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnATpZwWK9TBK%2BESWOZ%2BAgA7qU4AN%3DYc3bn7nh%3DqofkC-4A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [grpc-io] GSOC 2018 ideas

['Nathaniel Manista' via grpc.io]

On Sat, Feb 24, 2018 at 4:41 PM, Charles Cui 
wrote:

>This is Charles from Columbia University,
>

Pleased to meet you!

I saw gRPC is selected in GSOC 2018, and pretty interested in the idea of
> implementing early OK semantics for gRPC. So, I want to prepare for a
> proposal based on this idea. Can I have more detailed information of what
> documents
>

There's not a whole lot of documentation around the proposed feature,
because it's not so much a new feature as much as filling in a missing
corner. Consider a two-by-two matrix of "have read all requests/have not
read all requests" and "respond with not-OK status/respond with OK status".
It is currently the case that a service-side application:

1) after having read all requests can respond with not-OK status,
2) after having read all requests can respond with OK status,
3) without having read all requests can respond with not-OK status,
4) without having read all requests *cannot* respond with OK status.

So there's not a whole lot of documentation needed to say "this fourth case
should be made to work like the other three".

or code that I need to understand?
>

The code to understand is gRPC Core - how are RPCs serviced? How do status
codes get handled in the course of RPC service?

Any comments or suggestions are welcomed!
>

Two suggestions spring to mind for this idea:
1) After building gRPC Core and running its tests yourself, draft
additional test coverage for the changed behavior, run it, and observe that
it fails. Do any tests in the current test corpus depend upon the current
behavior?
2) Is there any history to how this case was missed? Are you able to dig
through source control history and find anything interesting? How does it
look like the current behavior came to be?

Thanks for getting in touch!
-Nathaniel

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnAShWaaER_HdJ%2BtNZ%3DarTT%3DWvGYbmX_cF-OeYq6-a5fgNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [grpc-io] Regarding working on gRPC for gsoc 2018

['Nathaniel Manista' via grpc.io]

On Wed, Feb 14, 2018 at 11:16 PM, shubham dubey  wrote:

> I am shubham from India and would like to work
> on gRPC python project "Support static type-checking of both gRPC Python
> itself and of code that uses gRPC Python" for gsoc 2018.
>

Pleased to meet you!

Currently, I am thinking about getting familiar with the codebase.
> But, would like to listen few suggestions on how to proceed further and
> little more details about the idea.
>

For gRPC Python generally: there are currently a few "help wanted" Python
issues
;
those might be the right place. Other questions that have occurred to me in
the recent past: why do our Cython files have that ".pxi" extension? Is
that idiomatic for Cython or are we doing something weird? How is our
coverage reporting supposed to work and why doesn't it?


For static analysis of gRPC Python in particular: what's the difference
between mypy  and pytype
? Are there ways in which the gRPC Python
API is shaped
 that might
not fit well into the current static typing offerings
?
-N

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnARqZTVRLpUVTaxfDqeKrTC8tL09VHJMw4_S3SHEwz-fpg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [grpc-io] gRPC Persistent Connections??

['Nathaniel Manista' via grpc.io]

With what programming language(s) are you working with gRPC? It's likely
that the details of the answers to your questions will vary from one
programming language to another.

On Mon, Mar 12, 2018 at 2:15 AM, Lokanadham M 
wrote:

> If gRPC connections are persistent, a single client will always talk to a
> single backend in that configuration. It's fine if you have lots of clients
> but how load balancing is done?
>

Sometimes load balancing is done in the client by representing as one
connection to the application what are actually several connections on the
wire to several different servers. Sometimes load balancing is done but
putting a running a load balancer on a host between clients and servers.

How server is able to handle so many open connections?? Won't it hit open
> file discriptors limit??
> Can some one please explain how it was implemented at socket level to
> handle that many connections??
>

I suspect that the answers to these questions are implementation-dependent.
-Nathaniel

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAEOYnAQ6tvz2OhWiQypk2TKDCn1%3DVHoi9Td_TG1zo7ZkRJ-rGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


smime.p7s
Description: S/MIME Cryptographic Signature

[grpc-io] gRPC Persistent Connections??

[Lokanadham M]

If gRPC connections are persistent, a single client will always talk to a 
single backend in that configuration. It's fine if you have lots of clients 
but how load balancing is done?
How server is able to handle so many open connections?? Won't it hit open 
file discriptors limit??
Can some one please explain how it was implemented at socket level to 
handle that many connections??

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/3b9f16f1-ec1b-4c7f-8911-362d68477455%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.