Re: gRPC, Better C and D
On Monday, 2 July 2018 at 18:06:43 UTC, bpr wrote: On Monday, 2 July 2018 at 10:25:07 UTC, Andre Pany wrote: Small update. With the help of evilrat I was able to translate the C headers to D. https://github.com/andre2007/grpc/tree/feature/d/src/d/source/grpc/c While the source code compiles without errors, everything is completely untested. As next step I will try to rewrite the python grpc code with D: https://github.com/andre2007/grpc/tree/feature/d/src/python/grpcio/grpc/_cython/_cygrpc That's really interesting! Are you thinking about translating some part of gRPC to DasBetterC? At the moment, no. Foremost priority is building a wrapper for D to open the gRPC ecosystem for D. Kind regards André
Re: gRPC, Better C and D
On Monday, 2 July 2018 at 10:25:07 UTC, Andre Pany wrote: Small update. With the help of evilrat I was able to translate the C headers to D. https://github.com/andre2007/grpc/tree/feature/d/src/d/source/grpc/c While the source code compiles without errors, everything is completely untested. As next step I will try to rewrite the python grpc code with D: https://github.com/andre2007/grpc/tree/feature/d/src/python/grpcio/grpc/_cython/_cygrpc That's really interesting! Are you thinking about translating some part of gRPC to DasBetterC?
Re: gRPC, Better C and D
On Monday, 2 July 2018 at 10:25:07 UTC, Andre Pany wrote: On Friday, 22 June 2018 at 18:10:54 UTC, Andre Pany wrote: [...] Small update. With the help of evilrat I was able to translate the C headers to D. https://github.com/andre2007/grpc/tree/feature/d/src/d/source/grpc/c While the source code compiles without errors, everything is completely untested. As next step I will try to rewrite the python grpc code with D: https://github.com/andre2007/grpc/tree/feature/d/src/python/grpcio/grpc/_cython/_cygrpc Very interesting news. gRPC in D is a must. If it'll get used for web services and other microseevices.
Re: gRPC, Better C and D
On Friday, 22 June 2018 at 18:10:54 UTC, Andre Pany wrote: Hi, the core of gRPC is written in C. I asked the developers whether they considered rewriting the core with Better C (https://github.com/grpc/grpc/issues/15786). They answered that they won't rewrite it with Better C but they are open for including D into their ecosystem. They have support for different languages and most of them are wrappers around the C Core functionality (C++, C#, Objective C, PHP, Python, Ruby). As far as I know there is no gRPC wrapper for D at the moment and if I remember correctly on DConf München there was a speaker talking about the need of a D wrapper for gRPC. My idea was to translate the C++ (https://github.com/grpc/grpc/tree/master/src/cpp) coding to D coding but my C++ knowledge is very limited. Is someone interested in adding D to gRPC? Kind regards André Small update. With the help of evilrat I was able to translate the C headers to D. https://github.com/andre2007/grpc/tree/feature/d/src/d/source/grpc/c While the source code compiles without errors, everything is completely untested. As next step I will try to rewrite the python grpc code with D: https://github.com/andre2007/grpc/tree/feature/d/src/python/grpcio/grpc/_cython/_cygrpc
Re: gRPC, Better C and D
On Friday, 22 June 2018 at 18:10:54 UTC, Andre Pany wrote: Hi, the core of gRPC is written in C. I asked the developers whether they considered rewriting the core with Better C (https://github.com/grpc/grpc/issues/15786). They answered that they won't rewrite it with Better C but they are open for including D into their ecosystem. They have support for different languages and most of them are wrappers around the C Core functionality (C++, C#, Objective C, PHP, Python, Ruby). As far as I know there is no gRPC wrapper for D at the moment and if I remember correctly on DConf München there was a speaker talking about the need of a D wrapper for gRPC. My idea was to translate the C++ (https://github.com/grpc/grpc/tree/master/src/cpp) coding to D coding but my C++ knowledge is very limited. Is someone interested in adding D to gRPC? Kind regards André I'm also very interested in a D binding. But I have no C binding skills. Maybe the direlict guys can help. gRPC is the gold standard for rpc in the interprese. I'm surprised we don't have much ctivity around it.
gRPC, Better C and D
Hi, the core of gRPC is written in C. I asked the developers whether they considered rewriting the core with Better C (https://github.com/grpc/grpc/issues/15786). They answered that they won't rewrite it with Better C but they are open for including D into their ecosystem. They have support for different languages and most of them are wrappers around the C Core functionality (C++, C#, Objective C, PHP, Python, Ruby). As far as I know there is no gRPC wrapper for D at the moment and if I remember correctly on DConf München there was a speaker talking about the need of a D wrapper for gRPC. My idea was to translate the C++ (https://github.com/grpc/grpc/tree/master/src/cpp) coding to D coding but my C++ knowledge is very limited. Is someone interested in adding D to gRPC? Kind regards André
Re: gRPC in D
On Monday, 4 June 2018 at 11:40:54 UTC, Nicholas Wilson wrote: On Monday, 4 June 2018 at 11:21:57 UTC, aberba wrote: At DConf 2018,there was a talk by ?? about blockchain and gRPC library for D came up. That was Kai Nacke. Thanks :)
Re: gRPC in D
On Monday, 4 June 2018 at 11:21:57 UTC, aberba wrote: At DConf 2018,there was a talk by ?? about blockchain and gRPC library for D came up. That was Kai Nacke.
gRPC in D
At DConf 2018,there was a talk by ?? about blockchain and gRPC library for D came up. In summary, grpc a universal rpc framework by Google and can be implemented in any language. It enables you to call methods on a remote server from a client as if they're both on the same host. Its use http/http2 as its protocol and protobuffer as the schema for api definition. Its quite an interesting subject in large scale enterprise development and microservices. gRPC is now a good reason to use Go or Java for development...at least for distributed services...including blockchain. Its actually very powerful. There's a protobuffer implementation at code.dlang.org What remains is a D grpc implementation or bindings using the c++ or c version. You may read more about it at https://grpc.io. Huge potential.
Re: gRPC in D good idea for GSOC 2018?
for grpc, we should add to dproto (which is pretty good protobuf library for D but lacks grpc) instead of starting from scratch, see https://github.com/msoucy/dproto/issues/113 [your advice/opinions on integrating with grpc?] On Mon, Jan 22, 2018 at 12:24 PM, Adrian Matoga via Digitalmars-d wrote: > On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: >> >> I know a project where D could benefit from gRPC in D, which is not among >> the supported languages: >> >> https://grpc.io/docs/ >> >> Do you think gRPC support is worth adding to GSOC 2018 ideas? >> >> https://wiki.dlang.org/GSOC_2018_Ideas >> >> Ali > > > I can share a fresh experience from mentoring a student in a similar (also > RPC) project last summer. We built native D-Bus bindings in D based on > libasync. The student had had no previous experience with D or RPC, and > within ~2.5 months of focused work she implemented the following: > > 1. (de)serialization of all D-Bus data types, including the use of > compile-time reflection to recursively marshall structs, arrays, and > variants. Except Variant, for which we decided to make our own > D-Bus-specific tagged union type, all other D-Bus types are mapped to > built-in D types. > 2. A class to connect to the bus via libasync sockets, read the incoming > messages and dispatch them to the registered handlers, and send messages to > the bus. > 3. Proxy (client) and server class templates that generate all the code > necessary to make the remote calls look almost like local ones (the return > value/out arguments are passed to a delegate that handles the output instead > of being returned synchronously). > > So, more or less an equivalent of vibe.d's REST interface generator, only > with fewer customization points. > > There were still some opportunities for refactorings and optimizations, so I > wouldn't consider it production ready. Also, some planned features weren't > implemented, such as a more convenient way for handling signals or allowing > transports other than unix sockets on libasync. On the other hand, what we > have is almost 100% covered with unit tests. This not only made adding > successive layers quite pleasant, as little (if any) debugging of previously > written stuff was ever necessary, but also helps to keep the stuff working > as we modify it. > > Based on my experience, I'd say that implementing gRPC may be of a right > size for a GSoC project, as long as you split it into smaller > components/layers, prioritize them, and focus on having at least the basic > stuff usable and tested, instead of expecting it to cover all usage cases > and be heavily optimized. >
Re: gRPC in D good idea for GSOC 2018?
On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali I can share a fresh experience from mentoring a student in a similar (also RPC) project last summer. We built native D-Bus bindings in D based on libasync. The student had had no previous experience with D or RPC, and within ~2.5 months of focused work she implemented the following: 1. (de)serialization of all D-Bus data types, including the use of compile-time reflection to recursively marshall structs, arrays, and variants. Except Variant, for which we decided to make our own D-Bus-specific tagged union type, all other D-Bus types are mapped to built-in D types. 2. A class to connect to the bus via libasync sockets, read the incoming messages and dispatch them to the registered handlers, and send messages to the bus. 3. Proxy (client) and server class templates that generate all the code necessary to make the remote calls look almost like local ones (the return value/out arguments are passed to a delegate that handles the output instead of being returned synchronously). So, more or less an equivalent of vibe.d's REST interface generator, only with fewer customization points. There were still some opportunities for refactorings and optimizations, so I wouldn't consider it production ready. Also, some planned features weren't implemented, such as a more convenient way for handling signals or allowing transports other than unix sockets on libasync. On the other hand, what we have is almost 100% covered with unit tests. This not only made adding successive layers quite pleasant, as little (if any) debugging of previously written stuff was ever necessary, but also helps to keep the stuff working as we modify it. Based on my experience, I'd say that implementing gRPC may be of a right size for a GSoC project, as long as you split it into smaller components/layers, prioritize them, and focus on having at least the basic stuff usable and tested, instead of expecting it to cover all usage cases and be heavily optimized.
Re: gRPC in D good idea for GSOC 2018?
On Monday, 22 January 2018 at 05:54:31 UTC, Dmitry Olshansky wrote: On Monday, 22 January 2018 at 04:40:53 UTC, Andrew Benton wrote: On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali An http/2 and gRPC solutions is probably necessary with tools like linkerd, envoy, and istio if D wants to be competitive in service mesh and distributed applications. http/2 and/or gRPC are both excellent ideas for GSoC 2018. I would consider them awful in a sense that there is no foundation to build them on. At best it will be a self-serving artifact poorly fitting with anything else. I think it would likely be built on top of vibe or collie instead of being standalone, but I see your point. There is not even a standard way on handling IO as of yet. Basically do we want fiber-aware IO or blocking IO or explicit async with future/promise? You're right. It's definitely a problem. Given ease of use and the learning curve, I'd be in favor of fiber-aware IO or async/await types. Do we even have protobuf package? We have at least two. dproto is a package that supports proto2. I've used it before and it's really nice. There's also the eponymous protobuf package, which integrates with protoc and can also do proto3. I haven't used it yet.
Re: gRPC in D good idea for GSOC 2018?
> Do we even have protobuf package? https://github.com/msoucy/dproto it could receive some attention, there are pending issues for RPC I've been using msgpackrpc since no gRPC was available. But would be nice to have gRPC. NOTE: capnproto is a very interesting newer alternative to protobuf; https://github.com/capnproto/capnproto-dlang shows: Missing RPC part of Cap'n Proto. helping out capnproto project (esp around RPC) could be another idea. we definitely need a good way to do RPC in D, otherwise hard to integrate D with other services. > I would consider them awful in a sense that there is no foundation to build > them on. At best it will be a self-serving artifact poorly fitting with > anything else. But it would enable using D in places that were not previously possible (integrating with services); we could imaging providing a (semi) stable interface for grpc in D code and change implementation to use better foundations later On Sun, Jan 21, 2018 at 9:54 PM, Dmitry Olshansky via Digitalmars-d wrote: > On Monday, 22 January 2018 at 04:40:53 UTC, Andrew Benton wrote: >> >> On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: >>> >>> I know a project where D could benefit from gRPC in D, which is not among >>> the supported languages: >>> >>> https://grpc.io/docs/ >>> >>> Do you think gRPC support is worth adding to GSOC 2018 ideas? >>> >>> https://wiki.dlang.org/GSOC_2018_Ideas >>> >>> Ali >> >> >> An http/2 and gRPC solutions is probably necessary with tools like >> linkerd, envoy, and istio if D wants to be competitive in service mesh and >> distributed applications. >> >> http/2 and/or gRPC are both excellent ideas for GSoC 2018. > > > I would consider them awful in a sense that there is no foundation to build > them on. At best it will be a self-serving artifact poorly fitting with > anything else. > > There is not even a standard way on handling IO as of yet. > Basically do we want fiber-aware IO or blocking IO or explicit async with > future/promise? > > Do we even have protobuf package? > >
Re: gRPC in D good idea for GSOC 2018?
On Monday, 22 January 2018 at 04:40:53 UTC, Andrew Benton wrote: On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali An http/2 and gRPC solutions is probably necessary with tools like linkerd, envoy, and istio if D wants to be competitive in service mesh and distributed applications. http/2 and/or gRPC are both excellent ideas for GSoC 2018. I would consider them awful in a sense that there is no foundation to build them on. At best it will be a self-serving artifact poorly fitting with anything else. There is not even a standard way on handling IO as of yet. Basically do we want fiber-aware IO or blocking IO or explicit async with future/promise? Do we even have protobuf package?
Re: gRPC in D good idea for GSOC 2018?
On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali An http/2 and gRPC solutions is probably necessary with tools like linkerd, envoy, and istio if D wants to be competitive in service mesh and distributed applications. http/2 and/or gRPC are both excellent ideas for GSoC 2018.
Re: gRPC in D good idea for GSOC 2018?
On Friday, 19 January 2018 at 10:14:15 UTC, yawniek wrote: On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali Problem here is that D lacks a proper http/2 stack. unfortunately due to time constraints my libh2o binding project got stuck. Have you seen Etienne's libhttp2? http://code.dlang.org/packages/libhttp2
Re: gRPC in D good idea for GSOC 2018?
On Friday, 19 January 2018 at 18:34:31 UTC, Ali Çehreli wrote: On 01/19/2018 02:14 AM, yawniek wrote: Could you please summarize what needs to be done and in what order. I would be happy to take active part in this effort. well, my personal opinion this should be designed/decided on a green field and not again make the mistake to try to include too much that is just laying around. i think vibe.d's eventcore may (or may not) be too opinionated and providing too much: 1a. define an overal strategy how/where protocols should be implemented and whether to focus on speed or "elegant" implementations. 1b. define how high performance servers should ideally be implemented and how that works together with D's concurrency models. plus provide the necessary things such as streams/ranges. ideally endorse/include/create one primary eventloop implementation or interface. possibly rusts tokyo.rs could be an inspiration? https://tokio.rs/docs/getting-started/tls/ 2a. create reference implementations for a http parser or e.g. port something like picohttpparser https://github.com/h2o/picohttpparser but this should adhere to the approach defined in 1. 3. implement crypto functions. e.g. port picotls ( https://github.com/h2o/picotls ) but also provide/maintain proper openssl bindings. 4. implement http/2 grpc also, think about what happens when the world starts switching to quic, the ietf wg makes good progress from what i saw. a totally different approach/decision would be to say, that D is a glue language and one should use C libraries (libuv, openssl, picohttpparser). but then i think tooling for integrating those should be improved/standardized (e.g. package manager, binding automation).
Re: gRPC in D good idea for GSOC 2018?
On Friday, 19 January 2018 at 18:34:31 UTC, Ali Çehreli wrote: On 01/19/2018 02:14 AM, yawniek wrote: > Just look how > beautiful Golangs protocol implementations are and the whole ecosystem > that focused effort spawned. I have said this here before. I agree and do have first-hand experience with that but I'm not experienced enough in this field to come up with a modern solution. If it were left to me, I would start looking at Go and Python solutions but I fear I wouldn't see design decisions that could have been better. Could you please summarize what needs to be done and in what order. I would be happy to take active part in this effort. Thank you, Ali
Re: gRPC in D good idea for GSOC 2018?
On 01/19/2018 02:14 AM, yawniek wrote: > Just look how > beautiful Golangs protocol implementations are and the whole ecosystem > that focused effort spawned. I agree and do have first-hand experience with that but I'm not experienced enough in this field to come up with a modern solution. If it were left to me, I would start looking at Go and Python solutions but I fear I wouldn't see design decisions that could have been better. Could you please summarize what needs to be done and in what order. I would be happy to take active part in this effort. Thank you, Ali
Re: gRPC in D good idea for GSOC 2018?
On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali Problem here is that D lacks a proper http/2 stack. unfortunately due to time constraints my libh2o binding project got stuck. Imo the real problem with D is still that phobos does not provide the important building blocks for today modern infrastrucutre: - fast and standardized eventloop compatible with Ds concurrency primitives - network protocol libs/ bindings ( http, http/2, tls, dns ) until there is effort to do this D will continue to be an ecosystem of hacks. I really like vibe.d but it comes with a lot of problems and it has the focus of providing a usable web framework and not implementing protocols in a fast and reusable way. so as long as "management" does not decide that internet protocols is something important in D nothing useful will come out. Just look how beautiful Golangs protocol implementations are and the whole ecosystem that focused effort spawned.
Re: gRPC in D good idea for GSOC 2018?
On Tuesday, 16 January 2018 at 17:58:40 UTC, jmh530 wrote: On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali Also, the deadline is in 1 week. Yes it's a good idea for GSoC18 and we still accept ideas. Reply here if you have more ideas ;-)
Re: gRPC in D good idea for GSOC 2018?
On Monday, 15 January 2018 at 19:28:08 UTC, Ali Çehreli wrote: I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali Also, the deadline is in 1 week.
gRPC in D good idea for GSOC 2018?
I know a project where D could benefit from gRPC in D, which is not among the supported languages: https://grpc.io/docs/ Do you think gRPC support is worth adding to GSOC 2018 ideas? https://wiki.dlang.org/GSOC_2018_Ideas Ali
Re: grpc
On Monday, 30 November 2015 at 14:55:53 UTC, yawniek wrote: would anyone be interested in doing the work for adding D / vibe.d to https://github.com/grpc/grpc ? we currently lack the manpower but would be able to sponsor it (or parts, depending on the effort needed). grpc is probably going to be the dominant rpc system for building a microsrvices architecture. even though i prefer msgpack-rpc i think it would be very valuable in terms of connecting to e.g. GO services. +1
grpc
would anyone be interested in doing the work for adding D / vibe.d to https://github.com/grpc/grpc ? we currently lack the manpower but would be able to sponsor it (or parts, depending on the effort needed). grpc is probably going to be the dominant rpc system for building a microsrvices architecture. even though i prefer msgpack-rpc i think it would be very valuable in terms of connecting to e.g. GO services.