On Mon, Jan 23, 2017 at 7:21 PM, Rudi Chiarito <[email protected]> wrote:
> On Fri, Jan 20, 2017 at 12:47 PM, 'Carl Mastrangelo' via grpc.io < > [email protected]> wrote: > >> Initial thoughts: >> >> * percentage needs to be declared to be an integer (as opposed to >> number). This will make it consistent internally and externally. >> > > What if 1% is too many? Can we have a deci-percent? I'm sure Mark will be > thrilled to deal with deci-quantities. ;-) > > More in general, is this "pull" design set in stone? By that, I mean that > clients generate a random number and figure if they fall e.g. in the 1% > that should try the new configuration. There's no guarantee that any > clients will pick it up (or that 10 out of 100, i.e. 10%, won't). > It's true that there's no guarantee. However, the service owner should be able to estimate how many clients they have (at least as an order of magnitude), which should allow setting the percentage to a reasonable value for getting the desired subset of canary clients. For services with a large number of clients, even if we don't hit the target percentage exactly, we'll likely be close enough that it makes no difference. For example, if there are 1000 clients and we set the percentage to 10, then it's probably fine if the actual number of canary clients is anywhere between 80 and 120 (which it very likely will be with any sort of decent hash algorithm). For services with a very small number of clients, service owners will need to carefully set the percentage such that they can be confident that at least one client is selected. For example, if there are 10 clients and we set the percentage to 10, it may not select any clients, but you can set it to 20 or 30 instead to avoid that. However, at this kind of small scale, if you need greater precision, it's probably not that hard to directly coordinate changes to individual clients (i.e., it nullifies some of the advantages of the service config mechanism anyway). > I started a similar discussion on canarying ConfigMaps under Kubernetes. > While this design is the way at least one push mechanism inside Google > works, there's also a more predictable, push-like one (GDP's), where you > pick N candidates, tell them to get a new config and watch them for T > minutes, making sure they don't die. That, of course, assumes you keep > track of the clients, through e.g. grpclb, and can also track their health > (Borg and Kubernetes both do). Would you consider that for future > developments? > The problem is that we have no mechanism for tracking clients that way. Tracking them via grpclb won't work, because clients should be able to use the service config without using grpclb. And more generally, any tracking mechanism like this would require a lot of communication between servers and clients, which would add a lot of complexity. I don't think that the advantage of getting slightly more accurate canary percentages is really worth that complexity. > > >> * TXT records are limitted to ASCII chars. What will happen if the >> method name, programming language, or load balancing policy is not pure >> ascii? >> > > To also tackle the 65K TXT record size limit, there's the old hack used by > a service that ended up with enormous command lines and ran into the Linux > 131072 byte command line limit: compress the whole thing, then encode that > stream in Base64 or similar. > I'd prefer to avoid that if possible, because I think it's valuable for debugging purposes to have the TXT record in human-readable form. However, we can certainly add support for something like this if/when people start running into the length limitation. > > * Are TXT records for a superdomain applicable? For example, if there was >> a SC for foo.bar.com, but not sub.foo.bar.com, does it apply? >> >> >> On Thursday, January 19, 2017 at 11:42:29 AM UTC-8, Mark D. Roth wrote: >>> >>> It's obviously going to have to be a heuristic, since we don't have any >>> way of knowing the full set of clients a priori. I was thinking that we >>> would take a hash of the client's hostname and pid, which unfortunately >>> wouldn't really be that deterministic. But I'd welcome suggestions for a >>> more deterministic algorithm. >>> >>> On Thu, Jan 19, 2017 at 9:18 AM, 'Craig Tiller' via grpc.io < >>> [email protected]> wrote: >>> >>>> How does the percentage field work? >>>> >>>> Do clients roll a die to determine if they're in the canary subset? Or >>>> is there a deterministic way of determining this? >>>> >>>> On Thu, Jan 19, 2017 at 8:57 AM 'Mark D. Roth' via grpc.io < >>>> [email protected]> wrote: >>>> >>>>> I've created a gRFC describing how service configs will be encoded in >>>>> DNS: >>>>> >>>>> https://github.com/grpc/proposal/pull/5 >>>>> >>>>> I'd welcome feedback, especially on the proposed use of TXT records. >>>>> >>>>> Please keep discussion in this thread. Thanks! >>>>> >>>>> -- >>>>> Mark D. Roth <[email protected]> >>>>> Software Engineer >>>>> Google, Inc. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "grpc.io" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/grpc-io/CAJgPXp5VBE%3DBVJq >>>>> 8JKXAdKV%3D3-nrjFDHFd0sTwUW%3DGOr%2B3q6Tw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/grpc-io/CAJgPXp5VBE%3DBVJq8JKXAdKV%3D3-nrjFDHFd0sTwUW%3DGOr%2B3q6Tw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "grpc.io" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/grpc-io. >>>> To view this discussion on the web visit https://groups.google.com/d/ms >>>> gid/grpc-io/CAAvp3oM7boP7X4GeGXmXf0C5pHCBSXZ8_vCF7jFGwceHyjD >>>> bDg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/grpc-io/CAAvp3oM7boP7X4GeGXmXf0C5pHCBSXZ8_vCF7jFGwceHyjDbDg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Mark D. Roth <[email protected]> >>> Software Engineer >>> Google, Inc. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/grpc-io. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/grpc-io/ccc7f85d-3f0d-4f59-a438-cedd6c8dd074%40googlegroups.com >> <https://groups.google.com/d/msgid/grpc-io/ccc7f85d-3f0d-4f59-a438-cedd6c8dd074%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Rudi Chiarito — Infrastructure — Clarifai, Inc. > "Trust me, I know what I'm doing." (Sledge Hammer!) > > -- > You received this message because you are subscribed to the Google Groups " > grpc.io" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/grpc-io. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/grpc-io/CAPzY348RNufro5TR%2BWneYSA%2BBPDVfPcMqt8YPL5XLY2f8UVUtw% > 40mail.gmail.com > <https://groups.google.com/d/msgid/grpc-io/CAPzY348RNufro5TR%2BWneYSA%2BBPDVfPcMqt8YPL5XLY2f8UVUtw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Mark D. Roth <[email protected]> Software Engineer Google, Inc. -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAJgPXp7%2BQOxXHJPXw6UObNGLxDdCvN49%2BuHq%2Bhu7nZLakbaJsg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
