Hi Chet,
TL;DR: Move on, send pull requests, iterate. We will probably crush it into pieces during review a few times :-), but it will be worth it. LONG VERSION (read to the end, don't stop in between) > I think you're incorrect. Maybe. We all are. I was just asking, haven't read the whole conversation. > (1) configuration data is -just- another form of data. And executable programs are just another. Now what does that tell us? Should we write binaries with Thrift instead of PE or ELF? Could it be that that will be slightly less efficient? (but hey, it's human readable!). > The reason I originally preferred Thrift to GRPC -- the reason I stick > with > Thrift, is that Thrift is -modular-. It allows me to install my own > transport, my own protocol. But actually, I can only install my own > protocol *within* *limits*. I MUST stick with field-ids as identifiers, > and -cannot- use field-names. And the fix for this is ... so trivial. > And would affect the peformance of other protocols ... not at all. Then do it. > (2) I'm not the first person to note that the Thrift JSON protocol is > horrendous, Sure. I agree with that. I would not call it "horrendous" though, but yes, there's room for improvement. For example, I never understood why the field types have to be three-letter codes. Why do't we write the numeric types as we do with other formats? > and what's really needed is some sort of Thrift JSON protocol > that looks like idiomatic JSON. Here I tend to disagree. As I see it, we already have two JSON fomats, but ... - the efficient one could still be better (i.e. even less bloated) - the readable one could be more ... idiomatic, if we stick to that term. More on that below. But these would still need to be two formats, plus at least one (the current TSimple) for comaptibility reasons. I see no way how we can have best efficiency and human readability in one. These are contradicting goals, so we still need two kinds of JSON. > that the JSON protocol that emerges, would be idiomatic JSON. > This is not without value. Added value is asking "what for?". For performance? For human readability? Save the whales? World peace? And what do I have to sacrifice while doing that? Without the "... for XYZ" part any sentence of "added value" becomes just meaningless. And the use of "idiomatic" does not make it any better to me. > GRPC/Proto3 is a "we know how you should use RPC systems" answer. It > -prescribes- answers to every question you might ask, and if your answer > is > different, well sucks to be you. THIS IS BAD. Yeah, that's Googles approach. Take it or leave it. If you do not like it, you just have the wrong problems. > Look: again, I'm a rabid partisan of Thrift, and think that GRPC is a > massive mistake. > This doesn't mean that I'm blind to the good ideas that Google came up > with. You're welcome, you came to the right project. Honestly. > Thrift is -only- and -ever- and -forever- and RPC system. It is never > going to be for de/marshalling data-at-rest. It is never going to > -attempt- to provide (at a performance cost, surely) a universal type > system for data. No, that's not what Thrift is about. > > Do you really want to sign up for that? Do we need PDF generation capabilities in grep? Thrift is a small, but open framework. It does one (actually two) things and it does it well. I like that approach, maybe because I'm a big fan of the KISS principle. That does not mean that I'm not open for improvements, and we surely have plenty of room for that. For example, with our release process, to name just one. Have fun, JensG -----Ursprüngliche Nachricht----- From: Chet Murthy Sent: Tuesday, October 3, 2017 2:47 AM To: [email protected] Subject: Re: Human-readable wire-format for Thrift? Jens, I think you're incorrect. Please let me try to convince you. (1) configuration data is -just- another form of data. In many, many systems, there is special code that reads config-files in whatever format was chosen (YAML? INI?) and transforms it into custom application-specific data-structures. This code has to be maintained and tested. It is a form of -demarshalling- code. And then when you get into multi-language systems, you have to write that code more-than-once. Maintain it as your config changes. Oh, groan. You may say that Thrift is the -wrong- tool for this job. But (2) I'm not the first person to note that the Thrift JSON protocol is horrendous, and what's really needed is some sort of Thrift JSON protocol that looks like idiomatic JSON. -An- advantage of what I propose, is that the JSON protocol that emerges, would be idiomatic JSON. This is not without value. (3) The change I'm suggesting is both minor, and adds function that your -most- -popular- competitor already supports. Look: please, please understand me. I HAVE GRAVE RESERVATIONS ABOUT PROTO3. Especially in Golang. Holey moley. GRPC/Proto3 is a "we know how you should use RPC systems" answer. It -prescribes- answers to every question you might ask, and if your answer is different, well sucks to be you. THIS IS BAD. The reason I originally preferred Thrift to GRPC -- the reason I stick with Thrift, is that Thrift is -modular-. It allows me to install my own transport, my own protocol. But actually, I can only install my own protocol *within* *limits*. I MUST stick with field-ids as identifiers, and -cannot- use field-names. And the fix for this is ... so trivial. And would affect the peformance of other protocols ... not at all. Look: I can see why one might not want to make this change: it breaks source-code compatibility (though that can be mitigated). There is only one other argument I can see. And that is this: Thrift is -only- and -ever- and -forever- and RPC system. It is never going to be for de/marshalling data-at-rest. It is never going to -attempt- to provide (at a performance cost, surely) a universal type system for data. No, that's not what Thrift is about. Do you really want to sign up for that? --chet-- P.S. Look: again, I'm a rabid partisan of Thrift, and think that GRPC is a massive mistake. This doesn't mean that I'm blind to the good ideas that Google came up with. On Mon, Oct 2, 2017 at 2:40 PM, Jens Geyer <[email protected]> wrote: > > Have you ever thought of using the wrong tool for the job? > > > -----Ursprüngliche Nachricht----- > From: Chet Murthy > Sent: Monday, October 2, 2017 9:54 PM > To: [email protected] > Subject: Re: Human-readable wire-format for Thrift? > > Randy, > > There is a different way that one could allow for Thrift IDL metadata to > be > accessible at protocol-time (which is really what I'm proposing): one > could > pass the entire metadata information the protocol constructor. Pros & > Cons: > > (1) for the config-file use-case I described, this is enough. > > (2) But this is NOT enough for the "human-readable wireline for Thrift > servers". I've noted that several times, people have asked for a JSON > wire-format that they could use to invoke Thrift servers, that is .... > well, more "normal JSON". And this is not advanced by passing the IDL > metadata to the T<NiceJSON>Protocol constructor, b/c that constructor is > invoked from the TProtocolFactory, and there's structure in place for > figuring out the right metadata to pass. > > BUT (3) one COULD imagine a new Thrift RPC stack, which would be > initialized with IDL metadata, and which would pass the right metadata > down > to the Protocol instances at creation time. > > This seems like much more work than the small change I'm proposiing. > > I -DO- see that the biggest con of what I'm proposing, is that it MIGHT > break source-code compatibility for all previously-generated Thrift code. > I think this is remediable in the following way: > > (a) add a new method readFieldBegin2(....) where the metadata is passed > as a (possibly NULL) pointer > > (b) old generated code continues to call readFieldBegin() > > (c) old protocol impls merely define readFieldBegin2() to call > readFieldBegin() > > (d) new generated code calls readFieldBegin2 (with that metadata) > > (e) and new protocols like TNiceJSONProtocol and TCompactTextProtocol > would implement readFieldBegin2 to do what I described in my first note, > while implementing readFieldBegin() (the old method) to panic with an > error-message. > > Uh ... this seems to solve the source-code compatibility issue. I don't > have a good understanding of what level of binary-code compatibility > Thrift > promises, so I can't comment on that. > > Thoughts? > --chet-- > >
