Hi David, That is a good point that we should clarify. I think we should not commit to guaranteeing the names or the format of the particular request and response schemas themselves, though we should guarantee that they are parseable as JSON. The pre-existing trace logging did not guarantee this either and it was more difficult to parse as it was not well structured. I believe for most structured logging use cases where we are analyzing cluster behavior this should be sufficient.
Thanks, Lucas On Thu, Oct 29, 2020 at 3:47 AM David Jacot <dja...@confluent.io> wrote: > Hi folks, > > I have looked at Anastasia's PR to implement this KIP and I was wondering > how far we want to go with the backward compatibility of this in the > future. > Now that we rely on the auto-generated protocol, the outputted requests and > responses use the name of the fields defined in the schema. Until now, we > have been renaming fields rather easily in the schemas as they were purely > internal. With this KIP, renaming a field will break the structured request > log. > > Does this imply that we will now consider the schemas as part of our public > API? We don't discuss this point in the KIP so it is subject to > interpretation. > > I think that we should be clear on that point and either commit to only > making > the request log parsable while not guaranteeing the format of the requests > and > the responses; or commit to making the schemas part of our public API. > > Personally, I lean towards the former at the moment. That is > probably sufficient > for the targeted use cases. Breaking changes are annoying but as this is > intended to be used for debugging purposes, that may be OK. > > What do you guys think? > > Best, > David > > > On Thu, Oct 8, 2020 at 7:19 PM Anastasia Vela <av...@confluent.io> wrote: > > > Thanks everyone for the votes. In summary, the vote passed and the KIP > was > > accepted with 3 binding and 3 non-binding +1s. > > > > Best, > > Anastasia > > > > On Mon, Oct 5, 2020 at 2:50 PM Jason Gustafson <ja...@confluent.io> > wrote: > > > > > +1 Thanks for the KIP! > > > > > > On Fri, Oct 2, 2020 at 4:37 PM Colin McCabe <cmcc...@apache.org> > wrote: > > > > > > > Thanks, Anastasia! This will be a lot easier to maintain. > > > > > > > > +1 (binding) > > > > > > > > best, > > > > Colin > > > > > > > > On Wed, Sep 30, 2020, at 23:57, David Jacot wrote: > > > > > Thanks for the KIP, Anastasia. > > > > > > > > > > +1 (non-binding) > > > > > > > > > > On Thu, Oct 1, 2020 at 8:06 AM Tom Bentley <tbent...@redhat.com> > > > wrote: > > > > > > > > > > > Thanks Anastasia, > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > > > On Thu, Oct 1, 2020 at 6:30 AM Gwen Shapira <g...@confluent.io> > > > wrote: > > > > > > > > > > > > > Thank you, this will be quite helpful. > > > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > > > On Wed, Sep 30, 2020 at 11:19 AM Anastasia Vela < > > > av...@confluent.io> > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > Thanks again for the discussion. I'd like to start the vote > for > > > > this > > > > > > KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-673%3A+Emit+JSONs+with+new+auto-generated+schema > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Anastasia > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Gwen Shapira > > > > > > > Engineering Manager | Confluent > > > > > > > 650.450.2760 | @gwenshap > > > > > > > Follow us: Twitter | blog > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
