Hi Fredrik, There is no need to parse the documentation-- the Kafka protocol now has a protocol specification here:
https://github.com/apache/kafka/tree/trunk/clients/src/main/resources/common/message One caveat is that there are no protocol specifications for the message format yet. best, Colin On Mon, Mar 18, 2019, at 11:49, Fredrik Arvidsson wrote: > Hi, > > I'm automating a low level C# kafka protocol definition by parsing the > Kafka protocol guide (inkl the Record Batch documentation), and I've > found some inconsistencies in the protocol guide: > > 1. Common Request and Response Structure > > RequestOrResponse => Size (RequestMessage | ResponseMessage) > Size => int32 > > FIELD DESCRIPTION > message_size The message_size field gives the size of the subsequent > request or response message in bytes. The client can read requests by > first reading this 4 byte size as an integer N, and then reading and > parsing the subsequent N bytes of the request. > > * int32 does not follow the primitive type name INT32 > > * RequestMessage and ResponseMessage is never defined > > * message_size is not an expression. Correct name should be Size > > 2. Inconsistent usage of ARRAY expressions. > > Sometimes an array is expressed as a field surrounded with brackets, > i.e. [topics] and sometimes it is expresses as ARRAY(INT32). Brackets in > BNF usually indicates that the term is optional, which makes the usage > of brackets in the context of an array confusing. Parenthesis is a > grouping expression, so that is also confusing. Why not skip the use of > an ARRAY primitive type and instead use a repetition expression in BNF, > like the following example? > > |Metadata Request (Version: 1) => topics topics => INT32 topic* topic => > STRING | > > 3. Record batch documentation > > Would it not be nice to use the BNFÂ format here as well? It is being > used under the "5.3.3 Old Message Format" but not for "5.3.1 Record > Batch" and "5.3.2 Record". > > Best regards > > //Fredrik > >
