"In practice, it is very rare for schema evolution to change the order of the fields." - I'll say. Since we are talking about a protocol that is deliberately not self-describing we cannot just pluck out what we want - how would such code get to it? This is why the standard advice in these situations is to never reorder, remove or rename fields and to always add new stuff to the end.
On Fri, 12 Jan 2024 at 13:19, Thiruvalluvan MG <thiru...@yahoo.com.invalid> wrote: > Out-of-order fields are not handled transparently in C++ if you are > manually using the resolving decoder. (It's the same situation in Java as > well). > But, in C++ and in Java, if you generate code for the given Avro schema, > the generated code takes care of the field ordering issue. Similarly, in > both bindings, Generic data structures work properly with the field-order. > If you are using the resolving decoder in your code directly, care must be > exercises, If the reader schema and writer schema are both records and they > have fields in different order (it is okay to insert or remove fields), the > protocol is to first get the field-order array (which is essentially a > permutation of the reader field ids 0 to n-1) from the resolving decoder > and then read the fields of the reader schema in the order specified in the > field-order array. This is done in order to avoid buffering by the decoder. > Buffering can take a large number of allocations if the out-of-order fields > is an array or map. > In practice, it is very rare for schema evolution to change the order of > the fields. > Thanks > Thiru On Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov < > mgrigo...@apache.org> wrote: > > Hi Vivek, > > I am not sure there is anyone to give you an exact answer. The C++ SDK has > not been actively developed in the last few years. > The best is to try it for your use cases and see if it works or not. The > next step is to contribute Pull Requests for the missing functionalities! > > Martin > > On Thu, Jan 11, 2024 at 8:59 AM Vivek Kumar < > vivek.ku...@eclipsetrading.com.invalid> wrote: > > > +dev > > > > > > Regards, > > Vivek Kumar > > > > [http://www.eclipsetrading.com/logo.png] > > > > Senior Software Developer > > 23/F One Hennessy > > 1 Hennessy Road > > Wan Chai > > Hong Kong > > www.eclipsetrading.com<http://www.eclipsetrading.com/> > > +852 2108 7352 > > > > Follow us today on our online platforms > > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]< > > https://www.linkedin.com/company/eclipse-trading>[Instagram]< > > https://www.instagram.com/eclipsetrading> > > ________________________________ > > From: Vivek Kumar <vivek.ku...@eclipsetrading.com> > > Sent: Thursday, January 11, 2024 11:07 AM > > To: user@avro.apache.org <user@avro.apache.org> > > Subject: Avro schema evolution support in AVRO CPP > > > > Hi Avro team, > > > > I am writing this email to check the support of Avro schema evolution in > > CPP - i.e. provide both the producer and consumer schema when decoding > the > > data. > > > > I can see that there's a resolvingDecoder function in AVRO CPP that takes > > two schemas. See > > > > > https://avro.apache.org/docs/1.10.2/api/cpp/html/index.html#ReadingDifferentSchema > > > > But there's a FIXME comment in this function. See > > https://issues.apache.org/jira/browse/AVRO-3720 and > > > https://github.com/apache/avro/blob/main/lang/c%2B%2B/api/Decoder.hh#L218. > > Does this mean resolvingDecoder does not work properly? Could you please > > explain what scenarios are not covered by resolvingDecoder and how can we > > use it to support "Avro Schema Evolution" in c++? > > > > Thanks > > > > > > Regards, > > Vivek Kumar > > > > [http://www.eclipsetrading.com/logo.png] > > > > Senior Software Developer > > 23/F One Hennessy > > 1 Hennessy Road > > Wan Chai > > Hong Kong > > www.eclipsetrading.com<http://www.eclipsetrading.com/> > > +852 2108 7352 > > > > Follow us today on our online platforms > > [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]< > > https://www.linkedin.com/company/eclipse-trading>[Instagram]< > > https://www.instagram.com/eclipsetrading> > > > -- Regards, Andrew Marlow http://www.andrewpetermarlow.co.uk
