[GitHub] [pulsar] eolivelli commented on pull request #9895: Implement Schema#OBJECT - support for Consumer

Sun, 14 Mar 2021 08:09:25 -0700


eolivelli commented on pull request #9895:
URL: https://github.com/apache/pulsar/pull/9895#issuecomment-798923804


   @codelipenghui 
   if you see this feature from a "new user"'s point of view, you will find 
this:
   
   ```
   Consumer<Object> consumer = pulsarClient.newConsumer(Schema.OBJECT())
                   .subscriptionName("test")
                   .topic(topic)
                   .subscribe();
           Message<Object> message = consumer.receive();
           Object value = message.getValue();
           
   ```
   
   that is really cool ! and it is self-explaining.
   
   > why GenericRecord can't deal with the primary types?
   
   Problems with GenericRecord approach:
   1) we are adding a new wrapper object, and this is a waste of resources
   2) A Record is a structure with "fields", but primitive types, lists and 
also KeyValue does not have "fields"
   3) it is not intuitive that a String is a GenericRecord without "fields"
   
   > In my opinion, the OBJECT schema will bring more complexity to the Pulsar 
Schema, 2 schema types want to deal with the same problem.
   
   I believe that AUTO_CONSUME will still stay for the case that you know that 
you are dealing with Structures like AVRO, JSON, Protobuf, but if you want to 
write "generic" code, at least in Java, you really would like to use "Object" 
and not a custom Pulsar interface.
   So for new users I imagine that their eyes will fall into "Object" as first 
chance, and they will discover "AUTO_CONSUME" only if they are interested in 
dealing directly with GenericRecord
   
   In Avro itself you cannot map a primitive value or an array type to 
GenericRecord, see:
   
https://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericRecord.html
   
   If we really want a generic container I would add a new interface, like 
GenericObject o ObjectWithSchema
   ```
   interface ObjectWithSchema {
       Object asObject();
       Schema getSchema();
   }
   ```
   By the way we cannot retrofit AUTO_CONSUME to Schema<ObjectWithSchema> as it 
is not a compatible change.
   As we need to add a new "name" to the API (Schema.XXXX) let be this new name 
"OBJECT", that is very simple for Java users to understand
   
   
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to