sijie commented on issue #3741: POJO AvroSchema always allowNull
URL: https://github.com/apache/pulsar/issues/3741#issuecomment-469563201
 
 
   @jerrypeng  correct. I added one more to complete the picture.
   
   ```
           log.info("Pulsar AllowNull                  :{}", new 
String(AvroSchema.of(User.class).getSchemaInfo().getSchema()));
           log.info("Pulsar shaded AVRO NOT AllowNull  :{}", 
org.apache.pulsar.shade.org.apache.avro.reflect.ReflectData.get().getSchema(User.class));
           log.info("Avro AllowNull                    :{}", 
ReflectData.AllowNull.get().getSchema(User.class));
           log.info("Avro Not AllowNull                :{}", 
ReflectData.get().getSchema(User.class));
   ```
   
   ```
   Pulsar AllowNull                  
:{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":["null","string"],"default":null},{"name":"favorite_number","type":["null","int"],"default":null},{"name":"favorite_color","type":["null","string"],"default":null},{"name":"age","type":"int"}]}
   Pulsar shaded AVRO NOT AllowNull  
:{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":"int"},{"name":"favorite_color","type":"string"},{"name":"age","type":"int"}]}
   Avro AllowNull                    
:{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}
   Avro Not AllowNull                
:{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}
   ```
   
   As you described above, AllowNull doesn't matter if the POJO is generated by 
AVRO, because AVRO will try to use `SCHEMA$` field to retrieve the schema. But 
If the POJO is not generated by AVRO, AVRO will attempt to parse it and 
AllowNull will generating two different incompatible schemas.
   
   my tests avoid covered all the cases.
   
   1) `Avro AllowNull` and `Avro Not AllowNull` are using unshaded avro library 
to parse avro generated POJO. in this way, they will always produce the same 
schema no matter `AllowNull` used or not. 
   
   2) `Pulsar AllowNull` and `Pulsar shade avro Not Allow Null` are using 
shaded avro library to parse avro generated POJO. In this way, they will 
produce a completely different schema. `AllowNull` matters here. 
   
   The 2) indicates two problems:
   
   a) for normal POJO (not generated by AVRO), `AllowNull` is producing 
incompatible schemas. so we need #3752 
   
   b) for avro generated POJO, shaded AVRO library (the one used by Pulsar) 
doesn't recognize it is an AVRO generated POJO. We need a separate fix to make 
sure Pulsar shaded AVRO working with avro generated POJO.
   
   
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to