[GitHub] [pulsar] eolivelli opened a new pull request #10034: Pulsar IO: Allow develop Sinks that support Schema but without setting it at build time (Sink)

Wed, 24 Mar 2021 08:42:34 -0700


eolivelli opened a new pull request #10034:
URL: https://github.com/apache/pulsar/pull/10034


   ### Motivation
   
   Support users that want to implement a Sink that is aware of the Schema but 
it is not tied to a specific schema type at compile time.
   
   This patch is based on #9895 that implements support for Schema<Object>, 
using the same code of AutoConsumeSchema, that is able to dynamically download 
the schema (and new versions of the schema).
   
   With this patch you can implement a Sink that is aware of the Schema but it 
is not tied to a particular schema type at compile time.
   
   ```
   public class ObjectSink implements Sink<Object> {
   
       @Override
       public void write(Record<Object> record) throws Exception {
           try {
               Schema<?> schema = record.getSchema();
               SchemaType type = schema.getSchemaInfo().getType();
               Object value = record.getValue();
               log.info("Record {} type {} schema {} value {}", record, type, 
schema, value);
               record.ack();
           } catch (Throwable t) {
               .....
           }
       }
   ```
   
   
   ### Modifications
   
   Based on #9895. 
   We are adding support in TopicSchema.java  to deal with java.lang.Object and 
set Schema.OBJECT().
   
   ### Verifying this change
   
   I will add test cases and integration tests as soon as the community accepts 
this feature.
   
   ### Does this pull request potentially affect one of the following parts:
   
   With this feature Sink developers will be able to write advanced Sinks
   
   ### Documentation
   
     - Does this pull request introduce a new feature? yes
     - If yes, how is the feature documented? docs
   
   I will be happy to provide documentation as soon as this feature lands to 
Pulasr


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to