Anonymitaet commented on code in PR #1056: URL: https://github.com/apache/pulsar-client-go/pull/1056#discussion_r1264787777
########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. Review Comment: ```suggestion Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article focuses on the comparison between Pulsar's Go Client and Java Client in schema compatibility. It focuses on four common schema types: Avro, JSON, Proto, and Proto native schema. ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. Review Comment: ```suggestion Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string JSON, to create schemas. However, Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define Review Comment: ```suggestion // Compatible with defining a schema in Java ``` Do you intend to convey this meaning? ########## README.md: ########## @@ -195,3 +195,9 @@ similar to the following, and then run `go mod tidy`. ``` github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220630195735-e95cf0633348 // indirect ``` +### Schema compatibility with JAVA client +Due to the differences in programming language features and data representation, +schema incompatibility may arise between different languages. +We have conducted an analysis of the schema compatibility between Java and Go, +and we have provided some feasible solutions. Review Comment: ```suggestion The analysis of the schema compatibility between Java and Go has been conducted and the corresponding solutions have been listed below. ``` In technical writing: - Avoid the first-person pronouns I and we - Use the second-person pronoun (you, your, yours, or yourself) as much as possible. The subject of an imperative sentence is understood to be you. Details see https://docs.google.com/document/d/1lc5j4RtuLIzlEYCBo97AC8-U_3Erzs_lxpkDuseU0n4/edit#bookmark=id.gfwms5zehcak ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +Producer := NewAvroSchema(exampleSchemaDef, nil) + +``` +JAVA: +```java +@AllArgsConstructor +@NoArgsConstructor +public static class Example { + public String name; + public int id; +} + +Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)) + .topic(topic).create(); +``` + +#### JSON Schema +The situation with JSON schema is similar to Avro Schema. +```go +// Compatible with Java schema define Review Comment: ```suggestion // Compatible with defining a schema in Java ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. Review Comment: ```suggestion Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article focuses on the comparison between Pulsar's Go Client and Java Client in schema compatibility. It focuses on four common schema types: Avro, JSON, Proto, and Proto native schema. ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +Producer := NewAvroSchema(exampleSchemaDef, nil) + +``` +JAVA: +```java +@AllArgsConstructor +@NoArgsConstructor +public static class Example { + public String name; + public int id; +} + +Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)) + .topic(topic).create(); +``` + +#### JSON Schema +The situation with JSON schema is similar to Avro Schema. +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":[\"null\", \"string\"]}]}" + +consumerJSCompatible := NewJSONSchema(exampleSchemaDefCompatible, nil) +// Not compatible with Java schema define Review Comment: ```suggestion // Not compatible with defining a schema in Java ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +Producer := NewAvroSchema(exampleSchemaDef, nil) + +``` +JAVA: +```java +@AllArgsConstructor +@NoArgsConstructor +public static class Example { + public String name; + public int id; +} + +Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)) + .topic(topic).create(); +``` + +#### JSON Schema +The situation with JSON schema is similar to Avro Schema. +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":[\"null\", \"string\"]}]}" + +consumerJSCompatible := NewJSONSchema(exampleSchemaDefCompatible, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}" + +consumerJSIncompatible := NewJSONSchema(exampleSchemaDefIncompatible, nil) +``` + +To achieve compatibility, modify the `exampleSchemaDefIncompatible` to allow null fields and ensure that the variable names in the Java Example class match the schema definition. + +#### Proto Schema +Proto and ProtoNative schemas exhibit some incompatibility between Go and Java clients. This is because Avro Proto currently does not provide full compatibility between Java and Go. + +```proto +message TestMessage { + string stringField = 1; + int32 intField = 2; +} +``` + +Schema define in java，which can be parsed by a Class. Review Comment: ```suggestion Defining a schema in Java can be parsed by a class. ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +Producer := NewAvroSchema(exampleSchemaDef, nil) + +``` +JAVA: +```java +@AllArgsConstructor +@NoArgsConstructor +public static class Example { + public String name; + public int id; +} + +Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)) + .topic(topic).create(); +``` + +#### JSON Schema +The situation with JSON schema is similar to Avro Schema. +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":[\"null\", \"string\"]}]}" + +consumerJSCompatible := NewJSONSchema(exampleSchemaDefCompatible, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}" + +consumerJSIncompatible := NewJSONSchema(exampleSchemaDefIncompatible, nil) +``` + +To achieve compatibility, modify the `exampleSchemaDefIncompatible` to allow null fields and ensure that the variable names in the Java Example class match the schema definition. + +#### Proto Schema +Proto and ProtoNative schemas exhibit some incompatibility between Go and Java clients. This is because Avro Proto currently does not provide full compatibility between Java and Go. + +```proto +message TestMessage { + string stringField = 1; + int32 intField = 2; +} +``` + +Schema define in java，which can be parsed by a Class. +```json +protoSchemaDef = "{\"type\":\"record\",\"name\":\"TestMessage\",\"namespace\":\"org.apache.pulsar.client.api.schema.proto.Test\",\"fields\":[{\"name\":\"stringField\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"default\":\"\"},{\"name\":\"intField\",\"type\":\"int\",\"default\":0}]}" + +``` + +Schema define in GO, which need to be manually written. +```json +protoSchemaDef = "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"num\",\"type\":\"int\"},{\"name\":\"msf\",\"type\":\"string\"}]}" +``` +To address the incompatibility between Proto and ProtoNative types, you can follow this approach: +1. In the Java client, parse the message using the Avro Proto library to obtain the Schema Definition. Review Comment: ```suggestion 1. In the Java client, parse the message using the Avro Proto library to obtain the schema definition. ``` ########## docs/schema-compatibility.md: ########## @@ -0,0 +1,89 @@ +### Description +Pulsar is a high-performance, persistent message middleware that supports multiple programming languages and data models. This article will focus on the comparison between Pulsar's GO Client and JAVA Client in Schema compatibility. We'll focus on four common schema types: Avro, JSON, Proto, and Proto native schema. +#### Avro Schema +Avro schema in Go and Java are compatible, but there are some differences in how the schemas are defined. Go typically uses schema definitions, a string Json, to create schemas, whereas Java often uses class types for schema creation. As a result, Java allows non-primitive fields to be nullable by default, while in Go's schema definition, the nullability of fields needs to be explicitly stated. +GO: +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["null","string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := NewAvroSchema(`{"fields": + [ + {"name":"id","type":"int"},{"default":null,"name":"name","type":["string"]} + ], + "name":"MyAvro","namespace":"schemaNotFoundTestCase","type":"record"}`, nil) +Producer := NewAvroSchema(exampleSchemaDef, nil) + +``` +JAVA: +```java +@AllArgsConstructor +@NoArgsConstructor +public static class Example { + public String name; + public int id; +} + +Producer<Example> producer = pulsarClient.newProducer(Schema.AVRO(Example.class)) + .topic(topic).create(); +``` + +#### JSON Schema +The situation with JSON schema is similar to Avro Schema. +```go +// Compatible with Java schema define +exampleSchemaDefCompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":[\"null\", \"string\"]}]}" + +consumerJSCompatible := NewJSONSchema(exampleSchemaDefCompatible, nil) +// Not compatible with Java schema define +exampleSchemaDefIncompatible := "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," + + "\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}" + +consumerJSIncompatible := NewJSONSchema(exampleSchemaDefIncompatible, nil) +``` + +To achieve compatibility, modify the `exampleSchemaDefIncompatible` to allow null fields and ensure that the variable names in the Java Example class match the schema definition. + +#### Proto Schema +Proto and ProtoNative schemas exhibit some incompatibility between Go and Java clients. This is because Avro Proto currently does not provide full compatibility between Java and Go. + +```proto +message TestMessage { + string stringField = 1; + int32 intField = 2; +} +``` + +Schema define in java，which can be parsed by a Class. +```json +protoSchemaDef = "{\"type\":\"record\",\"name\":\"TestMessage\",\"namespace\":\"org.apache.pulsar.client.api.schema.proto.Test\",\"fields\":[{\"name\":\"stringField\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"default\":\"\"},{\"name\":\"intField\",\"type\":\"int\",\"default\":0}]}" + +``` + +Schema define in GO, which need to be manually written. Review Comment: ```suggestion Defining a schema in Go needs to write manually. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
