satishkotha commented on a change in pull request #2453:
URL: https://github.com/apache/hudi/pull/2453#discussion_r559030870
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/config/SerializableSchema.java
##########
@@ -62,12 +62,17 @@ private void readObject(ObjectInputStream in) throws
IOException {
// create a public write method for unit test
public void writeObjectTo(ObjectOutputStream out) throws IOException {
- out.writeUTF(schema.toString());
+ // Note: writeUTF cannot support string length > 64K. So use writeObject
which has small overhead (relatively).
+ out.writeObject(schema.toString());
}
// create a public read method for unit test
public void readObjectFrom(ObjectInputStream in) throws IOException {
- schema = new Schema.Parser().parse(in.readUTF());
+ try {
+ schema = new Schema.Parser().parse(in.readObject().toString());
Review comment:
This was initial implementation. Unfortunately, it doesnt well with java
serialization because we have 'out.defaultWriteObject();' in front. Per
Vinoth's suggestion I used writeUTF, see
https://issues.apache.org/jira/browse/HUDI-1443 for some context.
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/config/SerializableSchema.java
##########
@@ -62,12 +62,17 @@ private void readObject(ObjectInputStream in) throws
IOException {
// create a public write method for unit test
public void writeObjectTo(ObjectOutputStream out) throws IOException {
- out.writeUTF(schema.toString());
+ // Note: writeUTF cannot support string length > 64K. So use writeObject
which has small overhead (relatively).
+ out.writeObject(schema.toString());
Review comment:
This was initial implementation. Unfortunately, it doesnt well with java
serialization because we have 'out.defaultWriteObject();' in front. Per
Vinoth's suggestion I used writeUTF, see
https://issues.apache.org/jira/browse/HUDI-1443 for some context.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
