alexeykudinkin commented on code in PR #6017:
URL: https://github.com/apache/hudi/pull/6017#discussion_r956400078
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala:
##########
@@ -162,11 +163,11 @@ object HoodieSparkUtils extends SparkAdapterSupport {
if (rows.isEmpty) {
Iterator.empty
} else {
+ val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
val transform: GenericRecord => GenericRecord =
if (sameSchema) identity
else {
- val readerAvroSchema = new
Schema.Parser().parse(readerAvroSchemaStr)
- rewriteRecord(_, readerAvroSchema)
Review Comment:
BTW, one miss for the new API is that previously `rewriteRecord` was
validating that the record adheres to the new schema while new method doesn't
do that (this obscures the issues when conversion is not following Avro
evolution rules)
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala:
##########
@@ -162,11 +163,11 @@ object HoodieSparkUtils extends SparkAdapterSupport {
if (rows.isEmpty) {
Iterator.empty
} else {
+ val readerAvroSchema = new Schema.Parser().parse(readerAvroSchemaStr)
val transform: GenericRecord => GenericRecord =
if (sameSchema) identity
else {
- val readerAvroSchema = new
Schema.Parser().parse(readerAvroSchemaStr)
- rewriteRecord(_, readerAvroSchema)
Review Comment:
@xiarixiaoyao since we're changing this, shall we also revisit all the other
places that use `rewriteRecord` and consider rebasing them onto the new methods?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
