voonhous commented on code in PR #19463:
URL: https://github.com/apache/hudi/pull/19463#discussion_r3889397271
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java:
##########
@@ -119,6 +119,10 @@ private static boolean checkIfHudiTable(final InputSplit
split, final JobConf jo
public RecordReader<NullWritable, ArrayWritable> getRecordReader(final
InputSplit split, final JobConf job,
final
Reporter reporter) throws IOException {
HoodieRealtimeInputFormatUtils.addProjectionField(job,
job.get(hive_metastoreConstants.META_TABLE_PARTITION_COLUMNS, "").split("/"));
+ // The bootstrap and schema-evolution paths below parse the read-column
ids with Integer#parseInt, so the
+ // blank ids HIVE-22438 leaves in the conf have to be dropped here too;
neither path goes through a
+ // realtime input format. The call is idempotent, so the realtime formats'
own call becomes a no-op.
+ HoodieRealtimeInputFormatUtils.cleanProjectionColumnIds(job);
Review Comment:
Correcting my note above: I did get this suite to run, after rebasing onto
master.
`TestHiveTableSchemaEvolution` passes 5/5 with the change, and with
`hudi-hadoop-mr/src/main` reverted to the base 2 of the 5 fail -- both arms of
`testHiveReadSchemaEvolutionTable` -- with `NumberFormatException: For input
string: ""`. So this arm discriminates the new `cleanProjectionColumnIds` call
rather than only executing it.
The `NoClassDefFoundError` I hit earlier was local `.m2` skew, not the
suite. The bootstrap arm is still uncovered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
