danny0405 commented on code in PR #7355:
URL: https://github.com/apache/hudi/pull/7355#discussion_r1185655172
##########
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.java:
##########
@@ -103,10 +104,21 @@ void addProjectionToJobConf(final RealtimeSplit
realtimeSplit, final JobConf job
// For e:g _hoodie_record_key would be missing and merge step would
throw exceptions.
// TO fix this, hoodie columns are appended late at the time
record-reader gets built instead of construction
// time.
+ List<String> fieldsToAdd = new ArrayList<>();
if (!realtimeSplit.getDeltaLogPaths().isEmpty()) {
-
HoodieRealtimeInputFormatUtils.addRequiredProjectionFields(jobConf,
realtimeSplit.getVirtualKeyInfo(),
- StringUtils.isNullOrEmpty(preCombineKey) ? Option.empty() :
Option.of(preCombineKey));
+
HoodieRealtimeInputFormatUtils.addRequiredProjectionFields(jobConf,
realtimeSplit.getVirtualKeyInfo());
+ String preCombineKey = tableConfig.getPreCombineField();
+ if (!StringUtils.isNullOrEmpty(preCombineKey)) {
Review Comment:
Is it okay to miss the check
```java
String preCombineKey = preCombineKeyOpt.get();
List<String> columnNameList =
Arrays.stream(configuration.get(serdeConstants.LIST_COLUMNS).split(",")).collect(Collectors.toList());
int pos = columnNameList.indexOf(preCombineKey);
if (pos != -1) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
