dixingxing0 commented on a change in pull request #2053:
URL: https://github.com/apache/iceberg/pull/2053#discussion_r553901117
##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java
##########
@@ -82,10 +82,17 @@ public void initialize(@Nullable Configuration
configuration, Properties serDePr
}
String[] selectedColumns =
ColumnProjectionUtils.getReadColumnNames(configuration);
- Schema projectedSchema = selectedColumns.length > 0 ?
tableSchema.select(selectedColumns) : tableSchema;
+ Schema projectedSchema = tableSchema;
+
+ boolean caseSensitive =
configuration.getBoolean(InputFormatConfig.CASE_SENSITIVE,
Review comment:
Agree with you, i'll make here go with `false`, i don't have a specific
use-case.
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/serde/objectinspector/IcebergRecordObjectInspector.java
##########
@@ -35,21 +35,30 @@
public final class IcebergRecordObjectInspector extends StructObjectInspector {
private static final IcebergRecordObjectInspector EMPTY =
- new IcebergRecordObjectInspector(Types.StructType.of(),
Collections.emptyList());
+ new IcebergRecordObjectInspector(Types.StructType.of(),
Collections.emptyList(), true);
private final List<IcebergRecordStructField> structFields;
+ private final List<IcebergRecordStructField> structFieldsInLowercase;
Review comment:
You are right!
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/serde/objectinspector/IcebergRecordObjectInspector.java
##########
@@ -35,21 +35,30 @@
public final class IcebergRecordObjectInspector extends StructObjectInspector {
private static final IcebergRecordObjectInspector EMPTY =
- new IcebergRecordObjectInspector(Types.StructType.of(),
Collections.emptyList());
+ new IcebergRecordObjectInspector(Types.StructType.of(),
Collections.emptyList(), true);
private final List<IcebergRecordStructField> structFields;
+ private final List<IcebergRecordStructField> structFieldsInLowercase;
Review comment:
You are right! I'll change it.
##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java
##########
@@ -82,10 +82,17 @@ public void initialize(@Nullable Configuration
configuration, Properties serDePr
}
String[] selectedColumns =
ColumnProjectionUtils.getReadColumnNames(configuration);
- Schema projectedSchema = selectedColumns.length > 0 ?
tableSchema.select(selectedColumns) : tableSchema;
+ Schema projectedSchema = tableSchema;
+
+ boolean caseSensitive =
configuration.getBoolean(InputFormatConfig.CASE_SENSITIVE,
Review comment:
Agree with you, i'll make here go with `false`, i don't have a specific
use-case.
Since we make hive go with `false`, i think iceberg.mr.case.sensitive should
use `true` as a default value, is that ok?
##########
File path:
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -150,6 +150,27 @@ public void testScanTable() throws IOException {
Assert.assertArrayEquals(new Object[] {"Alice", 0L}, descRows.get(2));
}
+ @Test
Review comment:
OK, i'll remove the test code in
TestHiveIcebergStorageHandlerLocalScan.java
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/serde/objectinspector/IcebergRecordObjectInspector.java
##########
@@ -48,7 +49,15 @@ public IcebergRecordObjectInspector(Types.StructType
structType, List<ObjectInsp
for (Types.NestedField field : structType.fields()) {
ObjectInspector oi = objectInspectors.get(position);
- IcebergRecordStructField structField = new
IcebergRecordStructField(field, oi, position);
+
+ IcebergRecordStructField structField;
+ if (caseSensitive) {
+ structField = new IcebergRecordStructField(field, oi, position);
+ } else {
+ Types.NestedField fieldInLowercase =
Types.NestedField.of(field.fieldId(), field.isOptional(),
+ field.name().toLowerCase(), field.type(), field.doc());
+ structField = new IcebergRecordStructField(fieldInLowercase, oi,
position);
+ }
Review comment:
Got it!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
