vvysotskyi commented on a change in pull request #1819: DRILL-7313: Use Hive
schema for MaprDB native reader when field was empty
URL: https://github.com/apache/drill/pull/1819#discussion_r300435181
##########
File path:
contrib/storage-hive/core/scrMapr/main/java/org/apache/drill/exec/planner/sql/logical/ConvertHiveMapRDBJsonScanToDrillMapRDBJsonScan.java
##########
@@ -114,23 +123,35 @@ Hive does not always give correct costing (i.e. for
external tables Hive does no
/**
* Helper method which creates a DrillScanRel with native Drill HiveScan.
*/
- private DrillScanRel createNativeScanRel(DrillScanRel hiveScanRel,
PlannerSettings settings) {
+ private DrillScanRel createNativeScanRel(DrillScanRel hiveScanRel,
PlannerSettings settings) throws IOException {
RelDataTypeFactory typeFactory = hiveScanRel.getCluster().getTypeFactory();
HiveScan hiveScan = (HiveScan) hiveScanRel.getGroupScan();
- Map<String, String> parameters =
hiveScan.getHiveReadEntry().getHiveTableWrapper().getParameters();
+ HiveReadEntry hiveReadEntry = hiveScan.getHiveReadEntry();
+ Map<String, String> parameters =
hiveReadEntry.getHiveTableWrapper().getParameters();
JsonScanSpec scanSpec = new
JsonScanSpec(parameters.get(MAPRDB_TABLE_NAME), null, null);
List<SchemaPath> hiveScanCols = hiveScanRel.getColumns().stream()
.map(colNameSchemaPath -> replaceOverriddenSchemaPath(parameters,
colNameSchemaPath))
.collect(Collectors.toList());
+
+ HiveToRelDataTypeConverter dataTypeConverter = new
HiveToRelDataTypeConverter(typeFactory);
+ TupleSchema schema = new TupleSchema();
+ List<FieldSchema> tableSchemaColumns =
hiveReadEntry.getTable().getColumnListsCache().getTableSchemaColumns();
+ tableSchemaColumns.forEach(column ->
Review comment:
Updated code to handle correctly Drill maps (hive structs) and throw an
exception for union and hive map types.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
