Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2299#discussion_r188624043
--- Diff:
core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletDataMapFactory.java
---
@@ -115,13 +123,55 @@ public DataMapBuilder createBuilder(Segment segment,
String shardName) {
Set<TableBlockIndexUniqueIdentifier> tableBlockIndexUniqueIdentifiers =
segmentMap.get(segment.getSegmentNo());
if (tableBlockIndexUniqueIdentifiers == null) {
+ CarbonTable carbonTable = this.getCarbonTable();
+ if (!carbonTable.getTableInfo().isTransactionalTable()) {
+ // For NonTransactional table, compare the schema of all index
files with inferred schema.
+ // If there is a mismatch throw exception. As all files must be of
same schema.
+ validateSchemaForNewTranscationalTableFiles(segment, carbonTable);
+ }
tableBlockIndexUniqueIdentifiers =
BlockletDataMapUtil.getTableBlockUniqueIdentifiers(segment);
segmentMap.put(segment.getSegmentNo(),
tableBlockIndexUniqueIdentifiers);
}
return tableBlockIndexUniqueIdentifiers;
}
+ private void validateSchemaForNewTranscationalTableFiles(Segment
segment, CarbonTable carbonTable)
+ throws IOException {
+ SchemaConverter schemaConverter = new
ThriftWrapperSchemaConverterImpl();
+ Map<String, String> indexFiles = segment.getCommittedIndexFile();
+ for (Map.Entry<String, String> indexFileEntry : indexFiles.entrySet())
{
+ Path indexFile = new Path(indexFileEntry.getKey());
--- End diff --
In below logs, we need to print just the index file name. Hence path object
getName() is used.
---
