mcvsubbu commented on a change in pull request #4954: Support schema evolution
for consuming segments
URL: https://github.com/apache/incubator-pinot/pull/4954#discussion_r368326346
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/indexsegment/mutable/MutableSegmentImpl.java
##########
@@ -467,13 +483,22 @@ public SegmentMetadata getSegmentMetadata() {
return physicalColumnNames;
}
+ @Override
+ public Set<String> getColumnNamesForSelectStar() {
+ return Sets.union(getPhysicalColumnNames(),
_newlyAddedColumnsFieldMap.keySet());
+ }
+
@Override
public ColumnDataSource getDataSource(String columnName) {
FieldSpec fieldSpec = _schema.getFieldSpecFor(columnName);
- if (fieldSpec.isVirtualColumn()) {
+ if ((fieldSpec == null &&
_newlyAddedColumnsFieldMap.containsKey(columnName)) ||
fieldSpec.isVirtualColumn()) {
Review comment:
We should have a different column provider here that should be optimized
better than the virtual column provider. The new column provider should not be
built every time, but should stay in memory once built. What may be updated is
the number of rows. That will minimize additional allocation during query
processing.
Add a new column provider factory (say, ConstantValueColumnProviderFactory)
that has a map from the col name to the provider. The map is built lazily as
columns are added to the query.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
