paul-rogers commented on a change in pull request #2026: DRILL-7330: Implement
metadata usage for all format plugins
URL: https://github.com/apache/drill/pull/2026#discussion_r392608447
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/file/FileMetadataColumnDefn.java
##########
@@ -53,7 +53,7 @@ public String toString() {
public MajorType dataType() {
return MajorType.newBuilder()
.setMinorType(MinorType.VARCHAR)
- .setMode(DataMode.REQUIRED)
+ .setMode(defn.isOptional() ? DataMode.OPTIONAL : DataMode.REQUIRED)
Review comment:
Does the `isOptional()` apply per-file or per-query? That is, if you've
added a new implicit column `foo`, will `foo` be set on some files but not
others? Or, is it either available if using metadata, but not available without
metadata?
If the value is per-file optional, then using an `OPTIONAL` mode makes
sense. Though, it is hard to see how we have file attributes available for some
files but not others. If per-query (query planned with metadata or without),
then the column could be handled using the "missing columns" mechanism: if the
implicit value is not available, just treat it as missing and let the "null
column manager" fill it with nulls.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
