wgtmac commented on a change in pull request #990:
URL: https://github.com/apache/orc/pull/990#discussion_r778606490
##########
File path: c++/include/orc/Common.hh
##########
@@ -122,6 +122,11 @@ namespace orc {
StreamKind_BLOOM_FILTER_UTF8 = 8
};
+ enum ArrayReadIntent {
Review comment:
Should we extend the support to MAP type?
##########
File path: c++/include/orc/Reader.hh
##########
@@ -149,6 +149,26 @@ namespace orc {
*/
RowReaderOptions& includeTypes(const std::list<uint64_t>& types);
+ /**
+ * A map of <typeId, ArrayReadIntent> used as a parameter in
+ * RowReaderOptions::includeTypesWithIntents.
+ */
+ typedef std::map<uint64_t, ArrayReadIntent> TypeReadIntents;
+
+ /**
+ * Selects which type ids to read and specific ArrayReadIntents for each
+ * type id. The root type is always 0 and the rest of the types are
+ * labeled in a preorder traversal of the tree. The parent types are
+ * automatically selected, but the children are not.
+ *
+ * This option clears any previous setting of the selected columns or
+ * types.
+ * @param typesWithIntents a map of <typeId, ArrayReadIntent>.
+ * @return this
+ */
+ RowReaderOptions&
+ includeTypesWithIntents(const TypeReadIntents& typesWithIntents);
Review comment:
What is the scenario that some LIST columns read offsets only but other
LIST types read all?
Does it make sense to apply same ReadIntent to all LIST types? If all LIST
columns apply the same logic, we can still support column pruning by all three
ways: includeTypes, include(const std::list<uint64_t>&), and include(const
std::list<std::string>&).
##########
File path: c++/include/orc/Reader.hh
##########
@@ -565,6 +591,15 @@ namespace orc {
*/
virtual void seekToRow(uint64_t rowNumber) = 0;
+ /**
+ * Get an ArrayReadIntent for a given typeId.
+ * @param typeId the type id to look up.
+ * @return an ArrayReadIntent that was specified for given typeId through
+ * RowReaderOptions::includeTypesWithIntents. If no ArrayReadIntent was
+ * specified for typeId, return ArrayReadIntent_ALL.
+ */
+ virtual ArrayReadIntent getReadIntent(uint64_t typeId) const = 0;
Review comment:
What if the column of typeId is not selected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@orc.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
