magibney commented on code in PR #1996:
URL: https://github.com/apache/solr/pull/1996#discussion_r1355222328
##########
solr/core/src/java/org/apache/solr/handler/export/ExportWriter.java:
##########
@@ -844,4 +844,61 @@ public String getMessage() {
return "Early Client Disconnect";
}
}
+
+ /**
+ * Creates a complete field list using the provided field list by expanding
any glob patterns into
+ * field names
+ *
+ * @param fields the original set of fields provided
+ * @param searcher an index searcher to access schema info
+ * @return a complete list of fields included any fields matching glob
patterns
+ * @throws IOException if a provided field does not exist or cannot be
retrieved from the schema
+ * info
+ */
+ private List<SchemaField> expandFieldList(String[] fields, SolrIndexSearcher
searcher)
+ throws IOException {
+ List<SchemaField> expandedFields = new ArrayList<>(fields.length);
+ Set<String> fieldsProcessed = new HashSet<>();
+ for (String field : fields) {
+ try {
+ if (field.contains("*")) {
+ getGlobFields(field, searcher, fieldsProcessed, expandedFields);
+ } else {
+ if (fieldsProcessed.add(field)) {
+ expandedFields.add(searcher.getSchema().getField(field));
+ }
+ }
+ } catch (Exception e) {
+ throw new IOException(e);
+ }
+ }
+
+ return expandedFields;
+ }
+
+ /**
+ * Create a list of schema fields that match a given glob pattern
+ *
+ * @param fieldPattern the glob pattern to match
+ * @param searcher an index search to access schema info
+ * @param fieldsProcessed the set of field names already processed to avoid
duplicating
+ * @param expandedFields the list of fields to add expanded field names into
+ */
+ private void getGlobFields(
+ String fieldPattern,
+ SolrIndexSearcher searcher,
+ Set<String> fieldsProcessed,
+ List<SchemaField> expandedFields) {
+ for (FieldInfo fi : searcher.getFieldInfos()) {
+ if (GlobPatternUtil.matches(fieldPattern, fi.getName())) {
+ SchemaField schemaField = searcher.getSchema().getField(fi.getName());
+ if (fieldsProcessed.add(fi.getName())
+ && schemaField.hasDocValues()
+ && (!(schemaField.getType() instanceof SortableTextField)
Review Comment:
As @justinrsweeney foreshadowed, I am in favor of parity in how this is
handled between select and export.
The purpose of udvas as I understand it is not simply to mitigate
performance issues in select (and in fact, performance should be comparable
between select and export as of #1938), but also as a config option to allow
the configuration of docValues for a field without requiring that those values
be returned when a field matches a glob pattern.
There are plenty of use cases where, e.g., a derivative string field is
configured solely for the purpose of sort, faceting, [or other purpose that
would make even less sense to return on export]. It should be possible to
configure a field that's intended for one of the many other purposes docValues
serve, without forcing it to be returned when the field name matches a glob
pattern. Note also, if both `stored=true` and `useDocValuesAsStored=true`,
stored fields will be used for select and docValues will be used for export.
I think what I'm missing is: how does this help usability? Are there cases
where one would want docValues to be used for field value retrieval, but could
not simply set that field to `useDocValuesAsStored=true`? And udvas defaults to
true anyway (for all fields except SortableTextField) iirc.
I'm curious to hear others' thoughts on this as well!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
