Special characters attribute filter search behaviour

[Nallapati, Sreenivasulu]

Hello Atlas team,

We are facing some performance issue with respect to the Entity Type attribute 
filter search.

[cid:image001.png@01D4F899.68EC7A40]


What is the reason behind  for going  in-memory or graph query in case of 
special characters in string attribute filter query, even though the attribute 
is indexed? . Its taking very very long time to get the results back and we are 
getting gateway timeouts.
The below code snippet says “in-memory or graph query in case” but doesn’t 
explain what is the reason behind it.

    private boolean isIndexSearchable(FilterCriteria filterCriteria, 
AtlasStructType structType) throws AtlasBaseException {
        String      qualifiedName = 
structType.getQualifiedAttributeName(filterCriteria.getAttributeName());
        Set<String> indexedKeys   = context.getIndexedKeys();
        boolean     ret           = indexedKeys != null && 
indexedKeys.contains(qualifiedName);

        if (ret) { // index exists
            // for string type attributes, don't use index query in the 
following cases:
            //   - operation is NEQ, as it might return fewer entries due to 
tokenization of vertex property value
            //   - value-to-compare has special characters
            AtlasType attributeType = 
structType.getAttributeType(filterCriteria.getAttributeName());

            if 
(AtlasBaseTypeDef.ATLAS_TYPE_STRING.equals(attributeType.getTypeName())) {
                if (filterCriteria.getOperator() == 
SearchParameters.Operator.NEQ) {
                    if (LOG.isDebugEnabled()) {
                        LOG.debug("NEQ operator found for string attribute {}, 
deferring to in-memory or graph query (might cause poor performance)", 
qualifiedName);
                    }

                    ret = false;
                } else if 
(hasIndexQuerySpecialChar(filterCriteria.getAttributeValue())) {
                    if (LOG.isDebugEnabled()) {
                        LOG.debug("special characters found in filter value {}, 
deferring to in-memory or graph query (might cause poor performance)", 
filterCriteria.getAttributeValue());
                    }

                    ret = false;
                }
            }
        }

        if (LOG.isDebugEnabled()) {
            if (!ret) {
                LOG.debug("Not using index query for: attribute='{}', 
operator='{}', value='{}'", qualifiedName, filterCriteria.getOperator(), 
filterCriteria.getAttributeValue());
            }
        }

        return ret;
    }




---
Regards,
Nallapati, Sreenivasulu,
+91 99020 65578