clintropolis opened a new issue #6680: json-path filter by path existence broken due to optimization URL: https://github.com/apache/incubator-druid/issues/6680 #6654 introduced a bug that causes filter expressions that are checking for path existence. Using this example from the docs: ``` $..book[?(@.isbn)] ``` it incorrectly will return all 4 books, even though only 2 of them have the 'isbn' property, due to special handling of the `UNDEFINED` placeholder object in this case. Unfortunately, I cannot find a way to workaround this issue, so we might want to consider reverting that commit, or if we are ok with the loss of this functionality, then we should document this behavior. Also of note, the type of transform that would probably be done to actually use the 'isbn' value during ingestion, e.g. ``` $..book[?(@.isbn)].isbn ``` say to select a list of isbn numbers to use as a multi-value dimension or whatever, does function correctly with the optimization since `null` values are filtered out, so this might not be a big deal, but I am uneasy that it broke full behavioral compatibility. Still, the performance difference of the optimization is significant so maybe careful consideration is in order? I would put the chances of getting the performance fixed upstream at very low because of lack of development over the last year (see https://github.com/json-path/JsonPath/issues/500)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
