npawar edited a comment on issue #5264: Flatten fields during ingestion URL: https://github.com/apache/incubator-pinot/issues/5264#issuecomment-614976122 Initial thoughts: Groovy functions are capable of handling non-flat objects (list, map, complex object). For example: ``` ObjectMapper objectMapper = new ObjectMapper(); String json = "{ \"firstName\":\"Hello\", \"lastName\":\"Human\" }"; JsonNode jsonNode = objectMapper.readTree(json); GenericRow genericRow = new GenericRow(); genericRow.putValue("name", jsonNode); GroovyExpressionEvaluator groovyExpressionEvaluator = new GroovyExpressionEvaluator( "Groovy({name.get('firstName').asText() + ' ' + name.get('lastName').asText()}, name)"); Object result = groovyExpressionEvaluator.evaluate(genericRow); ``` The only piece missing right now is the RecordExtractors expect single value or Collection. A complex object will not be correctly forwarded to the Transformer. If RecordExtractors correctly pass forward the complex objects, groovy should flatten it. This is partially being done in AVRORecordExtractor, where we also handle Map.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
