Hello everyone! I am working on the following data enrichment scenario. I am receiving a streaming data from which I am extracting several attributes. Now I want to use three of these attributes as keys to query a csv file and retrieve two values from it. (Something equivalent to this SQL statement) SELECT value1, value2 FROM csv_file WHERE key1=att_key_1 and key2=att_key_2 and key3=att_key_3 I made this scenario work using QueryRecord, however, I need to read the csv file every time I receive the streaming data - which does not seem really scalable solution.
I have looked for other options and read about LookupRecord[1] and LookupAttribute[2] which are build for such scenarios but unfortunately the respective LookupServices does not seem to allow specifying more than one key. I always get an ERROR saying that the expected key should be one of [my csv file attributes]. (Using one key at a time also results in an error saying that the values for it repeat which is the case as only the combination of the three is unique) So I am wondering if someone has encountered a similar problem and can provide any advice. (For now I would prefer not to implement a custom processor but if nothing works any directions and templates that would help will be also very appreciated.) Many thanks in advance, Valentina [1] https://community.cloudera.com/t5/Community-Articles/Data-flow-enrichment-with-NiFi-part-1-LookupRecord-processor/ta-p/246940 [2] https://community.cloudera.com/t5/Community-Articles/Data-flow-enrichment-with-NiFi-part-2-LookupAttribute/ta-p/247072
