Hi Dima, Thanks for this. I have some knowledge of Nifi, but I'm still early on the learning curve.
Our current implementation plan is to use a collection of pre-existing log servers and feed that into a Kafka cluster. In the model you describe would that mean inserting NIfi between the log servers and Kafka? Cheers, Tom. On 2020-05-05 17:25:01-07:00 Dima Kovalyov wrote: I would drop them on ingestion using NiFi's RouteOnContent. On Tue, May 5, 2020, 17:53 Yerex, Tom <tom.ye...@ubc.ca<mailto:tom.ye...@ubc.ca>> wrote: Good afternoon, Our incoming data is not always perfect, in some cases events are simply missing fields. We would like a way to drop events when particular fields are empty (or have values we don't care about). One way we thought to do this might be to write a custom Stellar function. Does anyone know of another solution? Thank you, Tom. - Dima
