Github user kaspersorensen commented on the issue:
https://github.com/apache/metamodel/pull/178
To address your two discussion points:
1. Yes it is very typical to store some data structure in Kafka. I
definitely think that a natural improvement here would be to extend the module
here with more pluggable serialization support. I wonder if the already
existing Converters utils would work. I think Iâll try that first.
Btw this is not very different from how our Cassandra module works today
since Cassandra also exposes just the stored bytes as the base of its API.
2. Certainly some queries would not work well here. Using ORDER BY on a
stream means that we have to exhaust the stream first. But OTOH many queries
will work quite well and thatâs what I was looking for. It does do streaming
of the records out to the client, so it wonât have to materialize everything
in memory to serve those queries.
---
