Dengpan Yin created SAMZA-2383:
----------------------------------
Summary: Make Samza internal state (backed by RocksDB) queryable
Key: SAMZA-2383
URL: https://issues.apache.org/jira/browse/SAMZA-2383
Project: Samza
Issue Type: Task
Components: container
Reporter: Dengpan Yin
Assignee: Dengpan Yin
Samza is using RocksDB as the KV store for internal state, both key and values
are encoded as byte array. Some application owners want to query the internal
state when the job is running for troubleshooting. To do so, the application
owner needs to access the cluster where the container is running. For security
reasons, the application owner usually does not haver permission to access the
shared cluster, which prevent the application owner from querying the RocksDB
directly.
Samza-beam-runner also include some meta information in the key, such as the
window information. The application owner usually does not know the window
information when perform the query. The application owner needs to understand
the encoding format in order to do a range query.
It will be helpful to provide a convenient interface to the user to query the
state and show the results. For instance, the user can specify the container
ID, partition#, the state name, the query key and SERDE. The request will be
processed by the container and present the results to the user.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
