jerryshao opened a new issue, #7132:
URL: https://github.com/apache/gravitino/issues/7132
### Describe the proposal
Metadata is the brain of the data, also it is the treasure for the company.
Currently, Gravitino already supports managing structured data (table),
unstructured data (fileset), messaging data (topic), and ML models (model), so
what is the next step, how to leverage metadata to empower the business?
Intuitively, the next step is to make decisions and take actions based on
the managed metadata in Gravitino. With this, we can support lots of different
scenarios, for example, like:
1. Table, column, fileset (sub-directories), and model data clean based on
the TTL policy.
2. Multiple location fileset synchronization based on the local-remote R/W
strategy.
3. Table maintenance, like data file compaction, data file rewrite, snapshot
expiration, orphan file removal, etc.
4. Regulation supports, like column masking/pruning, column deletion, data
encryption.
5. Various actions that are based on the existing metadata.
So, Gravitino is already the brain of the data, the next step is to build
arms and legs for Gravitino to truly empower the business. With this, we will
kick off a new key project in Gravitino - a metadata-driven action system.
## Goal
1. Build a generic action system in Gravitino. This action system is not
only limited to table maintenance but also can support other various actions
based on the provided metadata in Gravitino. Also, users can build and submit
their actions with our provided framework.
2. Build the framework in a self-contained, generic, and atomic way:
* Each component in this framework is self-contained, and can be reused in
other scenarios.
* Each component should provide REST APIs, and can be integrated with an
agentic framework, like MCP, to make it a tool or MCP server.
### Task list
Will be split later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
