Chinmay Kulkarni created PHOENIX-5943:
-----------------------------------------
Summary: Consider splitting up SYSTEM.TASK into 2 tables - one for
queued jobs and one for task history
Key: PHOENIX-5943
URL: https://issues.apache.org/jira/browse/PHOENIX-5943
Project: Phoenix
Issue Type: Improvement
Affects Versions: 4.15.0
Reporter: Chinmay Kulkarni
Fix For: 4.16.0
We can split SYSTEM.TASK into 2 tables:
# SYSTEM.TASK_QUEUE: used as an ephemeral task queue with potentially a
smaller TTL which contains tasks that are new and yet to be picked up.
# SYSTEM.TASK_HISTORY: used to store tasks that have already been picked up
and may or may not have succeeded, along with more data about the task (stack
trace, job IDs, etc.). This table can have a longer TTL and can be used for
auditing/debugging purposes.
The advantages of doing this are:
* Makes SYSTEM.TASK_QUEUE more lightweight.
* Less data scanned when picking up new tasks.
* Combined with a PK change, we can introduce a natural priority queue using
the TASK_PRIORITY field.
* Different TTL value on the TASK_HISTORY table so we can maintain past tasks
for a longer period of time.
* Potentially makes this more in-line with other use-cases that want to
maintain a similar kind of task and task history table (for ex: View TTL's
async delete functionality).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
