Zameer Manji created AURORA-1674:
------------------------------------
Summary: Migration from MemTaskStore to DbTaskStore can take hours
Key: AURORA-1674
URL: https://issues.apache.org/jira/browse/AURORA-1674
Project: Aurora
Issue Type: Bug
Reporter: Zameer Manji
Migrating an Aurora instance which has about 330k tasks in MemTaskStore to
DbTaskStore took over 13h.
This can be replicated as follows:
# Schedule thousands of tasks with a scheduler on MemTaskStore
# Use {{aurora_admin}} to create a snapshot.
# Redeploy the scheduler with DbTaskStore enabled
# Watch as Snapshot recovery and inserting tasks takes several hours
Stack dumps of this process indicates that all of the time was spent in H2
handing the consequences of inserting hundreds of thousands of records.
I wasn't able to figure out what precisely was the root cause but here are some
possible suspects
* Generating the SQL for delete tasks which would involve creating a delete
statement with 330k task ids.
* H2 transaction/locking configuration making this mass insert very slow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
