I'm building a couchdb DB to store and manage user, group and role data.
I have an external feed for the data.
The external data is delivered as a file containing all the records for
each data type, daily.
I need to keep the information in the database current:
- if there is a user defined in the current data that doesn't yet
exist in the database, a document needs to be created.
- if there a feed-created user in the database that doesn't exist in
the current data, that user document needs to be set 'inactive'.
Similar logic needs to exist for the groups and roles. (Whether roles
should be stored in the user document or separately, I'm not yet sure)
Given that there are ~200k users, ~150k roles, and ~3k groups, how would
you suggest this update process be approached?
Regards,
-Patrick Barnes
