Issue #864 has been reported by Trevor Fong. ---------------------------------------- Feature #864: More Efficient Processing for Database Source http://tools.lsc-project.org/issues/864
Author: Trevor Fong Status: New Priority: Normal Assigned to: Category: Target version: Currently, LSC processes sync from a database source in the following fashion: 1. Run query for requestNameForList node to get a list of ID's to process from the source database 2. For each of those ID's run query for requestNameForObject to obtain the object to sync 3. Sync the object to the destination This could be made more efficient by doing the following: 1. Run query for requestNameForData node to get a result set from the source database containing all rows and columns to process 2. Iterate through the result set rows (each row being the object to sync) 3. Sync the object to the destination For the current method, for a database containing N number of accounts to sync, N+1 database connections and queries must be made, whereas for the proposed method, only 1 database connection and query need be made. If each List query takes M seconds and each Object query takes P seconds, total time for database activity would be P+(NxM) seconds. For the proposed method, total time for database activity would be P seconds. For example, if there were 1000 records, the List query takes 1 sec, and each Object database query returns in 0.1 sec: - current: total database time will be 1 + (1000 x 0.1) = 101 sec. - proposed: total database time will be 1 sec. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://tools.lsc-project.org/my/account
_______________________________________________________________ Ldap Synchronization Connector (LSC) - http://lsc-project.org lsc-dev mailing list [email protected] http://lists.lsc-project.org/listinfo/lsc-dev
