Hi Gary, The motivation for this patch was to remove the unnecessary querying that was being done to quantum during the initial sync, previously a get_subnets/_ports was done for each network in addition to an update for the dhcp port even if it didn't need to). In my opinion the amount of data being sent to the agents is not a big deal. Even in extremely large deployments we are talking about on order of megs of data that are sent and this is only done on initial sync. That said, I'll update the patch to remove subnets that are not dhcp enabled as you are right we don't need to send these down.
In regards to rewriting the hosts file for each update I don't think this is really a big deal either. Implementing logic to hash the files to see if it needs to be updated seems like overkill as it will only save us on the initial sync. Currently, with the code on review i can restart the dhcp and have it complete sync_state() with 2k networks 8k ports in ~20 seconds where it previously took 30+ minutes. On the same subject though another performance issue that I think we should fix is the agent currently updates the ipallocation table for each renewed lease which happens every 120 seconds by the default value. This generates a ton of API calls to quantum and grows the more vms that are deployed(!) In my opinion it would be worth while to batch these and provide a bulk update method that is done every 60 seconds rather than immediately when it gets a renew. I'm not sure about the left over port issue you are seeing. I've not noticed this issue on my end. I'll keep an eye out though as well. Best, Aaron On Sun, Jun 9, 2013 at 12:10 AM, Gary Kotton <[email protected]> wrote: > Hi, > In addition to Aarons changes there are still a lot of issues which I > think are problematic with the current implementation. The problems that I > see at the moment are as follows: > 1. The amount of data that sent on the network to the agents > 2. The way in which the data is parsed on the agent: > i. Networks that are disabled are sent to the agent > ii. Subnets that have DHCP disabled are sent to the agent > iii. The host file is rebuilt with each and every change. This means > looping over all of the ports in the network each time there is an update. > 3. Message reliability. At the moment I am investigating a bug where > deleting 10 networks leaves the host file with details of a port that has > already been deleted. ( > I think that a way of addressing the above may be as follows: > > 1. Enable the plugin to prepare the information for the agents. This can > leverage the database filtering and reduce the amount of data on the > network. In addition to this the plugin can/may have drivers that are > specific to the agents implementation - for example in the case of dnsmasq, > it will be preparing the host file. In the case of the dnsmasq we can send > host file deltas. A further reduction can be compressing the host file. > 2. In order to prevent very heavy operations when either agents/services > are restarted we can have a hash of the above file. This can ensure that it > is synced (this is something that is specific to the driver > implementations). > > Thoughts? > > Thanks > Gary > > > -- > Mailing list: > https://launchpad.net/~**quantum-core<https://launchpad.net/~quantum-core> > Post to : > [email protected].**net<[email protected]> > Unsubscribe : > https://launchpad.net/~**quantum-core<https://launchpad.net/~quantum-core> > More help : > https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp> >
-- Mailing list: https://launchpad.net/~quantum-core Post to : [email protected] Unsubscribe : https://launchpad.net/~quantum-core More help : https://help.launchpad.net/ListHelp
