On 08/13/2012 09:53 AM, Stephen Gran wrote: > Hi, > > I think user_data is probably reasonably common - most people who use, > eg, cloud-init will use it (we do). > > As the 64k limit is a MySQL limitation, and not a nova limitation, why > not just say, "if you want more storage, use postgres" (or similar)? I > have no issue with making the size guarded in the application, with a > configurable limit, but the particular problem that started this off is > an implementation issue rather than a code issue.
Or just set the column to the LONGTEXT type and both MySQL and PostgreSQL will be just as happy. > Storing the user_data in some place like the database is fairly > important for making things like launch configs for autoscale groups > work. I'd like to not make that harder to implement. Why is storing user_data in the database fairly important? You say above you don't want an implementation issue to be misconceived as a code issue -- and then go on to say that an implementation issue (storing user_data in a database) isn't a code issue. I don't think you can have it both ways. :) Now, I totally buy the argument that there is a large existing cloud-init userbase out there that relies on the EC2 Metadata API service living on the hard-coded 169.254.169.254 address, and we shouldn't do anything to mess up that experience. But I totally think that config-drive or disk-injection is a better way to handle this stuff -- and certainly doesn't force an implementation that has proven to be a major performance and scaling bottleneck (the EC2 Metadata service) Best, -jay > Cheers, > > On Mon, 2012-08-13 at 09:12 -0400, Dan Prince wrote: >> >> ----- Original Message ----- >>> From: "Michael Still" <michael.st...@canonical.com> >>> To: openstack@lists.launchpad.net, openstack-operat...@lists.openstack.org >>> Sent: Saturday, August 11, 2012 5:12:22 AM >>> Subject: [Openstack] [Nova] How common is user_data for instances? >>> >>> Greetings. >>> >>> I'm seeking information about how common user_data is for instances >>> in >>> nova. Specifically for large deployments (rackspace and HP, here's >>> looking at you). What sort of costs would be associated with changing >>> the data type of the user_data column in the nova database? >>> >>> Bug 1035055 [1] requests that we allow user_data of more than 65,535 >>> bytes per instance. Note that this size is a base64 encoded version >>> of >>> the data, so that's only a bit under 50k of data. This is because the >>> data is a sqlalchemy Text column. >>> >>> We could convert to a LongText column, which allows 2^32 worth of >>> data, >>> but I want to understand the cost to operators of that change some >>> more. >>> Is user_data really common? Do you think people would start uploading >>> much bigger user_data? Do you care? >> >> Nova has configurable quotas on most things so if we do increase the size of >> the DB column we should probably guard it in a configurable manner with >> quotas as well. >> >> My preference would actually be that we go the other way though and not have >> to store user_data in the database at all. That unfortunately may not be >> possible since some images obtain user_data via the metadata service which >> needs a way to look it up. Other methods of injecting metadata via disk >> injection, agents and/or config drive however might not need it to be store >> in the database right? >> >> As a simpler solution: >> >> Would setting a reasonable limit (hopefully smaller) and returning a HTTP >> 400 bad request if incoming requests exceed that limit be good enough to >> resolve this ticket? That way we don't have to increase the DB column at all >> and end users would be notified up front that user_data is too large (not >> silently truncated). They way I see it user_data is really for bootstrapping >> instances... we probably don't need it to be large enough to write an entire >> application, etc. >> >> >>> >>> Mikal >>> >>> 1: https://bugs.launchpad.net/nova/+bug/1035055 >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >>> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
