Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
config all times is not good. On 10/21/2013 05:12 PM, Tim Simpson wrote: Thanks for the feedback Andrey. 2. Got this case in irc, and decided to pass type and version together to avoid confusing. I don't understand how allowing the user to only pass the version would confuse anyone. Could you elaborate? 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. Hmm. Does anyone remember the reason for this? 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. If 'active' is allowed to be set for multiple rows of the 'datastore_versions' table then it isn't a good substitute for the functionality I'm seeking, which is to allow operators to specify a *single* default version for each datastore_type in the database. I still think we should still add a 'default_version_id' field to the 'datastore_types' table. Thanks, Tim *From:* Andrey Shestakov [ashesta...@mirantis.com] *Sent:* Monday, October 21, 2013 7:15 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance 1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
+1 to Vipul's suggestion. Since all compabilities based upon config, we could send response to user based upon available guestagent managers, but this means, that all datastore managers parameter should migrate to taskmanager.? Manager description(fq classpath) would delivered to instance through guest_info. I think using the config files like that goes too far- there was an earlier email by Ilya which touched on something similar. In general I think if we plan on storing rich configuration details for potentially dozens of database types and versions then plain text files may be an option but we shouldn't try to morph the current Oslo config files to serve that purpose. From: Denis Makogon [dmako...@mirantis.com] Sent: Monday, October 28, 2013 1:05 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance Small suggestion on listing datastore_type: Since all compabilities based upon config, we could send response to user based upon available guestagent managers, but this means, that all datastore managers parameter should migrate to taskmanager. Manager description(fq classpath) would delivered to instance through guest_info. 2013/10/28 Vipul Sabhaya vip...@gmail.commailto:vip...@gmail.com On Sun, Oct 27, 2013 at 4:29 PM, Ilya Sviridov isviri...@mirantis.commailto:isviri...@mirantis.com wrote: Totally agree, however current concept supposes working with type and version as different entities, even if it is the attributes on one thing - the configuration. The reason for storing it as separate models can be cases when we are going to use them separately. Sounds really reasonable to keep it as one model, but another question comes to mind. How 'list datastore_type' will look like. That is important because API should be inambiguous. Following openstack tenets, each entity exposed via API has an id and can referenced by it. If we are storing datastore as one entity, we are not able to query versions or types only with their ids. But it is agreed as API /{tenant_id}/datastore_types /{tenant_id}/datastore_types/{datastore_type}/versions /{tenant_id}/datastore_types/versions/{id} I am wondering why we even need the last route. /{tenant_id}/datastore_types/versions/{id} If we assume that a datastore_type is the parent resource of versions, we could change that route to: /{tenant_id}/datastore_types/{datastore_type}/versions/{id}. Although I don’t know if this route is even necessary - since listing all available versions of a certain type is all users really need. This will allow us to group the type and version, making version no longer independent as Nikhil suggests. So, with current concept it seems better to keep version and type as separate entities in database. With best regards, Ilya Sviridov http://www.mirantis.ru/ On Fri, Oct 25, 2013 at 10:25 PM, Nikhil Manchanda nik...@manchanda.memailto:nik...@manchanda.me wrote: It seems strange to me to treat both the datastore_type and version as two separate entities, when they aren't really independent of each other. (You can't really deploy a mysql type with a cassandra version, and vice-versa, so why have separate datastore-list and version-list calls?) I think it's a better idea to store in the db (and list) actual representations of the datastore type/versions that an image we can deploy supports. Any disambiguation could then happen based on what entries actually exist here. Let me illustrate what I'm trying to get at with a few examples: Database has: id | type | version | active -- a | mysql | 5.6.14 | 1 b | mysql | 5.1.0 | 0 c | postgres | 9.3.1 | 1 d | redis | 2.6.16 | 1 e | redis | 2.6.15 | 1 f | cassandra | 2.0.1 | 1 g | cassandra | 2.0.0 | 0 Config specifies: default_datastore_id = a 1. trove-cli instance create ... Just works - Since nothing is specified, this uses the default_datastore_id from the config (mysql 5.6.14 a) . No need for disambiguation. 2. trove-cli instance create --datastore_id e The datastore_id specified always identifies a unique datastore type / version so no other information is needed for disambiguation. (In this case redis 2.6.15, identified by e) 3. trove-cli instance create --datastore_type postgres The datastore_type in this case uniquely identifies postgres 9.3.1 c, so no disambiguation is necessary. 4. trove-cli instance create --datastore_type cassandra In this case, there is only one _active_ datastore with the given datastore_type, so no further disambiguation is needed and cassandra 2.0.1 f is uniquely identified. 5. trove-cli instance create --datastore_type redis In this case, there are _TWO_ active versions of the specified datastore_type (2.6.16, and 2.6.17) so the call should return that further disambiguation _is_ needed. 6
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Oslo config files is the higher abstraction of trove service configuration. Since common/cfg.py configuration is almost independent from *.conf files, then yes, we could use parameters from there, but, to be precise, we should intialize that parameter in a right way, and this means we should store dictionary with managers fq classname at taskmanager.conf, or even at trove.conf - it is up to us. 2013/10/29 Tim Simpson tim.simp...@rackspace.com +1 to Vipul's suggestion. Since all compabilities based upon config, we could send response to user based upon available guestagent managers, but this means, that all datastore managers parameter should migrate to taskmanager.? Manager description(fq classpath) would delivered to instance through guest_info. I think using the config files like that goes too far- there was an earlier email by Ilya which touched on something similar. In general I think if we plan on storing rich configuration details for potentially dozens of database types and versions then plain text files may be an option but we shouldn't try to morph the current Oslo config files to serve that purpose. -- *From:* Denis Makogon [dmako...@mirantis.com] *Sent:* Monday, October 28, 2013 1:05 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance Small suggestion on listing datastore_type: Since all compabilities based upon config, we could send response to user based upon available guestagent managers, but this means, that all datastore managers parameter should migrate to taskmanager. Manager description(fq classpath) would delivered to instance through guest_info. 2013/10/28 Vipul Sabhaya vip...@gmail.com On Sun, Oct 27, 2013 at 4:29 PM, Ilya Sviridov isviri...@mirantis.comwrote: Totally agree, however current concept supposes working with type and version as different entities, even if it is the attributes on one thing - the configuration. The reason for storing it as separate models can be cases when we are going to use them separately. Sounds really reasonable to keep it as one model, but another question comes to mind. How 'list datastore_type' will look like. That is important because API should be inambiguous. Following openstack tenets, each entity exposed via API has an id and can referenced by it. If we are storing datastore as one entity, we are not able to query versions or types only with their ids. But it is agreed as API /{tenant_id}/datastore_types /{tenant_id}/datastore_types/{datastore_type}/versions /{tenant_id}/datastore_types/versions/{id} I am wondering why we even need the last route. /{tenant_id}/datastore_types/versions/{id} If we assume that a datastore_type is the parent resource of versions, we could change that route to: /{tenant_id}/datastore_types/{datastore_type}/versions/{id}. Although I don’t know if this route is even necessary - since listing all available versions of a certain type is all users really need. This will allow us to group the type and version, making version no longer independent as Nikhil suggests. So, with current concept it seems better to keep version and type as separate entities in database. With best regards, Ilya Sviridov http://www.mirantis.ru/ On Fri, Oct 25, 2013 at 10:25 PM, Nikhil Manchanda nik...@manchanda.me wrote: It seems strange to me to treat both the datastore_type and version as two separate entities, when they aren't really independent of each other. (You can't really deploy a mysql type with a cassandra version, and vice-versa, so why have separate datastore-list and version-list calls?) I think it's a better idea to store in the db (and list) actual representations of the datastore type/versions that an image we can deploy supports. Any disambiguation could then happen based on what entries actually exist here. Let me illustrate what I'm trying to get at with a few examples: Database has: id | type | version | active -- a | mysql | 5.6.14 | 1 b | mysql | 5.1.0 | 0 c | postgres | 9.3.1 | 1 d | redis | 2.6.16 | 1 e | redis | 2.6.15 | 1 f | cassandra | 2.0.1 | 1 g | cassandra | 2.0.0 | 0 Config specifies: default_datastore_id = a 1. trove-cli instance create ... Just works - Since nothing is specified, this uses the default_datastore_id from the config (mysql 5.6.14 a) . No need for disambiguation. 2. trove-cli instance create --datastore_id e The datastore_id specified always identifies a unique datastore type / version so no other information is needed for disambiguation. (In this case redis 2.6.15, identified by e) 3. trove-cli instance create --datastore_type postgres The datastore_type in this case uniquely identifies postgres 9.3.1 c, so
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
add a 'default_version_id' field to the 'datastore_types' table. Thanks, Tim *From:* Andrey Shestakov [ashesta...@mirantis.com] *Sent:* Monday, October 21, 2013 7:15 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance 1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
conversation rejected this case, i cant remember exactly reason. Hmm. Does anyone remember the reason for this? 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. If 'active' is allowed to be set for multiple rows of the 'datastore_versions' table then it isn't a good substitute for the functionality I'm seeking, which is to allow operators to specify a *single* default version for each datastore_type in the database. I still think we should still add a 'default_version_id' field to the 'datastore_types' table. Thanks, Tim *From:* Andrey Shestakov [ashesta...@mirantis.com] *Sent:* Monday, October 21, 2013 7:15 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance 1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
It seems strange to me to treat both the datastore_type and version as two separate entities, when they aren't really independent of each other. (You can't really deploy a mysql type with a cassandra version, and vice-versa, so why have separate datastore-list and version-list calls?) I think it's a better idea to store in the db (and list) actual representations of the datastore type/versions that an image we can deploy supports. Any disambiguation could then happen based on what entries actually exist here. Let me illustrate what I'm trying to get at with a few examples: Database has: id | type | version | active -- a | mysql | 5.6.14 | 1 b | mysql | 5.1.0 | 0 c | postgres | 9.3.1 | 1 d | redis | 2.6.16 | 1 e | redis | 2.6.15 | 1 f | cassandra | 2.0.1 | 1 g | cassandra | 2.0.0 | 0 Config specifies: default_datastore_id = a 1. trove-cli instance create ... Just works - Since nothing is specified, this uses the default_datastore_id from the config (mysql 5.6.14 a) . No need for disambiguation. 2. trove-cli instance create --datastore_id e The datastore_id specified always identifies a unique datastore type / version so no other information is needed for disambiguation. (In this case redis 2.6.15, identified by e) 3. trove-cli instance create --datastore_type postgres The datastore_type in this case uniquely identifies postgres 9.3.1 c, so no disambiguation is necessary. 4. trove-cli instance create --datastore_type cassandra In this case, there is only one _active_ datastore with the given datastore_type, so no further disambiguation is needed and cassandra 2.0.1 f is uniquely identified. 5. trove-cli instance create --datastore_type redis In this case, there are _TWO_ active versions of the specified datastore_type (2.6.16, and 2.6.17) so the call should return that further disambiguation _is_ needed. 6. trove-cli instance create --datastore_type redis --datastore_version 2.6.16 We have both datastore_type and datastore_version, and that uniquely identifies redis 2.6.16 e. No further disambiguation is needed. 7. trove-cli instance create --datastore_type cassandra --version 2.0.0, or trove-cli instance create --datastore_id g Here, we are attempting to deploy a datastore which is _NOT_ active and this call should fail with an appropriate error message. Cheers, -Nikhil Andrey Shestakov writes: 2. it can be confusing coz not clear to what type version belongs (possible add type field in version). also if you have default type, then specified version recognizes as version of default type (no lookup in version.datastore_type_id) but i think we can do lookup in version.datastore_type_id before pick default. 4. if default version is need, then it should be specified in db, coz switching via versions can be frequent and restart service to reload config all times is not good. On 10/21/2013 05:12 PM, Tim Simpson wrote: Thanks for the feedback Andrey. 2. Got this case in irc, and decided to pass type and version together to avoid confusing. I don't understand how allowing the user to only pass the version would confuse anyone. Could you elaborate? 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. Hmm. Does anyone remember the reason for this? 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. If 'active' is allowed to be set for multiple rows of the 'datastore_versions' table then it isn't a good substitute for the functionality I'm seeking, which is to allow operators to specify a *single* default version for each datastore_type in the database. I still think we should still add a 'default_version_id' field to the 'datastore_types' table. Thanks, Tim *From:* Andrey Shestakov [ashesta...@mirantis.com] *Sent:* Monday, October 21, 2013 7:15 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance 1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
So, we have 2 places for configuration management - database and config file Config file for tunning all datasource type behavior during installation and database for all changeable configurations during usage and administration of Trove installation. Database usecases: - update/custom image - update/custom packages - activating/deactivating datastore_type Config file usecases: - security group policy - provisioning mechanism - guest configuration parameters per database engine - provisioning parameters, templates - manager class ... In case if i need to register one more MySQL installation with following customization: - custom heat template - custom packages and additional monitoring tool package - open specific port for working with my monitoring tool on instance According to current concept should i add one more section in addition to existing mysql like below? [monitored_mysql] mount_point=/var/lib/mysql #8080 is port of my monitoring tool trove_security_group_rule_ports = 3306, 8080 heat_template=/etc/trove/heat_templates/monitored_mysql.yaml ... and put additional packages to database configuration? With best regards, Ilya Sviridov http://www.mirantis.ru/ On Wed, Oct 23, 2013 at 9:37 PM, Michael Basnight mbasni...@gmail.comwrote: On Oct 23, 2013, at 10:54 AM, Ilya Sviridov wrote: Besides the strategy of selecting the default behavior. Let me share with you my ideas of configuration management in Trove and how the datastore concept can help with that. Initially there was only one database and all configuration was in one config file. With adding of new databases, heat provisioning mechanism, we are introducing more options. Not only assigning specific image_id, but custom packages, heat templates, probably specific strategies of working with security groups. Such needs already exist because we have a lot of optional things in config, and any new feature is implemented with back sight to already existing legacy installations of Trove. What is actually datastore_type + datastore_version? The model which glues all the bricks together, so let us use it for all variable part of *service type* configuration. from current config file # Trove DNS trove_dns_support = False # Trove Security Groups for Instances trove_security_groups_support = True trove_security_groups_rules_support = False trove_security_group_rule_protocol = tcp trove_security_group_rule_port = 3306 trove_security_group_rule_cidr = 0.0.0.0/0 #guest_config = $pybasedir/etc/trove/trove-guestagent.conf.sample #cloudinit_location = /etc/trove/cloudinit block_device_mapping = vdb device_path = /dev/vdb mount_point = /var/lib/mysql All that configurations can be moved to data_strore (some defined in heat templates) and be manageable by operator in case if any default behavior should be changed. The trove-config becomes core functionality specific only. Its fine for it to be in the config or the heat templates… im not sure it matters. what i would like to see is that specific thing to each service be in their own config group in the configuration. [mysql] mount_point=/var/lib/mysql … [redis] volume_support=False ….. and so on. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
So if we decide to support any number of config options for each various datastore version, eventually we'll have large config files that will be hard to manage. What about storing the extra config info for each datastore version in its own independent config file? So rather than having one increasingly bloated config file used by everything, you could optionally specify a file in the datastore_versions table of the database that would be looked up similar to how we load template files on demand. - Tim From: Ilya Sviridov [isviri...@mirantis.com] Sent: Thursday, October 24, 2013 7:40 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance So, we have 2 places for configuration management - database and config file Config file for tunning all datasource type behavior during installation and database for all changeable configurations during usage and administration of Trove installation. Database usecases: - update/custom image - update/custom packages - activating/deactivating datastore_type Config file usecases: - security group policy - provisioning mechanism - guest configuration parameters per database engine - provisioning parameters, templates - manager class ... In case if i need to register one more MySQL installation with following customization: - custom heat template - custom packages and additional monitoring tool package - open specific port for working with my monitoring tool on instance According to current concept should i add one more section in addition to existing mysql like below? [monitored_mysql] mount_point=/var/lib/mysql #8080 is port of my monitoring tool trove_security_group_rule_ports = 3306, 8080 heat_template=/etc/trove/heat_templates/monitored_mysql.yaml ... and put additional packages to database configuration? With best regards, Ilya Sviridov http://www.mirantis.ru/ On Wed, Oct 23, 2013 at 9:37 PM, Michael Basnight mbasni...@gmail.commailto:mbasni...@gmail.com wrote: On Oct 23, 2013, at 10:54 AM, Ilya Sviridov wrote: Besides the strategy of selecting the default behavior. Let me share with you my ideas of configuration management in Trove and how the datastore concept can help with that. Initially there was only one database and all configuration was in one config file. With adding of new databases, heat provisioning mechanism, we are introducing more options. Not only assigning specific image_id, but custom packages, heat templates, probably specific strategies of working with security groups. Such needs already exist because we have a lot of optional things in config, and any new feature is implemented with back sight to already existing legacy installations of Trove. What is actually datastore_type + datastore_version? The model which glues all the bricks together, so let us use it for all variable part of *service type* configuration. from current config file # Trove DNS trove_dns_support = False # Trove Security Groups for Instances trove_security_groups_support = True trove_security_groups_rules_support = False trove_security_group_rule_protocol = tcp trove_security_group_rule_port = 3306 trove_security_group_rule_cidr = 0.0.0.0/0http://0.0.0.0/0 #guest_config = $pybasedir/etc/trove/trove-guestagent.conf.sample #cloudinit_location = /etc/trove/cloudinit block_device_mapping = vdb device_path = /dev/vdb mount_point = /var/lib/mysql All that configurations can be moved to data_strore (some defined in heat templates) and be manageable by operator in case if any default behavior should be changed. The trove-config becomes core functionality specific only. Its fine for it to be in the config or the heat templates… im not sure it matters. what i would like to see is that specific thing to each service be in their own config group in the configuration. [mysql] mount_point=/var/lib/mysql … [redis] volume_support=False ….. and so on. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
I am 1% behind that idea. It makes things easier to manage. On Thu, Oct 24, 2013 at 10:44 AM, Tim Simpson tim.simp...@rackspace.comwrote: So if we decide to support any number of config options for each various datastore version, eventually we'll have large config files that will be hard to manage. What about storing the extra config info for each datastore version in its own independent config file? So rather than having one increasingly bloated config file used by everything, you could optionally specify a file in the datastore_versions table of the database that would be looked up similar to how we load template files on demand. - Tim -- *From:* Ilya Sviridov [isviri...@mirantis.com] *Sent:* Thursday, October 24, 2013 7:40 AM *To:* OpenStack Development Mailing List *Subject:* Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance So, we have 2 places for configuration management - database and config file Config file for tunning all datasource type behavior during installation and database for all changeable configurations during usage and administration of Trove installation. Database usecases: - update/custom image - update/custom packages - activating/deactivating datastore_type Config file usecases: - security group policy - provisioning mechanism - guest configuration parameters per database engine - provisioning parameters, templates - manager class ... In case if i need to register one more MySQL installation with following customization: - custom heat template - custom packages and additional monitoring tool package - open specific port for working with my monitoring tool on instance According to current concept should i add one more section in addition to existing mysql like below? [monitored_mysql] mount_point=/var/lib/mysql #8080 is port of my monitoring tool trove_security_group_rule_ports = 3306, 8080 heat_template=/etc/trove/heat_templates/monitored_mysql.yaml ... and put additional packages to database configuration? With best regards, Ilya Sviridov http://www.mirantis.ru/ On Wed, Oct 23, 2013 at 9:37 PM, Michael Basnight mbasni...@gmail.comwrote: On Oct 23, 2013, at 10:54 AM, Ilya Sviridov wrote: Besides the strategy of selecting the default behavior. Let me share with you my ideas of configuration management in Trove and how the datastore concept can help with that. Initially there was only one database and all configuration was in one config file. With adding of new databases, heat provisioning mechanism, we are introducing more options. Not only assigning specific image_id, but custom packages, heat templates, probably specific strategies of working with security groups. Such needs already exist because we have a lot of optional things in config, and any new feature is implemented with back sight to already existing legacy installations of Trove. What is actually datastore_type + datastore_version? The model which glues all the bricks together, so let us use it for all variable part of *service type* configuration. from current config file # Trove DNS trove_dns_support = False # Trove Security Groups for Instances trove_security_groups_support = True trove_security_groups_rules_support = False trove_security_group_rule_protocol = tcp trove_security_group_rule_port = 3306 trove_security_group_rule_cidr = 0.0.0.0/0 #guest_config = $pybasedir/etc/trove/trove-guestagent.conf.sample #cloudinit_location = /etc/trove/cloudinit block_device_mapping = vdb device_path = /dev/vdb mount_point = /var/lib/mysql All that configurations can be moved to data_strore (some defined in heat templates) and be manageable by operator in case if any default behavior should be changed. The trove-config becomes core functionality specific only. Its fine for it to be in the config or the heat templates… im not sure it matters. what i would like to see is that specific thing to each service be in their own config group in the configuration. [mysql] mount_point=/var/lib/mysql … [redis] volume_support=False ….. and so on. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Besides the strategy of selecting the default behavior. Let me share with you my ideas of configuration management in Trove and how the datastore concept can help with that. Initially there was only one database and all configuration was in one config file. With adding of new databases, heat provisioning mechanism, we are introducing more options. Not only assigning specific image_id, but custom packages, heat templates, probably specific strategies of working with security groups. Such needs already exist because we have a lot of optional things in config, and any new feature is implemented with back sight to already existing legacy installations of Trove. What is actually datastore_type + datastore_version? The model which glues all the bricks together, so let us use it for all variable part of *service type* configuration. from current config file # Trove DNS trove_dns_support = False # Trove Security Groups for Instances trove_security_groups_support = True trove_security_groups_rules_support = False trove_security_group_rule_protocol = tcp trove_security_group_rule_port = 3306 trove_security_group_rule_cidr = 0.0.0.0/0 #guest_config = $pybasedir/etc/trove/trove-guestagent.conf.sample #cloudinit_location = /etc/trove/cloudinit block_device_mapping = vdb device_path = /dev/vdb mount_point = /var/lib/mysql All that configurations can be moved to data_strore (some defined in heat templates) and be manageable by operator in case if any default behavior should be changed. The trove-config becomes core functionality specific only. What do you think about it? With best regards, Ilya Sviridov http://www.mirantis.ru/ On Tue, Oct 22, 2013 at 8:21 PM, Michael Basnight mbasni...@gmail.comwrote: On Oct 22, 2013, at 9:34 AM, Tim Simpson wrote: It's not intuitive to the User, if they are specifying a version alone. You don't boot a 'version' of something, with specifying what that some thing is. I would rather they only specified the datastore_type alone, and not have them specify a version at all. I agree for most users just selecting the datastore_type would be most intutive. However, when they specify a version it's going to a be GUID which they could only possibly know if they have recently enumerated all versions and thus *know* the version is for the given type they want. In that case I don't think most users would appreciate having to also pass the type- it would just be redundant. So in that case why not make it optional?] im ok w/ making either optional if the criteria for selecting the _other_ is not ambiguous. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Oct 23, 2013, at 10:54 AM, Ilya Sviridov wrote: Besides the strategy of selecting the default behavior. Let me share with you my ideas of configuration management in Trove and how the datastore concept can help with that. Initially there was only one database and all configuration was in one config file. With adding of new databases, heat provisioning mechanism, we are introducing more options. Not only assigning specific image_id, but custom packages, heat templates, probably specific strategies of working with security groups. Such needs already exist because we have a lot of optional things in config, and any new feature is implemented with back sight to already existing legacy installations of Trove. What is actually datastore_type + datastore_version? The model which glues all the bricks together, so let us use it for all variable part of *service type* configuration. from current config file # Trove DNS trove_dns_support = False # Trove Security Groups for Instances trove_security_groups_support = True trove_security_groups_rules_support = False trove_security_group_rule_protocol = tcp trove_security_group_rule_port = 3306 trove_security_group_rule_cidr = 0.0.0.0/0 #guest_config = $pybasedir/etc/trove/trove-guestagent.conf.sample #cloudinit_location = /etc/trove/cloudinit block_device_mapping = vdb device_path = /dev/vdb mount_point = /var/lib/mysql All that configurations can be moved to data_strore (some defined in heat templates) and be manageable by operator in case if any default behavior should be changed. The trove-config becomes core functionality specific only. Its fine for it to be in the config or the heat templates… im not sure it matters. what i would like to see is that specific thing to each service be in their own config group in the configuration. [mysql] mount_point=/var/lib/mysql … [redis] volume_support=False ….. and so on. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Are you saying you must have a default version defined to have 1 active versions? No, my point was using a default version field in the db rather than also picking from active versions may be confusing. From: Michael Basnight [mbasni...@gmail.com] Sent: Monday, October 21, 2013 4:04 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance On Oct 21, 2013, at 1:40 PM, Tim Simpson wrote: 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. Can't we do both? If a user wants a specific version, most likely they had to enumerate all datastore_versions, spot it in a list, and grab the guid. Why force them to also specify the datastore_type when we can easily determine what that is? Fair enough. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. Imagine that an operator sets up Trove and only has one active version. They then somehow fumble setting up the default_version, but think they succeeded as the API works for users the way they expect anyway. Then they go to add another active version and suddenly their users get error messages. If we only use the default_version field of the datastore_type to define a default would honor the principle of least surprise. Are you saying you must have a default version defined to have 1 active versions? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
This is also true that we dont want to define the _need_ to have custom images for the datastores. You can, quite easily, deploy mysql or redis on a vanilla image. Additionally there could be server code at some point soon that will need to know what datastore type is associated with an instance to determine what db engine is in use. So for example, if a call such as users isn't supported by a certain datastore used by an instance, the server side code will be able to determine that and something such as a bad request or not found status code. From: Michael Basnight [mbasni...@gmail.com] Sent: Monday, October 21, 2013 4:05 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance On Oct 21, 2013, at 1:57 PM, Nikhil Manchanda wrote: The image approach works fine if Trove only supports deploying a single datastore type (mysql in your case). As soon as we support deploying more than 1 datastore type, Trove needs to have some knowledge of which guestagent manager classes to load. Hence the need for having a datastore type API. The argument for needing to keep track of the version is similar. Potentially a version increment -- especially of the major version -- may require for a different guestagent manager. And Trove needs to have this information. This is also true that we dont want to define the _need_ to have custom images for the datastores. You can, quite easily, deploy mysql or redis on a vanilla image. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
It's not intuitive to the User, if they are specifying a version alone. You don't boot a 'version' of something, with specifying what that some thing is. I would rather they only specified the datastore_type alone, and not have them specify a version at all. I agree for most users just selecting the datastore_type would be most intutive. However, when they specify a version it's going to a be GUID which they could only possibly know if they have recently enumerated all versions and thus *know* the version is for the given type they want. In that case I don't think most users would appreciate having to also pass the type- it would just be redundant. So in that case why not make it optional? From: Vipul Sabhaya [vip...@gmail.com] Sent: Monday, October 21, 2013 5:09 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance On Mon, Oct 21, 2013 at 2:04 PM, Michael Basnight mbasni...@gmail.commailto:mbasni...@gmail.com wrote: On Oct 21, 2013, at 1:40 PM, Tim Simpson wrote: 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. Can't we do both? If a user wants a specific version, most likely they had to enumerate all datastore_versions, spot it in a list, and grab the guid. Why force them to also specify the datastore_type when we can easily determine what that is? Fair enough. It's not intuitive to the User, if they are specifying a version alone. You don't boot a 'version' of something, with specifying what that some thing is. I would rather they only specified the datastore_type alone, and not have them specify a version at all. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. Imagine that an operator sets up Trove and only has one active version. They then somehow fumble setting up the default_version, but think they succeeded as the API works for users the way they expect anyway. Then they go to add another active version and suddenly their users get error messages. If we only use the default_version field of the datastore_type to define a default would honor the principle of least surprise. Are you saying you must have a default version defined to have 1 active versions? I think it makes sense to have a 'Active' flag on every version -- and a default flag for the version that should be used as a default in the event the user doesn't specify. It also makes sense to require the deployer to set this accurately, and if one doesn't exist instance provisioning errors out. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Oct 22, 2013, at 9:34 AM, Tim Simpson wrote: It's not intuitive to the User, if they are specifying a version alone. You don't boot a 'version' of something, with specifying what that some thing is. I would rather they only specified the datastore_type alone, and not have them specify a version at all. I agree for most users just selecting the datastore_type would be most intutive. However, when they specify a version it's going to a be GUID which they could only possibly know if they have recently enumerated all versions and thus *know* the version is for the given type they want. In that case I don't think most users would appreciate having to also pass the type- it would just be redundant. So in that case why not make it optional?] im ok w/ making either optional if the criteria for selecting the _other_ is not ambiguous. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Thanks for the feedback Andrey. 2. Got this case in irc, and decided to pass type and version together to avoid confusing. I don't understand how allowing the user to only pass the version would confuse anyone. Could you elaborate? 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. Hmm. Does anyone remember the reason for this? 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. If 'active' is allowed to be set for multiple rows of the 'datastore_versions' table then it isn't a good substitute for the functionality I'm seeking, which is to allow operators to specify a *single* default version for each datastore_type in the database. I still think we should still add a 'default_version_id' field to the 'datastore_types' table. Thanks, Tim From: Andrey Shestakov [ashesta...@mirantis.com] Sent: Monday, October 21, 2013 7:15 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance 1. Good point 2. Got this case in irc, and decided to pass type and version together to avoid confusing. 3. Names of types and maybe versions can be good, but in irc conversation rejected this case, i cant remember exactly reason. 4. Actually, active field in version marks it as default in type. Specify default version in config can be usefull if you have more then one active versions in default type. But how match active version in type depends on operator`s configuration. And what if default version in config will marked as inactive? On 10/18/2013 10:30 PM, Tim Simpson wrote: Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Oct 18, 2013, at 12:30 PM, Tim Simpson wrote: 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: I agree with this. 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. Id prefer this honestly. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
What is the major motivation not to simply use a glance image named MySQL 5.5 or MongoDB 2.4? Wouldn't that give service providers all the flexibility they need for providing different types? For example, I could offer a simple MySQL image that creates a MySQL instance. If all my users use the one MySQL image then I can update that image deploy the latest version (or any version that I, as the service provider, want to deploy). Alternatively, my users could have a choice of versions if I roll a MySQL 5.1 and MySQL 5.5 image. Want to deactivate a version: delete the image. Want to offer a new version: create a new image. It seems like this is parallel to a NOVA deploy offering multiple version of the same OS (Ubuntu 12 vs Ubuntu 13). Images work nicely for that. Why couldn't they work for us? On 10/21/13 3:12 PM, Michael Basnight mbasni...@gmail.com wrote: On Oct 18, 2013, at 12:30 PM, Tim Simpson wrote: 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: I agree with this. 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. Id prefer this honestly. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
The image approach works fine if Trove only supports deploying a single datastore type (mysql in your case). As soon as we support deploying more than 1 datastore type, Trove needs to have some knowledge of which guestagent manager classes to load. Hence the need for having a datastore type API. The argument for needing to keep track of the version is similar. Potentially a version increment -- especially of the major version -- may require for a different guestagent manager. And Trove needs to have this information. Hope this helps, Cheers, -Nikhil Kevin Conway writes: What is the major motivation not to simply use a glance image named MySQL 5.5 or MongoDB 2.4? Wouldn't that give service providers all the flexibility they need for providing different types? For example, I could offer a simple MySQL image that creates a MySQL instance. If all my users use the one MySQL image then I can update that image deploy the latest version (or any version that I, as the service provider, want to deploy). Alternatively, my users could have a choice of versions if I roll a MySQL 5.1 and MySQL 5.5 image. Want to deactivate a version: delete the image. Want to offer a new version: create a new image. It seems like this is parallel to a NOVA deploy offering multiple version of the same OS (Ubuntu 12 vs Ubuntu 13). Images work nicely for that. Why couldn't they work for us? On 10/21/13 3:12 PM, Michael Basnight mbasni...@gmail.com wrote: On Oct 18, 2013, at 12:30 PM, Tim Simpson wrote: 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: I agree with this. 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. Id prefer this honestly. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Oct 21, 2013, at 1:40 PM, Tim Simpson wrote: 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. Can't we do both? If a user wants a specific version, most likely they had to enumerate all datastore_versions, spot it in a list, and grab the guid. Why force them to also specify the datastore_type when we can easily determine what that is? Fair enough. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. Imagine that an operator sets up Trove and only has one active version. They then somehow fumble setting up the default_version, but think they succeeded as the API works for users the way they expect anyway. Then they go to add another active version and suddenly their users get error messages. If we only use the default_version field of the datastore_type to define a default would honor the principle of least surprise. Are you saying you must have a default version defined to have 1 active versions? signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Oct 21, 2013, at 1:57 PM, Nikhil Manchanda wrote: The image approach works fine if Trove only supports deploying a single datastore type (mysql in your case). As soon as we support deploying more than 1 datastore type, Trove needs to have some knowledge of which guestagent manager classes to load. Hence the need for having a datastore type API. The argument for needing to keep track of the version is similar. Potentially a version increment -- especially of the major version -- may require for a different guestagent manager. And Trove needs to have this information. This is also true that we dont want to define the _need_ to have custom images for the datastores. You can, quite easily, deploy mysql or redis on a vanilla image. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Mon, Oct 21, 2013 at 11:40 PM, Tim Simpson tim.simp...@rackspace.com wrote: 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. Imagine that an operator sets up Trove and only has one active version. They then somehow fumble setting up the default_version, but think they succeeded as the API works for users the way they expect anyway. Then they go to add another active version and suddenly their users get error messages. If we only use the default_version field of the datastore_type to define a default would honor the principle of least surprise. if default version is inactive? there will more cases for error also, i think we have to use default version only if type contains 1 active version. And default version should be active, otherwise - error. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
On Mon, Oct 21, 2013 at 2:04 PM, Michael Basnight mbasni...@gmail.comwrote: On Oct 21, 2013, at 1:40 PM, Tim Simpson wrote: 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: When i brought this up it was generally discussed as being confusing. Id like to use type and rely on having a default (or active) version behind the scenes. Can't we do both? If a user wants a specific version, most likely they had to enumerate all datastore_versions, spot it in a list, and grab the guid. Why force them to also specify the datastore_type when we can easily determine what that is? Fair enough. It's not intuitive to the User, if they are specifying a version alone. You don't boot a 'version' of something, with specifying what that some thing is. I would rather they only specified the datastore_type alone, and not have them specify a version at all. 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: Since we have determined from this email thread that we have an active status, and that 1 version can be active, we have to think about the precedence of active vs default. My question would be, if we have a default_version_id and a active version, what do we choose on behalf of the user? If there is 1 active version and a user does not specify the version, the api will error out, unless a default is defined. We also need a default_type in the config so the existing APIs can maintain compatibility. We can re-discuss this for v2 of the API. Imagine that an operator sets up Trove and only has one active version. They then somehow fumble setting up the default_version, but think they succeeded as the API works for users the way they expect anyway. Then they go to add another active version and suddenly their users get error messages. If we only use the default_version field of the datastore_type to define a default would honor the principle of least surprise. Are you saying you must have a default version defined to have 1 active versions? I think it makes sense to have a 'Active' flag on every version -- and a default flag for the version that should be used as a default in the event the user doesn't specify. It also makes sense to require the deployer to set this accurately, and if one doesn't exist instance provisioning errors out. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Hi Tim, I do think your recommendation in 3 4 makes a lot of sense and improves the usability of the API. Given that Trove currently only supports a single datastore deployment per control system, does the current work also allow for a default type/version to be defined so that operators of Trove can set this as a property to maintain the current API compatibility/behavior? Josh From: Tim Simpson tim.simp...@rackspace.commailto:tim.simp...@rackspace.com Reply-To: OpenStack Development Mailing List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Friday, October 18, 2013 2:30 PM To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance
Hi Josh, Given that Trove currently only supports a single datastore deployment per control system, does the current work also allow for a default type/version to be defined so that operators of Trove can set this as a property to maintain the current API compatibility/behavior? Yes, the current pull request to support this allows for a default type, which, if there is only a single version for that type in the Trove infrastructure database, means that the existing behavior would be preserved. However as soon as an operator adds more than one datastore version of the default type then API users would need to always include the version ID. This would be fixed by recommendation #4 in my original message. Thanks, Tim From: Josh Odom [josh.o...@rackspace.com] Sent: Friday, October 18, 2013 3:16 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance Hi Tim, I do think your recommendation in 3 4 makes a lot of sense and improves the usability of the API. Given that Trove currently only supports a single datastore deployment per control system, does the current work also allow for a default type/version to be defined so that operators of Trove can set this as a property to maintain the current API compatibility/behavior? Josh From: Tim Simpson tim.simp...@rackspace.commailto:tim.simp...@rackspace.com Reply-To: OpenStack Development Mailing List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Friday, October 18, 2013 2:30 PM To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Trove] How users should specify a datastore type when creating an instance Hello fellow Trovians, There has been some good work recently to figure out a way to specify a specific datastore when using Trove. This is essential to supporting multiple datastores from the same install of Trove. I have an issue with some elements of the proposed solution though, so I decided I'd start a thread here so we could talk about it. As a quick refresher, here is the blue print for this work (there are some gists ammended to the end but I figured the mailing list would be an easier venue for discussion): https://wiki.openstack.org/wiki/Trove/trove-versions-types One issue I have is with the way the instance create call will change to support different data stores. For example, here is the post call: { instance : { flavorRef : 2, name : as, datastore_type : e60153d4-8ac4-414a-ad58-fe2e0035704a, datastore_version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b, volume : { size : 1 } } } 1. I think since we have two fields in the instance object we should make a new object for datastore and avoid the name prefixing, like this: { instance : { flavorRef : 2, name : as, datastore: { type : e60153d4-8ac4-414a-ad58-fe2e0035704a, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 2. I also think a datastore_version alone should be sufficient since the associated datastore type will be implied: { instance : { flavorRef : 2, name : as, datastore: { version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 3. Additionally, while a datastore_type should have an ID in the Trove infastructure database, it should also be possible to pass just the name of the datastore type to the instance call, such as mysql or mongo. Maybe we could allow this in addition to the ID? I think this form should actually use the argument type, and the id should then be passed as type_id instead. { instance : { flavorRef : 2, name : as, datastore: { type : mysql, version : 94ed1f9f-6c1a-4d6e-87e9-04ecff37b64b } volume : { size : 1 } } } 4. Additionally, in the current pull request to implement this it is possible to avoid passing a version, but only if no more than one version of the datastore_type exists in the database. I think instead the datastore_type row in the database should also have a default_version_id property, that an operator could update to the most recent version or whatever other criteria they wish to use, meaning the call could become this simple: { instance : { flavorRef : 2, name : as, datastore: { type : mysql } volume : { size : 1 } } } Thoughts? Thanks, Tim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev