There is already a ticket logged for this, https://issues.apache.org/jira/browse/CLOUDSTACK-4942
Fix should be introducing a new table say storage_tag and follow the design of host_tag and also upgrade steps for existing storage tags present in the details. Prachi -----Original Message----- From: Marcus Sorensen [mailto:shadow...@gmail.com] Sent: Monday, January 27, 2014 11:32 AM To: dev@cloudstack.apache.org Subject: Re: Tags on storagePool Right, for some reason I was thinking in the moment that all details would have a value of 'true'. Since that's not the case, and this field is clearly not a boolean field, we should just change them to storage_tag for a value, as suggested. On Mon, Jan 27, 2014 at 12:28 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> wrote: > I agree...we should fix it in 4.4. > > Would you write up a JIRA ticket for it, Chris? > > Thanks! > > > On Mon, Jan 27, 2014 at 12:26 PM, SuichII, Christopher < > chris.su...@netapp.com> wrote: > >> I suspect the reason you don't have this problem is that you don't >> have any storage pool details whose value is 'true'. That is the only >> case where this problem arises. >> >> I think we do have a problem differentiating, though. If you list all >> of the storage tags for a storage pool, then any storage pool details >> whose value is 'true' will be assumed to be a storage tag. The only >> way around this problem I see is to cross reference the results there >> with all of the storage tags from disk offerings, and if you find a >> storage pool detail that doesn't match a disk offering storage tag, >> then assume it isn't a storage tag. >> >> Unfortunately it sounds like this is way out of scope for 4.3. I >> think we need to start thinking about how we can fix this issue >> moving forward now that there will be plugins out there impacted by this. >> >> -Chris >> -- >> Chris Suich >> chris.su...@netapp.com >> NetApp Software Engineer >> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat >> >> On Jan 27, 2014, at 2:21 PM, Marcus Sorensen <shadow...@gmail.com> wrote: >> >> > I'm ok with whatever. It does seem like we have the ability to >> > differentiate real storage tags from details, since we can >> > list/edit those storage tags (I don't see all of my details showing >> > up in the UI when I look at the tags for my primary storage), >> > though I'm not immediately sure why or how at the moment. I think >> > this affects Mike the most, since he's shipping a plugin to >> > customers. I can just look at whatever changes are made and update my >> > pools manually, if need be. >> > >> > On Mon, Jan 27, 2014 at 11:43 AM, Mike Tutkowski >> > <mike.tutkow...@solidfire.com> wrote: >> >> I agree...it certainly is not an ideal situation. >> >> >> >> Have you assessed the risk involved with changing storage tags >> >> from >> using >> >> true as a value to using something like storage_tag as a value? >> >> >> >> If so, do you feel it is an acceptable amount of risk for 4.3 now >> >> that >> we >> >> have already begun spinning up RC builds? >> >> >> >> Thanks >> >> >> >> >> >> On Mon, Jan 27, 2014 at 8:40 AM, SuichII, Christopher < >> >> chris.su...@netapp.com> wrote: >> >> >> >>> I think my only real concern with working around it in this >> >>> manner is >> that >> >>> once a fix is applied, we now have a bunch of settings that do >> >>> not >> conform >> >>> to the true/false pattern. In order to migrate them from using >> >>> t/f, >> yes/no >> >>> or enabled/disabled back to using true/false, we would have to >> >>> write >> some >> >>> kind of DB upgrade/migration in our plugin to update these >> >>> values. In addition, I would have to change the ConfigKey from >> >>> being a boolean to >> a >> >>> string and then back to a boolean once a solution is implemented. >> >>> It is just some technical debt that would pile up for plugins >> >>> with a pretty >> nasty >> >>> fix down the road. >> >>> >> >>> -Chris >> >>> -- >> >>> Chris Suich >> >>> chris.su...@netapp.com >> >>> NetApp Software Engineer >> >>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat >> >>> >> >>> On Jan 27, 2014, at 10:33 AM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> >> >>> wrote: >> >>> >> >>>> It is a problem, but I think - at the moment - only NetApp, >> >>>> SolidFire, >> >>> and >> >>>> Marcus have storage plug-ins. >> >>>> >> >>>> We could document this issue and the easy workaround of not >> >>>> using true >> >>> and >> >>>> false as a value and try to address it in 4.4. >> >>>> >> >>>> >> >>>> On Mon, Jan 27, 2014 at 7:53 AM, SuichII, Christopher < >> >>>> chris.su...@netapp.com> wrote: >> >>>> >> >>>>> Any final thoughts on this? Letting this go into 4.3 could >> potentially >> >>>>> cause issues with anyone using the configuration system for >> >>>>> plugins >> in >> >>> 4.3. >> >>>>> >> >>>>> -Chris >> >>>>> -- >> >>>>> Chris Suich >> >>>>> chris.su...@netapp.com >> >>>>> NetApp Software Engineer >> >>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red Hat >> >>>>> >> >>>>> On Jan 24, 2014, at 3:34 PM, SuichII, Christopher < >> >>> chris.su...@netapp.com> >> >>>>> wrote: >> >>>>> >> >>>>>> This is the approach we'll have to take if we can't fix this >> >>>>>> in >> 4.3. It >> >>>>> certainly works, but it isn't ideal. >> >>>>>> >> >>>>>> Does anyone else have any thoughts on the impact this might >> >>>>>> have to >> >>> 4.3? >> >>>>>> >> >>>>>> -Chris >> >>>>>> -- >> >>>>>> Chris Suich >> >>>>>> chris.su...@netapp.com >> >>>>>> NetApp Software Engineer >> >>>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & Red >> >>>>>> Hat >> >>>>>> >> >>>>>> On Jan 24, 2014, at 10:11 AM, Mike Tutkowski < >> >>>>> mike.tutkow...@solidfire.com> wrote: >> >>>>>> >> >>>>>>> This isn't a great solution, but you could also change the >> >>>>>>> value >> for >> >>>>> your >> >>>>>>> plug-in from true or false to something like t or f. >> >>>>>>> >> >>>>>>> >> >>>>>>> On Fri, Jan 24, 2014 at 8:08 AM, Mike Tutkowski < >> >>>>>>> mike.tutkow...@solidfire.com> wrote: >> >>>>>>> >> >>>>>>>> Edison could confirm this perhaps, but I doubt any current >> >>> installation >> >>>>>>>> would have true for the value unless it was for a storage >> >>>>>>>> tag (the >> >>>>> plug-in >> >>>>>>>> framework just came out in 4.2). >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Fri, Jan 24, 2014 at 8:05 AM, Mike Tutkowski < >> >>>>>>>> mike.tutkow...@solidfire.com> wrote: >> >>>>>>>> >> >>>>>>>>> I think your idea would be acceptable risk for 4.3. The >> >>>>>>>>> upgrade >> >>> logic >> >>>>>>>>> would have to perform this true-to-storage_tag conversion, >> >>>>>>>>> too, >> >>>>> though. >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On Fri, Jan 24, 2014 at 8:00 AM, SuichII, Christopher < >> >>>>>>>>> chris.su...@netapp.com> wrote: >> >>>>>>>>> >> >>>>>>>>>> I think the quickest, easiest change would be to keep >> >>>>>>>>>> using the >> tag >> >>>>> name >> >>>>>>>>>> as the key in the details table, but use a unique value >> >>>>>>>>>> like >> >>>>> 'storage_tag' >> >>>>>>>>>> instead of 'true'. This wouldn't require any major logic >> changes, >> >>>>> just the >> >>>>>>>>>> value used to check whether something is a storage tag. >> >>>>>>>>>> >> >>>>>>>>>> It is quite risky for 4.3. However my concern is that if >> >>>>>>>>>> we let >> >>> this >> >>>>>>>>>> ship with 4.3, then any plugin that wants to use a storage >> >>>>>>>>>> pool >> >>>>> detail with >> >>>>>>>>>> the value 'true' will make updating the storage tag system >> >>>>>>>>>> MUCH >> >>> more >> >>>>>>>>>> difficult. As far as I can tell, there is no other way to >> determine >> >>>>> if >> >>>>>>>>>> something is a storage tag than checking if the details >> >>>>>>>>>> table >> value >> >>>>> is >> >>>>>>>>>> 'true'. If there are other details with the value 'true', >> >>>>>>>>>> then >> we >> >>>>> wouldn't >> >>>>>>>>>> be able to differentiate between them for a DB upgrade >> >>>>>>>>>> between >> >>>>> versions. >> >>>>>>>>>> >> >>>>>>>>>> -Chris >> >>>>>>>>>> -- >> >>>>>>>>>> Chris Suich >> >>>>>>>>>> chris.su...@netapp.com >> >>>>>>>>>> NetApp Software Engineer >> >>>>>>>>>> Data Center Platforms - Cloud Solutions Citrix, Cisco & >> >>>>>>>>>> Red Hat >> >>>>>>>>>> >> >>>>>>>>>> On Jan 24, 2014, at 9:53 AM, Mike Tutkowski < >> >>>>>>>>>> mike.tutkow...@solidfire.com> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> I think at some point we need to use a key/value for >> >>>>>>>>>>> storage >> tags >> >>>>> such >> >>>>>>>>>> as >> >>>>>>>>>>> the following: >> >>>>>>>>>>> >> >>>>>>>>>>> storageTags=value1,value2,value3 >> >>>>>>>>>>> >> >>>>>>>>>>> The problem with that is you have to parse the value cell >> >>>>>>>>>>> each >> >>> time >> >>>>> you >> >>>>>>>>>>> pull it out of the DB. >> >>>>>>>>>>> >> >>>>>>>>>>> That might be too risky of a change, though, for 4.3 at >> >>>>>>>>>>> this >> >>> point. >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> On Fri, Jan 24, 2014 at 5:24 AM, SuichII, Christopher < >> >>>>>>>>>>> chris.su...@netapp.com> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> I have found an additional issue related to this. The >> allocators >> >>> do >> >>>>>>>>>>>> properly ignore any storage pool details whose value is >> >>>>>>>>>>>> true >> that >> >>>>> is >> >>>>>>>>>> not >> >>>>>>>>>>>> actually a storage pool. However, the list storage pools >> >>>>>>>>>>>> API >> does >> >>>>>>>>>> NOT. When >> >>>>>>>>>>>> creating the StoragePoolResponse, it is still assumed >> >>>>>>>>>>>> that any >> >>>>>>>>>> storage pool >> >>>>>>>>>>>> detail with the value 'true' is a storage tag. >> >>>>>>>>>>>> >> >>>>>>>>>>>> For my plugin targeting 4.3, we create a storage pool >> >>>>>>>>>>>> detail >> >>> with a >> >>>>>>>>>>>> true/false value, so this causes a problem with the >> >>>>>>>>>>>> storage >> pool >> >>>>> UI. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Any thoughts on how to fix this? >> >>>>>>>>>>>> >> >>>>>>>>>>>> -Chris >> >>>>>>>>>>>> -- >> >>>>>>>>>>>> Chris Suich >> >>>>>>>>>>>> chris.su...@netapp.com >> >>>>>>>>>>>> NetApp Software Engineer Data Center Platforms - Cloud >> >>>>>>>>>>>> Solutions Citrix, Cisco & Red Hat >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Oct 23, 2013, at 6:43 PM, Alena Prokharchyk < >> >>>>>>>>>>>> alena.prokharc...@citrix.com> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> Filed >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-4942 >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Alena. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> From: Prachi Damle <prachi.da...@citrix.com<mailto: >> >>>>>>>>>>>> prachi.da...@citrix.com>> >> >>>>>>>>>>>>> Date: Wednesday, October 23, 2013 2:04 PM >> >>>>>>>>>>>>> To: Alena Prokharchyk <alena.prokharc...@citrix.com<mailto: >> >>>>>>>>>>>> alena.prokharc...@citrix.com>>, >> >>>>>>>>>>>> "dev@cloudstack.apache.org >> >>> <mailto: >> >>>>>>>>>>>> dev@cloudstack.apache.org>" <dev@cloudstack.apache.org >> <mailto: >> >>>>>>>>>>>> dev@cloudstack.apache.org>> >> >>>>>>>>>>>>> Subject: RE: Tags on storagePool >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Alena, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I don't know why it was designed this way in first >> >>>>>>>>>>>>> place - a >> >>>>> design >> >>>>>>>>>> like >> >>>>>>>>>>>> host_tags where we have separate table to store tags is >> >>>>>>>>>>>> much >> >>> better >> >>>>>>>>>> for >> >>>>>>>>>>>> Allocators to work on. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> It is a bug, but will cause problem only if we land up >> >>>>>>>>>>>>> with >> >>>>> situation >> >>>>>>>>>>>> explained below: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Given the existing design of storage tags, the >> >>>>>>>>>>>>> Allocators >> search >> >>>>> the >> >>>>>>>>>>>> details table using the name = >> >>>>>>>>>>>> <tag-provided-in-disk-offering> >> >>> and >> >>>>>>>>>> value >> >>>>>>>>>>>> =true >> >>>>>>>>>>>>> Thus this will cause a problem in placement only if >> >>>>>>>>>>>>> some >> other >> >>>>>>>>>> storage >> >>>>>>>>>>>> pool detail happen to have the same 'name' as a >> >>>>>>>>>>>> storage-tag >> and >> >>>>> also a >> >>>>>>>>>>>> value = true. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Prachi >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> From: Alena Prokharchyk >> >>>>>>>>>>>>> Sent: Wednesday, October 23, 2013 1:36 PM >> >>>>>>>>>>>>> To: Prachi Damle; dev@cloudstack.apache.org<mailto: >> >>>>>>>>>>>> dev@cloudstack.apache.org> >> >>>>>>>>>>>>> Subject: Tags on storagePool >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I came across a potential bug in the way allocators do >> volumes >> >>>>>>>>>> placement >> >>>>>>>>>>>> on storage, based on storage tags. Prachi, can you >> >>>>>>>>>>>> please >> confirm >> >>>>> if >> >>>>>>>>>> the >> >>>>>>>>>>>> bug is real. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> The tags are passed to the createStoragePool API in >> >>>>>>>>>>>>> form of >> >>>>>>>>>>>> tags='tag1,tag2,..': >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> ?command=createStoragePool&...&tags=alena >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> and stored in the storage_pool_details db table as: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> mysql> select *from storage_pool_details where >> >>>>>>>>>>>>> mysql> pool_id=2; >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | id | pool_id | name | value | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | 2 | 2 | alenatags | true | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> 1 row in set (0.00 sec) >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Allocator code assumes that everything stored in >> >>>>> storage_pool_details >> >>>>>>>>>>>> table, having value=true - is a storage pool tag. And >> >>>>>>>>>>>> this is >> >>>>>>>>>> incorrect, as >> >>>>>>>>>>>> the storage_pool_details table is used for storing diff >> >>>>>>>>>>>> kinds >> of >> >>>>>>>>>> storage >> >>>>>>>>>>>> pool details - not just tags - that can be later used by >> various >> >>>>>>>>>> cloudStack >> >>>>>>>>>>>> managers. Like for example we save Storage level config >> >>> parameters >> >>>>> in >> >>>>>>>>>> this >> >>>>>>>>>>>> table ( >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>> >> >>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+for+Granula >> r+Global+Configuration+Parameters >> >>>>>>>>>>>> ). >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> The correct way to fix it would be - store tags as: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | id | pool_id | name | value | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> | 2 | 2 | tag | alena | >> >>>>>>>>>>>>> +----+---------+-----------+-------+ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> and fix StorageManager to retrive all the tags by the "tag" >> key. >> >>>>> We >> >>>>>>>>>> also >> >>>>>>>>>>>> have to fix the DB upgrade, which can be tricky as we >> >>>>>>>>>>>> will >> have >> >>> to >> >>>>>>>>>> figure >> >>>>>>>>>>>> out which detail is a tag. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> -Alena. >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> -- >> >>>>>>>>>>> *Mike Tutkowski* >> >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>>>>> e: mike.tutkow...@solidfire.com >> >>>>>>>>>>> o: 303.746.7302 >> >>>>>>>>>>> Advancing the way the world uses the >> >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >> >>>>>>>>>>> *(tm)* >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> -- >> >>>>>>>>> *Mike Tutkowski* >> >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>>> e: mike.tutkow...@solidfire.com >> >>>>>>>>> o: 303.746.7302 >> >>>>>>>>> Advancing the way the world uses the cloud< >> >>>>> http://solidfire.com/solution/overview/?video=play> >> >>>>>>>>> *(tm)* >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> *Mike Tutkowski* >> >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>>> e: mike.tutkow...@solidfire.com >> >>>>>>>> o: 303.746.7302 >> >>>>>>>> Advancing the way the world uses the cloud< >> >>>>> http://solidfire.com/solution/overview/?video=play> >> >>>>>>>> *(tm)* >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> *Mike Tutkowski* >> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>>>>> e: mike.tutkow...@solidfire.com >> >>>>>>> o: 303.746.7302 >> >>>>>>> Advancing the way the world uses the >> >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >> >>>>>>> *(tm)* >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>>> -- >> >>>> *Mike Tutkowski* >> >>>> *Senior CloudStack Developer, SolidFire Inc.* >> >>>> e: mike.tutkow...@solidfire.com >> >>>> o: 303.746.7302 >> >>>> Advancing the way the world uses the >> >>>> cloud<http://solidfire.com/solution/overview/?video=play> >> >>>> *(tm)* >> >>> >> >>> >> >> >> >> >> >> -- >> >> *Mike Tutkowski* >> >> *Senior CloudStack Developer, SolidFire Inc.* >> >> e: mike.tutkow...@solidfire.com >> >> o: 303.746.7302 >> >> Advancing the way the world uses the >> >> cloud<http://solidfire.com/solution/overview/?video=play> >> >> *(tm)* >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *(tm)*
