I agree with this last email from Rawlin. I don't think that imposing structure here will gain us very much, and one can still save JSON/whatever in the proposed field without the additional complexity as Rawlin pointed out. In the discussion thus far, we used a lot of prior examples as reasons to not do what Anuj proposed, yet none of the prior examples actually do what he needs. Instead, they showed how we have been undisciplined in the past with our schema, and were used as reasons to not move forward. I don't agree with that sentiment.
Specifically, Anuj would like a text field to store "service/serial id of the servers and any specific info for a server for which no field is available." The key part of this is the second half, not the first half, which we could clearly accomplish with existing fields. I view this specifically as a place to store data "for which no field is available." I think that's a reasonable request. How this field is used, or not, is solely up to the operator of the CDN. If we have problems with the schema generally, instead of being -1 on any new changes, we should evaluate the two issues separately. If we don't like how the various fields or other aspects of the schema work, then let's work toward resolving that instead of being -1 on new ideas that may provide tangible benefit for our users. Perhaps this field would need to move elsewhere if we refactored the server table in the future, but let's not let that prevent us from moving forward today. In general I'm fine with this feature request. I'm going to refrain from using a +1 since this isn't a real vote. :) -- Thanks, Jeff On Tue, Nov 20, 2018 at 3:13 PM Rawlin Peters <[email protected]> wrote: > > I think there's naturally going to be a lot of data overlap (IP > addresses, hostnames, etc) between TO and whatever asset management > system you're using for all your CDN servers, so I can see the benefit > of trying to just stick it all in one place (TO) rather than having to > deal with importing/exporting data to/from multiple asset management > systems and keeping the data synced across all the systems. > > However, maybe we shouldn't require the field to be a json blob. If > it's just an arbitrary text "description" field, you can really put > whatever your heart desires into it (json, key/value pairs, > base64-encoded data, etc). Then as a project we're not on the hook for > making it easy to edit arbitrary json via the Traffic Portal UI, but > end users can still build whatever they want into those fields for > their own purposes. > > - Rawlin > > On Tue, Nov 20, 2018 at 2:36 PM Robert Butts <[email protected]> wrote: > > > > This sounds like an Asset Management thing, not a CDN thing. We currently > > have a number of "Asset Management" fields (like ASNs) which aren't > > necessary for the CDN itself, and just asset tracking. Personally, I'd > > rather see us move toward the CDN being a CDN, and operators using an Asset > > Management system if they need one. Separation of Concerns and all that. > > > > The first hit searching for "open source asset management system" is > > https://snipeitapp.com - but there are many. Shouldn't be too hard for an > > operator to put TC names and IDs in one for cross-reference. > > > > Likewise for things like Ansible variables. Seems like the TO database > > should be the TO/TC database, and data for other systems should be > > maintained by them. Rather than shoving everything from every system into > > the CDN database. > > > > But that's my opinion. If the quorum disagrees, it's not a hill I'm going > > to die on. > > > > On Tue, Nov 20, 2018 at 2:30 PM Steve Malenfant <[email protected]> > > wrote: > > > > > +1 json would help us with Ansible as well to store various variables. > > > would be nice to have by cache group, servers, type... maybe not possible. > > > > > > On Tue, Nov 20, 2018 at 12:52 PM Rawlin Peters <[email protected]> > > > wrote: > > > > > > > I think I'm mostly +1 on the compromise of adding an arbitrary jsonb > > > > column. Then it is up to whatever downstream components might be using > > > > the column to make sure it works for their purposes. In the API we > > > > could just validate that it's actually valid JSON, and I don't think > > > > we'd need any other validation on it. Then it would be up to the > > > > operator to come up with their own versioned JSON schema for the jsonb > > > > column for their downstream components to use. Editing JSON directly > > > > via TP is not ideal, but I'm sure there could be a way to generate a > > > > usable input form given a JSON schema. The schema would be > > > > version-controlled and placed down with TP via whatever configuration > > > > management tool you're using. Then you can easily add whatever > > > > arbitrary data you want to add to a server without polluting the DB > > > > with multiple columns that aren't actually consumed by TC components. > > > > Columns like all the ilo_* ones, rack, mgmt_ip_*, etc. would be a good > > > > fit for that I think. Those columns aren't consumed by other TC > > > > components directly and are mostly just informational or consumed by > > > > things outside of Traffic Control, so maybe they'd be better off as > > > > second-class citizens. > > > > > > > > So basically this: > > > > If the new column is consumed by a TC component for the purpose of > > > > control flow, it deserves a first-class column in the DB. If the new > > > > field is not consumed by TC components, meant to be consumed by non-TC > > > > components (e.g. configuration management), or is purely informational > > > > for human consumption, it can just be relegated to the jsonb column. > > > > > > > > What do you all think? > > > > > > > > - Rawlin > > > > > > > > On Tue, Nov 20, 2018 at 9:30 AM Gray, Jonathan > > > > <[email protected]> wrote: > > > > > > > > > > DS lifecycle tracking was supposed to modeled via DSR Comments. Also, > > > > we do have 3 existing comment fields on a DS. They've all been co-opted > > > > for other various purposes already. That's why they use different names > > > in > > > > TP than in the database. > > > > > > > > > > Jonathan G > > > > > > > > > > > > > > > On 11/20/18, 8:54 AM, "Jason Tucker" <[email protected]> wrote: > > > > > > > > > > Right, we do... but try reading or writing paragraphs of info in > > > > them. The > > > > > DB fields may support it, but the UI not so much. > > > > > > > > > > __Jason > > > > > > > > > > On Tue, Nov 20, 2018 at 9:27 AM Fieck, Brennan < > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > We have three such fields for Delivery Services, afaik :P > > > > > > ________________________________________ > > > > > > From: Jason Tucker <[email protected]> > > > > > > Sent: Tuesday, November 20, 2018 7:25 AM > > > > > > To: [email protected] > > > > > > Subject: Re: [EXTERNAL] Re: Adding a text field in Servers > > > > > config > > > > of TP > > > > > > > > > > > > I'm actually a fan of arbitrary text boxes for more than just > > > > server > > > > > > objects. I've been hoping for something like this in delivery > > > > service > > > > > > objects as well, as this sort of field can be used to help > > > document > > > > > > unusual/custom/snowflake behavior which may not necessarily be > > > > obvious to > > > > > > those who come later with the intention of troubleshooting. > > > Should > > > > be used > > > > > > for communicating with humans, rather than systems. In lieu of > > > > versioning > > > > > > of configs, it could be used to keep a change log for the object > > > > as well. > > > > > > But, again, that's more applicable to DS objects rather than > > > Server > > > > > > objects, I think. > > > > > > > > > > > > __Jason > > > > > > > > > > > > On Mon, Nov 19, 2018 at 9:48 PM Gray, Jonathan < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > I'm -1 depending on what the intended use case is. > > > > > > > > > > > > > > Generic text fields should only be useful to human operators. > > > > In the > > > > > > case > > > > > > > where you intend anything to programmatically access that > > > > information and > > > > > > > it's generally useful, you're better off with specific columns > > > > per point > > > > > > of > > > > > > > data. This is how we ended up with unparsable, yet critical, > > > > data in the > > > > > > > comment fields of physical location table when we should have > > > > added real > > > > > > > columns. The example in the issue is delivery services. The > > > > description > > > > > > > field that I think is being referenced is one of LongDesc, > > > > LongDesc_1, or > > > > > > > LongDesc_2 in the database. Columns should have one purpose > > > and > > > > one > > > > > > > meaning that is clear to a new developer working in the code > > > and > > > > > > > conceptually plausible to anyone else trying to understand how > > > > the system > > > > > > > works. One compromise, I'm not a huge fan of, would be to > > > allow > > > > for > > > > > > > arbitrary structured data via a column of type jsonb instead > > > > > of > > > > text. > > > > > > > That's not a great answer from a usability or db theory > > > > perspective, but > > > > > > > it's slightly better than regex parsing. > > > > > > > > > > > > > > Jonathan G > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/19/18, 3:09 PM, "Dave Neuman" <[email protected]> wrote: > > > > > > > > > > > > > > +1, I am fine with it. That table already has a lot of > > > > columns, > > > > > > > what's one > > > > > > > more!? > > > > > > > > > > > > > > On Mon, Nov 19, 2018 at 2:59 PM Jeremy Mitchell < > > > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Sounds like server "notes" or a server "description". > > > > Seems like a > > > > > > > fair > > > > > > > > ask. I don't see the harm in adding an optional column > > > > > to > > > > the > > > > > > server > > > > > > > table > > > > > > > > with type=text for this data. > > > > > > > > > > > > > > > > On Mon, Nov 19, 2018 at 2:16 PM Anuj Tyagi < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hello Traffic Controllers, > > > > > > > > > > > > > > > > > > I discussed this with a couple of ATC users. We have > > > > multiple > > > > > > > > > Description/text fields in Delivery Service > > > > configuration of TP. > > > > > > > > Similarly, > > > > > > > > > We should also have one text field in servers > > > > configuration. My > > > > > > > use case > > > > > > > > is > > > > > > > > > to keep the service/serial id of the servers and any > > > > specific > > > > > > info > > > > > > > for a > > > > > > > > > server for which no field is available. > > > > > > > > > > > > > > > > > > I have created an issue on GitHub for it earlier: > > > > > > > > > https://github.com/apache/trafficcontrol/issues/2764 > > > > > > > > > <https://github.com/apache/trafficcontrol/issues/2764> > > > > > > > > > > > > > > > > > > It's not a major change so shouldn't be a problem. If > > > > everyone > > > > > > > agrees, > > > > > > > > I'd > > > > > > > > > be interested to add that. > > > > > > > > > > > > > > > > > > Thank you > > > > > > > > > Anuj Tyagi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
