>So actually we don't really need to add a new API endpoint at all Sure. I don't object to renaming, if people want to. But the existing one does work, yes.
>Those endpoints should be sufficient enough to support the legacy format import/export There's no reason to support legacy export, just import. I'm 100% +1 on deprecating and removing the old export. >Going forward the import/export in TP should be done with the /api/*/profiles format (assuming we add parameters support to that API), with a separate button for "legacy" import which uses the deprecated /api/*/profiles/import format +1 >But IMO there's not really a good reason to export profiles and save them outside of TO for long periods of time I personally don't have a use case either; but I'm sure someone does, or will. On Mon, Sep 17, 2018 at 4:11 PM Rawlin Peters <[email protected]> wrote: > So actually we don't really need to add a new API endpoint at all if > my understanding is correct. We currently have > /api/*/profiles/:id/export and /api/*/profiles/import. Those endpoints > should be sufficient enough to support the legacy format import/export > without the need for new endpoints, right? The real issue is exposing > them via TP. Going forward the import/export in TP should be done with > the /api/*/profiles format (assuming we add parameters support to that > API), with a separate button for "legacy" import which uses the > deprecated /api/*/profiles/import format. But IMO there's not really a > good reason to export profiles and save them outside of TO for long > periods of time, so I think the whole import/export thing is really > meant for default profiles or for quickly exporting profiles from one > TO instance and importing into another. As long as the default > profiles are updated to the new format, I think that would solve most > of the problem for our users. > > - Rawlin > > On Mon, Sep 17, 2018 at 3:05 PM Robert Butts <[email protected]> wrote: > > > > >It just seems wrong to me to add a new API endpoint to support a > > deprecated format. > > > > So people who've exported their data are just out of luck? We just "don't > > support" migrating your data forward? It seems egregiously wrong to me > > _not_ to support importing old data, for any app or service, ever. If we > > gave people a way to create exported data, it's our job to continue to > > provide them a way to import it. Without them having to pay for > development > > time for an engineer to do a data conversion. > > > > >IMO if we do add a new endpoint just to support the deprecated legacy > > format, then it should also be marked "deprecated" for removal in the > next > > release. > > > > The entire point of supporting importing legacy data is _not_ to > deprecate > > it. This isn't about API versioning, and deprecating old undesirable > > protocols. That's absolutely the right way to progress the API itself. > This > > is about providing a way to move data forward. Users have exported files, > > from an export we gave them. It's the job of any well-behaved product to > > provide a mechanism to import old formats, when the format changes. > > > > How would you feel if you were using a product, say Google Docs, or > > Microsoft Excel, and they suddenly said, "Sorry, your old documents don't > > work in the latest version." "Oh, but we provide a command-line script to > > migrate them, you just have to open MS-DOS (or maybe Powershell?), and > run > > this script, with these arguments. On all your documents." Sure, as an > > engineer you could do that, however inconvenient; how would you feel if > you > > read that and were an average user, who'd never seen a command line > before, > > and only ever used the GUI? > > > > > > On Mon, Sep 17, 2018 at 2:44 PM Rawlin Peters <[email protected]> > > wrote: > > > > > It just seems wrong to me to add a new API endpoint to support a > > > deprecated format. There's much more involved in maintaining an API > > > endpoint as opposed to a one-off script to convert formats. IMO if we > > > do add a new endpoint just to support the deprecated legacy format, > > > then it should also be marked "deprecated" for removal in the next > > > release. > > > > > > - Rawlin > > > > > > On Mon, Sep 17, 2018 at 1:51 PM Robert Butts <[email protected]> wrote: > > > > > > > > >Do we really want to add a new API endpoint just to support the > legacy > > > > format > > > > > > > > An endpoint and UI page are easier to use for Self-Service CDN > consumers. > > > > Some CDN tenants might not know how to run a script or use a command > > > line. > > > > I'm not sure I understand the objection. Why is it such a big deal > to add > > > > an endpoint? And/or to support importing old formats? IMO importing > old > > > > data formats should be a first-class citizen, in any API or UI. Data > > > jails > > > > are bad. > > > > > > > > >traffic_ops/app/bin/config2json > > > > > > > > The difference is that the TO config is used by system > administrators. As > > > > Self-Service moves forward, profiles and other API data will be used > by > > > CDN > > > > tenants, who may have very little technical knowledge. We need to > make > > > sure > > > > anything a Self-Service tenant may need to do has a simple graphical > way > > > to > > > > do it, and also an API so people deploying TC can write their own > UIs if > > > > necessary. > > > > > > > > > > > > On Mon, Sep 17, 2018 at 1:23 PM Rawlin Peters < > [email protected]> > > > > wrote: > > > > > > > > > Do we really want to add a new API endpoint just to support the > legacy > > > > > format? Couldn't we just provide a script that converts between the > > > > > two formats, similar to what we did for cdn.conf with > > > > > traffic_ops/app/bin/config2json? > > > > > > > > > > - Rawlin > > > > > On Mon, Sep 17, 2018 at 11:04 AM Robert Butts <[email protected]> > wrote: > > > > > > > > > > > > Ah, excellent. It wasn't clear from the docs that `POST > > > > > /api/1.2/profiles` > > > > > > and `GET /api/1.2/profiles/:id` accept parameters. We should fix > > > that. > > > > > > > > > > > > That said, I would be in favor of adding a new endpoint, to > import > > > in the > > > > > > old format, `/api/profile/import-legacy` or something. Just so > people > > > > > with > > > > > > old exports can import them back in. Data jails are bad (I know > it's > > > > > > human-readable and not that difficult to convert, but it'd still > be > > > > > > frustrating). > > > > > > > > > > > > I made Github issues for both: > > > > > > https://github.com/apache/trafficcontrol/issues/2827 > > > > > > https://github.com/apache/trafficcontrol/issues/2826 > > > > > > > > > > > > On Mon, Sep 17, 2018 at 10:08 AM Dan Kirkwood <[email protected] > > > > > wrote: > > > > > > > > > > > > > yes -- that's really what I meant. Import/Export should not > be a > > > > > separate > > > > > > > operation as documented here: > > > > > > > > > > > > > > > > > > > > > > > https://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/default_profiles.html?highlight=import > > > > > > > , > > > > > > > but should be part of the "normal" API. The default profile > files > > > we > > > > > keep > > > > > > > in http://trafficcontrol.apache.org/downloads/profiles/ are > in the > > > > > format > > > > > > > I'm describing. They should be in the more standard form and > > > should > > > > > > > associate all parameters in the file with the profile. > > > > > > > > > > > > > > The Go `POST api/1.3/profiles` endpoint already loads the > > > parameters > > > > > into > > > > > > > memory, but ignores the parameters list. > > > > > > > > > > > > > > -dan > > > > > > > > > > > > > > On Mon, Sep 17, 2018 at 9:58 AM Jeremy Mitchell < > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Maybe this is what Dan said but imo there should be no > > > import/export > > > > > > > > endpoints but these should do the job: > > > > > > > > > > > > > > > > GET /api/*/profiles/:id <-- this is essentially an export. it > > > gives > > > > > you > > > > > > > the > > > > > > > > json of a profile (along with all the parameters). save the > json > > > to a > > > > > > > file > > > > > > > > and you've essentially exported the profile. > > > > > > > > POST /api/*/profiles <-- this is how you create profiles. > if you > > > > > supply > > > > > > > > parameters as well you've essentially done an import. > > > > > > > > > > > > > > > > jeremy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 17, 2018 at 9:16 AM Robert Butts <[email protected] > > > > > wrote: > > > > > > > > > > > > > > > > > Can you clarify, are you referring to `/profile/import` and > > > > > > > > > `/profile/:id/export`? Or > `/api/$version/profiles/:id/export` > > > and > > > > > > > > > `/api/$version/profiles/import`? > > > > > > > > > > > > > > > > > > We should definitely deprecate and remove the non-API > > > endpoints. > > > > > But > > > > > > > IMO > > > > > > > > we > > > > > > > > > need to keep a way to create and get an entire profile, > with > > > > > > > parameters, > > > > > > > > in > > > > > > > > > a single request. Users shouldn't have to make a dozen > > > requests to > > > > > > > import > > > > > > > > > and export a profile. > > > > > > > > > > > > > > > > > > Also +1 on changing them to real booleans (not sure if the > api > > > > > > > > > export/import is, they appear to be undocumented). Though > we > > > can't > > > > > > > break > > > > > > > > > the current API; we could make the POST accept either, but > the > > > GET > > > > > > > would > > > > > > > > > have to be a new endpoint I think. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 15, 2018 at 3:15 PM Dan Kirkwood < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I’d like to propose deprecating the import/export format > of > > > > > profiles. > > > > > > > > The > > > > > > > > > > current format (Perl-based) is inconsistent with the > standard > > > > > POST > > > > > > > > > > api/x.x/profiles format. Import/Export can/should be done > > > using > > > > > the > > > > > > > > same > > > > > > > > > > API. Traffic Portal Import should use the standard API > > > > > endpoint. > > > > > > > > > > > > > > > > > > > > Import/export (shown below) has this form which includes > the > > > > > > > "profile" > > > > > > > > > key > > > > > > > > > > at the top level. The "secure" option in the > "parameters" > > > secion > > > > > > > uses > > > > > > > > a > > > > > > > > > > 0/1 rather than a boolean true/false. These 2 things > make it > > > > > > > > > inconsistent. > > > > > > > > > > > > > > > > > > > > Opinions are welcome... > > > > > > > > > > > > > > > > > > > > Dan > > > > > > > > > > > > > > > > > > > > { > > > > > > > > > > "profile": { > > > > > > > > > > "name": "myname", > > > > > > > > > > ... > > > > > > > > > > }, > > > > > > > > > > "parameters": [ > > > > > > > > > > { > > > > > > > > > > "name": "foo", > > > > > > > > > > "secure": 0, > > > > > > > > > > ... > > > > > > > > > > ] > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
