Re: Traffic Server Secondary Streaming IPs Design

[Dewayne Richardson]

I agree with Rob, please explore the Go implementation, we're reaching
critical mass (most of the critical foundational routes are complete) with
the API endpoints written in Go with the big one /deliveryservices being
the biggest and most important to complete.

-Dew

On Fri, Apr 20, 2018 at 8:37 AM, Robert Butts 
wrote:

> @weifensh We're hoping to have most of the API endpoints, not including ATS
> config files, in the next month or so. I'm currently working on
> `deliveryservices`, and @dylanvoltz is on `servers`, the two biggest, and
> they're both mostly done. They should be code-complete by the end of next
> week, and then maybe a week for testing. Most of the rest are
> straightforward DB->JSON->HTTP mappings without much logic, they should go
> pretty fast.
>
> Do you have a list of endpoints you'll have to modify for this feature? We
> can certainly prioritize them, if you give us a list.
>
> Will you be modifying any ATS config files generated by Traffic Ops? Those
> are lower on our list, but again, if you have some, we can definitely
> prioritize them.
>
> I'd highly recommend not doing anything new in Perl. At Comcast, we just
> recently allocated people for Self Service, and that's going to require all
> endpoints in Go, so the Perl->Go migration just got bumped to the top of
> our priority list. Since we're not using this "Secondary IP" feature, it's
> going to be harder for us to be sure it's working right when we
> transliterate Perl to Go, and we're likely to break it for you, and we'll
> have to figure out why. It'll just be more painful for everyone to do new
> things in Perl.
>
> You can also convert Perl endpoints to Go yourself, as you need to modify
> them, if you like. Just be sure to coordinate with us, so we don't
> duplicate work.
>
>
> On Fri, Apr 20, 2018 at 3:57 AM, John Shen (weifensh) 
> wrote:
>
> > Hi Nir and all,
> >
> > Do you know when will the full GO version of Traffic Ops be ready in the
> > master branch? The reason I raise this question is that we are starting
> to
> > implement this feature, and if some of the APIs involving this feature
> are
> > still in Perl and will not be ported to GO very soon, we are planning
> > implement them in the Perl. For other APIs, we will implement them in GO.
> > Is there any objection to this plan?
> >
> > Thanks,
> > John
> >
> >
> > On 2018/4/10, 4:22 PM, "Zhilin Huang (zhilhuan)" 
> > wrote:
> >
> > To be clear, "immutable" here means the IP could not be removed, but
> > allow modification.
> >
> > And streaming "priority" is better be Delivery Service based than
> > server based.
> >
> > Thanks,
> > Zhilin
> >
> >
> > On 10/04/2018, 4:12 PM, "Zhilin Huang (zhilhuan)" <
> zhilh...@cisco.com>
> > wrote:
> >
> > Hey Nir,
> >
> > Thanks a lot for your comments. Please see my replies inline.
> >
> > On 10/04/2018, 3:14 AM, "Nir Sopher"  wrote:
> >
> >  Hey Zhilin,
> >
> > Regarding the ports configuration. Even though I believe
> > modeling will be
> > cleaner if the port and IP are set together, you are probably
> > correct - it
> > is reasonable to consider the Port per IP flexibility as a
> > future extension
> > and avoid it for now.
> > Still, I would suggest to at-least module the cr-config with
> > the Port
> > specified per IP (delivery-unit). It is more flexible as well
> > as simplify
> > the router and monitor code.
> > ZH> I understand your consideration about the flexibility. But I
> > still think port is a server lever configuration, do not see the needs of
> > multiple ports in the near future. Anyway, if we want to add port
> > configured together with IP, it is easy to add a new field into the json
> of
> > RESTful API or cr-config, since adding new field is easy to be backward
> > compatible. So I would like to leave this change to future when there is
> > use case required.
> >
> >
> > Regarding the crud of the server configuration, I believe the
> > API should
> > change, but with backward compatibility.
> > Maybe we should have
> > (GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/interfaces
> > And
> > (GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/
> > interfaces/{:ifId}/delivery-units
> > ZH> I went thru those APIs again, and agree with your points. The
> > design doc has been updated to reflect this change in section 3.1.1.3 and
> > 3.1.1.4. A little different than your suggestion is I used "ips" instead
> of
> > "delivery-units" to allow manipulation of management IP and ILO IP as
> well.
> >
> > These APIs will allow us to manipulate all interfaces and all
> > IPs
> > (delivery-units). Note that as I see it, there is no special
> > "primary" IP
> > 

Re: Traffic Server Secondary Streaming IPs Design

[Robert Butts]

@weifensh We're hoping to have most of the API endpoints, not including ATS
config files, in the next month or so. I'm currently working on
`deliveryservices`, and @dylanvoltz is on `servers`, the two biggest, and
they're both mostly done. They should be code-complete by the end of next
week, and then maybe a week for testing. Most of the rest are
straightforward DB->JSON->HTTP mappings without much logic, they should go
pretty fast.

Do you have a list of endpoints you'll have to modify for this feature? We
can certainly prioritize them, if you give us a list.

Will you be modifying any ATS config files generated by Traffic Ops? Those
are lower on our list, but again, if you have some, we can definitely
prioritize them.

I'd highly recommend not doing anything new in Perl. At Comcast, we just
recently allocated people for Self Service, and that's going to require all
endpoints in Go, so the Perl->Go migration just got bumped to the top of
our priority list. Since we're not using this "Secondary IP" feature, it's
going to be harder for us to be sure it's working right when we
transliterate Perl to Go, and we're likely to break it for you, and we'll
have to figure out why. It'll just be more painful for everyone to do new
things in Perl.

You can also convert Perl endpoints to Go yourself, as you need to modify
them, if you like. Just be sure to coordinate with us, so we don't
duplicate work.


On Fri, Apr 20, 2018 at 3:57 AM, John Shen (weifensh) 
wrote:

> Hi Nir and all,
>
> Do you know when will the full GO version of Traffic Ops be ready in the
> master branch? The reason I raise this question is that we are starting to
> implement this feature, and if some of the APIs involving this feature are
> still in Perl and will not be ported to GO very soon, we are planning
> implement them in the Perl. For other APIs, we will implement them in GO.
> Is there any objection to this plan?
>
> Thanks,
> John
>
>
> On 2018/4/10, 4:22 PM, "Zhilin Huang (zhilhuan)" 
> wrote:
>
> To be clear, "immutable" here means the IP could not be removed, but
> allow modification.
>
> And streaming "priority" is better be Delivery Service based than
> server based.
>
> Thanks,
> Zhilin
>
>
> On 10/04/2018, 4:12 PM, "Zhilin Huang (zhilhuan)" 
> wrote:
>
> Hey Nir,
>
> Thanks a lot for your comments. Please see my replies inline.
>
> On 10/04/2018, 3:14 AM, "Nir Sopher"  wrote:
>
>  Hey Zhilin,
>
> Regarding the ports configuration. Even though I believe
> modeling will be
> cleaner if the port and IP are set together, you are probably
> correct - it
> is reasonable to consider the Port per IP flexibility as a
> future extension
> and avoid it for now.
> Still, I would suggest to at-least module the cr-config with
> the Port
> specified per IP (delivery-unit). It is more flexible as well
> as simplify
> the router and monitor code.
> ZH> I understand your consideration about the flexibility. But I
> still think port is a server lever configuration, do not see the needs of
> multiple ports in the near future. Anyway, if we want to add port
> configured together with IP, it is easy to add a new field into the json of
> RESTful API or cr-config, since adding new field is easy to be backward
> compatible. So I would like to leave this change to future when there is
> use case required.
>
>
> Regarding the crud of the server configuration, I believe the
> API should
> change, but with backward compatibility.
> Maybe we should have
> (GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/interfaces
> And
> (GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/
> interfaces/{:ifId}/delivery-units
> ZH> I went thru those APIs again, and agree with your points. The
> design doc has been updated to reflect this change in section 3.1.1.3 and
> 3.1.1.4. A little different than your suggestion is I used "ips" instead of
> "delivery-units" to allow manipulation of management IP and ILO IP as well.
>
> These APIs will allow us to manipulate all interfaces and all
> IPs
> (delivery-units). Note that as I see it, there is no special
> "primary" IP
> (but IPs has priorities).
> ZH> I think there must be a so called "default" streaming IP bind
> with a server until server deleted. A server with no streaming is useless.
> That's how I understand the "primary" IP, which is immutable with a server.
> A "secondary" streaming IP could be add and removed afterwards.
> For the streaming IP priority, I think it is out of the scope of
> this feature, we could add that in some future features. The concept
> streaming "priority" itself could be co-existing with the
> "primary/secondary" concept, in different 

Re: Traffic Server Secondary Streaming IPs Design

[John Shen (weifensh)]

Hi Nir and all,

Do you know when will the full GO version of Traffic Ops be ready in the master 
branch? The reason I raise this question is that we are starting to implement 
this feature, and if some of the APIs involving this feature are still in Perl 
and will not be ported to GO very soon, we are planning implement them in the 
Perl. For other APIs, we will implement them in GO. Is there any objection to 
this plan?
 
Thanks,
John

 
On 2018/4/10, 4:22 PM, "Zhilin Huang (zhilhuan)"  wrote:

To be clear, "immutable" here means the IP could not be removed, but allow 
modification.

And streaming "priority" is better be Delivery Service based than server 
based.

Thanks,
Zhilin


On 10/04/2018, 4:12 PM, "Zhilin Huang (zhilhuan)"  
wrote:

Hey Nir,

Thanks a lot for your comments. Please see my replies inline.

On 10/04/2018, 3:14 AM, "Nir Sopher"  wrote:

 Hey Zhilin,

Regarding the ports configuration. Even though I believe modeling 
will be
cleaner if the port and IP are set together, you are probably 
correct - it
is reasonable to consider the Port per IP flexibility as a future 
extension
and avoid it for now.
Still, I would suggest to at-least module the cr-config with the 
Port
specified per IP (delivery-unit). It is more flexible as well as 
simplify
the router and monitor code.
ZH> I understand your consideration about the flexibility. But I still 
think port is a server lever configuration, do not see the needs of multiple 
ports in the near future. Anyway, if we want to add port configured together 
with IP, it is easy to add a new field into the json of RESTful API or 
cr-config, since adding new field is easy to be backward compatible. So I would 
like to leave this change to future when there is use case required.


Regarding the crud of the server configuration, I believe the API 
should
change, but with backward compatibility.
Maybe we should have
(GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/interfaces
And
(GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/
interfaces/{:ifId}/delivery-units
ZH> I went thru those APIs again, and agree with your points. The 
design doc has been updated to reflect this change in section 3.1.1.3 and 
3.1.1.4. A little different than your suggestion is I used "ips" instead of 
"delivery-units" to allow manipulation of management IP and ILO IP as well.

These APIs will allow us to manipulate all interfaces and all IPs
(delivery-units). Note that as I see it, there is no special 
"primary" IP
(but IPs has priorities).
ZH> I think there must be a so called "default" streaming IP bind with 
a server until server deleted. A server with no streaming is useless. That's 
how I understand the "primary" IP, which is immutable with a server. A 
"secondary" streaming IP could be add and removed afterwards.
For the streaming IP priority, I think it is out of the scope of this 
feature, we could add that in some future features. The concept streaming 
"priority" itself could be co-existing with the "primary/secondary" concept, in 
different dimension. The priority is a streaming load balance concept. And any 
streaming IP, no matter primary or secondary, could assign a higher priority 
than others.

The old /api/1.2/servers/{:svrId} API can be backward compatible. 
We need
to think it through but just an example:
ZH> Agree, I would like the old APIs will not see the "secondary" IPs 
and interfaces. No behavior change.

   - Server "GET" will return the IP of the server's delivery unit 
with the
   lowest ID
   - Server "PUT" will allow empty IP, but if IP is set, it verify 
there is
   exactly 1 IP record for the server, and work against it. O.w. 
fails.

Another option can be to have a global param that enables multiple 
IPs per
server.  When enabled, API changes - IP is removed from the server 
API.

Nir



On Sun, Apr 8, 2018 at 9:18 AM, Zhilin Huang (zhilhuan) 

wrote:

> Hi Jifeng,
>
> I do not think we need to change the APIs. Current CRUD 
/api/1.2/servers
> will configure the primary IP and interface.
>
> I do not think we want to change this due to:
> 1) backward compatibility
> 2) there should always be a default (primary) IP and interface 
configured
> when creating a