Re: Updated TO API guidelines

[John Rushford]

Why the change?  It’s my understanding that path parameters should be used to 
specify a particular resource
and query parameters should be used to sort/filter the query.  Why use a query 
parameter to specify a particular
resource?  Is this REST API best practice?

What about sub resource queries such as using the following:

GET api/1.3/deliveryservices/{xmlID}/urisigning

where you are requesting a particular urisigning keys sub resource for the 
particular deliveryservice resource. You can make it work
with an xmlid query parameter but what do you return if the query parameter is 
left off, all uri signing keys?  Is that useful?

John

> On Apr 4, 2018, at 3:23 PM, Jeremy Mitchell  wrote:
> 
> tbh i'm not sure about versioning. I was just trying to suggest that new
> routes be formulated this way per the new API guidelines:
> 
> GET /foos[?id, name, etc=]
> POST /foos
> PUT /foos [?id, name, etc=]
> DELETE /foos [?id, name, etc=]
> 
> instead of the old way:
> 
> GET /foos
> GET /foos/:id
> POST /foos
> PUT /foos/:id
> DELETE /foos/:id
> 
> The difference being the use of query params over route/path params.
> 
> Technically, adding new routes does not break old stuff right so i don't
> think that warrants a major version roll.
> 
> While we're on the subject, what does everyone think if we took this one
> step further and made routes handle a request payload with one or more
> items. For example:
> 
> GET /foos[?id, name, etc=]
> POST /foos <-- takes in an array of foos to create
> PUT /foos <-- takes in an array of foos to update
> DELETE /foos <-- takes in an array of foos to delete
> 
> in this scenario, query params only pertain to the GET. The POST, PUT and
> DELETE rely on the contents of the request json...
> 
> Jeremy
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Wed, Apr 4, 2018 at 1:55 PM, Robert Butts 
> wrote:
> 
>> That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.
>> 
>> Just to clarify, changing to query parameters breaks compatibility with 1.2
>> and older, so new APIs in that format have to be a new major version, i.e.
>> 2.0, per Semantic Versioning, right?
>> 
>> On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell 
>> wrote:
>> 
>>> FYI - I've updated the TO API guidelines to reflect our desire to move
>> away
>>> from route/path params and embrace query params in the Golang API.
>>> 
>>> https://cwiki.apache.org/confluence/display/TC/API+Guidelines
>>> 
>>> Jeremy
>>> 
>> 

Re: Traffic Server Secondary Streaming IPs Design

[Jifeng Yang (jifyang)]

Due to this change, the Traffic Ops APIs may also need change:

(GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/2ndintfs
Need change
Suggestion: /api/1.2/servers/{:svrId}/interfaces

(GET/POST/PUT/DELETE)  /api/1.2/servers/{:svrId}/2ndips
Don't need change.

Thanks,
Jifeng

On 04/04/2018, 11:56, "Zhilin Huang (zhilhuan)"  wrote:

Updated the DB schema in section 3.1.1.4

Thanks,
Zhilin


On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)"  
wrote:

Good points. I am happy to make this change in the design doc.

Thanks,
Zhilin


On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" 
 wrote:

I would prefer a consistent way to store all interface and IP 
address information. Its good database design practice to store similar 
information in similar tables (i.e. all IP info in 1 table) rather than keep 
some IPs in the server table and some IPs in another table. 

I also think this refactoring will give us greater flexibility for 
more changes in the future. Outside of this particular use case, we might have 
additional features like sharing edges between public/private networks or 
having multiple (equal priority) streaming interfaces on a cache.

These future features would be easier if the interface data and IP 
data is all organized into separate tables. 

I’d also like to see the delivery service to IP mapping be a many 
to many mapping in the DB. For this particular feature we will only assign a 
single IP (and we can restrict that in the API if we want), but I am near 
certain that in the future we would like the ability to assign a DS to multiple 
IPs on the same cache. 


—Eric



> On Apr 3, 2018, at 2:42 AM, Zhilin Huang (zhilhuan) 
 wrote:
> 
> Hi Mark,
> 
> Thanks for your comments. Please check my reply in another thread:
> 
> If we all agreed to use unified tables for all IPs and/or 
interfaces: primary, management, secondary, then there need to be two tables: 
IP and interface.
> And in the server table, we need to replace the original 
"interface_xxx", "ip_xxx", "ip6_xxx" fields with a "primary_ip_id" field. And 
do similar things to management IP.
> 
> Thanks,
> Zhilin
> 
> 
> On 03/04/2018, 7:08 AM, "Mark Torluemke"  
wrote:
> 
>I would support an 'interfaces' table (adding some sort of a 
'type' column)
>that would include moving the management and lights out 
management
>interfaces to that table as well.
> 
>Cheers,
>Mark
> 
>On Mon, Apr 2, 2018 at 2:39 PM, Nir Sopher  
wrote:
> 
>> Hi Zhilin,
>> 
>> I took a quick look into the spec. Hope to have the opportunity 
to dive
>> deeper into it soon so we can further discuss it.
>> 
>> For now I have a 2 questions.
>> In the spec, you refer to "secondary interfaces", and you have a 
list of
>> secondary interfaces added.
>> IIUC the secondary interfaces are used as long as they are 
available, and
>> when down, you move to the primary interface.
>> 
>> Why not, instead of holding a secondary interfaces table, move 
all
>> interfaces to a separate table? Primary and secondary.
>> For each interface you can hold:
>> 
>>   - Server id
>>   - name (e.g. eth0)
>>   - IPv6
>>   - IPv4
>>   - Priority (Integer as flexible as you wish: e.g. "1" for 
"secondary",
>>   "2" for "primary" in your example,)
>> 
>> 
>> Additionally, it is not clear to me what happens if one of the 
interfaces
>> fails?
>> Does every interface has a unique DNS name? If an interface 
fails, are
>> redirects
>> sent only to the available (secondary) interfaces?
>> 
>> Thanks,
>> Nir
>> 
>> 
>> On Mon, Apr 2, 2018 at 10:21 AM, Zhilin Huang (zhilhuan) <
>> zhilh...@cisco.com
>>> wrote:
>> 
>>> Hi Guys,
>>> 
>>> This was originally posted in another discussion. Resend this 
in a
>>> standalone topic to catch more awareness. The link for the 
design doc:
>>> 

Re: Updated TO API guidelines

[Robert Butts]

Ah, sorry, I misunderstood. I thought when you said "a route that never
existed in 1.2 before" you meant e.g. /api/1.3/servers "didn't exist in
Golang or 1.3 before".

Right, for entirely new routes, e.g. deliveryservices/{xmlID}/urisignkeys,
path params are fine, and still conform to SemVer for the same major
version, e.g. 1.3.


On Wed, Apr 4, 2018 at 4:12 PM, Robert Butts 
wrote:

> You're right, in our code today, we don't use raw major versions
> (/api/1/foo) in our API, and our clients have hard-coded minor versions.
>
> If we feel like that's how we want to design APIs and clients, and say
> "our minor versions are not compatible with each-other", we can do that.
> But it's not Semantic Versioning.
>
> Semantic Versioning says: (semver.org)
> > MAJOR version when you make incompatible API changes,
> > MINOR version when you add functionality in a backwards-compatible manner
>
> Even if our current code doesn't strictly need SemVer, using a well-known
> standard makes it easier to understand. People know at a glance which
> versions are compatible, which have changed, and when they need to go read
> the docs again. Maybe someone else wants to write their own TO client, that
> expects SemVer.
>
> Is there a disadvantage to using SemVer? Even if we want to break
> backwards compatibility, we can just keep rolling major versions, /api/2/,
> /api/3/, /api/42/. I'd rather we didn't break compatibility so often, but
> if we do, there's no harm in the number itself.
>
>
> On Wed, Apr 4, 2018 at 4:00 PM, Volz, Dylan 
> wrote:
>
>> " a request from a client which is built to understand say /api/1.2/, if
>> that same request is made to /api/1.3/, it should still work."
>>
>> But this is a route that never existed in 1.2 before, not a new version
>> of a route in 1.3,
>> a client built to understand 1.2 will just not have this route at all and
>> would require
>>  new dev work to support, or maybe I am misunderstanding you?
>>
>> On 4/4/18, 3:49 PM, "Robert Butts"  wrote:
>>
>> >Technically, adding new routes does not  break old stuff right so i
>> don't
>> think that warrants a major version roll.
>>
>> New routes that use query params instead of path params will
>> definitely
>> break old stuff.
>>
>> When you say "break old stuff", the idea behind Semantic Versioning,
>> is
>> that a request from a client which is built to understand say
>> /api/1.2/, if
>> that same request is made to /api/1.3/, it should still work. If not,
>> it's
>> not Semantic Versioning.
>>
>> That also allows you to serve your latest minor version at /api/1/,
>> /api/2/, etc, and clients who know the Server is newer than them can
>> hit
>> the major endpoint, and be guaranteed to work. We don't currently do
>> that,
>> but it'd be nice if we did.
>>
>> I'm +1 on query params, but also +1 on https://semver.org/
>>
>>
>> On Wed, Apr 4, 2018 at 3:23 PM, Jeremy Mitchell <
>> mitchell...@gmail.com>
>> wrote:
>>
>> > tbh i'm not sure about versioning. I was just trying to suggest
>> that new
>> > routes be formulated this way per the new API guidelines:
>> >
>> > GET /foos[?id, name, etc=]
>> > POST /foos
>> > PUT /foos [?id, name, etc=]
>> > DELETE /foos [?id, name, etc=]
>> >
>> > instead of the old way:
>> >
>> > GET /foos
>> > GET /foos/:id
>> > POST /foos
>> > PUT /foos/:id
>> > DELETE /foos/:id
>> >
>> > The difference being the use of query params over route/path params.
>> >
>> > Technically, adding new routes does not break old stuff right so i
>> don't
>> > think that warrants a major version roll.
>> >
>> > While we're on the subject, what does everyone think if we took
>> this one
>> > step further and made routes handle a request payload with one or
>> more
>> > items. For example:
>> >
>> > GET /foos[?id, name, etc=]
>> > POST /foos <-- takes in an array of foos to create
>> > PUT /foos <-- takes in an array of foos to update
>> > DELETE /foos <-- takes in an array of foos to delete
>> >
>> > in this scenario, query params only pertain to the GET. The POST,
>> PUT and
>> > DELETE rely on the contents of the request json...
>> >
>> > Jeremy
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Apr 4, 2018 at 1:55 PM, Robert Butts <
>> robert.o.bu...@gmail.com>
>> > wrote:
>> >
>> > > That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.
>> > >
>> > > Just to clarify, changing to query parameters breaks
>> compatibility with
>> > 1.2
>> > > and older, so new APIs in that format have to be a new major
>> version,
>> > i.e.
>> > > 2.0, per Semantic Versioning, right?
>> > >
>> > > On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell <
>> 

Re: Updated TO API guidelines

[Robert Butts]

You're right, in our code today, we don't use raw major versions
(/api/1/foo) in our API, and our clients have hard-coded minor versions.

If we feel like that's how we want to design APIs and clients, and say "our
minor versions are not compatible with each-other", we can do that. But
it's not Semantic Versioning.

Semantic Versioning says: (semver.org)
> MAJOR version when you make incompatible API changes,
> MINOR version when you add functionality in a backwards-compatible manner

Even if our current code doesn't strictly need SemVer, using a well-known
standard makes it easier to understand. People know at a glance which
versions are compatible, which have changed, and when they need to go read
the docs again. Maybe someone else wants to write their own TO client, that
expects SemVer.

Is there a disadvantage to using SemVer? Even if we want to break backwards
compatibility, we can just keep rolling major versions, /api/2/, /api/3/,
/api/42/. I'd rather we didn't break compatibility so often, but if we do,
there's no harm in the number itself.


On Wed, Apr 4, 2018 at 4:00 PM, Volz, Dylan  wrote:

> " a request from a client which is built to understand say /api/1.2/, if
> that same request is made to /api/1.3/, it should still work."
>
> But this is a route that never existed in 1.2 before, not a new version of
> a route in 1.3,
> a client built to understand 1.2 will just not have this route at all and
> would require
>  new dev work to support, or maybe I am misunderstanding you?
>
> On 4/4/18, 3:49 PM, "Robert Butts"  wrote:
>
> >Technically, adding new routes does not  break old stuff right so i
> don't
> think that warrants a major version roll.
>
> New routes that use query params instead of path params will definitely
> break old stuff.
>
> When you say "break old stuff", the idea behind Semantic Versioning, is
> that a request from a client which is built to understand say
> /api/1.2/, if
> that same request is made to /api/1.3/, it should still work. If not,
> it's
> not Semantic Versioning.
>
> That also allows you to serve your latest minor version at /api/1/,
> /api/2/, etc, and clients who know the Server is newer than them can
> hit
> the major endpoint, and be guaranteed to work. We don't currently do
> that,
> but it'd be nice if we did.
>
> I'm +1 on query params, but also +1 on https://semver.org/
>
>
> On Wed, Apr 4, 2018 at 3:23 PM, Jeremy Mitchell  >
> wrote:
>
> > tbh i'm not sure about versioning. I was just trying to suggest that
> new
> > routes be formulated this way per the new API guidelines:
> >
> > GET /foos[?id, name, etc=]
> > POST /foos
> > PUT /foos [?id, name, etc=]
> > DELETE /foos [?id, name, etc=]
> >
> > instead of the old way:
> >
> > GET /foos
> > GET /foos/:id
> > POST /foos
> > PUT /foos/:id
> > DELETE /foos/:id
> >
> > The difference being the use of query params over route/path params.
> >
> > Technically, adding new routes does not break old stuff right so i
> don't
> > think that warrants a major version roll.
> >
> > While we're on the subject, what does everyone think if we took this
> one
> > step further and made routes handle a request payload with one or
> more
> > items. For example:
> >
> > GET /foos[?id, name, etc=]
> > POST /foos <-- takes in an array of foos to create
> > PUT /foos <-- takes in an array of foos to update
> > DELETE /foos <-- takes in an array of foos to delete
> >
> > in this scenario, query params only pertain to the GET. The POST,
> PUT and
> > DELETE rely on the contents of the request json...
> >
> > Jeremy
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Apr 4, 2018 at 1:55 PM, Robert Butts <
> robert.o.bu...@gmail.com>
> > wrote:
> >
> > > That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.
> > >
> > > Just to clarify, changing to query parameters breaks compatibility
> with
> > 1.2
> > > and older, so new APIs in that format have to be a new major
> version,
> > i.e.
> > > 2.0, per Semantic Versioning, right?
> > >
> > > On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell <
> mitchell...@apache.org>
> > > wrote:
> > >
> > > > FYI - I've updated the TO API guidelines to reflect our desire
> to move
> > > away
> > > > from route/path params and embrace query params in the Golang
> API.
> > > >
> > > > https://cwiki.apache.org/confluence/display/TC/API+Guidelines
> > > >
> > > > Jeremy
> > > >
> > >
> >
>
>
>

Re: Traffic Control Doc's Proposal

[Jeremy Mitchell]

Can you clarify what a "howto" is?

Yes, it looks like our README's are a bit out of control. What if we
consolidated them like such:

./README.md <-- this one links to general information about TC
(readthedocs) and also links to the following:

./traffic_portal/README.md
./traffic_monitor/README.md
./traffic_ops/README.md <-- this links to ./traffic_ops_db/README.md
./traffic_router/README.md
./traffic_server/README.md
./traffic_stats/README.md

^^ each of these readme's include 3 things at the very least:

- How to install
- How to develop
- A link to component info found in readthedocs (rst)

Jeremy

On Wed, Apr 4, 2018 at 9:48 AM, Dewayne Richardson 
wrote:

> I've been going through the Traffic Ops API doc's and revamping them with
> Swagger and noticed this trend with our documentation.   "Guides" (Howto's,
> Installs, Developer's)  growing because they are easier to maintain
> (because they are less fancy) as markdown files in the Github project.  My
> concern is the RST equivalents here:
> http://traffic-control-cdn.readthedocs.io/en/latest/development/index.html
> are going stale and they are also being duplicated in the markdown files.
> I also worry that the markdown files aren't surfaced properly to our
> community because there is valuable information in them as well.
>
> Therefore I propose the following documentation "rules of thumb":
>
> 1. Howto's, Installs, Developer/Admin Guides are done in Markdown but have
> RST links (or some reference to an index page to surface all of them)
> 2. High level informative documentation like API docs, and Traffic Control
> features are done in RST as we have been to surface that information for
> "Users of Traffic Control"
>
> Please provide your feedback,
>
> -Dew
>
>
> Here is a quick list of the Markdown files I've found in the project so
> far:
>
>
> ./misc/git/README.md
> ./CODE_OF_CONDUCT.md
> ./experimental/traffic_router_golang/README.md
> ./test/traffic_ops_cfg/Readme.md
> ./test/router/dnssec/Readme.md
> ./test/router/Readme.md
> ./CHANGELOG.md
> ./traffic_portal/v1/README.md
> ./traffic_portal/test/end_to_end/README.md
> ./traffic_portal/README.md
> ./traffic_portal/build/README.md
> ./traffic_monitor_java/README.md
> ./traffic_monitor/tools/testcaches/README.md
> ./traffic_monitor/README.md
> ./traffic_server/plugins/astats_over_http/README.md
> ./traffic_server/README.md
> ./README.md
> ./traffic_ops_db/pg-migration/README.md
> ./traffic_ops/experimental/goto/README.md
> ./traffic_ops/experimental/auth/README.md
> ./traffic_ops/experimental/server/README.md
> ./traffic_ops/experimental/webfront/README.md
> ./traffic_ops/INSTALL.md
> ./traffic_ops/README.md
> ./traffic_ops/testing/compare/README.md
> ./traffic_ops/testing/api/docker/README.md
> ./traffic_ops/testing/api/README.md
> ./traffic_ops/build/README.md
> ./traffic_ops/client_tests/README.md
> ./traffic_ops/traffic_ops_golang/README.md
> ./traffic_ops/traffic_ops_golang/swaggerdocs/v13/README.md
> ./CONTRIBUTING.md
> ./BUILD.md
> ./build/README.md
> ./infrastructure/docker/README.md
> ./infrastructure/docker/build/README.md
> ./infrastructure/test/integration/README.md
> ./infrastructure/test/README.md
> ./traffic_router/core/src/test/resources/Readme.md
> ./traffic_router/neustar/README.md
>

Re: Updated TO API guidelines

[Robert Butts]

>Technically, adding new routes does not  break old stuff right so i don't
think that warrants a major version roll.

New routes that use query params instead of path params will definitely
break old stuff.

When you say "break old stuff", the idea behind Semantic Versioning, is
that a request from a client which is built to understand say /api/1.2/, if
that same request is made to /api/1.3/, it should still work. If not, it's
not Semantic Versioning.

That also allows you to serve your latest minor version at /api/1/,
/api/2/, etc, and clients who know the Server is newer than them can hit
the major endpoint, and be guaranteed to work. We don't currently do that,
but it'd be nice if we did.

I'm +1 on query params, but also +1 on https://semver.org/


On Wed, Apr 4, 2018 at 3:23 PM, Jeremy Mitchell 
wrote:

> tbh i'm not sure about versioning. I was just trying to suggest that new
> routes be formulated this way per the new API guidelines:
>
> GET /foos[?id, name, etc=]
> POST /foos
> PUT /foos [?id, name, etc=]
> DELETE /foos [?id, name, etc=]
>
> instead of the old way:
>
> GET /foos
> GET /foos/:id
> POST /foos
> PUT /foos/:id
> DELETE /foos/:id
>
> The difference being the use of query params over route/path params.
>
> Technically, adding new routes does not break old stuff right so i don't
> think that warrants a major version roll.
>
> While we're on the subject, what does everyone think if we took this one
> step further and made routes handle a request payload with one or more
> items. For example:
>
> GET /foos[?id, name, etc=]
> POST /foos <-- takes in an array of foos to create
> PUT /foos <-- takes in an array of foos to update
> DELETE /foos <-- takes in an array of foos to delete
>
> in this scenario, query params only pertain to the GET. The POST, PUT and
> DELETE rely on the contents of the request json...
>
> Jeremy
>
>
>
>
>
>
>
>
>
>
>
> On Wed, Apr 4, 2018 at 1:55 PM, Robert Butts 
> wrote:
>
> > That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.
> >
> > Just to clarify, changing to query parameters breaks compatibility with
> 1.2
> > and older, so new APIs in that format have to be a new major version,
> i.e.
> > 2.0, per Semantic Versioning, right?
> >
> > On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell 
> > wrote:
> >
> > > FYI - I've updated the TO API guidelines to reflect our desire to move
> > away
> > > from route/path params and embrace query params in the Golang API.
> > >
> > > https://cwiki.apache.org/confluence/display/TC/API+Guidelines
> > >
> > > Jeremy
> > >
> >
>

Re: Updated TO API guidelines

[Jeremy Mitchell]

tbh i'm not sure about versioning. I was just trying to suggest that new
routes be formulated this way per the new API guidelines:

GET /foos[?id, name, etc=]
POST /foos
PUT /foos [?id, name, etc=]
DELETE /foos [?id, name, etc=]

instead of the old way:

GET /foos
GET /foos/:id
POST /foos
PUT /foos/:id
DELETE /foos/:id

The difference being the use of query params over route/path params.

Technically, adding new routes does not break old stuff right so i don't
think that warrants a major version roll.

While we're on the subject, what does everyone think if we took this one
step further and made routes handle a request payload with one or more
items. For example:

GET /foos[?id, name, etc=]
POST /foos <-- takes in an array of foos to create
PUT /foos <-- takes in an array of foos to update
DELETE /foos <-- takes in an array of foos to delete

in this scenario, query params only pertain to the GET. The POST, PUT and
DELETE rely on the contents of the request json...

Jeremy











On Wed, Apr 4, 2018 at 1:55 PM, Robert Butts 
wrote:

> That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.
>
> Just to clarify, changing to query parameters breaks compatibility with 1.2
> and older, so new APIs in that format have to be a new major version, i.e.
> 2.0, per Semantic Versioning, right?
>
> On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell 
> wrote:
>
> > FYI - I've updated the TO API guidelines to reflect our desire to move
> away
> > from route/path params and embrace query params in the Golang API.
> >
> > https://cwiki.apache.org/confluence/display/TC/API+Guidelines
> >
> > Jeremy
> >
>

Re: Updated TO API guidelines

[Robert Butts]

That document doesn't mention versions, 1.2 vs 1.3 vs 2.0.

Just to clarify, changing to query parameters breaks compatibility with 1.2
and older, so new APIs in that format have to be a new major version, i.e.
2.0, per Semantic Versioning, right?

On Tue, Apr 3, 2018 at 3:26 PM, Jeremy Mitchell 
wrote:

> FYI - I've updated the TO API guidelines to reflect our desire to move away
> from route/path params and embrace query params in the Golang API.
>
> https://cwiki.apache.org/confluence/display/TC/API+Guidelines
>
> Jeremy
>

[VOTE] Release Apache Traffic Control (incubating) 2.2.0-RC4

[Robert Butts]

Hello All,

I've prepared a release for v2.2.0-RC4

The vote is open for at least 72 hours and passes if a majority of at least
3 +1 PPMC votes are cast.

[ ] +1 Approve the release

[ ] -1 Do not release this package because ...

Changes since 2.1.0:
https://github.com/apache/incubator-trafficcontrol/compare/RELEASE-2.1.0...
RELEASE-2.2.0-RC4

This corresponds to git:
 Hash: 45151f8e518fe92dfa64c7311b877cb13299debc
 Tag: RELEASE-2.2.0-RC4

Which can be verified with the following: git tag -v RELEASE-2.2.0-RC4

My code signing key is available here:
http://keys.gnupg.net/pks/lookup?search=0xFDD04F7F=vindex

Make sure you refresh from a key server to get all relevant signatures.

The source .tar.gz file, pgp signature (.asc signed with my key from
above), and sha512 checksum are provided here:

https://dist.apache.org/repos/dist/dev/incubator/trafficcontrol/2.2.0/RC4



Thanks!

Re: Traffic Server Secondary Streaming IPs Design

[Nir Sopher]

Eric,
Great, IP as delivery unit.
Note that I believe the port is part of this unit, and not a common
settings to all IPs in the interface.

+1 for Rob's suggestion (stats are collected on the interface level, and
health/heartbeat on the ip level)

Rob/Jeff
I believe we need to verify this entire concept fits well with monitor and
router localization.

Nir

On Wed, Apr 4, 2018, 17:23 Robert Butts  wrote:

> @nbaoping
>
> > So I suggest the change to the current TM to be like:
> > 1) Separate the current polling of cache servers into two different
> pollings, one for the keep alive, the other for the stat query.
>
> The Golang Monitor already does this. We call it the "health" and "stat"
> polls in the code. The stat poll is the full stats, and the health poll is
> just the system stats. Does that work for your keep-alive poll? The health
> poll is slightly more than just establishing a TCP connection, but it's
> very small, and it also gives us the interface data.
>
> > We need record the availability for each configured IP so that if it’s
> assigned, the router can check if it can redirect the client request to
> that assigned IP or not.
> > if we have multiple interfaces support, we should check the bandwidth
> availability for each interface
>
> Because the health poll has interface data, I'd suggest modifying Traffic
> Monitor to poll a single arbitrary IP for the Stat poll, as you suggest;
> and to poll all IPs on the Health poll. Then, because the system stats are
> in the health poll, the Monitor can figure out which interface that IP is
> on, and track the availability of that interface from that health poll
> data.
>
> If you aren't familiar with the Health vs Stat polls in the new Golang
> Monitor, see:
> https://traffic-control-cdn.readthedocs.io/en/latest/
> development/traffic_monitor_golang.html#architecture
> https://github.com/apache/incubator-trafficcontrol/blob/
> master/traffic_monitor/manager/manager.go
> https://github.com/apache/incubator-trafficcontrol/blob/
> master/traffic_monitor/manager/health.go
> https://github.com/apache/incubator-trafficcontrol/blob/
> master/traffic_monitor/manager/stat.go
>
> Does that work?
>
>
> On Wed, Apr 4, 2018 at 5:07 AM, Eric Friedrich (efriedri) <
> efrie...@cisco.com> wrote:
>
> > Hey Nir-
> >   For our particular use case, we are looking at making an IP the
> delivery
> > unit. We would like to use a single interface with multiple IPs. DSs
> would
> > be assigned to one of the IPs on that interface.  Interface (or IP)
> > priority does not come into play here as there is no failover between IPs
> >
> > —Eric
> >
> >
> > > On Apr 4, 2018, at 1:23 AM, Nir Sopher  wrote:
> > >
> > > +1
> > > Note that beyond the DB, the change should also be reflected into the
> > > cr-config.
> > > As I see it, a flexible model may be built of the below items:
> > > 1 - Edge server.
> > > 2- Interface
> > > 3. IPs
> > >
> > > The Interface (or should it be called "delivery unit") is the element
> we
> > > redirect the traffic to and which is monitored by the traffic-monitor:
> > > * Each server may have multiple interfaces
> > > * Each interface may have multiple IPs
> > > * Interfaces has priorities (abstraction for primary/secondary)
> > > * Each interface is given a seperate DNS name by the router. Single
> name
> > > for the multiple IPs.
> > > * Each interface is monitored and reported seperately by the traffic
> > > monitor, including health an stats.
> > >
> > >
> > > The router "redirect target decision" may look as follows
> > > 0. Select cache as we do today taking into account the consistent
> hash. A
> > > server is in the selection group only if one of its interfaces is found
> > to
> > > be healthy
> > > 1. Once we have server selected, select an interface out of all
> > interfaces
> > > of the server with max available priority.
> > >
> > > An additional improvement, may assign DS to interfaces instead of
> > servers.
> > > A server serves DS X iff one of its interfaces is assigned to the DS.
> > >
> > > Nir
> > >
> > >
> > > On Apr 4, 2018 6:56 AM, "Zhilin Huang (zhilhuan)" 
> > > wrote:
> > >
> > > Updated the DB schema in section 3.1.1.4
> > >
> > > Thanks,
> > > Zhilin
> > >
> > >
> > >
> > > On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)" <
> zhilh...@cisco.com>
> > > wrote:
> > >
> > >Good points. I am happy to make this change in the design doc.
> > >
> > >Thanks,
> > >Zhilin
> > >
> > >
> > >On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" <
> > efrie...@cisco.com>
> > > wrote:
> > >
> > >I would prefer a consistent way to store all interface and IP
> > > address information. Its good database design practice to store similar
> > > information in similar tables (i.e. all IP info in 1 table) rather than
> > > keep some IPs in the server table and some IPs in another table.
> > >
> > >I also think this refactoring will give us greater 

Traffic Control Doc's Proposal

[Dewayne Richardson]

I've been going through the Traffic Ops API doc's and revamping them with
Swagger and noticed this trend with our documentation.   "Guides" (Howto's,
Installs, Developer's)  growing because they are easier to maintain
(because they are less fancy) as markdown files in the Github project.  My
concern is the RST equivalents here:
http://traffic-control-cdn.readthedocs.io/en/latest/development/index.html
are going stale and they are also being duplicated in the markdown files.
I also worry that the markdown files aren't surfaced properly to our
community because there is valuable information in them as well.

Therefore I propose the following documentation "rules of thumb":

1. Howto's, Installs, Developer/Admin Guides are done in Markdown but have
RST links (or some reference to an index page to surface all of them)
2. High level informative documentation like API docs, and Traffic Control
features are done in RST as we have been to surface that information for
"Users of Traffic Control"

Please provide your feedback,

-Dew


Here is a quick list of the Markdown files I've found in the project so far:


./misc/git/README.md
./CODE_OF_CONDUCT.md
./experimental/traffic_router_golang/README.md
./test/traffic_ops_cfg/Readme.md
./test/router/dnssec/Readme.md
./test/router/Readme.md
./CHANGELOG.md
./traffic_portal/v1/README.md
./traffic_portal/test/end_to_end/README.md
./traffic_portal/README.md
./traffic_portal/build/README.md
./traffic_monitor_java/README.md
./traffic_monitor/tools/testcaches/README.md
./traffic_monitor/README.md
./traffic_server/plugins/astats_over_http/README.md
./traffic_server/README.md
./README.md
./traffic_ops_db/pg-migration/README.md
./traffic_ops/experimental/goto/README.md
./traffic_ops/experimental/auth/README.md
./traffic_ops/experimental/server/README.md
./traffic_ops/experimental/webfront/README.md
./traffic_ops/INSTALL.md
./traffic_ops/README.md
./traffic_ops/testing/compare/README.md
./traffic_ops/testing/api/docker/README.md
./traffic_ops/testing/api/README.md
./traffic_ops/build/README.md
./traffic_ops/client_tests/README.md
./traffic_ops/traffic_ops_golang/README.md
./traffic_ops/traffic_ops_golang/swaggerdocs/v13/README.md
./CONTRIBUTING.md
./BUILD.md
./build/README.md
./infrastructure/docker/README.md
./infrastructure/docker/build/README.md
./infrastructure/test/integration/README.md
./infrastructure/test/README.md
./traffic_router/core/src/test/resources/Readme.md
./traffic_router/neustar/README.md

Re: [VOTE] Resolution for Traffic Control graduation to TLP

[Jeremy Mitchell]

+1

Jeremy

On Mon, Apr 2, 2018 at 10:28 PM, Jason Tucker 
wrote:

> +1
>
> On Tue, Apr 3, 2018 at 2:58 AM, John Rushford 
> wrote:
>
> > +1
> >
> > > On Apr 2, 2018, at 7:07 PM, Eric Friedrich (efriedri) <
> > efrie...@cisco.com> wrote:
> > >
> > > +1
> > >> On Apr 2, 2018, at 4:11 PM, David Neuman 
> > wrote:
> > >>
> > >> Dear Traffic Control community members:
> > >>
> > >> I would like to call a vote on the resolution for Traffic Control to
> > >> graduate from to an Apache TLP.  We have already voted on whether or
> > not we
> > >> should start the graduation process [1] and this is the next step.
> > Please
> > >> see the resolution below and vote as follows:
> > >>
> > >> [ ] +1 Graduate Traffic Control from the incubator
> > >> [ ] +0 No Opinion
> > >> [ ] -1 Do not graduate Traffic Control from the incubator (please
> > provide a
> > >> reason)
> > >>
> > >> The vote is open for a minimum of 72 hours and will need at least 3
> > more +1
> > >> votes than -1 votes from PMC members to succeed.
> > >>
> > >> If this VOTE succeeds, a similar VOTE will be started on the
> > >> gene...@incubator.apache.org mailing list. If that VOTE succeeds, a
> > >> resolution will be included in the next Apache Board Meeting.
> > >>
> > >> Please feel free to reach out with any questions.
> > >>
> > >> Thanks,
> > >> Dave
> > >>
> > >> [1]
> > >> https://lists.apache.org/thread.html/fb1fae0785feb6568cef6deb6fa207
> > 23eba54ed63a445462d44564d3@%3Cdev.trafficcontrol.apache.org%3E
> > >>
> > >> -
> > >>
> > >> Establish the Apache Traffic Control Project
> > >>
> > >> WHEREAS, the Board of Directors deems it to be in the best interests
> of
> > >> the Foundation and consistent with the Foundation's purpose to
> establish
> > >> a Project Management Committee charged with the creation and
> maintenance
> > >> of open-source software, for distribution at no charge to the public,
> > >> related to building, monitoring, configuring, and provisioning a large
> > >> scale content delivery network (CDN)..
> > >>
> > >> NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> > >> (PMC), to be known as the "Apache Traffic Control Project", be and
> > >> hereby is established pursuant to Bylaws of the Foundation; and be it
> > >> further
> > >>
> > >> RESOLVED, that the Apache Traffic Control Project be and hereby is
> > >> responsible for the creation and maintenance of software related to
> > >> building, monitoring, configuring, and provisioning a large scale
> > >> content delivery network (CDN).; and be it further
> > >>
> > >> RESOLVED, that the office of "Vice President, Apache Traffic Control"
> be
> > >> and hereby is created, the person holding such office to serve at the
> > >> direction of the Board of Directors as the chair of the Apache Traffic
> > >> Control Project, and to have primary responsibility for management of
> > >> the projects within the scope of responsibility of the Apache Traffic
> > >> Control Project; and be it further
> > >>
> > >> RESOLVED, that the persons listed immediately below be and hereby are
> > >> appointed to serve as the initial members of the Apache Traffic
> Control
> > >> Project:
> > >>
> > >> * Dan Kirkwood   
> > >> * David Neuman   
> > >> * Dewayne Richardson 
> > >> * Eric Covener   
> > >> * Eric Friedrich 
> > >> * Hank Beatty
> > >> * Jan van Doorn  
> > >> * Jeff Elsloo
> > >> * Jeremy Mitchell
> > >> * Leif Hedstrom  
> > >> * Mark Torluemke 
> > >> * Phil Sorber
> > >> * Steve Malenfant
> > >>
> > >> NOW, THEREFORE, BE IT FURTHER RESOLVED, that David Neuman be appointed
> > >> to the office of Vice President, Apache Traffic Control, to serve in
> > >> accordance with and subject to the direction of the Board of Directors
> > >> and the Bylaws of the Foundation until death, resignation, retirement,
> > >> removal or disqualification, or until a successor is appointed; and be
> > >> it further
> > >>
> > >> RESOLVED, that the initial Apache Traffic Control PMC be and hereby is
> > >> tasked with the creation of a set of bylaws intended to encourage open
> > >> development and increased participation in the Apache Traffic Control
> > >> Project; and be it further
> > >>
> > >> RESOLVED, that the Apache Traffic Control Project be and hereby is
> > >> tasked with the migration and rationalization of the Apache Incubator
> > >> Traffic Control podling; and be it further
> > >>
> > >> RESOLVED, that all responsibilities pertaining to the Apache Incubator
> > >> Traffic Control podling encumbered upon the Apache Incubator PMC are
> > >> hereafter 

Re: Traffic Server Secondary Streaming IPs Design

[Robert Butts]

@nbaoping

> So I suggest the change to the current TM to be like:
> 1) Separate the current polling of cache servers into two different
pollings, one for the keep alive, the other for the stat query.

The Golang Monitor already does this. We call it the "health" and "stat"
polls in the code. The stat poll is the full stats, and the health poll is
just the system stats. Does that work for your keep-alive poll? The health
poll is slightly more than just establishing a TCP connection, but it's
very small, and it also gives us the interface data.

> We need record the availability for each configured IP so that if it’s
assigned, the router can check if it can redirect the client request to
that assigned IP or not.
> if we have multiple interfaces support, we should check the bandwidth
availability for each interface

Because the health poll has interface data, I'd suggest modifying Traffic
Monitor to poll a single arbitrary IP for the Stat poll, as you suggest;
and to poll all IPs on the Health poll. Then, because the system stats are
in the health poll, the Monitor can figure out which interface that IP is
on, and track the availability of that interface from that health poll data.

If you aren't familiar with the Health vs Stat polls in the new Golang
Monitor, see:
https://traffic-control-cdn.readthedocs.io/en/latest/
development/traffic_monitor_golang.html#architecture
https://github.com/apache/incubator-trafficcontrol/blob/
master/traffic_monitor/manager/manager.go
https://github.com/apache/incubator-trafficcontrol/blob/
master/traffic_monitor/manager/health.go
https://github.com/apache/incubator-trafficcontrol/blob/
master/traffic_monitor/manager/stat.go

Does that work?


On Wed, Apr 4, 2018 at 5:07 AM, Eric Friedrich (efriedri) <
efrie...@cisco.com> wrote:

> Hey Nir-
>   For our particular use case, we are looking at making an IP the delivery
> unit. We would like to use a single interface with multiple IPs. DSs would
> be assigned to one of the IPs on that interface.  Interface (or IP)
> priority does not come into play here as there is no failover between IPs
>
> —Eric
>
>
> > On Apr 4, 2018, at 1:23 AM, Nir Sopher  wrote:
> >
> > +1
> > Note that beyond the DB, the change should also be reflected into the
> > cr-config.
> > As I see it, a flexible model may be built of the below items:
> > 1 - Edge server.
> > 2- Interface
> > 3. IPs
> >
> > The Interface (or should it be called "delivery unit") is the element we
> > redirect the traffic to and which is monitored by the traffic-monitor:
> > * Each server may have multiple interfaces
> > * Each interface may have multiple IPs
> > * Interfaces has priorities (abstraction for primary/secondary)
> > * Each interface is given a seperate DNS name by the router. Single name
> > for the multiple IPs.
> > * Each interface is monitored and reported seperately by the traffic
> > monitor, including health an stats.
> >
> >
> > The router "redirect target decision" may look as follows
> > 0. Select cache as we do today taking into account the consistent hash. A
> > server is in the selection group only if one of its interfaces is found
> to
> > be healthy
> > 1. Once we have server selected, select an interface out of all
> interfaces
> > of the server with max available priority.
> >
> > An additional improvement, may assign DS to interfaces instead of
> servers.
> > A server serves DS X iff one of its interfaces is assigned to the DS.
> >
> > Nir
> >
> >
> > On Apr 4, 2018 6:56 AM, "Zhilin Huang (zhilhuan)" 
> > wrote:
> >
> > Updated the DB schema in section 3.1.1.4
> >
> > Thanks,
> > Zhilin
> >
> >
> >
> > On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)" 
> > wrote:
> >
> >Good points. I am happy to make this change in the design doc.
> >
> >Thanks,
> >Zhilin
> >
> >
> >On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" <
> efrie...@cisco.com>
> > wrote:
> >
> >I would prefer a consistent way to store all interface and IP
> > address information. Its good database design practice to store similar
> > information in similar tables (i.e. all IP info in 1 table) rather than
> > keep some IPs in the server table and some IPs in another table.
> >
> >I also think this refactoring will give us greater flexibility for
> > more changes in the future. Outside of this particular use case, we might
> > have additional features like sharing edges between public/private
> networks
> > or having multiple (equal priority) streaming interfaces on a cache.
> >
> >These future features would be easier if the interface data and IP
> > data is all organized into separate tables.
> >
> >I’d also like to see the delivery service to IP mapping be a many
> > to many mapping in the DB. For this particular feature we will only
> assign
> > a single IP (and we can restrict that in the API if we want), but I am
> near
> > certain that in the future we would like the ability to 

Re: Traffic Server Secondary Streaming IPs Design

[Eric Friedrich (efriedri)]

Hey Nir-
  For our particular use case, we are looking at making an IP the delivery 
unit. We would like to use a single interface with multiple IPs. DSs would be 
assigned to one of the IPs on that interface.  Interface (or IP) priority does 
not come into play here as there is no failover between IPs

—Eric


> On Apr 4, 2018, at 1:23 AM, Nir Sopher  wrote:
> 
> +1
> Note that beyond the DB, the change should also be reflected into the
> cr-config.
> As I see it, a flexible model may be built of the below items:
> 1 - Edge server.
> 2- Interface
> 3. IPs
> 
> The Interface (or should it be called "delivery unit") is the element we
> redirect the traffic to and which is monitored by the traffic-monitor:
> * Each server may have multiple interfaces
> * Each interface may have multiple IPs
> * Interfaces has priorities (abstraction for primary/secondary)
> * Each interface is given a seperate DNS name by the router. Single name
> for the multiple IPs.
> * Each interface is monitored and reported seperately by the traffic
> monitor, including health an stats.
> 
> 
> The router "redirect target decision" may look as follows
> 0. Select cache as we do today taking into account the consistent hash. A
> server is in the selection group only if one of its interfaces is found to
> be healthy
> 1. Once we have server selected, select an interface out of all interfaces
> of the server with max available priority.
> 
> An additional improvement, may assign DS to interfaces instead of servers.
> A server serves DS X iff one of its interfaces is assigned to the DS.
> 
> Nir
> 
> 
> On Apr 4, 2018 6:56 AM, "Zhilin Huang (zhilhuan)" 
> wrote:
> 
> Updated the DB schema in section 3.1.1.4
> 
> Thanks,
> Zhilin
> 
> 
> 
> On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)" 
> wrote:
> 
>Good points. I am happy to make this change in the design doc.
> 
>Thanks,
>Zhilin
> 
> 
>On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" 
> wrote:
> 
>I would prefer a consistent way to store all interface and IP
> address information. Its good database design practice to store similar
> information in similar tables (i.e. all IP info in 1 table) rather than
> keep some IPs in the server table and some IPs in another table.
> 
>I also think this refactoring will give us greater flexibility for
> more changes in the future. Outside of this particular use case, we might
> have additional features like sharing edges between public/private networks
> or having multiple (equal priority) streaming interfaces on a cache.
> 
>These future features would be easier if the interface data and IP
> data is all organized into separate tables.
> 
>I’d also like to see the delivery service to IP mapping be a many
> to many mapping in the DB. For this particular feature we will only assign
> a single IP (and we can restrict that in the API if we want), but I am near
> certain that in the future we would like the ability to assign a DS to
> multiple IPs on the same cache.
> 
> 
>—Eric
> 
> 
> 
>> On Apr 3, 2018, at 2:42 AM, Zhilin Huang (zhilhuan) <
> zhilh...@cisco.com> wrote:
>> 
>> Hi Mark,
>> 
>> Thanks for your comments. Please check my reply in another thread:
>> 
>> If we all agreed to use unified tables for all IPs and/or
> interfaces: primary, management, secondary, then there need to be two
> tables: IP and interface.
>> And in the server table, we need to replace the original
> "interface_xxx", "ip_xxx", "ip6_xxx" fields with a "primary_ip_id" field.
> And do similar things to management IP.
>> 
>> Thanks,
>> Zhilin
>> 
>> 
>> On 03/04/2018, 7:08 AM, "Mark Torluemke" 
> wrote:
>> 
>>   I would support an 'interfaces' table (adding some sort of a
> 'type' column)
>>   that would include moving the management and lights out
> management
>>   interfaces to that table as well.
>> 
>>   Cheers,
>>   Mark
>> 
>>   On Mon, Apr 2, 2018 at 2:39 PM, Nir Sopher 
> wrote:
>> 
>>> Hi Zhilin,
>>> 
>>> I took a quick look into the spec. Hope to have the opportunity
> to dive
>>> deeper into it soon so we can further discuss it.
>>> 
>>> For now I have a 2 questions.
>>> In the spec, you refer to "secondary interfaces", and you have a
> list of
>>> secondary interfaces added.
>>> IIUC the secondary interfaces are used as long as they are
> available, and
>>> when down, you move to the primary interface.
>>> 
>>> Why not, instead of holding a secondary interfaces table, move
> all
>>> interfaces to a separate table? Primary and secondary.
>>> For each interface you can hold:
>>> 
>>>  - Server id
>>>  - name (e.g. eth0)
>>>  - IPv6
>>>  - IPv4
>>>  - Priority (Integer as flexible as you wish: e.g. "1" for
> "secondary",
>>>  "2" for "primary" in your example,)
>>> 
>>> 
>>> Additionally, it is not clear to me what happens if one of the
> interfaces
>>> fails?
>>> Does every interface has a 

Re: Traffic Control Last-resort Routing

[Eric Friedrich (efriedri)]

Sounds good John. 

If you can provide a path, we shouldn’t call it Error Domain :-) We should call 
it “Error Redirect URL”.

—Eric

> On Apr 3, 2018, at 11:02 PM, John Shen (weifensh)  wrote:
> 
> Hey Eric,
> 
> Thanks a lot for your comments. Please refer to my reply inline.
> 
> Thanks,
> John
> 
> 
> On 2018/4/3, 8:36 PM, "Eric Friedrich (efriedri)"  wrote:
> 
>Hey John-
>1)“Last Resort Alternate domain” is incredibly similar to the Bypass FQDN. 
> I’d rather see us enhance the Bypass FQDN with an optional scheme and port 
> number, rather than add something so close in functionality. 
>[John] Yeah, we had the same thought. The only disadvantage is that we are 
> planning to use "DS Profile" instead of changing the GUI and DB schema to 
> support the new parameters, thus we will have the new options (e.g. scheme 
> and port) configured by "DS profile" and Bypass FQDN configured through GUI 
> for each DS. If no one has objection with this mixed configuration solution, 
> we would implement the Alternate domain/Bypass FQDN in this way.
> 
>2) Is there a definite need for the “To content origin” checkbox? If this 
> is only used when Bypass FQDN/Alternate domain is not configured, then we 
> could just ask the user to configure the Origin FQDN in this field. 
>[John] Yes, “To content origin” is only used when alternate domain is not 
> configured. Asking user to configure OFQDN in the alternate domain would make 
> the implementation easier. It's just that a user has to manually fill the 
> field.
> 
>3) Alternate Domain and Error domain also seem incredibly similar in the 
> proposal. Would error domain be better described as “Redirect location for 
> unknown delivery service FQDNs or blocked clients. If not set, TR would 
> typically return a 503” 
>[John] One difference is that for Error domain, user can configure a file 
> to redirect to, e.g. to "http://error-domain.org/404.html;. Error domain 
> (with filename) is the redirect location when there is no available cache in 
> a DS.
> 
>—Eric
> 
>> On Apr 2, 2018, at 7:43 AM, John Shen (weifensh)  wrote:
>> 
>> Hi guys,
>> 
>> We are planning to add a Last-resort routing feature to Traffic Control. It 
>> will make TR redirect a request to an alternate domain, an error domain, or 
>> an origin server when there is not suitable cache available (e.g. all caches 
>> are busy). It is similar with current “Bypass FQDN” in TC, but it adds more 
>> parameters like redirecting to error domains or origin servers.
>> 
>> All the new parameters will be configured through “DS Profile” (to avoid GUI 
>> and DB schema changes), which is different with the current GUI 
>> configuration of “Bypass FQDN”. So the design will not change the current 
>> “Bypass FQDN” feature, and just add Last-resort as another extra feature. If 
>> Last-resort feature is not enabled (by default), the current behavior of 
>> Traffic Router will not change.
>> 
>> Please refer to following doc for more details: 
>> https://docs.google.com/document/d/1ttrZUGoGZTdCT6g78v62xBVoDCC_cWesPzxfLaHBcp0/edit
>> 
>> Any comments are welcome.
>> 
>> Thanks,
>> John
>> 
> 
> 
> 

Re: Traffic Server Secondary Streaming IPs Design

[Neil Hao (nbaoping)]

Hi guys,

It’s really exciting that we are moving to one flexible solution to have the IP 
and interface tables separately. Based on this, I have some ideas for the 
change of the Traffic Monitor(TM). Actually, the TM have two different purposes 
to polling the cache servers. One is to make sure the connectivity is ok, the 
other is to query the stat of the interface. The connectivity is actually for 
the IP (with port) connectivity, while the stat is for the interface. Since we 
will have multiple interfaces and may have multiple IPs for each interface, I 
think the polling to the cache server should be like:
1) Check the connectivity of all the configured IPs of the cache server
  This function works like the keep alive to make sure all the configured IPs 
is reachable.
  
2) Query the stat for all the configured interfaces of the cache server
 This function is to query the interfaces stat, it doesn’t care which IP 
will be used to query the stat.

So I suggest the change to the current TM to be like:
1) Separate the current polling of cache servers into two different pollings, 
one for the keep alive, the other for the stat query.
  For the keep alive polling, we can use some specific url to check if the IP 
with port is opened or not; For the stat query, we will reuse the astats 
plugin, but need a little change to the uri to support returning multiple 
interfaces stat in the same request.
  If we don’t separate it, we will have to issue each stat query for each IP, 
and all the IPs of one interface will return the same big response, which is 
unnecessary, and will bring much more unnecessary load to the ATS and TM. If we 
separate it, we could make the keep alive check very light weighted, and also 
can use one request to query all the configured interfaces stat. Besides, we 
could also use different polling intervals for the keep alive and stat query.

2) The health availability of the cache server will be based on the configured 
IPs of the cache server
 Currently, we only have the availability of one cache server, but it will 
not be enough when we add multiple interfaces and multiple IPs support. We need 
record the availability for each configured IP so that if it’s assigned, the 
router can check if it can redirect the client request to that assigned IP or 
not.

3) The max available bandwidth check should be changed to per interface.
  Currently, the max available bandwidth is only for the primary interface of 
the cache server, if we have multiple interfaces support, we should check the 
bandwidth availability for each interface. If one interface is overload, we 
will mark all the IPs of that interface to be unavailable.

4) We will pick up one available IP to query the stat.
 Currently, we use the fqdn of the cache server to query the stat, since we 
will have multiple IPs, we could pick up one available IP to query.
 

Best regards,
Neil

On 4/4/18, 1:23 PM, "Nir Sopher"  wrote:

+1
Note that beyond the DB, the change should also be reflected into the
cr-config.
As I see it, a flexible model may be built of the below items:
1 - Edge server.
2- Interface
3. IPs

The Interface (or should it be called "delivery unit") is the element we
redirect the traffic to and which is monitored by the traffic-monitor:
* Each server may have multiple interfaces
* Each interface may have multiple IPs
* Interfaces has priorities (abstraction for primary/secondary)
* Each interface is given a seperate DNS name by the router. Single name
for the multiple IPs.
* Each interface is monitored and reported seperately by the traffic
monitor, including health an stats.


The router "redirect target decision" may look as follows
0. Select cache as we do today taking into account the consistent hash. A
server is in the selection group only if one of its interfaces is found to
be healthy
1. Once we have server selected, select an interface out of all interfaces
of the server with max available priority.

An additional improvement, may assign DS to interfaces instead of servers.
A server serves DS X iff one of its interfaces is assigned to the DS.

Nir


On Apr 4, 2018 6:56 AM, "Zhilin Huang (zhilhuan)" 
wrote:

Updated the DB schema in section 3.1.1.4

Thanks,
Zhilin



On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)" 
wrote:

Good points. I am happy to make this change in the design doc.

Thanks,
Zhilin


On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" 
wrote:

I would prefer a consistent way to store all interface and IP
address information. Its good database design practice to store similar
information in similar tables (i.e. all IP info in 1 table) rather than
keep some IPs