Re: Idea for the Wiki
On Fri, Feb 22, 2019 at 01:54:00AM +0100, Tim Düsterhus wrote: > I suggest to create new pages using the web interface only to make sure > it can handle it. Editing can be done using git. I agree. I'm seeing antoher benefit to this, which is that it will guarantee that we only use simple things that everyone can modify using the same web interface. Willy
Re: Idea for the Wiki
Baptiste, Am 20.02.19 um 07:44 schrieb Baptiste: > I just cloned the repo :) > How should we organize directories and pages? > IE for TLS offloading: > /common/acceleration/tls_offloading.md ? > I think it's quite important to agree on it now, because the folders will > be part of the URL indexed by google :) Be careful here: While the Wiki technically is a git repository that supports folders the web interface of GitHub does not. [timwolla@/t/test.wiki (master)]find . -not -path './.git/*' . ./Home.md ./Folder2 ./Folder2/Test.md ./.git ./Foo-bar.md ./Folder ./Folder/Test.md -> https://github.com/TimWolla/test/wiki/Test There are two pages called `Test` in the sidebar, but only the one in `Folder` can be accessed. The one in `Folder2` can't. I suggest to create new pages using the web interface only to make sure it can handle it. Editing can be done using git. Best regards Tim Düsterhus
Re: Idea for the Wiki
On Wed, Feb 20, 2019 at 07:44:46AM +0100, Baptiste wrote: > How should we organize directories and pages? > IE for TLS offloading: > /common/acceleration/tls_offloading.md ? > I think it's quite important to agree on it now, because the folders will > be part of the URL indexed by google :) I agree, but if you put too many levels, you'll end up on a tre-like doc where nobody can find anything because depending on what you're looking for you don't expect to find it at this place. You could simply have a big "SSL/TLS" section and put all related knowledge there, with a few links to some extra info present somewhere else. > I am not fan of the "advanced use cases" title, but we can brainstorm this > later. No problem. It could also be "integration with third party products", which probably more accurately translates what will land there. > And I wonder how / where we should put integration with third > parties (kubernetes, docker, consul, influxdb, grafana, prometheus, > etc...). Hehe, see above :-) > I would like to have a page for each of these items. Yes it makes sense. I'm pretty sure that over time some people will become the main editor for a set of pages, so it's important to split them accordingly so that it's easy for some people to regularly update "their" pages without having to depend on moving parts around. In this case this will be directly related to the ecosystem where haproxy is used, hence the product or set of products involved around. > This will also help third party maintainers to push their integration > documentation into this wiki, even if the page is just a link to their own > documentation. Good point, I hadn't thought about it. Thanks, willy
Re: Idea for the Wiki
On Tue, Feb 19, 2019 at 9:36 AM Willy Tarreau wrote: > Hi Baptiste, > > On Wed, Feb 06, 2019 at 03:55:37PM +0100, Baptiste wrote: > > I think one of the most important piece is guide lines on integrating > > HAProxy with third parties, IE: Observing HAProxy with influxdb, HAProxy > as > > a Kubernetes External Load-balancer, Service discovery with consul, and > so > > on. > > I don't really know where to put those in the summary you proposed, but > > that's what I want to see in such wiki :) > > For me it falls perfectly into the advanced use cases which aim at > covering interfacing with third-party products. > > Since I got no objection to the proposed plan, I've just created the > wiki's home page, and copy-pasted the proposed plan into a temporary > page that will serve as a guide about what can be worked on. I'll try > to devote a bit of time to this, those who always dreamed about > revamping the old architecture manual are welcome if they want to work > on this. My hope is that we can quickly delete the architecture.txt > file from the source repository :-) > > Cheers, > Willy > I just cloned the repo :) How should we organize directories and pages? IE for TLS offloading: /common/acceleration/tls_offloading.md ? I think it's quite important to agree on it now, because the folders will be part of the URL indexed by google :) I am not fan of the "advanced use cases" title, but we can brainstorm this later. And I wonder how / where we should put integration with third parties (kubernetes, docker, consul, influxdb, grafana, prometheus, etc...). I would like to have a page for each of these items. This will also help third party maintainers to push their integration documentation into this wiki, even if the page is just a link to their own documentation. Baptiste
Re: Idea for the Wiki
Hi Baptiste, On Wed, Feb 06, 2019 at 03:55:37PM +0100, Baptiste wrote: > I think one of the most important piece is guide lines on integrating > HAProxy with third parties, IE: Observing HAProxy with influxdb, HAProxy as > a Kubernetes External Load-balancer, Service discovery with consul, and so > on. > I don't really know where to put those in the summary you proposed, but > that's what I want to see in such wiki :) For me it falls perfectly into the advanced use cases which aim at covering interfacing with third-party products. Since I got no objection to the proposed plan, I've just created the wiki's home page, and copy-pasted the proposed plan into a temporary page that will serve as a guide about what can be worked on. I'll try to devote a bit of time to this, those who always dreamed about revamping the old architecture manual are welcome if they want to work on this. My hope is that we can quickly delete the architecture.txt file from the source repository :-) Cheers, Willy
Re: Idea for the Wiki
Hi Willy,
Thanks a lot for bringing up this topic, long time I wanted to spend time
on this!
We discussed my point during a lunch, but I want to share it here as well.
I think one of the most important piece is guide lines on integrating
HAProxy with third parties, IE: Observing HAProxy with influxdb, HAProxy as
a Kubernetes External Load-balancer, Service discovery with consul, and so
on.
I don't really know where to put those in the summary you proposed, but
that's what I want to see in such wiki :)
Baptiste
On Mon, Feb 4, 2019 at 6:33 PM Willy Tarreau wrote:
> Hi all,
>
> as discussed a few times in the past, we have the possibility to enable
> the Wiki on the github repository. In the past a few of us thought it
> would be a nice alternative to the obsolete architecture manual because
> it would allow a number of people to contribute to various areas with a
> relative ease.
>
> Thus today I've given it a deeper thought and figured that not only the
> architecture manual should go there but also some recommendations or
> indications about how to test certain things.
>
> So in order to help start with something, I'm proposing that we create
> sort of a plan vaguely following the one below (but I'm totally open to
> criticism, feel free to voice in). I'd rather avoid to have placeholders
> that will never be filled ("this site is under construction" for the old
> ones like me), and I'm not sure we can save drafts, so maybe we can
> simply place this into a separate document that serves as the initial
> plan for reference.
>
> If we manage to go far enough with this, we'll finally be able to kill
> the architecture manual 12 years after its last update.
>
> Please share ideas / dos / donts. However please keep in mind that wish
> lists are best served when the requester proposes to handle them by
> him/herself ;-)
>
> Thanks,
> Willy
>
> ---
>
> Project organization
>
>
> Team
>
> Places
> - haproxy.org
> - mailing list
> - discourse
> - github
>
> Release cycle
> - development
> - stable
> - LTS
>
> Contributing code
> - read CONTRIBUTING
> - read coding-style
> - read git log
>
> Participating with no code
> - read problem reports
> - review / adjust patches
> - help others
> - contribute to the wiki
> - test the code
> - suggest use cases
> - report issues, gdb traces
> - bisect issues
>
>
> Architecture manual
> ---
>
> Presentation
>
> How a proxy works
>
> Terminology
> - client
> - server
> - frontend / service
> - backend / farm
> - active / backup
> - connection, session, transaction, request, response
>
> Topologies
> - edge + short silos
> - central LB + a bunch of servers, multiple layers
> - service clusters (stacks of [haproxy + servers])
> - sidecar
>
> Setting up HA for haproxy
> - keepalived / ucarp / pacemaker ?
> - LVS
> - ECMP
> - ELB
>
> Common use cases
> 1) as a basic proxy
> - IPv6 to IPv4 gatewaying
> - port filtering
> - TLS enforcement / cert validation
> - protocol inspection. E.g. HTTP+SSH, SMTP banner delay
> - authentication
> - transparent proxying
> - logging / anomaly detection / time measurement
> - DoS protection (stick tables, tarpit)
> - traffic aggregation (multiple interfaces attachment)
> - traffic limitation (maxconn)
>
> 2) as an accelerating proxy
> - TLS offloading
> - traffic compression
> - response caching
>
> 3) as a load balancer
> - classical stateless L7 LB
> - classical stateful L7 LB
> - when to use round robin -> short requests / web applications
> - when to use least conn -> long sessions
> - when to use first-> ephemeral VMs, fast scale-in/scale-out
> - when to use hashing -> affinity (e.g. caches)
> - consistent vs map-based hashing
> - persistence vs hashing
> - inbound vs outbound load balancing
> - backup server(s)
> - grouping traffic to a single server (active/backup for data bases)
>
> Advanced use cases
> - providing TLS to Varnish (in + out)
> - caching clusters with consistent hashing and small object caching
> - H2 in front of Nginx (max-reuse)
> - using priorities to speed up critical parts of a site
> - service discovery via DNS, CLI, Lua
> - managing certificates at scale / let's encrypt
> - tuning for extreme loads. pitfalls.
> - accessing services inside Linux containers using namespaces
> - multi-site abuser eviction (stick-tables + peers)
>
> Scripting in Lua
>
> On the fly management
> - stats page
> - CLI
> - signals
> - master-worker
> - agent-check
> - add-acl/del-acl
> - DNS
>
> Operating system specificities
> - Linux >= 3.9 : SO_REUSEPORT
> - Linux >= 4.2 : IP_BIND_ADDRESS_NO_PORT
>
> Performance considerations
> - orders of magnitude for a few typical metrics
> - cost of processing for various operations
>
