[DISCUSS] Proposal: support more global rules for Apache APISIX Ingress Controller

[Xin Rong]

Background
Github Issues:
https://github.com/apache/apisix-ingress-controller/issues/1104

Currently APISIX Ingress only supports Turn on global rules for both
plugins (prometheus and skywalking). However, APISIX has complete global
rules. It is not enough to support only two. It cannot meet the needs of
most users. APISIX Ingress should support more global rules

apiVersion: apisix.apache.org/v2
kind: ApisixClusterConfig
metadata:
  name: default
spec:
  monitoring:
    prometheus:
      enable: true
    skywalking:
      enable: true
      sampleRatio: 0.5

Solution

   1. Added Plugins support on ApisixClusterConfig resources

ApisixClusterConfig in the design and use of some ambiguity, in the
multi-cluster architecture, ApisixClusterConfig currently used in the way
and semantics are incorrect, its evolution direction to be discussed, in
the unclear responsibilities are not recommended to change it.
detailed design
Similar to ApisixRoute, support more plugins by adding a Plugins field:

apiVersion: apisix.apache.org/v2
kind: ApisixClusterConfig
metadata:
  name: default
spec:
  plugins:
  - name: prometheus
    enable: true
  - name: limit-count
    enable: true
    config:
      time_window: 60


   1. Added Annotations support in ApisixPluginConfig

By adding Annotations to extend the ability of ApisixPluginConfig to act
globally, this approach improves the flexibility of ApisixPluginConfig.
global: true Turn on global mode, allowing plugins to be applied to global:
true rules.

apiVersion: apisix.apache.org/v2
kind: ApisixPluginConfig
metadata:
  name: plugin-config
  annotations:
   apisix.pache.org/global: true
spec:
  plugins:
  - name: prometheus
    enable: true
  - name: limit-count
    enable: true
    config:
      time_window: 60
      policy: "local"
      count: 2
      key: "remote_addr"
      rejected_code: 503


   1. ApisixGlobalRule

Added a new CRD resource, representing ApisixGlobalRule, which corresponds
to the concept of APISIX global rules, is easy to understand, and can
evolve with APISIX global rules.

apiVersion: apisix.apache.org/v2
kind: ApisixGlobalRule
metadata:
  name: apisix-plugins-cluster
spec:
  plugins:
  - name: prometheus
    enable: true
  - name: limit-count
    enable: true
    config:
      time_window: 60
      policy: "local"
      count: 2
      key: "remote_addr"
      rejected_code: 50

I think the third method is more appropriate because it is simpler and
easier to understand and can evolve with the global rules.