hi folks
Our task is to isolate cluster management from virtuals.
Nodes and virtuals (CT/QM) were at the same subnet before.
The following was already isolated by different subnet and VLAN before.
* 10.19.0.0/16 VLAN19 ... 2nd corosync ring
* 10.8.0.0/16 VLAN8 ... independent CEPH (not hyperconv)
===== ISOLATION
I used BRAIN from part 3.3.8 at
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_network_configuration
MANY thanks for these admin guides !
Really helpful source of PROXMOX community knowledge !
status
* pve-manager/7.3-3/c3928077 (running kernel: 5.15.74-1-pve)
* nodes&virtuals at OLD subnet 10.0.0.0/16 gateway 10.0.0.1
* nodes at NEW subnet=10.34.1.91/16 gateway=10.34.0.1 VLAN=34 (vmbr0.34)
* virtuals will continue to run in OLD subnet 10.0.0.0/16 (vmbr0)
Following config files are from a TESTING cluster 'minimox'
with 3 nodes (mox91,mox92,mox93)
example of host node isolation with
OLD IP 10.0.1.93/16 gateway 10.0.0.1
NEW IP 10.34.1.93/16 gateway 10.34.0.1 VLAN34
before isolation I was able to ping&nmap cluster node from inside of
CT/QM
after isolation it is NOT possible to ping&nmap cluster node from inside
of CT/QM ;-)
everything appears to work well
BUT I see a lot of rejected packages at syslog
so is it good ???
or should I do it different way at PRODUCTION cluster ???
BTW when I restarted corosync at ALL isolated nodes once more
there are NO more messages about rejected packets at syslog
any comments are appreciated
Nada
===== INTERFACES
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr1
iface vmbr1 inet static
address 10.8.3.93/16
bridge-ports vlan8
bridge-stp off
bridge-fd 0
#ceph
auto vmbr2
iface vmbr2 inet static
address 10.19.0.93/16
bridge-ports vlan19
bridge-stp off
bridge-fd 0
#corosync
auto vmbr0
iface vmbr0 inet static
address 10.0.1.93/16
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#LAN4virtuals
auto vmbr0.34
iface vmbr0.34 inet static
address 10.34.1.93/16
gateway 10.34.0.1
#LAN4management
auto vlan8
iface vlan8 inet manual
vlan-raw-device eno1
auto vlan19
iface vlan19 inet manual
vlan-raw-device eno1
==== COROSYNC
# cat /etc/pve/corosync.conf
logging {
debug: off
to_syslog: yes
}
nodelist {
node {
name: mox91
nodeid: 1
quorum_votes: 1
ring0_addr: 10.34.1.91
ring1_addr: 10.19.0.91
}
node {
name: mox92
nodeid: 2
quorum_votes: 1
ring0_addr: 10.34.1.92
ring1_addr: 10.19.0.92
}
node {
name: mox93
nodeid: 3
quorum_votes: 1
ring0_addr: 10.34.1.93
ring1_addr: 10.19.0.93
}
}
quorum {
provider: corosync_votequorum
}
totem {
cluster_name: minimox
config_version: 16
interface {
linknumber: 0
knet_link_priority: 100
}
interface {
linknumber: 1
knet_link_priority: 25
}
ip_version: ipv4
link_mode: passive
secauth: on
version: 2
}
===== REJECTED packets
# systemctl status corosync.service
● corosync.service - Corosync Cluster Engine
Loaded: loaded (/lib/systemd/system/corosync.service; enabled;
vendor preset: enabled)
Active: active (running) since Mon 2023-06-19 19:14:58 CEST; 19h
ago
Docs: man:corosync
man:corosync.conf
man:corosync_overview
Main PID: 1947 (corosync)
Tasks: 9 (limit: 18927)
Memory: 136.9M
CPU: 10min 38.124s
CGroup: /system.slice/corosync.service
└─1947 /usr/sbin/corosync -f
Jun 20 14:45:25 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.92:5405
Jun 20 14:45:25 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.91:5405
Jun 20 14:45:26 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.92:5405
Jun 20 14:45:26 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.91:5405
Jun 20 14:45:27 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.92:5405
Jun 20 14:45:27 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.91:5405
Jun 20 14:45:28 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.92:5405
Jun 20 14:45:28 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.91:5405
Jun 20 14:45:29 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.92:5405
Jun 20 14:45:29 mox93 corosync[1947]: [KNET ] rx: Packet rejected
from 10.34.1.91:5405
_______________________________________________
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
