Anyone have any help for clustering?
I've gone thru the Clustering Guide over and over, and can't see where I'm
going wrong.
I've tried 4 or 5 times now, starting with my own server builds, and with the
PF-ZEN appliances as a starting point, and always have errors when it comes to
starting the pf services.
When I get to the end of step 3 in the guide, and restart the pf services
(/usr/local/pf/bin/pfcmd service pf restart), I get an error writing to the L2
cache, and it hangs after restarting haproxy.
...
[root@PacketFence ~]# /usr/local/pf/bin/pfcmd service pf restart
service|command
carbon-cache|already stopped
carbon-relay|already stopped
collectd|already stopped
dhcpd|already stopped
haproxy|stop
httpd.aaa|already stopped
httpd.admin|stop
httpd.collector|already stopped
httpd.dispatcher|already stopped
httpd.graphite|already stopped
httpd.parking|already stopped
httpd.portal|already stopped
httpd.proxy|already stopped
httpd.webservices|already stopped
iptables|stop
keepalived|already stopped
p0f|already stopped
pfbandwidthd|already stopped
pfdetect|already stopped
pfdhcplistener|already stopped
pfdns|already stopped
pffilter|already stopped
pfmon|already stopped
pfqueue|already stopped
pfsetvlan|already stopped
pfsso|already stopped
radiusd-acct|already stopped
radiusd-auth|already stopped
radiusd-load_balancer|already stopped
radsniff|already stopped
redis_ntlm_cache|already stopped
redis_queue|already stopped
routes|stop
snmptrapd|already stopped
statsd|already stopped
winbindd|already stopped
Could not write namespace config::PfDefault to L2 cache !
Could not write namespace config::Documentation to L2 cache !
Could not write namespace config::Cluster to L2 cache !
Could not write namespace config::Pf to L2 cache !
Could not write namespace interfaces to L2 cache !
Could not write namespace interfaces::management_network to L2 cache !
Could not write namespace resource::local_secret to L2 cache !
Could not write namespace config::Network to L2 cache !
Could not write namespace config::Switch to L2 cache !
Could not write namespace config::Authentication to L2 cache !
Could not write namespace config::Domain to L2 cache !
Could not write namespace resource::authentication_sources to L2 cache !
Could not write namespace resource::fqdn to L2 cache !
Could not write namespace config::AdminRoles to L2 cache !
Could not write namespace config::ApacheFilters to L2 cache !
Could not write namespace config::Authentication to L2 cache !
Could not write namespace resource::authentication_config_hash to L2 cache !
Could not write namespace resource::authentication_lookup to L2 cache !
Could not write namespace resource::authentication_sources to L2 cache !
Could not write namespace resource::passthroughs to L2 cache !
Could not write namespace config::Profiles to L2 cache !
Could not write namespace resource::guest_self_registration to L2 cache !
Could not write namespace config::BillingTiers to L2 cache !
Could not write namespace config::Cluster to L2 cache !
Could not write namespace config::Pf to L2 cache !
Could not write namespace resource::CaptivePortal to L2 cache !
Could not write namespace resource::Database to L2 cache !
Could not write namespace resource::fqdn to L2 cache !
Could not write namespace config::Pfdetect to L2 cache !
Could not write namespace resource::trapping_range to L2 cache !
Could not write namespace resource::stats_levels to L2 cache !
Could not write namespace resource::passthroughs to L2 cache !
Could not write namespace interfaces to L2 cache !
Could not write namespace interfaces::listen_ints to L2 cache !
Could not write namespace interfaces::dhcplistener_ints to L2 cache !
Could not write namespace interfaces::ha_ints to L2 cache !
Could not write namespace interfaces::internal_nets to L2 cache !
Could not write namespace interfaces(PacketFence) to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::inline_enforcement_nets to L2 cache !
Could not write namespace interfaces::vlan_enforcement_nets to L2 cache !
Could not write namespace interfaces::monitor_int to L2 cache !
Could not write namespace interfaces::management_network to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace interfaces::portal_ints to L2 cache !
Could not write namespace interfaces::inline_nets to L2 cache !
Could not write namespace interfaces::routed_isolation_nets to L2 cache !
Could not write namespace interfaces::routed_registration_nets to L2 cache !
Could not write namespace interfaces::radius_ints to L2 cache !
Could not write namespace interfaces(PacketFence) to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace config::Pf(PacketFence) to L2 cache !
Could not write namespace resource::cluster_servers to L2 cache !
Could not write namespace resource::cluster_hosts to L2 cache !
Could not write namespace config::DNS_Filters to L2 cache !
Could not write namespace FilterEngine::DNS_Scopes to L2 cache !
Could not write namespace config::DhcpFilters to L2 cache !
Could not write namespace FilterEngine::DhcpScopes to L2 cache !
Could not write namespace config::Documentation to L2 cache !
Could not write namespace config::Domain to L2 cache !
Could not write namespace resource::domain_dns_servers to L2 cache !
Could not write namespace config::Domain(PacketFence) to L2 cache !
Could not write namespace config::Firewall_SSO to L2 cache !
Could not write namespace config::FloatingDevices to L2 cache !
Could not write namespace config::Network to L2 cache !
Could not write namespace interfaces to L2 cache !
Could not write namespace interfaces::listen_ints to L2 cache !
Could not write namespace interfaces::dhcplistener_ints to L2 cache !
Could not write namespace interfaces::ha_ints to L2 cache !
Could not write namespace interfaces::internal_nets to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::inline_enforcement_nets to L2 cache !
Could not write namespace interfaces::vlan_enforcement_nets to L2 cache !
Could not write namespace interfaces::monitor_int to L2 cache !
Could not write namespace interfaces::management_network to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace interfaces::portal_ints to L2 cache !
Could not write namespace interfaces::inline_nets to L2 cache !
Could not write namespace interfaces::routed_isolation_nets to L2 cache !
Could not write namespace interfaces::routed_registration_nets to L2 cache !
Could not write namespace interfaces::radius_ints to L2 cache !
Could not write namespace interfaces(PacketFence) to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace config::PKI_Provider to L2 cache !
Could not write namespace config::PfDefault to L2 cache !
Could not write namespace config::Pf to L2 cache !
Could not write namespace resource::CaptivePortal to L2 cache !
Could not write namespace resource::Database to L2 cache !
Could not write namespace resource::fqdn to L2 cache !
Could not write namespace config::Pfdetect to L2 cache !
Could not write namespace resource::trapping_range to L2 cache !
Could not write namespace resource::stats_levels to L2 cache !
Could not write namespace resource::passthroughs to L2 cache !
Could not write namespace interfaces to L2 cache !
Could not write namespace interfaces::listen_ints to L2 cache !
Could not write namespace interfaces::dhcplistener_ints to L2 cache !
Could not write namespace interfaces::ha_ints to L2 cache !
Could not write namespace interfaces::internal_nets to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::inline_enforcement_nets to L2 cache !
Could not write namespace interfaces::vlan_enforcement_nets to L2 cache !
Could not write namespace interfaces::monitor_int to L2 cache !
Could not write namespace interfaces::management_network to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace interfaces::portal_ints to L2 cache !
Could not write namespace interfaces::inline_nets to L2 cache !
Could not write namespace interfaces::routed_isolation_nets to L2 cache !
Could not write namespace interfaces::routed_registration_nets to L2 cache !
Could not write namespace interfaces::radius_ints to L2 cache !
Could not write namespace interfaces(PacketFence) to L2 cache !
Could not write namespace interfaces::internal_nets(PacketFence) to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace config::Pf(PacketFence) to L2 cache !
Could not write namespace config::Pfmon to L2 cache !
Could not write namespace config::PfmonDefault to L2 cache !
Could not write namespace config::Pfqueue to L2 cache !
Could not write namespace config::PortalModules to L2 cache !
Could not write namespace config::Profiles to L2 cache !
Could not write namespace FilterEngine::Profile to L2 cache !
Could not write namespace resource::URI_Filters to L2 cache !
Could not write namespace config::Provisioning to L2 cache !
Could not write namespace config::RadiusFilters to L2 cache !
Could not write namespace FilterEngine::RadiusScopes to L2 cache !
Could not write namespace config::Realm to L2 cache !
Could not write namespace config::Report to L2 cache !
Could not write namespace config::Roles to L2 cache !
Could not write namespace config::Scan to L2 cache !
Could not write namespace config::Violations to L2 cache !
Could not write namespace FilterEngine::Violation to L2 cache !
Could not write namespace resource::accounting_triggers to L2 cache !
Could not write namespace resource::bandwidth_expired_violations to L2 cache !
Could not write namespace config::VlanFilters to L2 cache !
Could not write namespace FilterEngine::VlanScopes to L2 cache !
Could not write namespace config::Wmi to L2 cache !
Could not write namespace resource::array_test to L2 cache !
Could not write namespace resource::local_secret to L2 cache !
Could not write namespace config::Switch to L2 cache !
Could not write namespace resource::default_switch to L2 cache !
Could not write namespace resource::switches_group to L2 cache !
Could not write namespace resource::switches_ranges to L2 cache !
Could not write namespace interfaces::management_network to L2 cache !
Could not write namespace interfaces::management_network(PacketFence) to L2
cache !
Could not write namespace resource::SwitchTypesConfigured to L2 cache !
Could not write namespace resource::cli_switches to L2 cache !
Could not write namespace resource::reverse_fqdn to L2 cache !
Could not write namespace resource::switches_list to L2 cache !
Created symlink from
/etc/systemd/system/packetfence-base.target.wants/packetfence-haproxy.service
to /usr/lib/systemd/system/packetfence-haproxy.service.
haproxy|start
Created symlink from
/etc/systemd/system/packetfence.target.wants/packetfence-httpd.aaa.service to
/usr/lib/systemd/system/packetfence-httpd.aaa.service.
<HANGS HERE>
...
My cluster.conf as follows:
# Cluster configuration file for active/active
# This file will have it deactivated by default
# To activate the active/active mode, set a management IP in the cluster section
# Before doing any changes to this file, read the documentation
[CLUSTER]
management_ip=172.16.1.16
[CLUSTER interface eth0]
ip=172.16.1.16
type=management,high-availability
[CLUSTER interface eth0.2]
ip=192.168.2.5
type=internal
[CLUSTER interface eth0.3]
ip=192.168.3.5
type=internal
[packetfence.earthcolor.com]
management_ip=172.16.1.50
[packetfence.earthcolor.com interface eth0]
ip=172.16.1.50
type=management,high-availability
mask=255.255.248.0
[packetfence.earthcolor.com interface eth0.2]
enforcement=vlan
ip=192.168.2.1
type=internal
mask=255.255.255.0
[packetfence.earthcolor.com interface eth0.3]
enforcement=vlan
ip=192.168.3.1
type=internal
mask=255.255.255.0
[packetfence2.earthcolor.com]
management_ip=172.16.1.49
[packetfence2.earthcolor.com interface eth0]
ip=172.16.1.49
type=management,high-availability
mask=255.255.248.0
[packetfence2.earthcolor.com interface eth0.2]
enforcement=vlan
ip=192.168.2.2
type=internal
mask=255.255.255.0
[packetfence2.earthcolor.com interface eth0.3]
enforcement=vlan
ip=192.168.3.2
type=internal
mask=255.255.255.0
[packetfence3.earthcolor.com]
management_ip=172.16.1.19
[packetfence3.earthcolor.com interface eth0]
ip=172.16.1.19
type=management,high-availability
mask=255.255.248.0
[packetfence3.earthcolor.com interface eth0.2]
enforcement=vlan
ip=192.168.2.3
type=internal
mask=255.255.255.0
[packetfence3.earthcolor.com interface eth0.3]
enforcement=vlan
ip=192.168.3.3
type=internal
mask=255.255.255.0
My pf.conf follows:
[general]
#
# general.domain
#
# Domain name of PacketFence system.
domain=earthcolor.com
#
# general.dhcpservers
#
# Comma-delimited list of DHCP servers. Passthroughs are created to allow DHCP
transactions from even "trapped" nodes.
dhcpservers=127.0.0.1,172.16.1.12,172.17.1.12,192.168.35.12
[alerting]
#
# alerting.emailaddr
#
# Email address to which notifications of rogue DHCP servers, violations with
an action of "email", or any other
# PacketFence-related message goes to.
[email protected]
[database]
#
# database.pass
#
# Password for the mysql database used by PacketFence. Changing this parameter
after the initial configuration will *not* change it in the database it self,
only in the configuration.
pass=<mypassword>
host=127.0.0.1
[monitoring]
db_host=127.0.0.1
[active_active]
galera_replication_username=pfcluster
galera_replication_password=<mypassword>
[interface eth0]
ip=172.16.1.50
type=management
mask=255.255.248.0
[interface eth0.2]
enforcement=vlan
ip=192.168.2.1
type=internal
mask=255.255.255.0
[interface eth0.3]
enforcement=vlan
ip=192.168.3.1
type=internal
mask=255.255.255.0
At first I had no error when checking config with pfcmd checkup. But after
restarting the database with "/usr/local/pf/sbin/pf-mariadb
--force-new-cluster" , I get error connecting to database:
# /usr/local/pf/bin/pfcmd checkup
Checking configuration sanity...
Could not write namespace config::PfDefault to L2 cache !
Could not write namespace config::Documentation to L2 cache !
Could not write namespace config::Cluster to L2 cache !
Could not write namespace config::PfDefault to L2 cache !
Could not write namespace config::Documentation to L2 cache !
Could not write namespace config::Cluster to L2 cache !
Radius configuration is missing from raddb directory. Assuming this is a first
run.
FATAL - Unable to connect to your database. Please verify your connection
settings in conf/pf.conf and make sure that it is started.
WARNING - unknown configuration parameter monitoring.db_host if you added the
parameter yourself make sure it is present in conf/documentation.conf
FATAL - Cannot connect to database to check schema version: unable to connect
to database: Lost connection to MySQL server at 'reading initial communication
packet', system error: 25 "Inappropriate ioctl for device" at
/usr/local/pf/lib/pf/version.pm line 42.
In /var/lib/mysql/PacketFence.err:
2017-05-09 16:39:49 140032874637056 [Note] WSREP: Flow-control interval: [16,
16]
2017-05-09 16:39:49 140032874637056 [Note] WSREP: Restored state OPEN -> JOINED
(0)
2017-05-09 16:39:49 140032874637056 [Note] WSREP: Member 0.0 (PacketFence)
synced with group.
2017-05-09 16:39:49 140032874637056 [Note] WSREP: Shifting JOINED -> SYNCED
(TO: 0)
2017-05-09 16:39:49 140033174883072 [Note] WSREP: New cluster view: global
state: 1e95c0b6-34d6-11e7-a3de-ae380f6439d6:0, view# 1: Primary, number of
nodes: 1, my index: 0, protocol version 3
2017-05-09 16:39:49 140033175202048 [Note] WSREP: SST complete, seqno: 0
2017-05-09 16:39:49 7f5c03aa2900 InnoDB: Warning: Using
innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in
future releases, together with the option innodb_use_sys_malloc and with the
InnoDB's internal memory allocator.
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Using mutexes to ref count
buffer pool pages
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: The InnoDB memory heap is
disabled
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Mutexes and rw_locks use GCC
atomic builtins
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: GCC builtin
__atomic_thread_fence() is used for memory barrier
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Compressed tables use zlib
1.2.7
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Using Linux native AIO
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Using SSE crc32 instructions
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Initializing buffer pool,
size = 500.0M
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Completed initialization of
buffer pool
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Highest supported file
format is Barracuda.
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: 128 rollback segment(s) are
active.
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Waiting for purge to start
2017-05-09 16:39:50 140033175202048 [Note] InnoDB: Percona XtraDB
(http://www.percona.com) 5.6.34-79.1 started; log sequence number 1724734259
2017-05-09 16:39:50 140031821842176 [Note] InnoDB: Dumping buffer pool(s) not
yet started
2017-05-09 16:39:50 140033175202048 [Note] Plugin 'FEEDBACK' is disabled.
2017-05-09 16:39:50 140033175202048 [Note] Server socket created on IP:
'172.16.1.19'.
2017-05-09 16:39:50 140033175202048 [Note] /usr/sbin/mysqld: ready for
connections.
Version: '10.1.21-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306
MariaDB Server
2017-05-09 16:39:50 140033174883072 [Note] WSREP: REPL Protocols: 7 (3, 2)
2017-05-09 16:39:50 140033174883072 [Note] WSREP: Assign initial position for
certification: 0, protocol version: 3
2017-05-09 16:39:50 140032932943616 [Note] WSREP: Service thread queue flushed.
2017-05-09 16:39:50 140033174883072 [Note] WSREP: GCache history reset:
old(00000000-0000-0000-0000-000000000000:0) ->
new(1e95c0b6-34d6-11e7-a3de-ae380f6439d6:0)
2017-05-09 16:39:50 140033174883072 [Note] WSREP: Synchronized with group,
ready for connections
Any help greatly appreciated
Thanks
From: Sokolowski, Darryl [mailto:[email protected]]
Sent: Wednesday, May 3, 2017 2:58 PM
To: [email protected]
Subject: [PacketFence-users] Creating PF 7 cluster radiusd errors
Hi all,
I am trying to create a PF 7 cluster following the Clustering Guide, and get
the following error when performing the checkup at the bottom of page 8:
...
/usr/local/pf/bin/pfcmd checkup
Checking configuration sanity...
WARNING - unknown configuration parameter monitoring.db_host if you added the
parameter yourself make sure it is present in conf/documentation.conf
...
I did have to add [monitoring] section to the pf.conf, as it didn't exist.
Is there something I need to add to documentation.conf?
If I attempt to start the pf services, radiusd-acct, radiusd-auth, and
radiusd-load_balancer, all fail to start. All show the similar message "start
request repeated too quickly, refusing to start"
...
# systemctl -l status packetfence-radiusd-acct.service
รข packetfence-radiusd-acct.service - PacketFence FreeRADIUS multi-protocol
accounting server
Loaded: loaded (/lib/systemd/system/packetfence-radiusd-acct.service;
enabled)
Active: failed (Result: start-limit) since Wed 2017-05-03 14:10:06 EDT; 3min
57s ago
Docs: man:radiusd(8)
man:radiusd.conf(5)
http://wiki.freeradius.org/
http://networkradius.com/doc/
Process: 5628 ExecStartPre=/usr/sbin/freeradius -d /usr/local/pf/raddb -n
acct -Cxm -lstdout (code=exited, status=1/FAILURE)
Process: 5581 ExecStartPre=/usr/local/pf/bin/pfcmd service radiusd
generateconfig (code=exited, status=0/SUCCESS)
Main PID: 1876 (code=exited, status=0/SUCCESS)
May 03 14:10:06 packetfence systemd[1]: Starting PacketFence FreeRADIUS
multi-protocol accounting server...
May 03 14:10:06 packetfence systemd[1]: packetfence-radiusd-acct.service start
request repeated too quickly, refusing to start.
May 03 14:10:06 packetfence systemd[1]: Failed to start PacketFence FreeRADIUS
multi-protocol accounting server.
May 03 14:10:06 packetfence systemd[1]: Unit packetfence-radiusd-acct.service
entered failed state.
May 03 14:10:11 packetfence systemd[1]: Starting PacketFence FreeRADIUS
multi-protocol accounting server...
May 03 14:10:11 packetfence systemd[1]: packetfence-radiusd-acct.service start
request repeated too quickly, refusing to start.
May 03 14:10:11 packetfence systemd[1]: Failed to start PacketFence FreeRADIUS
multi-protocol accounting server.
May 03 14:10:15 packetfence systemd[1]: Starting PacketFence FreeRADIUS
multi-protocol accounting server...
May 03 14:10:15 packetfence systemd[1]: packetfence-radiusd-acct.service start
request repeated too quickly, refusing to start.
May 03 14:10:15 packetfence systemd[1]: Failed to start PacketFence FreeRADIUS
multi-protocol accounting server.
...
In packetfence.log I see:
...
May 3 14:10:05 packetfence packetfence: WARN radsniff-wrapper(5613):
requesting member ips for an undefined interface... (pf::cluster::members_ips)
May 3 14:10:05 packetfence packetfence: FATAL radsniff-wrapper(5613): Use of
uninitialized value $_ in concatenation (.) or string at
/usr/local/pf/lib/pf/services/manager/radsniff.pm line 45.
(pf::services::manager::radsniff::make_filter)
May 3 14:10:06 packetfence packetfence: WARN radsniff-wrapper(5635):
requesting member ips for an undefined interface... (pf::cluster::members_ips)
May 3 14:10:06 packetfence packetfence: FATAL radsniff-wrapper(5635): Use of
uninitialized value $_ in concatenation (.) or string at
/usr/local/pf/lib/pf/services/manager/radsniff.pm line 45.
(pf::services::manager::radsniff::make_filter)
May 3 14:10:07 packetfence packetfence: INFO pfcmd.pl(5590): generating
/usr/local/pf/var/conf/ssl-certificates.conf
(pf::services::manager::httpd::generateCommonConfig)
May 3 14:10:07 packetfence packetfence: INFO pfcmd.pl(5590): generating
/usr/local/pf/var/conf/captive-portal-common
(pf::services::manager::httpd::generateCommonConfig)
May 3 14:10:07 packetfence packetfence: WARN radsniff-wrapper(5641):
requesting member ips for an undefined interface... (pf::cluster::members_ips)
May 3 14:10:07 packetfence packetfence: FATAL radsniff-wrapper(5641): Use of
uninitialized value $_ in concatenation (.) or string at
/usr/local/pf/lib/pf/services/manager/radsniff.pm line 45.
(pf::services::manager::radsniff::make_filter)
May 3 14:10:10 packetfence packetfence: WARN pfcmd.pl(5633): requesting member
ips for an undefined interface... (pf::cluster::m
...
Any help greatly appreciated.
Thanks
Darryl
________________________________
>>> CONFIDENTIALITY NOTICE <<<
This electronic mail (e-mail) message, including any and/or all attachments, is
for the sole use of the intended recipient(s), and may contain confidential
and/or privileged information, pertaining to business conducted under the
direction and supervision of EarthColor, Inc. All e-mail messages, which may
have been established as expressed views and/or opinions (stated either within
the e-mail message or any of its attachments), are left to the sole
responsibility of that of the sender, and are not necessarily attributed to
EarthColor, Inc. Unauthorized interception, review, use, disclosure or
distribution of any such information contained within this e-mail message
and/or its attachment(s), is(are) strictly prohibited. If you are not the
intended recipient, please contact the sender by replying to this e-mail
message, along with the destruction of all copies of the original e-mail
message (along with any attachments).
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
PacketFence-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/packetfence-users
