Hi, We're trying to set up a small (3 machine) geoserver cluster, and it's behaving oddly - some actions (creating or deleting workspaces, for example) are not completed or visible on all nodes in the cluster.
We are using the GeoServer Active Clustering Extension, with an external ActiveMQ broker and a shared filesystem between three peers (one of which we have set up for management through the UI; the other two are there for HA purposes). We are running the cluster in the P2P configuration as suggested here: https://docs.geoserver.geo-solutions.it/edu/en/clustering/clustering/active/index.html. We're running all three nodes as both masters and slaves (as implied by the P2P configuration), and they all share the same configuration filesystem. I include the cluster configuration settings at the end of this email. When we create a workspace or a data store, we are seeing occasionally (but often enough to be repeatable in a series of manual tests through the REST API) that the created resource isn't visible on every node in the cluster. It seems like sometimes the synchronisation message doesn't get through to all nodes, and they don't pick up the new resource, leading to subsequent failures in trying to access it. We can see through the ActiveMQ UI that one of the message queues (for one of the workers) isn't receiving the same number of messages as the other, which seems suspicious. The cluster is deployed in Kubernetes on Azure, with each node running in a different pod, and the nodes running on ringfenced VMs within the cluster, so no other pods will interfere with them. The shared filesystem is a CIFS filesystem from Azure, and the ActiveMQ instance is also in its own Kubernetes pod (in the same namespace). We've spent most of the day simply diagnosing this, and we're no further forward in working out what's going wrong. Has anybody seen this behaviour before, or knows what might be causing it? Thanks, Hugo. Cluster configuration settings: CLUSTER_CONFIG_DIR : "/usr/local/geoserver/cluster" instanceName : "75f234ba-59db-47e9-937a-d7b44b8b8485" readOnly : "disabled" empty : "false" brokerURL : "tcp://geoserver-broker.geoserver:61616" durable : "true" embeddedBroker : "disabled" toggleMaster : "true" connection.retry : "10" xbeanURL : "./broker.xml" embeddedBrokerProperties : "embedded-broker.properties" topicName : "VirtualTopic.geoserver" connection : "enabled" toggleSlave : "true" connection.maxwait : "500" group : "geoserver-cluster" _______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
