Hello Proxmox users! I'm in process of setting up 3-node HA cluster with shared ZFS-over-iSCSI storage. I ran into problem when I cannot setup properly my iSCSI storage. The setup follows:
1) Storage is a dedicated Linux server with ZFS filesystem. The server has 4 network cards. One network card is used for administration/general networking. Other three cards are cross-over connected to each of 3 Proxmox nodes. 2) Three identical servers running same latest PVE 4.1 Proxmox community version. Each of these Proxmox nodes is cross-over connected using UTP-6 cable to storage server's network cards. In other words node1 is connected to eth1 on storage, node2 is connected to the eth2 on storage, node3 is connected to eth3 on storage. My cluster is already configured - all nodes added, and all set. Storage server runs ZFS filesystem. Now I'm trying to setup shared storage on my cluster. First thing I thought is that I have to configure proper networking between nodes and storage server because all nodes have dedicated connection to storage server's network cards as described above. On storage system I created bridge interface "br0", added eth1,2,3 network cards into the bridge, assigned br0 IP address 192.168.4.1/24. Next thing, on the nodes I configured network card that is connected into storage server: eth1 192.168.4.10 (.11, and 12 on next nodes). I expected this to work - so that all nodes can access storage server using 192.168.4.1, but that did not work - nodes cannot see 192.168.4.1 (storage) in this setup. Then I decided I could use a hostname instead of IP address as a storage server target. So I re-configured networking on storage - each NIC was assigned /30 address, and each node was also assigned /30 address, so that storage and node could connect to each other using individual IPs using /30 subnet. Obviously /30 networking setup did the trick and now I could reach storage from each node. Next thing I did was to setup hostname mapping on nodes in /etc/hosts of every node: on node1 - /etc/hosts: 192.168.4.1 storage on node2 - /etc/hosts: 192.168.4.5 storage and so on. After that I went to Proxmox GUI to add shared ZFS-over-iSCSI storage. Before entering target name I configured SSH access using key name "storage" (/etc/pve/priv/zfs/storage_id_rsa) from every node. As a target name, I entered "storage" assuming that each node will use the "storage" name instead of IP and this way will allow each node to see storage from it's /30 subnet. Obviously this failed because Proxmox executed "zfs" commands to storage server using IP address and not hostnames as expected. The whole idea failed because node1 can access 192.168.4.1, but node2 cannot access this IP based on above setup. Sorry for making this story long, but hopefully I made it clear. How can I overcome this problem now? Perhaps I'm doing something wrong with bridge setup - I thought that bridging interfaces on storage will solve this issue, but it is not working. Thanks! _______________________________________________ pve-user mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
