Re: [Csync2] 3-node masters and root-owned files
On Wed, Aug 02, 2017 at 08:58:05PM -0500, Kevin Cackler wrote: > For those of you guys who are experiencing this issue: I noticed tonight, > after this happened yet again, that one of my files was owned by UID 1012 on > node3. The problem is that that uid does not exist on node3. That uid DOES > exist on node1, and was the affected user. So when csync2 copied this file > from node1 to node3, it copied the UID as well, and since that UID doesn't > exist on node3, the user on node3 did not have permission to access the > file. > > In other words: > > "user" on node1 has a UID of 1012 > "user" on node3 has a UID of 1007 > "user" updated a file on node1. When that file was copied to node3, the UID > of the owner and group was 1012 instead of 1007. This meant that "user" > could not access the file on node3. > > I really hope this can be resolved soon, as it's causing a ton of grief for > us. csync2 replicates numerical uid/gid. that has always been that way. If you find someone to write a "by name" feature, we are happy to take it. when authenticating against /etc/passwd and friends, you may want to put that into csync2 as well. otherwise, you have "centralized" authentication anyways (ldap or similar), and the id <-> name mapping is already covered. Lars ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2
Re: [Csync2] 3-node masters and root-owned files
For those of you guys who are experiencing this issue: I noticed tonight, after this happened yet again, that one of my files was owned by UID 1012 on node3. The problem is that that uid does not exist on node3. That uid DOES exist on node1, and was the affected user. So when csync2 copied this file from node1 to node3, it copied the UID as well, and since that UID doesn't exist on node3, the user on node3 did not have permission to access the file. In other words: "user" on node1 has a UID of 1012 "user" on node3 has a UID of 1007 "user" updated a file on node1. When that file was copied to node3, the UID of the owner and group was 1012 instead of 1007. This meant that "user" could not access the file on node3. I really hope this can be resolved soon, as it's causing a ton of grief for us. Kevin Cackler wrote: In my case all of my synced nodes had the bad ownership. After changing the ownership back to the correct value on one node, csync correctly fixed it on the rest of the nodes, no problem. In my case, the file with the bad ownership was one that was created by a web application. After discovering this problem, I caused the application to change the file again and the correct ownership was observed, so I don't think it's an application problem, at least not in my case. Roland van Laar wrote: Same here. There isn't a pattern I could deduce. The only weird thing that happened was that the root.root permissions are on the host where the file was put. So I didn't suspect csync, but the application doing the writing. Roland On 17-07-17 15:01, Aristedes Maniatis wrote: After years of reliable service, csync2 2.0 did exactly that for me just last week. One file suddenly owned by root. Ari On 17/7/17 10:32PM, Kevin Cackler wrote: Funnily enough, we are also experiencing this issue with the root owned files. Randomly, and without any definable pattern, so far as we can tell, we'll get a file that suddenly is owned by root:root with rw permissions and we have to go in and correct the permissions. So far we haven't been able to nail down the cause of this. Mark Hodge wrote: Hi, I've recently setup a csync cluster between 3 nodes and although the ring model works ok, it obviously fails when the middle server (node 2) is offline. Therefore I've been trying to get a working config that is something like this: node1 => node2 + node3 node2 => node1 + node3 node3 => node1 + node2 So at least if node2 is offline, node1+node3 are still syncing. Is this the best way to achieve this? using "master (slave)" pairs? I ended up putting csync on all nodes in a cron every minute (lsync would crash occasionally) when csync returned errors. I got lots of "Database is busy, sleeping a sec" errors, which I presumed was because csync was running at the same time on each node and causing db locks. So I staggered them, at 0, 20 and 40 sec in the minute which got rid of the "busy" errors. However, occasionally I will get random files appear one one or more nodes owned by "root" with perms "rw" only for owner. This means standard users cannot access these files. I suspect that csync is somehow failing to set uid/gid/perms after the copy. How can this happen? Mark. ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2
Re: [Csync2] 3-node masters and root-owned files
It's a pain as I'm now forced to re-evaluate our whole solution to 3-way
sync because of this issue...
When a file "test.txt" is updated, the logs show:
[12:22:01] Connecting to host node1 (PLAIN) ...
[12:22:01] Updating /test.txt on node1 ...
[12:22:01] File /test.txt is different on peer (cktxt char #0).
[12:22:01] File has been checked successfully (difference found).
[12:22:01] File is different on peer (rsync sig).
Local> PATCH
1Al8R_3ij5XXfI2e385IPPmvObt4OyXrHUxo0UnMvdZETeXIC2SGrFgQIeSMnSGu /test.txt\n
Peer> OK (send_data).\n
[12:22:01] Csync2 / Librsync: csync_rs_delta('/test.txt')
[12:22:01] Receiving sig_file from peer..
[12:22:01] Delta has been created successfully.
[12:22:02] While syncing file /opt/WeatherTest/test.txt:
[12:22:02] response from peer(/test.txt): node1 [1] <- OK (cmd_finished).
Local> SETOWN
1Al8R_3ij5XXfI2e385IPPmvObt4OyXrHUxo0UnMvdZETeXIC2SGrFgQIeSMnSGu
/test.txt 100904787 10513 \n
Peer> OK (cmd_finished).\n
Local> SETMOD
1Al8R_3ij5XXfI2e385IPPmvObt4OyXrHUxo0UnMvdZETeXIC2SGrFgQIeSMnSGu
/test.txt 33268\n
Peer> OK (cmd_finished).\n
Local> SETIME
1Al8R_3ij5XXfI2e385IPPmvObt4OyXrHUxo0UnMvdZETeXIC2SGrFgQIeSMnSGu
/test.txt 1499944915\n
Peer> OK (cmd_finished).\n
Local> BYE\n
Peer> OK (cu_later).\n
So I can only deduce that somewhere between "PATCH" and "SETOWN" csync
is crashing/failing/timing out.
In one case, I had the same file set to "root" on 2 nodes, but correct
permissions on the 3rd node.
I'm wondering if it's sqlite, or a locking issue, or network latency?
On 17/07/2017 14:01, Aristedes Maniatis wrote:
After years of reliable service, csync2 2.0 did exactly that for me just last
week. One file suddenly owned by root.
Ari
On 17/7/17 10:32PM, Kevin Cackler wrote:
Funnily enough, we are also experiencing this issue with the root owned files.
Randomly, and without any definable pattern, so far as we can tell, we'll get a
file that suddenly is owned by root:root with rw permissions and we have to go
in and correct the permissions. So far we haven't been able to nail down the
cause of this.
Mark Hodge wrote:
Hi,
I've recently setup a csync cluster between 3 nodes and although the
ring model works ok, it obviously fails when the middle server (node
2) is offline. Therefore I've been trying to get a working config that
is something like this:
node1 => node2 + node3
node2 => node1 + node3
node3 => node1 + node2
So at least if node2 is offline, node1+node3 are still syncing.
Is this the best way to achieve this? using "master (slave)" pairs?
I ended up putting csync on all nodes in a cron every minute (lsync
would crash occasionally) when csync returned errors.
I got lots of "Database is busy, sleeping a sec" errors, which I
presumed was because csync was running at the same time on each node
and causing db locks. So I staggered them, at 0, 20 and 40 sec in the
minute which got rid of the "busy" errors.
However, occasionally I will get random files appear one one or more
nodes owned by "root" with perms "rw" only for owner. This means
standard users cannot access these files. I suspect that csync is
somehow failing to set uid/gid/perms after the copy.
How can this happen?
Mark.
___
Csync2 mailing list
Csync2@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/csync2
___
Csync2 mailing list
Csync2@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/csync2
___
Csync2 mailing list
Csync2@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/csync2
___
Csync2 mailing list
Csync2@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/csync2
Re: [Csync2] 3-node masters and root-owned files
In my case all of my synced nodes had the bad ownership. After changing the ownership back to the correct value on one node, csync correctly fixed it on the rest of the nodes, no problem. In my case, the file with the bad ownership was one that was created by a web application. After discovering this problem, I caused the application to change the file again and the correct ownership was observed, so I don't think it's an application problem, at least not in my case. Roland van Laar wrote: Same here. There isn't a pattern I could deduce. The only weird thing that happened was that the root.root permissions are on the host where the file was put. So I didn't suspect csync, but the application doing the writing. Roland On 17-07-17 15:01, Aristedes Maniatis wrote: After years of reliable service, csync2 2.0 did exactly that for me just last week. One file suddenly owned by root. Ari On 17/7/17 10:32PM, Kevin Cackler wrote: Funnily enough, we are also experiencing this issue with the root owned files. Randomly, and without any definable pattern, so far as we can tell, we'll get a file that suddenly is owned by root:root with rw permissions and we have to go in and correct the permissions. So far we haven't been able to nail down the cause of this. Mark Hodge wrote: Hi, I've recently setup a csync cluster between 3 nodes and although the ring model works ok, it obviously fails when the middle server (node 2) is offline. Therefore I've been trying to get a working config that is something like this: node1 => node2 + node3 node2 => node1 + node3 node3 => node1 + node2 So at least if node2 is offline, node1+node3 are still syncing. Is this the best way to achieve this? using "master (slave)" pairs? I ended up putting csync on all nodes in a cron every minute (lsync would crash occasionally) when csync returned errors. I got lots of "Database is busy, sleeping a sec" errors, which I presumed was because csync was running at the same time on each node and causing db locks. So I staggered them, at 0, 20 and 40 sec in the minute which got rid of the "busy" errors. However, occasionally I will get random files appear one one or more nodes owned by "root" with perms "rw" only for owner. This means standard users cannot access these files. I suspect that csync is somehow failing to set uid/gid/perms after the copy. How can this happen? Mark. ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2 ___ Csync2 mailing list Csync2@lists.linbit.com http://lists.linbit.com/mailman/listinfo/csync2
