Hi,
yes, we already checked that
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 3055616 51143 3004473 2% /
none 2057204 720 2056484 1% /dev
none 2058311 1 2058310 1% /dev/shm
none 2058311 22 2058289 1% /var/run
none 2058311 2 2058309 1% /var/lock
none 2058311 3 2058308 1% /lib/init/rw
/dev/sdb 4818066560 4277250 4813789310 1% /storage/6
/dev/sda6 1220608 2232 1218376 1% /var
/dev/sda7 2445984 12 2445972 1% /tmp
We managed to free some more space (now ~1TB free) and creating files
seem to work again. However, during the migration of a user, some files
don't show up in his directory, although we are able to find them via ls
<filename>.
We decided to reverse the migration and try again, but then I was not
able to delete his directory:
rm: FATAL: directory `delete_me/rootfiles/dplusdminus' changed dev/ino
Also, when we tried chown -R <user>:<group> user/ the following error
popped up.
chown: fts_read failed: No such file or directory
Additionally, some directories show up in ls -l like that
?--------- ? ? ? ? ? b_dd
This all is really confusing to us. Can someone shed some light on this
mess?
Regards,
Daniel
On 01/19/2011 10:53 AM, Andrew Séguin wrote:
Hi,
Maybe a silly thought / maybe you have already thought of it, but is
there still some free inodes on the underlying filesystem (df -i)?
Regards,
Andrew
On Wed, Jan 19, 2011 at 10:44 AM, Daniel Zander
<[email protected]> wrote:
Hi,
we just made some space available on the affected brick (386 GB free), but
still the same problem remains. But I don't think that this is really a
glusterFS problem, as also root cannot create any directories directly on
the fileserver anymore.
Thanks so far,
Daniel
On 01/19/2011 10:12 AM, Mark "Naoki" Rogers wrote:
I think you might want to look into re-balance:
http://europe.gluster.org/community/documentation/index.php?title=Gluster_3.1:_Rebalancing_Volumes&redirect=no
<http://europe.gluster.org/community/documentation/index.php?title=Gluster_3.1:_Rebalancing_Volumes&redirect=no>
It's generally for adding/removing bricks but might re-distribute data
in a way that solves your disk space issue.
On 01/19/2011 04:43 PM, Daniel Zander wrote:
Hi!
Assuming you are doing a straight distribute there(?), if the user in
Yes's it's a distributed volume.
question is hashed onto the brick that is 100% full you'll get a space
Is there a way around this other than moving files away from this one
brick by hand?
error. Not sure I followed your migration details though, when you say
"user directories were moved into one of the above folders" do you mean
copied directly onto the individual storage bricks?
Yes, eg. user_a had the following directories
server5:/storage/5/user_a
server6:/storage/6/user_a
Then we performed a move:
ssh server5 "mv /storage/5/user_a/ /storage/5/cluster/user_a"
ssh server6 "mv /storage/6/user_a/ /storage/6/cluster/user_a"
This was done as it would not cause any network traffic. Then the volume
was created like that:
Brick1: 192.168.101.249:/storage/4/cluster
Brick2: 192.168.101.248:/storage/5/cluster
Brick3: 192.168.101.250:/storage/6/cluster
Brick4: 192.168.101.247:/storage/7/cluster
Brick5: 192.168.101.246:/storage/8/cluster
Regards,
Daniel
On 01/19/2011 05:01 AM, [email protected] wrote:
mv: cannot create regular file `/storage/cluster/<etc...>': No space
left on device
Doing df -h tells me, however:
glusterfs#192.168.101.247:/lemmy
104T 69T 36T 66% /storage/cluster
It may be of importance that one brick in the cluster is actually 100%
used. Others are almost completely empty. I am using GlusterFS.3.1.1,
the file servers are running debian lenny or ubuntu server 10.04,
clients are SLC4, SLC5, CentOS and ubuntu server 10.04.
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
