Hello and thanks for your answer! The problem was the amazon ec2-disassociate command, where the public IP became unreachable. I now invoke just the ec2-associate and everything is fine!!
Thanks to all for trying to help, Dimos. On 20 Φεβ 2012, at 5:42 μ.μ., [email protected] wrote: > Send Pvfs2-developers mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pvfs2-developers digest..." > > > Today's Topics: > > 1. no space left on device when running on amazon ec2 > (Dimos Stamatakis) > 2. Re: no space left on device when running on amazon ec2 > (Dimos Stamatakis) > 3. Re: no space left on device when running on amazon ec2 > (Dimos Stamatakis) > 4. Re: Re: no space left on device when running on amazon ec2 > (Becky Ligon) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 20 Feb 2012 02:17:01 +0200 > From: Dimos Stamatakis <[email protected]> > Subject: [Pvfs2-developers] no space left on device when running on > amazon ec2 > To: [email protected] > Message-ID: > <CAB8CW0ET2U3OtOXTgZMXBPRiBmvR4=0fpym8gzdg8s0a4ks...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hello! > I have successfully run a pvfs installation on a eucalyptus cloud, but when > I moved to Amazon EC2, I get a very strange error. > When I run a metadata server it says: > > [S 02/20 00:04] PVFS2 Server ready. > > and then it says: > > [E 02/20 00:04] batch_create request got: No space left on device > ....... And this error repeats ...... > > I checked all of my devices and there is plenty of space, so I don't think > there is not enough space left... > Can you explain that? > What is the batch_create function? And where is it trying to write? > > Here is the output of the df -h on the data node: > > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 9.9G 2.7G 6.8G 29% / > tmpfs 308M 0 308M 0% /lib/init/rw > udev 10M 108K 9.9M 2% /dev > tmpfs 308M 4.0K 308M 1% /dev/shm > > and on the meta data node: > > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 9.9G 2.0G 7.5G 21% / > tmpfs 308M 0 308M 0% /lib/init/rw > udev 10M 108K 9.9M 2% /dev > tmpfs 308M 4.0K 308M 1% /dev/shm > > Many thanks, > Dimos. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20120220/e5087dbc/attachment-0001.htm > > ------------------------------ > > Message: 2 > Date: Mon, 20 Feb 2012 03:49:50 +0200 > From: Dimos Stamatakis <[email protected]> > Subject: [Pvfs2-developers] Re: no space left on device when running > on amazon ec2 > To: [email protected] > Message-ID: > <CAB8CW0EvXb6012tqNWpAsg_bug4+ZuLniKv8cTY0tX1fnic=g...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I forgot to tell you that I use ec2-associate-address commands to tell the > new master to grab the elastic IP address. If I do not use replication and > I use the normal IP addresses it works fine! > Is there a way to have High availability to amazon EC2 without use of > elastic IPs?? > > Many thanks, > Dimos. > > > On Mon, Feb 20, 2012 at 2:17 AM, Dimos Stamatakis <[email protected]>wrote: > >> Hello! >> I have successfully run a pvfs installation on a eucalyptus cloud, but >> when I moved to Amazon EC2, I get a very strange error. >> When I run a metadata server it says: >> >> [S 02/20 00:04] PVFS2 Server ready. >> >> and then it says: >> >> [E 02/20 00:04] batch_create request got: No space left on device >> ....... And this error repeats ...... >> >> I checked all of my devices and there is plenty of space, so I don't think >> there is not enough space left... >> Can you explain that? >> What is the batch_create function? And where is it trying to write? >> >> Here is the output of the df -h on the data node: >> >> Filesystem Size Used Avail Use% Mounted on >> /dev/sda1 9.9G 2.7G 6.8G 29% / >> tmpfs 308M 0 308M 0% /lib/init/rw >> udev 10M 108K 9.9M 2% /dev >> tmpfs 308M 4.0K 308M 1% /dev/shm >> >> and on the meta data node: >> >> Filesystem Size Used Avail Use% Mounted on >> /dev/sda1 9.9G 2.0G 7.5G 21% / >> tmpfs 308M 0 308M 0% /lib/init/rw >> udev 10M 108K 9.9M 2% /dev >> tmpfs 308M 4.0K 308M 1% /dev/shm >> >> Many thanks, >> Dimos. >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20120220/ffdfe9c8/attachment-0001.htm > > ------------------------------ > > Message: 3 > Date: Mon, 20 Feb 2012 05:48:30 +0200 > From: Dimos Stamatakis <[email protected]> > Subject: [Pvfs2-developers] Re: no space left on device when running > on amazon ec2 > To: [email protected] > Message-ID: > <cab8cw0er_jzyh9uo7z0wqekxhk3_yrnn+kd700srkksr8qv...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hello again! > I want to help you realize what is going wrong by telling you that the > client blocks at a pvfs2-ls (it does not say connection refused > immediatelly). It can also ping the new elastic IP normally! > > I redirected the metadata server output to a file and when I checked it i > didn't find anything wrong... It did all the gets - puts that are happening > everytime the DB is created. Here is the output tail: > > get (handle: 4611686018427387903)()(key_sz:8) -> (511)(4) > put (handle: 4611686018427387903)()(key_sz:8) -> (512)(4) > [1329709072:419164][4413/140213703595776] TROVE:DBPF:Berkeley DB: bulk_msg: > Send buffer after copy due to PERM > [1329709072:419173][4413/140213703595776] TROVE:DBPF:Berkeley DB: > send_bulk: Send 160 (0xa0) bulk buffer bytes > [1329709072:419183][4413/140213703595776] TROVE:DBPF:Berkeley DB: > //pvfs2-storage-space/27c41225/ rep_send_message: msgv = 7 logv 19 gen = 1 > eid -1, type bulk_log, LSN [1][217660] perm > [1329709072:419193][4413/140213703595776] TROVE:DBPF:Berkeley DB: > rep_send_function returned: -30975 > > How can I find out why this metadata server refuses serving the client > requests? > > Thanks again, > Dimos. > > > > On Mon, Feb 20, 2012 at 3:49 AM, Dimos Stamatakis <[email protected]>wrote: > >> I forgot to tell you that I use ec2-associate-address commands to tell the >> new master to grab the elastic IP address. If I do not use replication and >> I use the normal IP addresses it works fine! >> Is there a way to have High availability to amazon EC2 without use of >> elastic IPs?? >> >> Many thanks, >> Dimos. >> >> >> >> On Mon, Feb 20, 2012 at 2:17 AM, Dimos Stamatakis <[email protected]>wrote: >> >>> Hello! >>> I have successfully run a pvfs installation on a eucalyptus cloud, but >>> when I moved to Amazon EC2, I get a very strange error. >>> When I run a metadata server it says: >>> >>> [S 02/20 00:04] PVFS2 Server ready. >>> >>> and then it says: >>> >>> [E 02/20 00:04] batch_create request got: No space left on device >>> ....... And this error repeats ...... >>> >>> I checked all of my devices and there is plenty of space, so I don't >>> think there is not enough space left... >>> Can you explain that? >>> What is the batch_create function? And where is it trying to write? >>> >>> Here is the output of the df -h on the data node: >>> >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/sda1 9.9G 2.7G 6.8G 29% / >>> tmpfs 308M 0 308M 0% /lib/init/rw >>> udev 10M 108K 9.9M 2% /dev >>> tmpfs 308M 4.0K 308M 1% /dev/shm >>> >>> and on the meta data node: >>> >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/sda1 9.9G 2.0G 7.5G 21% / >>> tmpfs 308M 0 308M 0% /lib/init/rw >>> udev 10M 108K 9.9M 2% /dev >>> tmpfs 308M 4.0K 308M 1% /dev/shm >>> >>> Many thanks, >>> Dimos. >>> >> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20120220/8f30dcd1/attachment-0001.htm > > ------------------------------ > > Message: 4 > Date: Mon, 20 Feb 2012 10:42:21 -0500 > From: Becky Ligon <[email protected]> > Subject: Re: [Pvfs2-developers] Re: no space left on device when > running on amazon ec2 > To: Dimos Stamatakis <[email protected]> > Cc: [email protected] > Message-ID: > <cajbnf338csi4q80xpccry7kpm+u7bzwkpmvfhs7j1uxuo2c...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > When a server is started, it sends a batch_create request to every other > server in the filesystem. The batch_create request asks the receiving > server to send back a list of unused data handles (owned by that particular > server). For those handles in the list, the receiving server sets an > attribute in the local database to indicate that the handle is in use. You > may see a batch_create request after your servers have been running for a > while, since a server will request another batch of handles if its current > store gets low. This entire process is a performance enhancement, which > allows a file's data handles to be assigned by the metadata server without > contacting the data handle servers, thus reducing the time it takes to > create a file. > > With all of that said, it seems that one of your servers is having trouble > accessing the database or communicating with another server. I'm not > exactly sure without further research. Think about the above description > and see if you can't pinpoint which server is causing the trouble. > > Becky > > On Sun, Feb 19, 2012 at 10:48 PM, Dimos Stamatakis <[email protected]>wrote: > >> Hello again! >> I want to help you realize what is going wrong by telling you that the >> client blocks at a pvfs2-ls (it does not say connection refused >> immediatelly). It can also ping the new elastic IP normally! >> >> I redirected the metadata server output to a file and when I checked it i >> didn't find anything wrong... It did all the gets - puts that are happening >> everytime the DB is created. Here is the output tail: >> >> get (handle: 4611686018427387903)()(key_sz:8) -> (511)(4) >> put (handle: 4611686018427387903)()(key_sz:8) -> (512)(4) >> [1329709072:419164][4413/140213703595776] TROVE:DBPF:Berkeley DB: >> bulk_msg: Send buffer after copy due to PERM >> [1329709072:419173][4413/140213703595776] TROVE:DBPF:Berkeley DB: >> send_bulk: Send 160 (0xa0) bulk buffer bytes >> [1329709072:419183][4413/140213703595776] TROVE:DBPF:Berkeley DB: >> //pvfs2-storage-space/27c41225/ rep_send_message: msgv = 7 logv 19 gen = 1 >> eid -1, type bulk_log, LSN [1][217660] perm >> [1329709072:419193][4413/140213703595776] TROVE:DBPF:Berkeley DB: >> rep_send_function returned: -30975 >> >> How can I find out why this metadata server refuses serving the client >> requests? >> >> Thanks again, >> Dimos. >> >> >> >> On Mon, Feb 20, 2012 at 3:49 AM, Dimos Stamatakis <[email protected]>wrote: >> >>> I forgot to tell you that I use ec2-associate-address commands to tell >>> the new master to grab the elastic IP address. If I do not use replication >>> and I use the normal IP addresses it works fine! >>> Is there a way to have High availability to amazon EC2 without use of >>> elastic IPs?? >>> >>> Many thanks, >>> Dimos. >>> >>> >>> >>> On Mon, Feb 20, 2012 at 2:17 AM, Dimos Stamatakis >>> <[email protected]>wrote: >>> >>>> Hello! >>>> I have successfully run a pvfs installation on a eucalyptus cloud, but >>>> when I moved to Amazon EC2, I get a very strange error. >>>> When I run a metadata server it says: >>>> >>>> [S 02/20 00:04] PVFS2 Server ready. >>>> >>>> and then it says: >>>> >>>> [E 02/20 00:04] batch_create request got: No space left on device >>>> ....... And this error repeats ...... >>>> >>>> I checked all of my devices and there is plenty of space, so I don't >>>> think there is not enough space left... >>>> Can you explain that? >>>> What is the batch_create function? And where is it trying to write? >>>> >>>> Here is the output of the df -h on the data node: >>>> >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/sda1 9.9G 2.7G 6.8G 29% / >>>> tmpfs 308M 0 308M 0% /lib/init/rw >>>> udev 10M 108K 9.9M 2% /dev >>>> tmpfs 308M 4.0K 308M 1% /dev/shm >>>> >>>> and on the meta data node: >>>> >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/sda1 9.9G 2.0G 7.5G 21% / >>>> tmpfs 308M 0 308M 0% /lib/init/rw >>>> udev 10M 108K 9.9M 2% /dev >>>> tmpfs 308M 4.0K 308M 1% /dev/shm >>>> >>>> Many thanks, >>>> Dimos. >>>> >>> >>> >> >> _______________________________________________ >> Pvfs2-developers mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers >> >> > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20120220/06b36737/attachment.htm > > ------------------------------ > > _______________________________________________ > Pvfs2-developers mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > > > End of Pvfs2-developers Digest, Vol 72, Issue 8 > *********************************************** _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
