Re: [Openstack] [SWIFT] raising network traffic o the storage node
On Mon, 8 Jul 2013 13:50:38 + Klaus Schürmann wrote: > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluster. > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png > > Can someone explain such behavior? At a guess, raising number of objects makes number of partitions to increase, and that increases the replication traffic, specifically the number of sent MD5s for partitions. It would be interesting to corellate the number of objects and number of non-empty and empty partitions with the amounts of traffic. If the increasing transfer of hashes is the reason, you could also verify by graphing the traffic to port 873 separately. Swift never replicates object bodies through its own HTTP interface, so this splits control traffic from data traffic for you. Data traffic should be driven by customer and node failures, not consistency checking. Be prepared to split up storage nodes, however. Even if there's no bug with replication, its aggregate traffic increases with the increases in object counts. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] raising network traffic o the storage node
Hi Klaus, Would you please grep your swift log for object-replicator? thx +Hugo Kuo+ h...@swiftstack.com tonyt...@gmail.com +886 935004793 2013/7/8 Klaus Schürmann > On Monday some more mailboxes store their Mails in the objectstorage. > But that only increased the raising. > > Traffic Storagenode: http://www.schuermann.net/temp/storagenode2.png > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver2.png > > > -Ursprüngliche Nachricht- > Von: Peter Portante [mailto:peter.a.porta...@gmail.com] > Gesendet: Montag, 8. Juli 2013 16:04 > An: Klaus Schürmann > Cc: openstack@lists.launchpad.net > Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node > > Can you zoom in past the spike, most recent 2 or three weeks and see > how it looks? > > My guess is that the proxy traffic is also rising. > > On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann > wrote: > > Hi, > > > > I use a swift storage as a mail-store. Now I have about 1.000.000 > objects > > stored in the cluster. > > > > > > > > I'm wondering about the raising network traffic on my storage nodes. The > > traffic from the proxy-server has a normal characteristic. > > > > > > > > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > > > > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png > > > > > > > > Can someone explain such behavior? > > > > > > > > Thanks > > > > Klaus > > > > > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp > > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] raising network traffic o the storage node
On Monday some more mailboxes store their Mails in the objectstorage. But that only increased the raising. Traffic Storagenode: http://www.schuermann.net/temp/storagenode2.png Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver2.png -Ursprüngliche Nachricht- Von: Peter Portante [mailto:peter.a.porta...@gmail.com] Gesendet: Montag, 8. Juli 2013 16:04 An: Klaus Schürmann Cc: openstack@lists.launchpad.net Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node Can you zoom in past the spike, most recent 2 or three weeks and see how it looks? My guess is that the proxy traffic is also rising. On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann wrote: > Hi, > > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluster. > > > > I'm wondering about the raising network traffic on my storage nodes. The > traffic from the proxy-server has a normal characteristic. > > > > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png > > > > Can someone explain such behavior? > > > > Thanks > > Klaus > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] raising network traffic o the storage node
Can you zoom in past the spike, most recent 2 or three weeks and see how it looks? My guess is that the proxy traffic is also rising. On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann wrote: > Hi, > > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluster. > > > > I’m wondering about the raising network traffic on my storage nodes. The > traffic from the proxy-server has a normal characteristic. > > > > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png > > > > Can someone explain such behavior? > > > > Thanks > > Klaus > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp