Re: [openstack-dev] [swift][swift3][s3] Keep containers unique among a cluster

On Thu, 10 May 2018 20:07:03 +0800 Yuxin Wang wrote: > I'm working on a swift project. Our customer cares about S3 compatibility > very much. I tested our swift cluster with ceph/s3-tests and analyzed the > failed cases. It turns out that lots of the failed cases

Re: [openstack-dev] swift3 Plugin Development

On Fri, 9 Jun 2017 10:37:15 +0530 Niels de Vos wrote: > > > we are looking for S3 plugin with ACLS so that we can integrate gluster > > > with that. > > > > Did you look into porting Ceph RGW on top of Gluster? > > This is one of the longer term options that we have under

Re: [openstack-dev] Swift3 Plugin Development

On Thu, 8 Jun 2017 17:06:02 +0530 Venkata R Edara wrote: > we are looking for S3 plugin with ACLS so that we can integrate gluster > with that. Did you look into porting Ceph RGW on top of Gluster? -- P

Re: [openstack-dev] [all] Embracing new languages in OpenStack

On Wed, 9 Nov 2016 11:14:32 + (GMT) Chris Dent wrote: > The conversations about additional languages in this community have > been one our most alarmingly regressive and patronizing. They seem > to be bred out of fear rather than hope and out of lack of faith in >

Re: [openstack-dev] [all] Embracing new languages in OpenStack

On Mon, 07 Nov 2016 15:53:51 -0800 Joshua Harlow wrote: > Standards though exist for a reason (and ini files are pretty common > across languages and such); though of course oslo.config has some very > tight integration with openstack, the underlying ini concept it >

Re: [Openstack] [Swift] Where can I get the libshss library ?

On Thu, 13 Oct 2016 11:58:58 +0900 Yu Watanabe wrote: > Oct 12 19:44:45 opstack-objstorage1 swift-account-server[27793]: Error: > [swift-hash]: both swift_hash_path_suffix and swift_hash_path_prefix are > m...ft.conf Leaving libshss aside, Swift being unable to read

Re: [Openstack] [Swift] 404 re-reading just created container

On Fri, 3 Jun 2016 16:09:06 +1200 Mark Kirkwood wrote: > This is a Swift 2.7.0.1 system with 2 proxies and 3 storage nodes (each > the latter with 6 devices). What is your replica count? > The proxies are load balanced behind Haproxy (which I'm guessing is >

Re: [openstack-dev] [swift][keystone] Using JSON as future ACL format

On Mon, 6 Jun 2016 13:05:46 -0700 "Thai Q Tran" wrote: > My intention is to spark discussion around this topic with the goal of > moving the Swift community toward accepting the JSON format. If would be productive if you came up with a specific proposal how to retrofit JSON

Re: [Openstack] Using swift-client in C with cURL to access a swift store

On Fri, 20 May 2016 10:24:38 -0700 Clay Gerrard wrote: > > Look at cf_xxx functions here: > > https://git.fedorahosted.org/cgit/iwhd.git/tree/backend.c > > Clone > > git://git.fedorahosted.org/iwhd.git > > > ^ should *also* go on the associated projects list! Naah,

Re: [Openstack] Using swift-client in C with cURL to access a swift store

On Wed, 18 May 2016 11:21:29 -0700 Clay Gerrard wrote: > I haven't heard much about folks using Swift bindings for C - there's no C > bindings listed on the associated projects page [1]. I'm sure just using The so-called "Image Warehouse" of the Aeolus project has a

Re: [Openstack] [swift] Object replication failure counts confusing in 2.7.0

On Wed, 18 May 2016 16:46:05 +1200 Mark Kirkwood wrote: > May 18 04:31:17 markir-dev-ostor002 object-server: object replication > failure 4, detail Traceback (most recent call last):#012 File >

Re: [openstack-dev] [tc] supporting Go

On Fri, 13 May 2016 10:14:02 +0200 Dmitry Tantsur wrote: > [...] If familiarity for Python > developers is an argument here, mastering Cython or making OpenStack run > on PyPy must be much easier for a random Python developer out there to > seriously bump the performance.

Re: [openstack-dev] [tc] supporting Go

On Mon, 9 May 2016 14:17:40 -0500 Edward Leafe wrote: > Whenever I hear claims that Python is “too slow for X”, I wonder > what’s so special about X that makes it so much more demanding than, > say, serving up YouTube. In case of Swift, the biggest issue was the scheduler. As

Re: [openstack-dev] [tc] supporting Go

On Mon, 9 May 2016 09:06:02 -0400 Rayson Ho wrote: > Since the Go toolchain is pretty self-contained, most people just follow > the official instructions to get it installed... by a one-step: > > # tar -C /usr/local -xzf go$VERSION.$OS-$ARCH.tar.gz I'm pretty certain the

Re: [openstack-dev] [tc] Swift api compat. Was: supporting Go

On Wed, 4 May 2016 21:52:49 + "Fox, Kevin M" wrote: > Swift is in a strange place where the api is implemented in a way to > favor one particular vendor backend implementation. Sorry, but I disagree with the above assessement. There is no one particular vendor like that,

Re: [openstack-dev] [tc] supporting Go

On Tue, 3 May 2016 22:37:30 + "Fox, Kevin M" wrote: > RadosGW has been excluded from joining the OpenStack community in part > due to its use of c++. Sounds like sheer lunacy. Nothing like that ever happened, AFAIK. -- Pete

Re: [openstack-dev] [tc] supporting Go

On Tue, 3 May 2016 22:11:06 + "Fox, Kevin M" wrote: > If we let go in, and there are no pluggable middleware, where does > RadosGW and other Swift api compatible implementations then stand? They remain where they are now. > Should we bless c++ too? As I understand it,

Re: [openstack-dev] [tc] supporting Go

On Tue, 3 May 2016 12:16:24 -0400 Rayson Ho wrote: > I like Go! However, Go does not offer binary compatibility between point > releases. For those who install from source it may not be a big issue, but > for commercial distributions that pre-package & pre-compile

Re: [Openstack] Swift accounts and container replication

On Mon, 04 Apr 2016 18:13:17 +0100 Carlos Rodrigues wrote: > I have two regions nodes of swift, geographically dispersed, and i have > storage policies for both regions.  > > How can i do to replicate the accounts and containers between two > regions? Policies do not apply to

Re: [Openstack] swift ringbuilder and disk size/capacity relationship

On Wed, 16 Mar 2016 13:23:31 +1300 Mark Kirkwood wrote: > So integrating swift-recon into regular monitoring/alerting > (collectd/nagios or whatever) is one approach (mind you most folk > already monitor disk usage data... and there is nothing overly special >

Re: [openstack-dev] swift missing X-Timestamp header commit review

On Wed, 20 Jan 2016 13:46:13 +0200 (EET) Mustafa ÇELİK (BİLGEM-BTE) wrote: > commit-1: This one is my patch for the bug. > https://review.openstack.org/#/c/268163/ > I need someone to review my commit-1. > Can somebody help me with code-review? Sure... I am

Re: [openstack-dev] [TripleO] Is Swift a good choice of database for the TripleO API?

On Tue, 22 Dec 2015 08:56:08 -0800 Clint Byrum wrote: > You could create a unique swift container, upload things to that, and > then update a pointer in a well-known location to point at that container > for the new plan only after you've verified it is available. This is a >

Re: [openstack-dev] Getting rid of suds, which is unmaintained, and which we want out of Debian

On Thu, 11 Jun 2015 11:08:55 +0300 Duncan Thomas duncan.tho...@gmail.com wrote: There's only one cinder driver using it (nimble storage), and it seems to be using only very basic features. There are half a dozen suds forks on pipi, or there's pisimplesoap that the debian maintainer recommends.

Re: [Openstack] Using swift with a single replica on software-defined storage

On Wed, 27 May 2015 10:28:53 +0200 Vincenzo Pii vinc@gmail.com wrote: My question is the following: when performance doesn't matter and reliability is taken care of below swift (so swift will always manage to read/write an object as devices will always be consistent and available), are

Re: [Openstack] [Swift] Using unevenly sized disks in a cluster

On Thu, 26 Feb 2015 12:40:32 -0800 Shrinand Javadekar shrin...@maginatics.com wrote: Let's say I start with two disks of 10GB each [...] At a later point, I increase the capacity of the cluster by adding one 500GB disk. [...] How does Swift place data such that it the as unique as possible

Re: [openstack-dev] [swift] Allow hostname for nodes in Ring

On Fri, 10 Oct 2014 04:56:55 + Osanai, Hisashi osanai.hisa...@jp.fujitsu.com wrote: Today the following patch was abandoned and I contacted with the author, so I would like to take it over if nobody else is chafing to take it. Is it OK? https://review.openstack.org/#/c/80421/ If it

Re: [Openstack] [Openstack-operators] [swift] How to encrypt account/container/object data that travels through storage nodes?

On Wed, 17 Sep 2014 15:16:22 -0300 Gui Maluf guimal...@gmail.com wrote: Replicas are copied between storage nodes and swift presume all storage nodes are running in a secure network. Taking any scenario of a Globally Distributed OpenStack Swift Cluster

Re: [Openstack] Implement swift service-list in python-swiftclient

On Tue, 9 Sep 2014 15:36:03 +0530 Ashish Chandra mail.ashishchan...@gmail.com wrote: 1) Do we have plans to include swift service-list in swiftclient ? If yes then I would be filing a blueprint in python-swiftclient to implement the same coz I require it to populate under the Admin - System

Re: [Openstack] [Swift] IP address of Swift nodes : need help

On Tue, 19 Aug 2014 00:12:43 +0530 Jyoti Ranjan jran...@gmail.com wrote: In other words, is it necessary to use static IP for Swift nodes? It is, but you can assign them with DHCP. In ISC DHCP the syntax is host r21s05 { fixed-address r21s05; hardware ethernet 68:9c:70:95:8e:51; }

Re: [openstack-dev] [swift] Use FQDN in Ring files instead of ip

On Wed, 23 Jul 2014 08:54:30 -0700 John Dickinson m...@not.mn wrote: So basically, it's a question of do we add the feature, knowing that most people who use it will in fact be making their lives more difficult, or do we keep it out, knowing that we won't be serving those who actually require

Re: [openstack-dev] About Swift as an object storage gateway, like Cinder in block storage

On Mon, 7 Jul 2014 11:05:40 +0800 童燕群 tyan...@qq.com wrote: The workflow of this middle-ware working with swift may be like this pic: Since you're plugging this into a/c/o nodes, there's no difference between this and Pluggable Back-ends. Note that PBE is already implemented in case of object

Re: [Openstack] [Swift] Running out of ports or fds?

On Tue, 8 Jul 2014 16:26:10 -0700 Shrinand Javadekar shrin...@maginatics.com wrote: I see that these servers do not use a persistent http connection between them. So every blob get/put/delete request will create a new connection, use it and tear it down. In a highly concurrent environment

Re: [openstack-dev] Swift: reason for using xfs on devices

On Wed, 2 Jul 2014 00:16:42 + Osanai, Hisashi osanai.hisa...@jp.fujitsu.com wrote: So I think if performance of swift is more important rather than scalability of it, it is a good idea to use ext4. The real problem is what happens when your drives corrupt the data. Both ext4 and XFS

Re: [Openstack] Swift generating thousands of rsyncs per second on a small cluster

On Tue, 1 Jul 2014 10:50:15 +0100 Diogo Vieira d...@eurotux.com wrote: Can you tell me if this is normal behaviour? If so, how will this scale when I add more objects? Will it keep getting more and more CPU usage? Dunno if it's normal or not, but clusters installed with default parameters do

Re: [Openstack] Swift and Keystone behind NAT Firewall

On Thu, 12 Jun 2014 09:52:59 +0100 Diogo Vieira d...@eurotux.com wrote: Ok, I guess that might work, but I have one problem with that approach. For a service I'm developing I have to know the public URL for an object in the store. For that I use Keystone to find the endpoint of Swift and I

Re: [Openstack] Swift and Keystone behind NAT Firewall

On Wed, 11 Jun 2014 18:52:43 +0100 Diogo Vieira d...@eurotux.com wrote: I have one Proxy Node in the same machine as the Keystone service is as well as a Storage Node. On the other machine I have only a Storage Node. What should be the approach used to make this publicly available? What

Re: [Openstack] swift don't start with ceilometer

On Wed, 04 Jun 2014 17:28:54 +0800 Yugang LIU 88.a...@gmail.com wrote: pkg_resources.VersionConflict: (happybase 0.7 (/usr/lib/python2.7/dist-packages), Requirement.parse('happybase=0.5,!=0.7')) it need happybase 0.5, but my system version is 0.7? You either need to go down on

Re: [OpenStack-Infra] [openstack-dev] Moving swift3 to stackforge (was: Re: Intermittent failures cloning noVNC from github.com/kanaka)

On Fri, 14 Mar 2014 09:03:22 +0100 Chmouel Boudjnah chmo...@enovance.com wrote: fujita (the maint of swift3 in CC of this email) has commented that he's been working on it. I think we should've not kicked it out. Maybe just re-fold it back into Swift? -- Pete

Re: [openstack-dev] Moving swift3 to stackforge (was: Re: [OpenStack-Infra] Intermittent failures cloning noVNC from github.com/kanaka)

On Fri, 14 Mar 2014 09:03:22 +0100 Chmouel Boudjnah chmo...@enovance.com wrote: fujita (the maint of swift3 in CC of this email) has commented that he's been working on it. I think we should've not kicked it out. Maybe just re-fold it back into Swift? -- Pete

[openstack-dev] Guru Meditation output seems useless

Dear Solly: I cobbled together a working prototype of Guru Meditation for Swift just to see how it worked. I did not use Oslo classes, but used the code from Dan's prototype and from your Nova review. Here's the Gerrit link: https://review.openstack.org/70513 Looking at the collected

Re: [Openstack] [SWIFT] What happens when the shard count on available disk is less than the total shard count?

On Fri, 28 Feb 2014 09:10:06 -0800 Stephen Wood smwo...@gmail.com wrote: However I realize that the shard count is completely different now. What is a shard count? Do you have a document that uses such terminology? I originally used a partition value of 15 but this now seems much to high for

Re: [Openstack] Capturing performance penalities Physical versus VM [Swift]

On Fri, 21 Feb 2014 13:47:31 -0800 Adam Lawson alaw...@aqorn.com wrote: So, are there benchmarks already published somewhere that demonstrate how/why a virtual infrastructure is such a bad idea with Swift? We all know Swift is designed to be a storage provider - not a storage consumer and

Re: [openstack-dev] Glance v1 and v2

On Tue, 18 Feb 2014 10:57:03 +0100 Joe Hakim Rahme joe.hakim.ra...@enovance.com wrote: Again, I have just spent a couple of days playing with it on a devstack. I'm by no means a reference on the subject of the API v2. I hope this will help you get a better idea of where it stands today.

[openstack-dev] Glance v1 and v2

Hello: does anyone happen to know, or have a detailed write-up, on the differences between so-called Glance v1 and Glance v2? In particular do we still need Glance Registry in Havana, or do we not? The best answer so far was to run the registry anyway, just in case, which does not feel entirely

[openstack-dev] [Glance] delayed delete and user credentials

Hi, guys: I looked briefly at a bug/fix, which looks exceedingly strange to me: https://review.openstack.org/59689 As much as I can tell, the problem (lp:1238604) is that pending delete fails because by the time the delete actually occurs, Glance API does not have proper permissions to talk to

Re: [Openstack] Building Swift cluster wiuth Ubuntu VM's

On Sun, 2 Feb 2014 20:08:02 -0800 Adam Lawson alaw...@aqorn.com wrote: (http://docs.openstack.org/developer/swift/howto_installmultinode.html) I'm running into issues where the instructions start to get really hazy/unclear after step #4 in Configure Proxy Server. Woops, there was a bug in

Re: [Openstack] [SWIFT] Memcached timeouts?

On Fri, 24 Jan 2014 12:51:45 -0800 Stephen Wood smwo...@gmail.com wrote: My memcached settings are basically stock. Any idea what could be causing these errors? I don't have a good idea, but I would start by backing up these commits:

Re: [Openstack] [Swift] Proxy server bottleneck

On Fri, 10 Jan 2014 15:25:02 -0800 Shrinand Javadekar shrin...@maginatics.com wrote: I see that the proxy-server already has a workers config option. However, looks like that is the # of threads in one proxy-server process. Not so. Workers are separate Linux processes. Look at os.fork() in

Re: [Openstack] [SWIFT] Unable to delete containers?

On Thu, 5 Dec 2013 10:06:14 -0800 Stephen Wood smwo...@gmail.com wrote: $ swift list ssbench_46 $ swift delete ssbench_46 Container 'ssbench_46' not found $ swift list ssbench_46 Container 'ssbench_46' not found How does one go about actually removing these

Re: [Openstack] [Swift] Swift on RHEL

On Thu, 5 Dec 2013 21:19:49 + Kotwani, Mukul mukul.g.kotw...@hp.com wrote: Has anyone used and/or deployed RHEL (5.8) with Swift? I was also looking for a Supported platforms for Swift, and I could not find it. I don't think an RDO for RHEL 5.x ever existed. First packages were built a

Re: [Openstack] [Swift] Changing local logging destinations?

On Wed, 27 Nov 2013 15:33:52 -0800 John Dickinson m...@not.mn wrote: Can someone give me advice on how to redirect logs from /var/log/syslog to /var/log/swift/{object|account|container}.logs? While a Swift-All-In-One certainly isn't something your should run in production, the SAIO

Re: [Openstack] [Swift] Container DB update after object PUT

On Thu, 28 Nov 2013 05:24:35 + Shao, Minglong minglong.s...@netapp.com wrote: 1. Proxy server sends three requests to three object servers. 2. One object server writes the object successfully, sends an update to the container DB and an “OK” reply to the proxy server. But the other

Re: [Openstack] WORM support

On Wed, 13 Nov 2013 16:47:29 + Perez, Daniel - ES daniel.pe...@exelisinc.com wrote: I am wondering if I can circumvent the use of tombstone files? Not without writing some code, but you can write your own Pluggable Back-end like Gluster does. In that case you're free to implement any delete

[openstack-dev] Swift account auditor duplicated code

Hi, Guys: Here's a weird piece of duplicated call to account_audit() in swift/account/auditor.py: for path, device, partition in all_locs: self.account_audit(path) if time.time() - reported = 3600: # once an hour self.logger.info(_('Since

Re: [openstack-dev] [Swift] question on Application class concurrency; paste.app_factory mechanism

On Wed, 24 Jul 2013 02:31:45 + Luse, Paul E paul.e.l...@intel.com wrote: I was thinking that each connection would get its own instance thus it would be sage to store connection-transient information there but I was surprised by my quick test. Yeah, you have it tracked in __call__,

Re: [openstack-dev] [Swift] erasure codes, digging deeper

On Thu, 18 Jul 2013 12:31:02 -0500 Chuck Thier cth...@gmail.com wrote: I'm with Chmouel though. It seems to me that EC policy should be chosen by the provider and not the client. For public storage clouds, I don't think you can make the assumption that all users/clients will understand the