On 09/17/2013 11:13 AM, Paul Robert Marino wrote:
Luis
well thats intresting because it was my impression that Gluster UFO
3.4 was based on the Grizzly version of Swift.
[LP] Sorry, the gluster-ufo RPM is Essex only.
Also I was previously unaware of this new rpm which doesnt seem to be
in a repo any where.
[LP] gluster-swift project RPMs have been submitted to Fedora and are
currently being reviewed.
also there is a line in this new howto that is extreamly unclear
"
/usr/bin/gluster-swift-gen-builders test
"
in place of "test" what should go there is it the tenant ID string,
the tenant name, or just a generic volume you can name whatever you
want?
in other words how should the Gluster volumes be named?
[LP] We will clarify that in the quick start guide. Thank you for
pointing it out. While we update the community site, please refer to
the documentation available here http://goo.gl/bQFI8o for a usage guide.
As for the tool, the format is:
gluster-swift-gen-buildes [VOLUME] [VOLUME...]
Where VOLUME is the name of the GlusterFS volume to use for object
storage. For example
if the following two GlusterFS volumes, volume1 and volume2, need to be
accessed over Swift,
then you can type the following:
# gluster-swift-gen-builders volume1 volume2
For more information please read: http://goo.gl/gd8LkW
Let us know if you have any more questions or comments.
- Luis
On Tue, Sep 17, 2013 at 10:10 AM, Luis Pabon <[email protected]> wrote:
First thing I can see is that you have Essex based gluster-ufo-* which has
been replaced by the gluster-swift project. We are currently in progress of
replacing the gluster-ufo-* with RPMs from the gluster-swift project in
Fedora.
Please checkout the following quickstart guide which show how to download
the Grizzly version of gluster-swift:
https://github.com/gluster/gluster-swift/blob/master/doc/markdown/quick_start_guide.md
.
For more information please visit: https://launchpad.net/gluster-swift
- Luis
On 09/16/2013 05:02 PM, Paul Robert Marino wrote:
Sorry for the delay on reporting the details. I got temporarily pulled
off the project and dedicated to a different project which was
considered higher priority by my employer. I'm just getting back to
doing my normal work today.
first here are the rpms I have installed
"
rpm -qa |grep -P -i '(gluster|swift)'
glusterfs-libs-3.4.0-8.el6.x86_64
glusterfs-server-3.4.0-8.el6.x86_64
openstack-swift-plugin-swift3-1.0.0-0.20120711git.el6.noarch
openstack-swift-proxy-1.8.0-2.el6.noarch
glusterfs-3.4.0-8.el6.x86_64
glusterfs-cli-3.4.0-8.el6.x86_64
glusterfs-geo-replication-3.4.0-8.el6.x86_64
glusterfs-api-3.4.0-8.el6.x86_64
openstack-swift-1.8.0-2.el6.noarch
openstack-swift-container-1.8.0-2.el6.noarch
openstack-swift-object-1.8.0-2.el6.noarch
glusterfs-fuse-3.4.0-8.el6.x86_64
glusterfs-rdma-3.4.0-8.el6.x86_64
openstack-swift-account-1.8.0-2.el6.noarch
glusterfs-ufo-3.4.0-8.el6.noarch
glusterfs-vim-3.2.7-1.el6.x86_64
python-swiftclient-1.4.0-1.el6.noarch
here are some key config files note I've changed the passwords I'm
using and hostnames
"
cat /etc/swift/account-server.conf
[DEFAULT]
mount_check = true
bind_port = 6012
user = root
log_facility = LOG_LOCAL2
devices = /swift/tenants/
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:gluster_swift_ufo#account
log_name = account-server
log_level = DEBUG
log_requests = true
[account-replicator]
vm_test_mode = yes
[account-auditor]
[account-reaper]
"
"
cat /etc/swift/container-server.conf
[DEFAULT]
devices = /swift/tenants/
mount_check = true
bind_port = 6011
user = root
log_facility = LOG_LOCAL2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:gluster_swift_ufo#container
[container-replicator]
vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync]
"
"
cat /etc/swift/object-server.conf
[DEFAULT]
mount_check = true
bind_port = 6010
user = root
log_facility = LOG_LOCAL2
devices = /swift/tenants/
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:gluster_swift_ufo#object
[object-replicator]
vm_test_mode = yes
[object-updater]
[object-auditor]
"
"
cat /etc/swift/proxy-server.conf
[DEFAULT]
bind_port = 8080
user = root
log_facility = LOG_LOCAL1
log_name = swift
log_level = DEBUG
log_headers = True
[pipeline:main]
pipeline = healthcheck cache authtoken keystone proxy-server
[app:proxy-server]
use = egg:gluster_swift_ufo#proxy
allow_account_management = true
account_autocreate = true
[filter:tempauth]
use = egg:swift#tempauth
# Here you need to add users explicitly. See the OpenStack Swift Deployment
# Guide for more information. The user and user64 directives take the
# following form:
# user_<account>_<username> = <key> [group] [group] [...] [storage_url]
# user64_<account_b64>_<username_b64> = <key> [group] [group]
[...] [storage_url]
# Where you use user64 for accounts and/or usernames that include
underscores.
#
# NOTE (and WARNING): The account name must match the device name specified
# when generating the account, container, and object build rings.
#
# E.g.
# user_ufo0_admin = abc123 .admin
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
[filter:keystone]
use = egg:swift#keystoneauth
#paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = Member,admin,swiftoperator
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
auth_host = keystone01.vip.my.net
auth_port = 35357
auth_protocol = http
admin_user = swift
admin_password = PASSWORD
admin_tenant_name = service
signing_dir = /var/cache/swift
service_port = 5000
service_host = keystone01.vip.my.net
[filter:swiftauth]
use = egg:keystone#swiftauth
auth_host = keystone01.vip.my.net
auth_port = 35357
auth_protocol = http
keystone_url = https://keystone01.vip.my.net:5000/v2.0
admin_user = swift
admin_password = PASSWORD
admin_tenant_name = service
signing_dir = /var/cache/swift
keystone_swift_operator_roles = Member,admin,swiftoperator
keystone_tenant_user_admin = true
[filter:catch_errors]
use = egg:swift#catch_errors
"
"
cat /etc/swift/swift.conf
[DEFAULT]
[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = gluster
#3d60c9458bb77abe
# The swift-constraints section sets the basic constraints on data
# saved in the swift cluster.
[swift-constraints]
# max_file_size is the largest "normal" object that can be saved in
# the cluster. This is also the limit on the size of each segment of
# a "large" object when using the large object manifest support.
# This value is set in bytes. Setting it to lower than 1MiB will cause
# some tests to fail. It is STRONGLY recommended to leave this value at
# the default (5 * 2**30 + 2).
# FIXME: Really? Gluster can handle a 2^64 sized file? And can the fronting
# web service handle such a size? I think with UFO, we need to keep with the
# default size from Swift and encourage users to research what size their
web
# services infrastructure can handle.
max_file_size = 18446744073709551616
# max_meta_name_length is the max number of bytes in the utf8 encoding
# of the name portion of a metadata header.
#max_meta_name_length = 128
# max_meta_value_length is the max number of bytes in the utf8 encoding
# of a metadata value
#max_meta_value_length = 256
# max_meta_count is the max number of metadata keys that can be stored
# on a single account, container, or object
#max_meta_count = 90
# max_meta_overall_size is the max number of bytes in the utf8 encoding
# of the metadata (keys + values)
#max_meta_overall_size = 4096
# max_object_name_length is the max number of bytes in the utf8 encoding of
an
# object name: Gluster FS can handle much longer file names, but the length
# between the slashes of the URL is handled below. Remember that most web
# clients can't handle anything greater than 2048, and those that do are
# rather clumsy.
max_object_name_length = 2048
# max_object_name_component_length (GlusterFS) is the max number of bytes in
# the utf8 encoding of an object name component (the part between the
# slashes); this is a limit imposed by the underlying file system (for XFS
it
# is 255 bytes).
max_object_name_component_length = 255
# container_listing_limit is the default (and max) number of items
# returned for a container listing request
#container_listing_limit = 10000
# account_listing_limit is the default (and max) number of items returned
# for an account listing request
#account_listing_limit = 10000
# max_account_name_length is the max number of bytes in the utf8 encoding of
# an account name: Gluster FS Filename limit (XFS limit?), must be the same
# size as max_object_name_component_length above.
max_account_name_length = 255
# max_container_name_length is the max number of bytes in the utf8 encoding
# of a container name: Gluster FS Filename limit (XFS limit?), must be the
same
# size as max_object_name_component_length above.
max_container_name_length = 255
"
The volumes
"
gluster volume list
cindervol
unified-storage-vol
a07d2f39117c4e5abdeba722cf245828
bd74a005f08541b9989e392a689be2fc
f6da0a8151ff43b7be10d961a20c94d6
"
if I run the command
"
gluster-swift-gen-builders unified-storage-vol
a07d2f39117c4e5abdeba722cf245828 bd74a005f08541b9989e392a689be2fc
f6da0a8151ff43b7be10d961a20c94d6
"
because of a change in the script in this version as compaired to the
version I got from
http://repos.fedorapeople.org/repos/kkeithle/glusterfs/ the
gluster-swift-gen-builders script only takes the first option and
ignores the rest.
other than the location of the config files none of the changes Ive
made are functionally different than the ones mentioned in
http://www.gluster.org/2012/09/howto-using-ufo-swift-a-quick-and-dirty-setup-guide/
The result is that the first volume named "unified-storage-vol" winds
up being used for every thing regardless of the tenant, and users and
see and manage each others objects regardless of what tenant they are
members of.
through the swift command or via horizon.
In a way this is a good thing for me it simplifies thing significantly
and would be fine if it just created a directory for each tenant and
only allow the user to access the individual directories, not the
whole gluster volume.
by the way seeing every thing includes the service tenants data so
unprivileged users can delete glance images without being a member of
the service group.
On Mon, Sep 2, 2013 at 9:58 PM, Paul Robert Marino <[email protected]>
wrote:
Well I'll give you the full details in the morning but simply I used the
stock cluster ring builder script that came with the 3.4 rpms and the old
version from 3.3 took the list of volumes and would add all of them the
version with 3.4 only takes the first one.
Well I ran the script expecting the same behavior but instead they all used
the first volume in the list.
Now I knew from the docs I read that the per tenant directories in a single
volume were one possible plan for 3.4 to deal with the scalding issue with a
large number of tenants, so when I saw the difference in the script and that
it worked I just assumed that this was done and I missed something.
-- Sent from my HP Pre3
________________________________
On Sep 2, 2013 20:55, Ramana Raja <[email protected]> wrote:
Hi Paul,
Currently, gluster-swift doesn't support the feature of multiple
accounts/tenants accessing the same volume. Each tenant still needs his own
gluster volume. So I'm wondering how you were able to observe the reported
behaviour.
How did you prepare the ringfiles for the different tenants, which use the
same gluster volume? Did you change the configuration of the servers? Also,
how did you access the files that you mention? It'd be helpful if you could
share the commands you used to perform these actions.
Thanks,
Ram
----- Original Message -----
From: "Vijay Bellur" <[email protected]>
To: "Paul Robert Marino" <[email protected]>
Cc: [email protected], "Luis Pabon" <[email protected]>, "Ramana Raja"
<[email protected]>, "Chetan Risbud" <[email protected]>
Sent: Monday, September 2, 2013 4:17:51 PM
Subject: Re: [rhos-list] Gluster UFO 3.4 swift Multi tenant question
On 09/02/2013 01:39 AM, Paul Robert Marino wrote:
I have Gluster UFO installed as a back end for swift from here
http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.0/RHEL/epel-6/
with RDO 3
Its working well except for one thing. All of the tenants are seeing
one Gluster volume which is some what nice, especially when compared
to the old 3.3 behavior of creating one volume per tenant named after
the tenant ID number.
The problem is I expected to see is sub directory created under the
volume root for each tenant but instead what in seeing is that all of
the tenants can see the root of the Gluster volume. The result is that
all of the tenants can access each others files and even delete them.
even scarier is that the tennants can see and delete each others
glance images and snapshots.
Can any one suggest options to look at or documents to read to try to
figure out how to modify the behavior?
Adding gluster swift developers who might be able to help.
-Vijay
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
