Re: [Openstack] Can any user add or delete OpenStack Swift middleware?

[Kota TSUYUZAKI]

Hi Quao,

See inline responses below, please.

(2018/09/27 0:58), Qiao Kang wrote:
> Kota,
> 
> Sorry for the late response, see more below:
> 
> On Fri, Sep 21, 2018 at 2:59 AM Kota TSUYUZAKI
>  wrote:
>>
>> Hi Qiao,
>>
>>> Thanks! I'm interested and would like to join, as well as contribute!
>>>
>>
>> One example, that is how the multi-READ works, is around [1], the storlets 
>> middleware can make a subrequest against to the backend Swift then,
>> attach the request input to the application in the Docker container by 
>> passing the file descriptor to be readable[2][3][4]*.
>> After all of the prepartion for the invocation, the input descriptors will 
>> be readable in the storlet app as the InputFile.
>>
>> * At first, the runtime prepares the extra source stub at [2], then creates 
>> a set of pipes for each sources to be communicated with the app
>> inside the docker daemon[3], then, the runtime module reads the extra data 
>> from Swift GET and flushes all buffers into the descriptor [4].
>>
>> 1: 
>> https://github.com/openstack/storlets/blob/master/storlets/swift_middleware/handlers/proxy.py#L294-L305
>> 2: 
>> https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L571-L581
>> 3: 
>> https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L665-L666
>> 4: 
>> https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L833-L840
>>
>> Following the mechanism, IMO, what we can do to enable multi-out is
>>
>> - add the capability to create some PUT subrequest at swift_middleware 
>> module (define the new API header too)
>> - create the extra communication write-able fds in the storlets runtime 
>> (perhaps, storlets daemon is also needed to be changed)
>> - pass all data from the write-able fds as to the sub PUT request input
>>
>>
>> If you have any nice idea rather than me, it's always welcome tho :)
> 
> I think your approach is clear and straightforward. One quick question:
>> - create the extra communication write-able fds in the storlets runtime 
>> (perhaps, storlets daemon is also needed to be changed)
> So the Storlet app will write to those fds? Are these fds temporary
> and need to be destroyed after PUT requests in step 3?

Good Question! I don't think we need new code to destory the descriptors.
Note that we have a couple of file descriptors that should be closed 
successfuly.

One is inside the container, that will be closed [1] at the storlet daemon 
agent that invoke the storlet app.
The other is the descripter to be read from at the swift middleware layer. That 
will be passed to a StorletResponse
object, then the reponse object can be closed by wsgi middleware[2][3]. 
Basically wsgi middleware
has the reponsibility to close the application response iterator so we expect 
the iterator will be closed around wsgi middleware
pipeline then it propergates to the read fd.

If you find any fd leak that is not intentioned, please feel free to report it 
as a bug, we had already fought to the leaking descriptors :)

1: 
https://github.com/openstack/storlets/blob/master/storlets/agent/daemon/server.py#L211-L212
2: 
https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L845
3: 
https://github.com/openstack/storlets/blob/master/storlets/gateway/common/stob.py#L119-L123


> 
>>
>>> Another potential use case: imagine I want to compress objects upon
>>> PUTs using two different algorithms X and Y, and use the future
>>> 'multi-write' feature to store three objects upon any single PUT
>>> (original copy, X-compressed copy and Y-compressed copy). I can
>>> install two Storlets which implement X and Y respectively. However,
>>> seems Storlets engine can only invoke one per PUT, so this is still
>>> not feasible. Is that correct?
>>>
>>
>> It sounds interesting. As you know, yes, one Storlet application can be 
>> invoked per one PUT.
>> On the other hand, Storlets has been capable to run several applications as 
>> you want.
>> One idea using the capability, if you develop an application like:
>>
>> - Storlet app invokes several multi threads with their output descriptor
>> - Storlet app reads the input stream, then pushes the data into the threads
>> - Each thread performs as you want (one does as X compression, the other 
>> does as Y compression)
>>   then, writes its own result to the output descriptor
>>
>> It might work for your use case.
> 
> Sounds great, I guess it should work as well.
> 
> I'm a

Re: [Openstack] Can any user add or delete OpenStack Swift middleware?

[Kota TSUYUZAKI]

Hi Qiao,

> Thanks! I'm interested and would like to join, as well as contribute!
>

One example, that is how the multi-READ works, is around [1], the storlets 
middleware can make a subrequest against to the backend Swift then,
attach the request input to the application in the Docker container by passing 
the file descriptor to be readable[2][3][4]*.
After all of the prepartion for the invocation, the input descriptors will be 
readable in the storlet app as the InputFile.

* At first, the runtime prepares the extra source stub at [2], then creates a 
set of pipes for each sources to be communicated with the app
inside the docker daemon[3], then, the runtime module reads the extra data from 
Swift GET and flushes all buffers into the descriptor [4].

1: 
https://github.com/openstack/storlets/blob/master/storlets/swift_middleware/handlers/proxy.py#L294-L305
2: 
https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L571-L581
3: 
https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L665-L666
4: 
https://github.com/openstack/storlets/blob/master/storlets/gateway/gateways/docker/runtime.py#L833-L840

Following the mechanism, IMO, what we can do to enable multi-out is

- add the capability to create some PUT subrequest at swift_middleware module 
(define the new API header too)
- create the extra communication write-able fds in the storlets runtime 
(perhaps, storlets daemon is also needed to be changed)
- pass all data from the write-able fds as to the sub PUT request input


If you have any nice idea rather than me, it's always welcome tho :)

> Another potential use case: imagine I want to compress objects upon
> PUTs using two different algorithms X and Y, and use the future
> 'multi-write' feature to store three objects upon any single PUT
> (original copy, X-compressed copy and Y-compressed copy). I can
> install two Storlets which implement X and Y respectively. However,
> seems Storlets engine can only invoke one per PUT, so this is still
> not feasible. Is that correct?
>

It sounds interesting. As you know, yes, one Storlet application can be invoked 
per one PUT.
On the other hand, Storlets has been capable to run several applications as you 
want.
One idea using the capability, if you develop an application like:

- Storlet app invokes several multi threads with their output descriptor
- Storlet app reads the input stream, then pushes the data into the threads
- Each thread performs as you want (one does as X compression, the other does 
as Y compression)
  then, writes its own result to the output descriptor

It might work for your use case.

Thanks,
Kota


(2018/09/19 5:52), Qiao Kang wrote:
> Dear Kota,
> 
> On Mon, Sep 17, 2018 at 11:43 PM Kota TSUYUZAKI
>  wrote:
>>
>> Hi Quio,
>>
>>> I know Storlets can provide user-defined computation functionalities,
>>> but I guess some capabilities can only be achieved using middleware.
>>> For example, a user may want such a feature: upon each PUT request, it
>>> creates a compressed copy of the object and stores both the original
>>> copy and compressed copy. It's feasible using middlware but I don't
>>> think Storlets provide such capability.
>>
>> Interesting, exactly currently it's not supported to write to multi objects 
>> for a PUT request but as well as other middlewares we could adopt the 
>> feasibility into Storlets if you prefer.
>> Right now, the multi read (i.e. GET from multi sources) is only available 
>> and I think we would be able to expand the logic to PUT requests too. IIRC, 
>> in those days, we had discussion on sort of the
>> multi-out use cases and I'm sure the data structure inside Storlets are 
>> designed to be capable to that expantion. At that time, we called them "Tee" 
>> application on Storlets, I could not find the
>> historical discussion logs about how to implement tho, sorry. I believe that 
>> would be an use case for storlets if you prefer the user-defined application 
>> flexibilities rather than operator defined
>> Swift middleware.
>>
>> The example of multi-read (GET from multi sources) are here:
>> https://github.com/openstack/storlets/blob/master/tests/functional/python/test_multiinput_storlet.py
>>
>> And if you like to try to write multi write, please join us, I'm happy to 
>> help you anytime.
>>
> 
> Thanks! I'm interested and would like to join, as well as contribute!
> 
> Another potential use case: imagine I want to compress objects upon
> PUTs using two different algorithms X and Y, and use the future
> 'multi-write' feature to store three objects upon any single PUT
> (original copy, X-compressed copy and Y-compressed copy). I can
> install two Storl

Re: [Openstack] Can any user add or delete OpenStack Swift middleware?

[Kota TSUYUZAKI]

Hi Quio,

> I know Storlets can provide user-defined computation functionalities,
> but I guess some capabilities can only be achieved using middleware.
> For example, a user may want such a feature: upon each PUT request, it
> creates a compressed copy of the object and stores both the original
> copy and compressed copy. It's feasible using middlware but I don't
> think Storlets provide such capability.

Interesting, exactly currently it's not supported to write to multi objects for 
a PUT request but as well as other middlewares we could adopt the feasibility 
into Storlets if you prefer.
Right now, the multi read (i.e. GET from multi sources) is only available and I 
think we would be able to expand the logic to PUT requests too. IIRC, in those 
days, we had discussion on sort of the
multi-out use cases and I'm sure the data structure inside Storlets are 
designed to be capable to that expantion. At that time, we called them "Tee" 
application on Storlets, I could not find the
historical discussion logs about how to implement tho, sorry. I believe that 
would be an use case for storlets if you prefer the user-defined application 
flexibilities rather than operator defined
Swift middleware.

The example of multi-read (GET from multi sources) are here:
https://github.com/openstack/storlets/blob/master/tests/functional/python/test_multiinput_storlet.py

And if you like to try to write multi write, please join us, I'm happy to help 
you anytime.


> Another example is that a user may want to install a Swift3-like
> middleware to provide APIs to a 3rd party, but she doesn't want other
> users to see this middleware.
>

If the definition can be made by operators, perhaps one possible solution that 
preparing different proxy-server endpoint for different users is available. 
i.e. an user uses no-s3api available proxy,
then the others use a different proxy-server endpoint that has the s3api in the 
pipeline.

Or, it sounds like kinda defaulter middleware[1], I don't think it has the 
scope turning on/off the middlewares for now.

1: https://review.openstack.org/#/c/342857/

Best,
Kota

(2018/09/18 11:34), Qiao Kang wrote:
> Kota,
> 
> Thanks for your reply, very helpful!
> 
> I know Storlets can provide user-defined computation functionalities,
> but I guess some capabilities can only be achieved using middleware.
> For example, a user may want such a feature: upon each PUT request, it
> creates a compressed copy of the object and stores both the original
> copy and compressed copy. It's feasible using middlware but I don't
> think Storlets provide such capability.
> 
> Another example is that a user may want to install a Swift3-like
> middleware to provide APIs to a 3rd party, but she doesn't want other
> users to see this middleware.
> 
> Regards,
> Qiao
> 
> On Mon, Sep 17, 2018 at 9:19 PM Kota TSUYUZAKI
>  wrote:
>>
>> With Storlets, users will be able to create their own applications that are 
>> able to run like as a Swift middeleware. The application (currently Python 
>> and Java are supported as the language but the
>> apps can calls any binaries in the workspace) can be uploaded as a Swift 
>> object, then, users can invoke them with just an extra header that specifies 
>> your apps.
>>
>> To fit your own use case, we may have to consider to invole or to integrate 
>> the system for you but I believe Storlets could be a choice for you.
>>
>> In detail, Storlets documantation is around there,
>>
>> Top Level Index: https://docs.openstack.org/storlets/latest/index.html
>> System Overview: 
>> https://docs.openstack.org/storlets/latest/storlet_engine_overview.html
>> APIs: https://docs.openstack.org/storlets/latest/api/overview_api.html
>>
>> Thanks,
>>
>> Kota
>>
>> (2018/09/17 8:59), John Dickinson wrote:
>>> You may be interested in Storlets. It's another OpenStack project, 
>>> maintained by a Swift core reviewer, that provides this sort of 
>>> user-defined middleware functionality.
>>>
>>> You can also ask about it in #openstack-swift
>>>
>>> --John
>>>
>>>
>>>
>>> On 16 Sep 2018, at 9:25, Qiao Kang wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm wondering whether Swift allows any user (not the administrator) to
>>>> specify which middleware that she/he wants his data object to go throught.
>>>> For instance, Alice wants to install a middleware but doesn't want Bob to
>>>> use it, where Alice and Bob are two accounts in a single Swift cluster.
>>>>
>>>> Or maybe all middlewares are pre-installed globally and cannot be
>>>> customized on a per-account basis?
&

Re: [Openstack] Can any user add or delete OpenStack Swift middleware?

[Kota TSUYUZAKI]

With Storlets, users will be able to create their own applications that are 
able to run like as a Swift middeleware. The application (currently Python and 
Java are supported as the language but the
apps can calls any binaries in the workspace) can be uploaded as a Swift 
object, then, users can invoke them with just an extra header that specifies 
your apps.

To fit your own use case, we may have to consider to invole or to integrate the 
system for you but I believe Storlets could be a choice for you.

In detail, Storlets documantation is around there,

Top Level Index: https://docs.openstack.org/storlets/latest/index.html
System Overview: 
https://docs.openstack.org/storlets/latest/storlet_engine_overview.html
APIs: https://docs.openstack.org/storlets/latest/api/overview_api.html

Thanks,

Kota

(2018/09/17 8:59), John Dickinson wrote:
> You may be interested in Storlets. It's another OpenStack project, maintained 
> by a Swift core reviewer, that provides this sort of user-defined middleware 
> functionality.
> 
> You can also ask about it in #openstack-swift
> 
> --John
> 
> 
> 
> On 16 Sep 2018, at 9:25, Qiao Kang wrote:
> 
>> Hi,
>>
>> I'm wondering whether Swift allows any user (not the administrator) to
>> specify which middleware that she/he wants his data object to go throught.
>> For instance, Alice wants to install a middleware but doesn't want Bob to
>> use it, where Alice and Bob are two accounts in a single Swift cluster.
>>
>> Or maybe all middlewares are pre-installed globally and cannot be
>> customized on a per-account basis?
>>
>> Thanks,
>> Qiao
>> ___
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack@lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  
NTT Software Innovation Center
Distributed Computing Technology Project
Phone  0422-59-2837
Fax0422-59-2965
---


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Struggling to get the s3 api interface to work with swift.

[Kota TSUYUZAKI]

Hi Shyam,


> s3curl: StringToSign='PUT\n\n\nFri, 01 Jun 2018 05:47:53
> +\n/s3server/testbucket'

It looks like your StringToSign doesn't include correct info for your 
request.(e.g. resource should be only "/testbucket")
See 
https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html 
for more info.

Your request should be:

> * Connected to s3server (20.20.20.229) port 8080 (#0)
>> PUT /testbucket HTTP/1.1
>> Host: s3server:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Fri, 01 Jun 2018 05:47:53 +
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=
>> Content-Length: 0

then, the string to sign should include the info corretly.

Looking at s3curl code, it looks to fall into cname case?

https://github.com/scality/S3/blob/master/tests/functional/s3curl/s3curl.pl#L311-L328

Once I played with s3curl the request with ordinary endpoint signing case 
worked correctly.
Again, I'm not an expert of s3curl so no idea why your setting went to the 
cname case tho.

Best,
Kota

(2018/06/01 14:49), Shyam Prasad N wrote:
> Hi Kota,
> 
> Thanks for the response.
> When I specify the URL as http://s3server:8080/testbucket, it throws back a
> SignatureDoesNotMatch error.
> 
> eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
> --createBucket -- http://s3server:8080/testbucket
> s3curl: Found the url: host=s3server; port=8080; uri=/testbucket; query=;
> s3curl: cname endpoint signing case
> s3curl: StringToSign='PUT\n\n\nFri, 01 Jun 2018 05:47:53
> +\n/s3server/testbucket'
> s3curl: exec curl -v -H 'Date: Fri, 01 Jun 2018 05:47:53 +' -H
> 'Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=' -L -H
> 'content-type: ' --data-binary  -X PUT http://s3server:8080/testbucket
> *   Trying 20.20.20.229...
> * TCP_NODELAY set
> * Connected to s3server (20.20.20.229) port 8080 (#0)
>> PUT /testbucket HTTP/1.1
>> Host: s3server:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Fri, 01 Jun 2018 05:47:53 +
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:uzQs5hJOnwY4dyFk0PIxHHwmVcA=
>> Content-Length: 0
>>
> < HTTP/1.1 403 Forbidden
> < x-amz-id-2: tx5c2ac9ea26a046ac96e4d-005b10de09
> < x-amz-request-id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < Content-Type: application/xml
> < X-Trans-Id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < X-Openstack-Request-Id: tx5c2ac9ea26a046ac96e4d-005b10de09
> < Date: Fri, 01 Jun 2018 05:47:53 GMT
> < Transfer-Encoding: chunked
> * HTTP error before end of send, stop sending
> <
> 
> * Closing connection 0
> SignatureDoesNotMatchThe request signature we
> calculated does not match the signature you provided. Check your key and
> signing
> method.tx5c2ac9ea26a046ac96e4d-005b10de09
> 
> Regards,
> Shyam
> 
> On Fri, Jun 1, 2018 at 11:02 AM, Kota TSUYUZAKI <
> tsuyuzaki.k...@lab.ntt.co.jp> wrote:
> 
>> Hi Shyam,
>>
>> You should specify the path starts from bucket. It will be like
>>
>> http://20.20.20.229:8080/testBucket
>>
>> assuming you didn't configure virtual-hosted style.
>> Even if your *Swift* endpoint is http://20.20.20.229:8080/v1/AUTH_
>> dc86f7d8787b46158268bd77098b6578,
>> swift3 doesn't require the account in the path so far as well as actual
>> amazon s3.
>>
>> Best,
>> Kota
>>
>> (2018/05/31 21:33), Shyam Prasad N wrote:
>>> Hi Yuxin,
>>>
>>> Thank you for sharing your configs.
>>> So I've managed to get past the Signature not matching error.
>>>
>>> Now the error is different. InvalidbucketName:
>>>
>>> eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
>>> --createBucket --
>>> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b65
>> 78/testBucket
>>> s3curl: Found the url: host=20.20.20.229; port=8080;
>>> uri=/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket; query=;
>>> s3curl: cname endpoint signing case
>>> s3curl: StringToSign='PUT\n\n\nThu, 31 May 2018 12:02:57 +\n/
>>> 20.20.20.229/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket'
>>> s3curl: exec curl -v -H 'Date: Thu, 31 May 2018 12:02:57 +' -H
>>> 'Authorization: AWS
>>> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=' -L -H
>>> 'content-type: ' --data-binary  -X PUT
>>> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b65
>> 78/testBucket
>>> *   Trying 20.20.20.229...
>>> * TCP_NODELAY set
>>> * Connected to 20.20.20.229 (20.20.2

Re: [Openstack] Struggling to get the s3 api interface to work with swift.

[Kota TSUYUZAKI]

Hi Shyam,

You should specify the path starts from bucket. It will be like

http://20.20.20.229:8080/testBucket

assuming you didn't configure virtual-hosted style.
Even if your *Swift* endpoint is 
http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578,
swift3 doesn't require the account in the path so far as well as actual amazon 
s3.

Best,
Kota

(2018/05/31 21:33), Shyam Prasad N wrote:
> Hi Yuxin,
> 
> Thank you for sharing your configs.
> So I've managed to get past the Signature not matching error.
> 
> Now the error is different. InvalidbucketName:
> 
> eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
> --createBucket --
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
> s3curl: Found the url: host=20.20.20.229; port=8080;
> uri=/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket; query=;
> s3curl: cname endpoint signing case
> s3curl: StringToSign='PUT\n\n\nThu, 31 May 2018 12:02:57 +\n/
> 20.20.20.229/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket'
> s3curl: exec curl -v -H 'Date: Thu, 31 May 2018 12:02:57 +' -H
> 'Authorization: AWS
> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=' -L -H
> 'content-type: ' --data-binary  -X PUT
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
> *   Trying 20.20.20.229...
> * TCP_NODELAY set
> * Connected to 20.20.20.229 (20.20.20.229) port 8080 (#0)
>> PUT /v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket HTTP/1.1
>> Host: 20.20.20.229:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Thu, 31 May 2018 12:02:57 +
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=
>> Content-Length: 0
>>
> < HTTP/1.1 400 Bad Request
> < x-amz-id-2: tx18266052d5044eb2a3bc7-005b0fe471
> < x-amz-request-id: tx18266052d5044eb2a3bc7-005b0fe471
> < Content-Type: application/xml
> < X-Trans-Id: tx18266052d5044eb2a3bc7-005b0fe471
> < X-Openstack-Request-Id: tx18266052d5044eb2a3bc7-005b0fe471
> < Date: Thu, 31 May 2018 12:02:57 GMT
> < Transfer-Encoding: chunked
> * HTTP error before end of send, stop sending
> <
> 
> * Closing connection 0
> InvalidBucketNameThe specified bucket is not
> valid.tx18266052d5044eb2a3bc7-005b0fe471v1eightkpc@objectstore1:~/s3curl$
> 
> 
> My specified endpoint is
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578
> What am I doing wrong?
> 
> Regards,
> Shyam
> 
> On Wed, May 30, 2018 at 7:32 PM, Yuxin Wang 
> wrote:
> 
>> Hi Shyam,
>>
>> No problem. The output of the commands is attached.
>>
>> And my test cluster is on Swift v2.15.1 with Swift3 v1.12
>>
>> Also, here is the common process when I'm creating an S3 credential and
>> using in s3curl. Hope it helps.
>>
>> 1. Create a user and a project, and assign a proper role.
>>
>> openstack project create testproject
>> openstack user create testuser --password 123
>> openstack role add --project testproject --user testuser _member_
>>
>> 2. Check accessibility to swift
>>
>> create a test-openrc file with above info
>> source test-openrc
>> swift list
>>
>> 3.Create a credential
>>
>> openstack credential create --type ec2 --project testproject testuser
>>  '{"access": "testaccess", "secret": "testsecret"}'
>>
>> 4. Use it in s3curl
>>
>> add the endpoint url to `my @endpoints` in s3curl.pl
>> add the credential to .s3curl config file
>>
>> do `s3curl.pl -i cred_name --debug -- http://endpoint -X GET`
>>
>>
>>
>>
>> 在 2018年5月25日，18:17，Shyam Prasad N  写道：
>>
>> Hi Yuxin,
>>
>> If you don't mind, can you share the output of the following commands in
>> your running swift3 setup?
>>
>> openstack credential list
>> openstack ec2 credentials list
>> cat /etc/swift/proxy-server.conf
>>
>> Also, what are the access keys and secret keys that you use?
>> I want to make sure that I'm not missing anything in configuration.
>>
>> Regards,
>> Shyam
>>
>> On Fri, May 25, 2018 at 3:05 PM, Shyam Prasad N 
>> wrote:
>>
>>> Tried that. Unfortunately same error.
>>> Is there anything I can do to troubleshoot this?
>>>
>>> On Fri, May 25, 2018 at 2:56 PM, Yuxin Wang 
>>> wrote:
>>>
 They can be any strings.

 Replace them with whatever you want.

 - Yuxin

 在 2018年5月25日，14:57，Shyam Prasad N  写道：

 Thanks. I'll try this.
 But what values do I use in place of ak and sk? I want to use some
 command to get those values, right?

 On Fri, May 25, 2018 at 9:52 AM, Yuxin Wang 
  wrote:

> I created ec2 credentials using command `openstack credential create`.
>
> i.e.
>
> openstack credential create --type ec2 --project proj user '{"access":
> "ak", "secret": "sk”}'
>
>
> It seems the two credentials are not the same thing.
>
> Ref:
>
> https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/
> com.ibm.spectrum.scale.v4r11.adv.doc/bl1adv_ConfigureOpensta
> ckEC2credentials.htm
> 

Re: [Openstack] Struggling to get the s3 api interface to work with swift.

[Kota TSUYUZAKI]

Hi, Shyam

> tester => {
> id => 'test:tester',
> key => 'testing',
> },

If you are using this id/password to get your token from keystone,
you should set them as access_key and secret key for your s3 client.
You don't have to set any token information from keystone for your client.
i.e. `./s3curl.pl --id=tester -- http://127.0.0.1:8080/` may work. I'm not
an expert of the s3curl client though.

Best,
Kota


(2018/05/24 20:48), Shyam Prasad N wrote:
> Hi,
> 
> I've been trying to get swift3 to work for several days now. But I haven't
> managed to get it running.
> Both with tempauth and keystoneauth, I'm getting the same error:
> 
> eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --id=testerks --
> http://127.0.0.1:8080/
> 
> SignatureDoesNotMatchThe request signature we
> calculated does not match the signature you provided. Check your key and
> signing
> method.txa691e7ca97a44d56bc4c2-005b06a292
> 
> May 24 11:31:30 localhost proxy-server: 127.0.0.1 127.0.0.1
> 24/May/2018/11/31/30 GET / HTTP/1.0 403 - curl/7.58.0 - - 277 -
> txa691e7ca97a44d56bc4c2-005b06a292 - 0.0200 - - 1527161490.543112040
> 1527161490.563107014 -
> May 24 11:31:30 localhost proxy-server: STDERR: 127.0.0.1 - - [24/May/2018
> 11:31:30] "GET / HTTP/1.1" 403 621 0.021979 (txn:
> txa691e7ca97a44d56bc4c2-005b06a292)
> 
> eightkpc@objectstore1:~$ cat .s3curl
> %awsSecretAccessKeys = (
> tester => {
> id => 'test:tester',
> key => 'testing',
> },
> testerks => {
> id => 'e6289a1b5692461388d0597a4873d054',
> key => '88bb706887094696b082f008ba133ad7',
> },
> );
> 
> eightkpc@objectstore1:~$ openstack ec2 credentials show
> e6289a1b5692461388d0597a4873d054
> +++
> | Field  |
> Value
> |
> +++
> | access |
> e6289a1b5692461388d0597a4873d054
> |
> | links  | {u'self': u'
> http://controller:5000/v3/users/d7df7b56343b4ea988869fc30efeda09/credentials/OS-EC2/e6289a1b5692461388d0597a4873d054'}
> |
> | project_id |
> dc86f7d8787b46158268bd77098b6578
> |
> | secret |
> 88bb706887094696b082f008ba133ad7
> |
> | trust_id   |
> None
> |
> | user_id|
> d7df7b56343b4ea988869fc30efeda09
> |
> +++
> 
> Can someone please let me know what is going on?
> 
> Regards,
> Shyam
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Not able to use openstack swift using the s3 api plugin...

[Kota TSUYUZAKI]

Hello Shyam,

I did not have so much experience with s3curl but it looks like the problem is 
caused by client but it seems like the s3token middleware, which is to get 
authentication with keystone.
So let us check "What authentication are you using", then check "your exisiting 
configuration". That seems a way to go.

And more, if you could change the reference from 
https://docs.openstack.org/mitaka/config-reference/object-storage/configure-s3.html,
please consider if you would refer the newest docs at the master of swift3 
repository, https://github.com/openstack/swift3.
That is because OpenStack mitaka was approximately 2 years ago and some of the 
docs looks weird to me. (e.g. recommend swauth by default in the pipeline 
config)

If you could follow up the Swift upstream, the good news is that swift3 got 
merged as "s3api" to the Swift upstream repository in recent,
https://github.com/openstack/swift/commit/636b922f3b2882f7dd6c656983d7862b274dcf98.
So you may consider to use the newest version if you prefer to do so.

Best,
Kota


(2018/05/03 16:22), Shyam Prasad N wrote:
> Hi,
> 
> I tried installing the swift3 plugin using the following link:
> https://docs.openstack.org/mitaka/config-reference/object-storage/configure-s3.html
> 
> However, I'm not able to perform the operations:
> # ./s3curl.pl --id=personal -get -- -s -v http://proxy-server:8080
> Unknown option: get
> * Rebuilt URL to: http://proxy-server:8080/
> *   Trying 20.20.20.220...
> * Connected to proxy-server (20.20.20.220) port 8080 (#0)
>> GET / HTTP/1.1
>> Host: proxy-server:8080
>> User-Agent: curl/7.47.0
>> Accept: */*
>> Date: Thu, 03 May 2018 06:07:40 +
>> Authorization: AWS
> 4579fb60db3a47069f289d8fd7fa3212:R4zTQJDhPB3G8wLCsNBHaWNjaZQ=
>>
> < HTTP/1.1 500 Internal Server Error
> < x-amz-id-2: txbbe4b8dc26904cbf880e3-005aeaa72c
> < x-amz-request-id: txbbe4b8dc26904cbf880e3-005aeaa72c
> < Content-Type: application/xml
> < X-Trans-Id: txbbe4b8dc26904cbf880e3-005aeaa72c
> < Date: Thu, 03 May 2018 06:07:41 GMT
> < Transfer-Encoding: chunked
> <
> 
> InternalErrorWe encountered an internal error.
> Please try
> again.txbbe4b8dc26904cbf880e3-005aeaa72c?xml
> version="1.0" encoding="UTF-8"?
> Error
>   CodeInvalidURI/Code
>   MessageCould not parse the specified URI/Message
> /Error
> * Connection #0 to host proxy-server left intact
> 
> Getting this in the logs...
> May  2 23:07:40 localhost proxy-server: STDERR: (20652) accepted
> ('20.20.20.220', 51030)
> May  2 23:07:41 localhost proxy-server:  encoding="UTF-8"?>#015#012#015#012  InvalidURI#015#012
> Could not parse the specified
> URI#015#012#015#012: #012Traceback (most recent call
> last):#012  File "/usr/lib/python2.7/dist-packages/swift3/middleware.py",
> line 81, in __call__#012resp = self.handle_request(req)#012  File
> "/usr/lib/python2.7/dist-packages/swift3/middleware.py", line 104, in
> handle_request#012res = getattr(controller, req.method)(req)#012  File
> "/usr/lib/python2.7/dist-packages/swift3/controllers/service.py", line 33,
> in GET#012resp = req.get_response(self.app, query={'format':
> 'json'})#012  File "/usr/lib/python2.7/dist-packages/swift3/request.py",
> line 686, in get_response#012headers, body, query)#012  File
> "/usr/lib/python2.7/dist-packages/swift3/request.py", line 665, in
> _get_response#012raise BadSwiftRequest(err_msg)#012BadSwiftRequest:
> #015#012#015#012
> InvalidURI#015#012  Could not parse the specified
> URI#015#012#015#012 (txn:
> txbbe4b8dc26904cbf880e3-005aeaa72c)
> May  2 23:07:41 localhost proxy-server: STDERR: 20.20.20.220 - -
> [03/May/2018 06:07:41] "GET / HTTP/1.1" 500 724 0.058274 (txn:
> txbbe4b8dc26904cbf880e3-005aeaa72c)
> 
> Can someone please tell me what's going on?
> Please let me know if some additional data is necessary.
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack][Swift] problem about swift proxy easurecode

[Kota TSUYUZAKI]

gt;>
>>>
>>>
>>> --
>>> ==
>>>  - Community : *Openstack Korea Community*
>>>  - Name  : Nalee Jang (장현정)
>>>  - Company: ASD Korea Cloudike System Engineer.
>>>  - Contact   : +82-10-3454-4260 / nalee...@gmail.com
>>>  - Blog: http://naleejang.tistory.com
>>>  - HomePage  : http://www.openstack.or.kr
>>>  - Facebook   : https://www.facebook.com/groups/openstack.kr/
>>> ==
>>>
>>> ___
>>> Mailing list: http://lists.openstack.org/cgi
>>> -bin/mailman/listinfo/openstack
>>> Post to : openstack@lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi
>>> -bin/mailman/listinfo/openstack
>>>
>>>
>>
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] [Swift] Does Swift3 support v4 signature for S3?

[Kota TSUYUZAKI]

Hello Alexandr,

Signature V4 has been supported since

https://github.com/openstack/swift3/commit/a1cc181bd8ef891d3e3b8ed71db9b41c179ced0e

It has been restricted with only keystone authentication though.
The docs in README.rst is wrong. I pushed the fix for the docs, 
https://review.openstack.org/#/c/387126/

Thanks,
Kota

(2016/10/15 6:00), Alexandr Porunov wrote:
> Hello,
> 
> On gitgub they tell us:
> Support AWS Signature Version 2 (Version 4 is not ready)
> 
> But in changelog they tell that they support v4:
> Amazon S3 Signature V4 support
> 
> I tried to use aws java s3 client and mino java client to send requests to
> swift but without success. Both show either 403 or 400 error..
> 
> When I use command line tool (s3curl.pl) it works.
> 
> Does Swift3 support v4? What can be wrong?
> 
> Sincerely,
> Alexandr
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] [Swift] How to use s3curl or s3cmd with Swift3?

[Kota TSUYUZAKI]

Hello again,

I'm not familier with s3curl in detail though,

> ./s3curl.pl - 'swift' -key '3d8b70ad194a4cf6857e59ec638d' -get - -s -v
> http://proxy:8080/v1/AUTH_3f842db875cc48b99d7ff246c27a2e6a
>

could you try:

./s3curl.pl - 'swift' -key 'swift' -get - -s -v http://proxy:8080/

IIRC, secret_key is not a swift token but a password you set in your keystone 
and swift3 doesn't require Swift url so you can request to the host directly.

Regards,
Kota


(2016/10/13 17:29), Alexandr Porunov wrote:
> Hello,
> 
> I try to use s3curl to test swift3 but I miss something.
> 
> Here is the command which I use:
> ./s3curl.pl - 'swift' -key '3d8b70ad194a4cf6857e59ec638d' -get - -s -v
> http://proxy:8080/v1/AUTH_3f842db875cc48b99d7ff246c27a2e6a
> 
> It shows:
> Can't locate Digest/HMAC_SHA1.pm in @INC (@INC contains:
> /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
> /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./
> s3curl.pl line 20.
> BEGIN failed--compilation aborted at ./s3curl.pl line 20.
> 
> I am not sure that I write the command correctly.
> 
> *swift* - is the account
> *3d8b70ad194a4cf6857e59ec638d* - is the X-Subject-Token I have got in
> the header with this command:
> 
> curl -i \
>   -H "Content-Type: application/json" \
>   -d '
> { "auth": {
> "identity": {
>   "methods": ["password"],
>   "password": {
> "user": {
>   "name": "swift",
>   "domain": { "name": "default" },
>   "password": "swift"
> }
>   }
> },
> "scope": {
>   "project": {
> "name": "service",
> "domain": { "name": "default" }
>   }
> }
>   }
> }' \
>   http://keystone:5000/v3/auth/tokens ; echo
> 
> *http://proxy:8080/v1/AUTH_3f842db875cc48b99d7ff246c27a2e6a
> <http://proxy:8080/v1/AUTH_3f842db875cc48b99d7ff246c27a2e6a>* - is the
> public endpoint
> 
> Am I doing correctly?
> 
> Why doesn't it work?
> 
> Sincerely,
> Alexandr
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] [Swift] How to install Swift3 middleware?

[Kota TSUYUZAKI]

Hi Alexandr,

Though I cannot see the actual reason why the proxy-server failed (your report 
looks just proxy-server exist with status code 1),
looking at your procedure, you might need to install dependencies (e.g. lxml) 
for swift3. I mean,

>>>>> git clone https://github.com/openstack/swift3
>>>>> cd swift3
>>>>> python setup.py install

sudo pip install -r requirements.txt
(you may need also "sudo pip install -r test-requirements.txt" for testing)

If you could share sort of logs why proxy-server got failed, it might be 
helpful to figure out more.

Thanks,
Kota


(2016/10/13 14:34), Alexandr Porunov wrote:
> No, I didn't.
> 
> I ran "python setup.py install" one more time. Here is the output:
> http://paste.openstack.org/show/585535/
> 
> On Thu, Oct 13, 2016 at 8:21 AM, John van Ommen <john.vanom...@gmail.com>
> wrote:
> 
>> To me, the errors you're receiving are indicative of the swift3
>> package not installing correctly.
>>
>> When you ran that first step (setup.py), did you get errors?
>>
>> John
>>
>> On Wed, Oct 12, 2016 at 10:03 PM, Alexandr Porunov
>> <alexandr.poru...@gmail.com> wrote:
>>> Hello,
>>>
>>> It still doesn't work... The same error is shown. Maybe I missed
>> something?
>>> Maybe before the installation I had to done something or something else.
>> I
>>> tried different things to run it. But when we do have swift3 or/and
>> s3token
>>> in pipeline it doesn't work.
>>>
>>> Sincerely,
>>> Alexandr
>>>
>>> On Thu, Oct 13, 2016 at 12:17 AM, Mark Kirkwood
>>> <mark.kirkw...@catalyst.net.nz> wrote:
>>>>
>>>>
>>>>
>>>> On 13/10/16 06:26, Alexandr Porunov wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I want to use S3 api for Swift.
>>>>>
>>>>> I have installed swift3 as here:
>>>>>
>>>>> git clone https://github.com/openstack/swift3
>>>>> cd swift3
>>>>> python setup.py install
>>>>>
>>>>> Then I have configured my proxy-server.conf as here:
>>>>>
>>>>> [pipeline:main]
>>>>> pipeline = catch_errors gatekeeper healthcheck proxy-logging cache
>>>>> container_sync bulk ratelimit swift3 s3token authtoken keystoneauth
>>>>> container-quotas account-quotas slo dlo versioned_writes proxy-logging
>>>>> proxy-server
>>>>>
>>>>> [filter:swift3]
>>>>> use = egg:swift3#swift3
>>>>>
>>>>> [filter:s3token]
>>>>> use = egg:swift3#s3token
>>>>> auth_uri = http://keystone:35357/
>>>>> reseller_prefix = AUTH_
>>>>>
>>>>> Then I tried to restart the proxy:
>>>>> systemctl restart openstack-swift-proxy
>>>>>
>>>>> And it failes.
>>>>>
>>>>>
>>>>
>>>> Yeah - for Mitaka (or thereabouts) onwards that s3token config needs to
>>>> change:
>>>>
>>>> [filter:s3token]
>>>> paste.filter_factory = keystoneclient.middleware.s3_
>> token:filter_factory
>>>> auth_uri = http://keystone:35357/
>>>> reseller_prefix = AUTH_
>>>>
>>>> Alternatively the older port/protocol/host variables still work instead
>> of
>>>> auth_uri:
>>>>
>>>> [filter:s3token]
>>>> paste.filter_factory = keystoneclient.middleware.s3_
>> token:filter_factory
>>>> auth_port = 35357
>>>> auth_host = keystone
>>>> auth_protocol = http
>>>>
>>>> The swift3 docs need updating, as it is not obvious what has gone wrong
>>>> when you hit this.
>>>>
>>>> regards
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ___
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to : openstack@lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>>
>>>
>>> ___
>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
>> openstack
>>> Post to : openstack@lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
>> openstack
>>>
>>
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [Swift] Where can I get the libshss library ?

[Kota TSUYUZAKI]

Correct. Thanks John for your quick response.

Kota

(2016/10/13 12:14), John Dickinson wrote:
> Unfortunately, no (as far as i know). It's a proprietary library used by NTT.
> 
> However, if you update to the latest version of liberasurecode, that warning 
> message is suppressed.
> 
> --John
> 
> 
> 
> On 12 Oct 2016, at 19:58, Yu Watanabe wrote:
> 
>> Hello.
>> 
>> I would like to ask question related to swift object storage node.
>> 
>> I have set up swift object storage node following the installation guide and 
>> now finalizing the installation using the guide.
>> 
>> http://docs.openstack.org/mitaka/install-guide-rdo/swift-finalize-installation.html
>> 
>> However, I get error when I start up the services.
>> 
>> systemctl status openstack-swift-account.service 
>> openstack-swift-account-auditor.service 
>> openstack-swift-account-reaper.service 
>> openstack-swift-account-replicator.service ●
>> openstack-swift-account.service - OpenStack Object Storage (swift) - Account 
>> Server Loaded: loaded 
>> (/usr/lib/systemd/system/openstack-swift-account.service; enabled; vendor 
>> preset: disabled) 
>> Active: failed (Result: exit-code) since Wed 2016-10-12 19:44:45 JST; 15h 
>> ago Process: 27793 ExecStart=/usr/bin/swift-account-server 
>> /etc/swift/account-server.conf (code=exited,
>> status=1/FAILURE) Main PID: 27793 (code=exited, status=1/FAILURE)
>> 
>> Oct 12 19:44:44 opstack-objstorage1 systemd[1]: Started OpenStack Object 
>> Storage (swift) - Account Server. Oct 12 19:44:44 opstack-objstorage1 
>> systemd[1]: Starting OpenStack Object Storage
>> (swift) - Account Server... Oct 12 19:44:45 opstack-objstorage1 
>> liberasurecode[27793]: liberasurecode_instance_create: dynamic linking error 
>> libshss.so.1: cannot open shared ...rectory Oct 12
>> 19:44:45 opstack-objstorage1 swift-account-server[27793]: Error: 
>> [swift-hash]: both swift_hash_path_suffix and swift_hash_path_prefix are 
>> m...ft.conf Oct 12 19:44:45 opstack-objstorage1
>> systemd[1]: openstack-swift-account.service: main process exited, 
>> code=exited, status=1/FAILURE Oct 12 19:44:45 opstack-objstorage1 
>> systemd[1]: Unit openstack-swift-account.service entered
>> failed state. Oct 12 19:44:45 opstack-objstorage1 systemd[1]: 
>> openstack-swift-account.service failed.
>> 
>> I have already installed libJerasure.so.2 and libisal.so.2 from source code 
>> but still not able to install libshss.so.1.
>> 
>> Below is information about my environment.
>> 
>> [root@opstack-objstorage1 system 2016.10.13 11:34:26]# cat 
>> /etc/redhat-release CentOS Linux release 7.2.1511 (Core) 
>> [root@opstack-objstorage1 system 2016.10.13 11:33:53]# rpm -qa | grep swift 
>> openstack-swift-container-2.7.0-2.el7.noarch 
>> openstack-swift-2.7.0-2.el7.noarch openstack-swift-object-2.7.0-2.el7.noarch 
>> openstack-swift-account-2.7.0-2.el7.noarch
>> 
>> Is there a web site where I can get the source code for libshss?
>> 
>> Thanks.
>> 
>> -- Yu Watanabe 渡辺 裕
>> 
>> LinkedIn : jp.linkedin.com/in/yuwatanabe1
> 
> 
>> ___ Mailing list: 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : 
>> openstack@lists.openstack.org Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 
> 
> 
> ___ Mailing list: 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : 
> openstack@lists.openstack.org Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] [Swift] Does Swift3 supports SLO (Manifest creation)?

[Kota TSUYUZAKI]

Hello Alexandr,

Swift3 supports Multipart Upload API for actual Amazon S3[1] which has similar 
functionality with Swift SLO manifest so that you can use any SDK for s3 
multiupload api.
It needs,

1. init a multipart Upload
2. upload segments (chunk you may call)
3. complete the multipart upload


 boto[2] known as python sdk also supports it. And it may be helpful to see 
swift3 functional test[3] to know how we can call the apis.

Thanks,
Kota


1: http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html
2: http://boto.cloudhackers.com/en/latest/ref/s3.html
3: 
https://github.com/openstack/swift3/blob/master/swift3/test/functional/test_multi_upload.py

(2016/10/12 4:32), Alexandr Porunov wrote:
> Hello,
> 
> I want to use S3 API for OpenStack Swift but I need chunked upload for my
> files. I want to save many small object (1-10 MB) and retrieve them as a
> single object. For it I need to be able to create a manifest file. Does
> Swift3 support manifest creation? If yes then where can I see an example?
> 
> Sincerely,
> Alexandr
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] FYI - gate completely borked on master and newton dsvm/grenade jobs

[Kota TSUYUZAKI]

Swift is affected too[1] (maybe since swift is using Cryptography). When trying 
to set upper constraint as pycparser<=2.13[1] in the requirements.txt, that 
worked for unit and functional tests[2] but
tempest/granade related to dsvm is still down.

1: https://bugs.launchpad.net/swift/+bug/1629765
2: https://review.openstack.org/#/c/380904/

https://review.openstack.org/#/c/380904/2/requirements.txt
(2016/10/03 14:57), Steven Dake (stdake) wrote:
> Also impacts Kolla (as in our gates are blocked).  At present we are using 
> the proposed workaround until the pycparser 2.14 wheel and package are synced 
> up.
> 
> Regards
> -steve
> 
> 
> From: Matt Riedemann <mrie...@linux.vnet.ibm.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Date: Sunday, October 2, 2016 at 5:54 PM
> To: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] FYI - gate completely borked on master and newton 
> dsvm/grenade jobs
> 
> There was a pycparser 2.14 package update on pypi today which is making
> cinder-manager db sync fail. This is making all dsvm/grenade jobs fail
> in master and stable/newton.
> 
> The upstream issue being tracked is:
> 
> https://github.com/eliben/pycparser/issues/147
> 
> --
> 
> Thanks,
> 
> Matt Riedemann
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Swift3] improve multi-delete performance

[Kota TSUYUZAKI]

Hello kirubakaran,

Thanks for contributing Swift3, that sounds great. I wonder if you could push 
the diff as a patch to gerrit code review which is the review system for 
openstack and openstack related projects.
The entry point for "how to contribute for openstack" is here, 
https://wiki.openstack.org/wiki/How_To_Contribute and specific git commands 
will be introduced at
http://docs.openstack.org/infra/manual/developers.html.

For your information, a patch[1] from Tim who is working for SwiftStack might 
help you to improve your patch because the patch is for enabling other swift 
middlewares to make concurrent requests, it
is for upstream Swift though :)

Best,
Kota


1: https://review.openstack.org/#/c/311817

(2016/05/24 13:47), Kirubakaran Kaliannan wrote:
> Hi,
> 
> 
> 
> The multi_delete in swift3, perform sequential DELETE.  In my 3
> storage-node configuration to delete a 1000 objects, it took 30 second.
> 
> 
> 
> Following code change to create 100 thread pool to delete 1000 object took
> only 12 second. (This may even reduce if more storage nodes in picture).
> 
> 
> 
> If the following code change look fine, How can we formally
> (propose/review/commit) take this to the swift3 github code base ?
> 
> 
> 
> 
> 
> *Code diff:*
> 
> 
> 
> diff --git a/swift3/swift-plugin-s3/swift3/controllers/multi_delete.py
> b/swift3/swift-plugin-s3/swift3/controllers/multi_delete.py
> 
> index 1bfde1d..5140529 100644
> 
> --- a/swift3/swift-plugin-s3/swift3/controllers/multi_delete.py
> 
> +++ b/swift3/swift-plugin-s3/swift3/controllers/multi_delete.py
> 
> @@ -21,9 +21,9 @@ from swift3.response import HTTPOk, S3NotImplemented,
> NoSuchKey, \
> 
> from swift3.cfg import CONF
> 
> from swift3.utils import LOGGER
> 
> -# Zadara-Begin
> 
> +from eventlet import GreenPool
> 
> +import copy
> 
> MAX_MULTI_DELETE_BODY_SIZE = 262144
> 
> -# Zadara-End
> 
> 
> 
>  class MultiObjectDeleteController(Controller):
> 
> @@ -44,6 +44,24 @@ class MultiObjectDeleteController(Controller):
> 
>  return tostring(elem)
> 
> +def async_delete(self, reqs, key, elem):
> 
> +req = copy.copy(reqs)
> 
> +req.object_name = key
> 
> +try:
> 
> +req.get_response(self.app, method='DELETE')
> 
> +except NoSuchKey:
> 
> +pass
> 
> +except ErrorResponse as e:
> 
> +error = SubElement(elem, 'Error')
> 
> +SubElement(error, 'Key').text = key
> 
> +SubElement(error, 'Code').text = e.__class__.__name__
> 
> +SubElement(error, 'Message').text = e._msg
> 
> +return
> 
> +
> 
> +if not self.quiet:
> 
> +deleted = SubElement(elem, 'Deleted')
> 
> +SubElement(deleted, 'Key').text = key
> 
> +
> 
>  @bucket_operation
> 
>  def POST(self, req):
> 
>  """
> 
> @@ -90,27 +108,17 @@ class MultiObjectDeleteController(Controller):
> 
>  body = self._gen_error_body(error, elem, delete_list)
> 
>  return HTTPOk(body=body)
> 
> +parallel_delete = 100
> 
> +run_pool = GreenPool(size=parallel_delete)
> 
>  for key, version in delete_list:
> 
>  if version is not None:
> 
>  # TODO: delete the specific version of the object
> 
>  raise S3NotImplemented()
> 
> -req.object_name = key
> 
> -
> 
> -try:
> 
> -req.get_response(self.app, method='DELETE')
> 
> -except NoSuchKey:
> 
> -pass
> 
> -except ErrorResponse as e:
> 
> -error = SubElement(elem, 'Error')
> 
> -SubElement(error, 'Key').text = key
> 
> -SubElement(error, 'Code').text = e.__class__.__name__
> 
> -SubElement(error, 'Message').text = e._msg
> 
> -continue
> 
> -
> 
> -if not self.quiet:
> 
> -deleted = SubElement(elem, 'Deleted')
> 
> -SubElement(deleted, 'Key').text = key
> 
> +run_pool.spawn(self.async_delete, req, key, elem)
> 
> +
> 
> +# Wait for all the process to complete
> 
> +run_pool.waitall()
> 
>  body = tostring(elem)
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] [Swift] How to add new regions/zones into a swift cluster?

[Kota TSUYUZAKI]

Hello Alexandr,

As the error message described, region and zone take an integer value[1] so 
probably you should set the value like as:

swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.57.51 
--port 6002 --device d1 --weight 100

Although you need to keep your own map like "region 1 means USA and zone 1 
means NewYork" by yourself.

Best,
Kota

1: 
https://github.com/openstack/swift/blob/master/swift/common/ring/utils.py#L559-L562
(2016/05/19 19:13), Alexandr Porunov wrote:
> Hello everyone!
> 
> I am trying to install swift without others products but I have problem
> with adding drives to the swift ring.
> I use this command:
> swift-ring-builder account.builder add --region USA --zone New_York --ip
> 192.168.57.51 --port 6002 --device d1 --weight 100
> 
> It shows me:
> "Usage: swift-ring-builder [options]. swift-ring-builder: error: option
> --region: invalid integer value: 'USA'".
> 
> What's wrong with a name of the region? How can I add new regions/zones
> into swift? Please help me
> 
> Best regards,
> Alexandr
> 
> 
> 
> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [Storlets] Swift copy middlleware

[Kota TSUYUZAKI]

Hey Eran,

This is what I was concerning in Bristol Hackathon :/

> As a quick and temporary resolution I have changes the tox.ini dependency
> to be 2.7.0
> Instead of master. We still need, however, to port the code accordingly,

For temprary, that would work but I thought we could (*not sure*) fix the issue 
just replace the order of pipeline, right? (i.e. storlets handler should be the 
left of copy middleware)
That is because the storlets middlware have the logic to translate 
COPY/PUT(X-COPY-FROM) into GET(no storelets running)/PUT(execute at Proxy). If 
it happens before the request reaching to copy
middleware, it looks like just PUT or GET at copy middleware itself (so nothing 
to oparate there).

I'll start to make sure my thought in this week but thanks to raise a flag to 
the community :)

Thanks,
Kota



(2016/05/16 3:42), Eran Rom wrote:
> Today the Swift team has merged copy middleware - congrats!
> For us, however, it breaks the copy code path, which in fact can get much 
> simpler now.
> 
> As a quick and temporary resolution I have changes the tox.ini dependency 
> to be 2.7.0
> Instead of master. We still need, however, to port the code accordingly,
> 
> Here is a suggestion:
> The copy middleware will process the COPY / PUT & X-Copy-From and will:
> 1. Do a GET of the source object
> 2. Do a PUT to the target object
> 
> I believe that for Storlets what would happen is that both PUT and GET
> cause a storlet invocation, where in fact we want that invocation to 
> happen
> Eithrer in the GET or in the PUT (but not both)
> I believe that if we are OK with running the storlet on the put, we can 
> use
> The swift_source SSC as an indicator that the get is generated from the
> Copy middleware and disregard the X-Run-Storlet header.
> 
> Thoughts?
> 
> Thanks,
> Eran
> 
> 
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [swift3][swift] What happens in swift3?

[Kota TSUYUZAKI]

Tim is sill working for making sure how the sigv4 patch works[1]. The couple of 
main reasons for the slow review is, right now, no way to make sure the sigv4 
in functests for jenkins job and no docs
about what's changed, what's constraint, what's we should change our config in 
those patches. In my opinion, the authentication is the most sensitive one of 
all modules because it can break everything
easily and I am still worried about the patch may break the exinsting 
environment for a bunch of users in all over the world using sigv2.

Regards,
Kota


1: https://bugs.launchpad.net/swift3/+bug/1557260
2: https://review.openstack.org/#/c/272516

(2016/03/14 19:01), Andrey Pavlov wrote:
> Hello,
> 
> I found that new Amazon tool for S3 started to use another way to sign 
> requests.
> I made changes and submitted my review [1] seven months ago.
> 
> This functionality is needed for ec2api project [2].
> It is needed for using new aws cli tool.
> 
> I asked to review my changes directly to Kota via email.
> My colleague asked him on previous summit about it.
> All dependent reviews (in keystone) already merged four months ago.
> On all comments were answered in the review.
> I tried to raise this question at swift3 meeting.
> 
> But still there is no answer.
> 
> Can anyone answer - Can it really be reviewed and merged?
> 
> [1] https://review.openstack.org/#/c/211933/
> [2] https://review.openstack.org/#/c/198571/
> 


-- 
------
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Swift] Erasure coding and geo replication

[Kota TSUYUZAKI]

Hello Mark,

AFAIK, a few reasons for that we still are in working progress for erasure code 
+ geo replication.

>> and expect to survive a region outage...
>>
>> With that I mind I did some experiments (Liberty swift) and it looks to me 
>> like if you have:
>>
>> - num_data_frags < num_nodes in (smallest) region
>>
>> and:
>>
>> - num_parity_frags = num_data_frags
>>
>>
>> then having a region fail does not result in service outage.

Good point but note that the PyECLib v1.0.7 (pinned to Kilo/Liberty stable) 
still have a problem which cannot decode the original data when all feed 
fragments are parity frags[1]. (i.e. if set
num_parity_frags = num_data frags and then, num_parity_frags comes into proxy 
for GET request, it will fail at the decoding) The problem was already resolved 
in the PyECLib/liberasurecode at master
branch and current swift master has the PyECLib>=1.0.7 dependencies so if you 
thought to use the newest Swift, it might be not
a matter.

In the Swift perspective, I think that we need more tests/discussion for geo 
replication around write/read affinity[2] which is geo replication stuff in 
Swift itself and performances.

For the write/read affinity, actually we didn't consider the affinity control 
to simplify the implementation until EC landed into Swift master[3] so I think 
it's time to make sure how we can use the
affinity control with EC but it's not done yet.

For the performance perspective, in my experiments, more parities causes quite 
performance degradation[4]. To prevent the degradation, I am working for the 
spec which makes duplicated copy from
data/parity fragments and spread them out into geo regions.

To sumurize, we've not done the work yet but we welcome to discuss and 
contribute for EC + geo replication anytime, IMO.

Thanks,
Kota

1: 
https://bitbucket.org/tsg-/liberasurecode/commits/a01b1818c874a65d1d1fb8f11ea441e9d3e18771
2: 
http://docs.openstack.org/developer/swift/admin_guide.html#geographically-distributed-clusters
3: 
http://docs.openstack.org/developer/swift/overview_erasure_code.html#region-support
4: 
https://specs.openstack.org/openstack/swift-specs/specs/in_progress/global_ec_cluster.html



(2016/02/15 18:00), Mark Kirkwood wrote:
> After looking at:
> 
> https://www.youtube.com/watch?v=9YHvYkcse-k
> 
> I have a question (that follows on from Bruno's) about using erasure coding 
> with geo replication.
> 
> Now the example given to show why you could/should not use erasure coding 
> with geo replication is somewhat flawed as it is immediately clear that you 
> cannot set:
> 
> - num_data_frags > num_devices (or nodes) in a region
> 
> and expect to survive a region outage...
> 
> With that I mind I did some experiments (Liberty swift) and it looks to me 
> like if you have:
> 
> - num_data_frags < num_nodes in (smallest) region
> 
> and:
> 
> - num_parity_frags = num_data_frags
> 
> 
> then having a region fail does not result in service outage.
> 
> So my real question is - it looks like it *is* possible to use erasure coding 
> in geo replicated situations - however I may well be missing something 
> significant, so I'd love some clarification here [1]!
> 
> Cheers
> 
> Mark
> 
> [1] Reduction is disk usage and net traffic looks attractive
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 


-- 
--
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.k...@lab.ntt.co.jp>
NTT Software Innovation Center
Cloud Solution Project
Phone  0422-59-2837
Fax0422-59-2965
---



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][ec2-api][swift] Moving EC2 Auth and S3Token to Externally supported

[Kota TSUYUZAKI]

Hello guys,

S3 compatibility API is now maitained in openstack namespace, it is in the 
external repository from original Swift though.
It works as Swift proxy middleware to translate S3 api into pure Swift api. 
While the translation, Swift3 makes a credential to retrieve actual auth token 
from keystone
(this works in s3_token middleware) and the credential is compatible with 
tempauth. (reference auth middleware maintained in Swift)

If the credential api in s3_token deprecated in keystonemiddleware, it will be 
hard to maintain the semantics with pure swift authentication system.

I'm not sure I followed up this conversation completely but hopefully I wonder 
if you give me more infomation (e.g. Why do we need this, how do we 
improve/migrate this) about this


Thanks,
Kota

1: https://github.com/openstack/swift3

(2016/02/07 9:02), Morgan Fainberg wrote:
> On Sat, Feb 6, 2016 at 11:06 AM, Tim Bell  wrote:
> 
>>
>> I support the move to deprecate but I don’t think the EC2API people should
>> be responsible for the S3 compatibility. This would seem to need an S3API
>> project (working with the SWIFT folk to do a smooth migration).
>>
>> Tim
>>
>>
> All on the table for discussion - right now the S3Token code relies on the
> EC2 API code in Keystone, so they cannot be separated without more work.
> We'll work on this in either case and figure the best way going forward.
> 
> Cheers,
> --Morgan
> 
> 
>> From: Morgan Fainberg
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> Date: Saturday 6 February 2016 at 17:57
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> Subject: Re: [openstack-dev] [keystone][ec2-api][swift] Moving EC2 Auth
>> and S3Token to Externally supported
>>
>> So based on this conversation and the need to support the legal
>> requirement of some deployers to totally strip the code from the install,
>> I've taken another approach. The base controller will emit a 403
>> (Forbidden) if the core of the compat code is not available for import. If
>> the legal demands (corp.legal) change for the org that requires the ease of
>> removing the compat code changes, we will remove the fallback to the hard
>> 403 response if the code is still in Keystone's tree. This type of fallback
>> will be handled only for the aws compat code since the legal requirements
>> have been relying on this , I do not expect this pattern to be expanded
>> beyond this specific AWS compat code in keystone.
>>
>> The move to deprecate and move this into the hands of the ec2api team will
>> continue to be discussed so we can hammer out details making sure we don't
>> break apps relying on the aws compat apis. Long term I fully expect this to
>> not be in Keystone's tree.
>>
>> --M
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Swift-proxy server failed

[Kota TSUYUZAKI]

Hi Venkatesh,

Your setting seems wrong for authtoken middleware in proxy-server.conf.
The log shows proxy-server was trying to look up "auth_token" due to your pipeline setting (probably there is "auth_token" string in your pipeline) but the section name of the middleware setting would 
be something like "authtoken" (no undersocre) which perhaps comes from recommendation example in master repository[1].


To use the trick for making filter pipeline, you must set up with same name both in 
"pipeline" and "section name" so I wonder if you could review your pipeline and 
section, again.

1: 
https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L284-L290

Best
Kota

(2015/09/15 7:21), venkatesh kotipalli wrote:

Hi Guys,

Successfully configured  swift. But when start the swift-proxy server. It
through  the error is :

openstack-swift-proxy.service - OpenStack Object Storage (swift) - Proxy
Server
Loaded: loaded (/usr/lib/systemd/system/openstack-swift-proxy.service;
enabled)
Active: failed (Result: exit-code) since Tue 2015-09-15 03:39:40 IST; 2s
ago
   Process: 27570 ExecStart=/usr/bin/swift-proxy-server
/etc/swift/proxy-server.conf (code=exited, status=1/FAILURE)
  Main PID: 27570 (code=exited, status=1/FAILURE)

Sep 15 03:39:40 controller swift-proxy-server[27570]: for name in
pipeline[:-1]]
Sep 15 03:39:40 controller swift-proxy-server[27570]: File
"/usr/lib/python2.7/site-packages/swift/common/wsgi.py", line 61, in
get_context
Sep 15 03:39:40 controller swift-proxy-server[27570]: object_type,
name=name, global_conf=global_conf)
Sep 15 03:39:40 controller swift-proxy-server[27570]: File
"/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 408, in
get_context
Sep 15 03:39:40 controller swift-proxy-server[27570]: object_type,
name=name)
Sep 15 03:39:40 controller swift-proxy-server[27570]: File
"/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 587, in
find_config_section
Sep 15 03:39:40 controller swift-proxy-server[27570]: self.filename))
Sep 15 03:39:40 controller swift-proxy-server[27570]: LookupError: No
section 'authtoken' (prefixed by 'filter') found in config
/etc/swift/proxy-server.conf
Sep 15 03:39:40 controller systemd[1]: openstack-swift-proxy.service: main
process exited, code=exited, status=1/FAILURE
Sep 15 03:39:40 controller systemd[1]: Unit openstack-swift-proxy.service
entered failed state.
[root@controller ~(keystone_admin)]#vi /etc/swift/proxy-server.conf

Please guys help me.

Regards,
Venkatesh.k



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [Swift] Meeting time change

[Kota TSUYUZAKI]

Thanks John and all swifters for taking care of Asian Pasific and Australian 
time zone. Much appreciated :)

Kota

(2015/05/26 4:32), John Dickinson wrote:

Based on discussion over at the summit and over the last few weeks, the Swift 
team meeting time has changed.

The new meeting time is 2100UTC on Wednesdays in #openstack-meeting.

The meeting agenda is tracked at https://wiki.openstack.org/wiki/Meetings/Swift


--John






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Swift] More Efficient Replication Slide in Juno Summit

[Kota TSUYUZAKI]

Hi, guys

I've uploaded my slide related to More Efficient Replication
in Juno Swift Design Summit to slideshare. Its URL is here.

http://www.slideshare.net/tsuyuzaki/open-stacks-ummitjuno

If anyone has questions or comments, please let me know by e-mail.

Regards,

Kota




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev