Re: [Openstack] Requirement - x86

2012-10-17 Thread Nandakumar Pezhery
Ganesh
For lab purpose a single box would also be fine. I started off that way.
However, adding more nodes would allow splitting services across them and
experimenting on a multinode setup.

Cheers
Nandakumar
On Oct 17, 2012 8:40 AM, Ganesh Hariharan ghariha...@gmail.com wrote:

 Nandakumar,

 thanks much... i have some more basic queries, please excuse :)

 Is the minimum hardware requirement (like good to have ) for a full
 fledged Internet based access to Compute infra  setups...

 also, like to dwelve more into other services that I can configure and
 test them as well.. essentially for learning lab purpose,,,

 thanks


 On 10/16/12, Nandakumar Pezhery nandakuma...@gmail.com wrote:
  Ganesh,
 
  you may refer the Folsom install guide
  http://docs.openstack.org/install/
 
  or
 
 https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/blob/master/OpenStack_Folsom_Install_Guide_WebVersion.rst
 
  or installation guide for Redhat
  http://docs.openstack.org/trunk/openstack-compute/install/yum/content/
 
  Rgds
  Nandakumar
 
 
 
  On Tue, Oct 16, 2012 at 1:55 PM, Ganesh Hariharan
  ghariha...@gmail.comwrote:
 
  Hi,
 
  I am a novice to openstack...
 
  Please guide me through compute and storage setupss... any ready
  reckoners...
 
  Thanks
 
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
 
 
 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Quantum OVS Agent as service

2012-10-17 Thread Srikanth Kumar Lingala
Hi all,
If we run Quantum OVS Agent as a service ... '*service
quantum-plugin-openvswitch-agent start*', it is unable to start the agent.
It is throwing the following error to '/var/log/quantum/agent-ovs.log'

*Stderr: 'Device br-eth1 does not exist.\n'*
*2012-10-16 12:48:10ERROR
[quantum.plugins.openvswitch.agent.ovs_quantum_agent] Bridge br-eth1 for
physical network default does not exist*
*
*
if we run the agent manually like the following command, it is able to
start:

*/usr/bin/quantum-openvswitch-agent --
--config-file=/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
--log-file=/var/log/quantum/agent-ovs.log
--config-file=/etc/quantum/quantum.conf*

Please suggest.
-- 

Srikanth.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Unconference CompatibleOne

2012-10-17 Thread Andy Edmonds
FYI: slides on OCCI that were presented at Open World Forum and CloudCamp Paris:

http://www.cloudcomp.ch/2012/10/icclabs-present-on-occi-at-owfcloudcamp-paris/

On 16 Oct 2012, at 15:48, jean-pierre laisne jplai...@gmail.com wrote:

 There will be a talk about interoperability and portability (multi-cloud, 
 federation, etc.) with CompatibleOne and OCCI during the unconference 
 sessions on Wednesday 17 at 11:00. If you are interested, please join us.
 
 Thanks,
 Jean-Pierre.
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Can you add new vNICs to running VMs?

2012-10-17 Thread Heinonen, Johanna (NSN - FI/Espoo)
Hi,

Is it possible to add more vNICs to running VMs?
For example a VM is started with one interface that is connected to
subnet1. If there is a need to add an another interface/subnet, is it
possible to do this without shutting down/ starting again the VM?

Best regards,
Johanna
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Can you add new vNICs to running VMs?

2012-10-17 Thread heut2008
yes,with support of devices hotplug,we can add ports created by
quantum to the instances,and I am working on this feather .

2012/10/17 Heinonen, Johanna (NSN - FI/Espoo) johanna.heino...@nsn.com:
 Hi,

 Is it possible to add more vNICs to running VMs?

 For example a VM is started with one interface that is connected to subnet1.
 If there is a need to add an another interface/subnet, is it possible to do
 this without shutting down/ starting again the VM?

 Best regards,

 Johanna


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp




-- 
Yaguang Tang

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] 回复: Can you add new vNICs to running VMs?

2012-10-17 Thread hao.1.w...@gmail.com
Hi Yaguang,

Do you mean even for vCPU it is the same case? If it is, which project will 
implement the similar feature then?

Regards,
Howard

- Reply message -
发件人: heut2008 heut2...@gmail.com
收件人: Heinonen, Johanna (NSN - FI/Espoo) johanna.heino...@nsn.com
抄送: openstack@lists.launchpad.net
主题: [Openstack] Can you add new vNICs to running VMs?
日期: 周三, 10 月 17 日, 2012 年 9:59 下午


yes,with support of devices hotplug,we can add ports created by
quantum to the instances,and I am working on this feather .

2012/10/17 Heinonen, Johanna (NSN - FI/Espoo) johanna.heino...@nsn.com:
 Hi,

 Is it possible to add more vNICs to running VMs?

 For example a VM is started with one interface that is connected to subnet1.
 If there is a need to add an another interface/subnet, is it possible to do
 this without shutting down/ starting again the VM?

 Best regards,

 Johanna


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp




-- 
Yaguang Tang

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] CLA Contributor list is missing from the wiki?

2012-10-17 Thread Pádraig Brady

I tried to approve a few new CLA signers and noticed
that the http://wiki.openstack.org/Contributors page is missing.
Trying to restore a previous version resulted in an error.
Could someone with appropriate admin rights look into this?

thanks,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Quantum OVS Agent as service

2012-10-17 Thread Dan Wendlandt
It sounds like your config file is saying that you have an OVS bridge
named br-eth1, but it looks like you do not.

Did you intend to do this, or is this a config copied from somewhere else?

If you actually intended to use VLANs that send traffic out eth1, you
can do the following:

# create bridge
ovs-vsctl add-br br-eth1

# add eth1 as a port on this bridge
ovs-vsctl add-port br-eth1 eth1

dan

On Wed, Oct 17, 2012 at 1:53 AM, Srikanth Kumar Lingala
srikanthkumar.ling...@gmail.com wrote:
 Hi all,
 If we run Quantum OVS Agent as a service ... 'service
 quantum-plugin-openvswitch-agent start', it is unable to start the agent. It
 is throwing the following error to '/var/log/quantum/agent-ovs.log'

 Stderr: 'Device br-eth1 does not exist.\n'
 2012-10-16 12:48:10ERROR
 [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Bridge br-eth1 for
 physical network default does not exist

 if we run the agent manually like the following command, it is able to
 start:

 /usr/bin/quantum-openvswitch-agent --
 --config-file=/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
 --log-file=/var/log/quantum/agent-ovs.log
 --config-file=/etc/quantum/quantum.conf

 Please suggest.
 --
 
 Srikanth.



 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp




-- 
~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] CLA Contributor list is missing from the wiki?

2012-10-17 Thread Michael Still
On 10/17/2012 09:41 AM, Pádraig Brady wrote:
 I tried to approve a few new CLA signers and noticed
 that the http://wiki.openstack.org/Contributors page is missing.
 Trying to restore a previous version resulted in an error.
 Could someone with appropriate admin rights look into this?

The list appears to be back now.

Mikal


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Michael Barton
Overall I've never been super enthusiastic about erasure codes for
swift.  Figuring out which blocks are missing then re-assembling them
is a lot more difficult and expensive than what we do now.

But if you can come up with a good scheme for identifying missing
blocks and it doesn't double the amount of code in Swift, I'm sure we
all have use cases where we'd trade latency for disk usage.

- Michael


On Mon, Oct 15, 2012 at 7:36 PM, Duan, Jiangang jiangang.d...@intel.com wrote:
 Some of our customers are interested in Erasure code than tri-replicate to 
 save disk space.
 We propose a BP Light weight Erasure code framework for swift, which can be 
 found here https://blueprints.launchpad.net/swift/+spec/swift-ec
 The general idea is to have some daemon on storage node to do offline scan - 
 select code object with big enough size to do EC.

 Will glad to hear any feedback on this.


 -jiangang




 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] quantum create_net- creates network for non-existing tenant also.

2012-10-17 Thread gong yong sheng
It seems that you are using quantum cli v1.0, which is not supported in 
new quantum version v2.0 API.


Are u using quantum v1.x, right?
On 10/16/2012 10:39 AM, Raja Gajju wrote:

Hi All,

I am testing the Quantum CLIs in my set up. When I am creating a new 
network using create_net CLI

for an non-existing tenant, it is showing successful creation.
It should not do so. It is something unexpected. And again when I am 
doing list_nets it is showing

that network under the specified non-existing tenant.

Here is the snippet of the commans and their results :

---

~/devstack$ quantum create_net hjbddsbfikeoqjroijmfgvkmgfv fake_net
Created a new Virtual Network with ID: 
f3b1d829-009d-45b5-9108-01722539135e

for Tenant: hjbddsbfikeoqjroijmfgvkmgfv

~/devstack$ quantum list_nets hjbddsbfikeoqjroijmfgvkmgfv
Virtual Networks for Tenant hjbddsbfikeoqjroijmfgvkmgfv
Network ID: f3b1d829-009d-45b5-9108-01722539135e

---

Any effort to explain this inconsistency issue will be highly 
appreciated. Many thanks in advance.



Thanks and Regards,
Girija Sharan Singh


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Duan, Jiangang
This requests sound reasonable.
This work may be worked together for the replicator-optimization patch. We will 
work on the patch.
Any other feedback?

-jiangang

-Original Message-
From: palr...@gmail.com [mailto:palr...@gmail.com] On Behalf Of Michael Barton
Sent: Thursday, October 18, 2012 2:29 AM
To: Duan, Jiangang
Cc: openstack@lists.launchpad.net; Zhou, Yuan
Subject: Re: [Openstack] ask for comments - Light weight Erasure code framework 
for swift

Overall I've never been super enthusiastic about erasure codes for swift.  
Figuring out which blocks are missing then re-assembling them is a lot more 
difficult and expensive than what we do now.

But if you can come up with a good scheme for identifying missing blocks and it 
doesn't double the amount of code in Swift, I'm sure we all have use cases 
where we'd trade latency for disk usage.

- Michael


On Mon, Oct 15, 2012 at 7:36 PM, Duan, Jiangang jiangang.d...@intel.com wrote:
 Some of our customers are interested in Erasure code than tri-replicate to 
 save disk space.
 We propose a BP Light weight Erasure code framework for swift, which 
 can be found here 
 https://blueprints.launchpad.net/swift/+spec/swift-ec
 The general idea is to have some daemon on storage node to do offline scan - 
 select code object with big enough size to do EC.

 Will glad to hear any feedback on this.


 -jiangang




 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Samuel Merritt

On 10/15/12 5:36 PM, Duan, Jiangang wrote:

Some of our customers are interested in Erasure code than tri-replicate to save 
disk space.
We propose a BP Light weight Erasure code framework for swift, which can be 
found here https://blueprints.launchpad.net/swift/+spec/swift-ec
The general idea is to have some daemon on storage node to do offline scan - 
select code object with big enough size to do EC.

Will glad to hear any feedback on this.


Here, in no particular order, are some thoughts I have.

- Object blocks (both data blocks and parity blocks) will need to be 
marked somehow so that 3 replicas of each block aren't kept. This is a 
pretty fundamental change to Swift; up until now, all objects are 
treated the same. It's essentially introducing the notion of tiered 
storage into Swift.


- Who's responsible for ensuring the presence of all the blocks? That 
is, assume you have an object that's been split into ten data blocks 
(D1, D2, ..., D10) and 2 parity blocks (P1, P2). The drive with D7 on it 
dies. Which replicator(s) is(are) responsible for rebuilding D7 and 
storing it on a handoff node?


If you have the replicators on each block's machine checking for 
failures, then you'll wind up with more people checking each replica. 
Here, it would be 11 replicators ensuring that each block is present. 
Compare that to the full-replication case, where there are 2 replicators 
checking on it. That's going to result in more traffic on the internal 
network.


- There will need to be throttles on the transformation daemons (replica 
- EC and vice versa), as that's very IO intensive. If a big bunch of 
data is uploaded at one time and then not accessed (think large 
backups), then that could be a ticking time bomb for my cluster 
performance. After those objects become cold, the transformation 
daemons will thrash my disks and network turning them into EC-type objects.


- Does this open up a Swift cluster to a DoS attack? If my objects are 
stored w/EC, then can someone go through and request a few bytes from 
each object in my cluster a few times and cause all my objects to get 
hot? Under the proposed scheme, this would turn my objects from 
EC-storage to replica-storage, filling up my disks and killing my 
cluster. To mitigate that, I'd have to keep enough disk around to hold 3 
replicas of everything, and at that point, I may as well just keep the 3 
replicas.


- Another thought for a resource-consumption attack: can someone slowly 
walk my objects and make a large fraction (say, 5%) of them hot each 
day? That seems like it would make the transformation daemons run at 
maximum capacity all the time trying to keep up.


- Retrieval of EC-stored objects becomes more failure-prone. With 
replica-stored objects, 1 out of 3 object servers has to be available 
for a GET request to work. With EC-stored objects and a 10:2 coding, 10 
out of 12 object servers have to be available. That makes network 
partitions much worse for data availability.


- EC-storage is at odds with geographic replication. Of course, Swift 
supports neither one today. However, with geographic replication, one 
wants to have a local replica of each each object in each geographic 
region, which results in more copies for lower latency. With EC-storage, 
less data is stored. When they're combined, the result is a whole lot of 
traffic across slow, expensive WAN links.


- Recombining EC-stored object chunks is going to chew up a ton more CPU 
on either the object or proxy servers, depending on which one does it. 
If the proxy, then it'll add more to an already CPU-heavy workload. If 
the object server, then it'll make using big storage boxes less 
practical (like one of the 48-drives-in-4U servers one can buy).


- Can one change the EC-coding level? That is, if I'm using 10:2 coding 
(so each object turns into 10 data blocks and 2 parity blocks), can I 
change that later? Will that have massive performance impacts on my 
cluster as more data blocks are computed?


It may be that this is like changing the replica count, and the answer 
is yes, but your cluster will thrash for a long time after you do it.


- Where's the original checksum stored? Clearly, each block will have 
its own checksum for the auditors to use. However, if a client issues a 
request like HEAD /a/c/o, that'll contain the checksum of the original 
file. Does that live somewhere, or will the proxy have to read all the 
bytes and determine the checksum?


- I wonder what effect this will have on internal-network traffic. With 
a replica-stored object, the proxy opens one connection to an object 
server, sends a request, gets a response, and streams the bytes out to 
the client.


With an EC-stored object, the proxy has to open connections to, say, 10 
different object servers. Further, if one of the data blocks is 
unavailable (say data block 5), then the proxy has to go ahead and 
re-request all the data blocks plus a parity block so that it can fill 
in the 

Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Eugene Kirpichov
Hi Sam,

My five cents.

Using Fountain codes, which are also a class of EC, one can make all
the blocks equivalent in role (no separation into data and parity
blocks).
http://en.wikipedia.org/wiki/Fountain_code

They resolve a few of the issues that you raised, however they may
raise others - e.g. it's more difficult to determine how many blocks
you need to fetch to reconstruct the data.

On Wed, Oct 17, 2012 at 4:24 PM, Samuel Merritt s...@swiftstack.com wrote:
 On 10/15/12 5:36 PM, Duan, Jiangang wrote:

 Some of our customers are interested in Erasure code than tri-replicate to
 save disk space.
 We propose a BP Light weight Erasure code framework for swift, which can
 be found here https://blueprints.launchpad.net/swift/+spec/swift-ec
 The general idea is to have some daemon on storage node to do offline scan
 - select code object with big enough size to do EC.

 Will glad to hear any feedback on this.


 Here, in no particular order, are some thoughts I have.

 - Object blocks (both data blocks and parity blocks) will need to be marked
 somehow so that 3 replicas of each block aren't kept. This is a pretty
 fundamental change to Swift; up until now, all objects are treated the same.
 It's essentially introducing the notion of tiered storage into Swift.

 - Who's responsible for ensuring the presence of all the blocks? That is,
 assume you have an object that's been split into ten data blocks (D1, D2,
 ..., D10) and 2 parity blocks (P1, P2). The drive with D7 on it dies. Which
 replicator(s) is(are) responsible for rebuilding D7 and storing it on a
 handoff node?

 If you have the replicators on each block's machine checking for failures,
 then you'll wind up with more people checking each replica. Here, it would
 be 11 replicators ensuring that each block is present. Compare that to the
 full-replication case, where there are 2 replicators checking on it. That's
 going to result in more traffic on the internal network.

 - There will need to be throttles on the transformation daemons (replica -
 EC and vice versa), as that's very IO intensive. If a big bunch of data is
 uploaded at one time and then not accessed (think large backups), then that
 could be a ticking time bomb for my cluster performance. After those objects
 become cold, the transformation daemons will thrash my disks and network
 turning them into EC-type objects.

 - Does this open up a Swift cluster to a DoS attack? If my objects are
 stored w/EC, then can someone go through and request a few bytes from each
 object in my cluster a few times and cause all my objects to get hot?
 Under the proposed scheme, this would turn my objects from EC-storage to
 replica-storage, filling up my disks and killing my cluster. To mitigate
 that, I'd have to keep enough disk around to hold 3 replicas of everything,
 and at that point, I may as well just keep the 3 replicas.

 - Another thought for a resource-consumption attack: can someone slowly walk
 my objects and make a large fraction (say, 5%) of them hot each day? That
 seems like it would make the transformation daemons run at maximum capacity
 all the time trying to keep up.

 - Retrieval of EC-stored objects becomes more failure-prone. With
 replica-stored objects, 1 out of 3 object servers has to be available for a
 GET request to work. With EC-stored objects and a 10:2 coding, 10 out of 12
 object servers have to be available. That makes network partitions much
 worse for data availability.

 - EC-storage is at odds with geographic replication. Of course, Swift
 supports neither one today. However, with geographic replication, one wants
 to have a local replica of each each object in each geographic region, which
 results in more copies for lower latency. With EC-storage, less data is
 stored. When they're combined, the result is a whole lot of traffic across
 slow, expensive WAN links.

 - Recombining EC-stored object chunks is going to chew up a ton more CPU on
 either the object or proxy servers, depending on which one does it. If the
 proxy, then it'll add more to an already CPU-heavy workload. If the object
 server, then it'll make using big storage boxes less practical (like one of
 the 48-drives-in-4U servers one can buy).

 - Can one change the EC-coding level? That is, if I'm using 10:2 coding (so
 each object turns into 10 data blocks and 2 parity blocks), can I change
 that later? Will that have massive performance impacts on my cluster as more
 data blocks are computed?

 It may be that this is like changing the replica count, and the answer is
 yes, but your cluster will thrash for a long time after you do it.

 - Where's the original checksum stored? Clearly, each block will have its
 own checksum for the auditors to use. However, if a client issues a request
 like HEAD /a/c/o, that'll contain the checksum of the original file. Does
 that live somewhere, or will the proxy have to read all the bytes and
 determine the checksum?

 - I wonder what effect this will have on 

Re: [Openstack] ask for comments - Light weight Erasure code framework for swift

2012-10-17 Thread Hao Wang
Hi Sam,

I got some thoughts from your mail. EC is more useful for a centered
storage solution but not for a distributed one. It will bring a heavy load
on internal network traffic.

Actually from network performance's point of view, the 3 copies are also
sort of the result of compromise. There should be a way to combine them
together and tune the parameters under different scenarios. They, however,
would bring different reliability and performance.

Regards,
Howard
On Thu, Oct 18, 2012 at 7:30 AM, Eugene Kirpichov ekirpic...@gmail.comwrote:

 Hi Sam,

 My five cents.

 Using Fountain codes, which are also a class of EC, one can make all
 the blocks equivalent in role (no separation into data and parity
 blocks).
 http://en.wikipedia.org/wiki/Fountain_code

 They resolve a few of the issues that you raised, however they may
 raise others - e.g. it's more difficult to determine how many blocks
 you need to fetch to reconstruct the data.

 On Wed, Oct 17, 2012 at 4:24 PM, Samuel Merritt s...@swiftstack.com
 wrote:
  On 10/15/12 5:36 PM, Duan, Jiangang wrote:
 
  Some of our customers are interested in Erasure code than tri-replicate
 to
  save disk space.
  We propose a BP Light weight Erasure code framework for swift, which
 can
  be found here https://blueprints.launchpad.net/swift/+spec/swift-ec
  The general idea is to have some daemon on storage node to do offline
 scan
  - select code object with big enough size to do EC.
 
  Will glad to hear any feedback on this.
 
 
  Here, in no particular order, are some thoughts I have.
 
  - Object blocks (both data blocks and parity blocks) will need to be
 marked
  somehow so that 3 replicas of each block aren't kept. This is a pretty
  fundamental change to Swift; up until now, all objects are treated the
 same.
  It's essentially introducing the notion of tiered storage into Swift.
 
  - Who's responsible for ensuring the presence of all the blocks? That is,
  assume you have an object that's been split into ten data blocks (D1, D2,
  ..., D10) and 2 parity blocks (P1, P2). The drive with D7 on it dies.
 Which
  replicator(s) is(are) responsible for rebuilding D7 and storing it on a
  handoff node?
 
  If you have the replicators on each block's machine checking for
 failures,
  then you'll wind up with more people checking each replica. Here, it
 would
  be 11 replicators ensuring that each block is present. Compare that to
 the
  full-replication case, where there are 2 replicators checking on it.
 That's
  going to result in more traffic on the internal network.
 
  - There will need to be throttles on the transformation daemons (replica
 -
  EC and vice versa), as that's very IO intensive. If a big bunch of data
 is
  uploaded at one time and then not accessed (think large backups), then
 that
  could be a ticking time bomb for my cluster performance. After those
 objects
  become cold, the transformation daemons will thrash my disks and
 network
  turning them into EC-type objects.
 
  - Does this open up a Swift cluster to a DoS attack? If my objects are
  stored w/EC, then can someone go through and request a few bytes from
 each
  object in my cluster a few times and cause all my objects to get hot?
  Under the proposed scheme, this would turn my objects from EC-storage to
  replica-storage, filling up my disks and killing my cluster. To mitigate
  that, I'd have to keep enough disk around to hold 3 replicas of
 everything,
  and at that point, I may as well just keep the 3 replicas.
 
  - Another thought for a resource-consumption attack: can someone slowly
 walk
  my objects and make a large fraction (say, 5%) of them hot each day? That
  seems like it would make the transformation daemons run at maximum
 capacity
  all the time trying to keep up.
 
  - Retrieval of EC-stored objects becomes more failure-prone. With
  replica-stored objects, 1 out of 3 object servers has to be available
 for a
  GET request to work. With EC-stored objects and a 10:2 coding, 10 out of
 12
  object servers have to be available. That makes network partitions much
  worse for data availability.
 
  - EC-storage is at odds with geographic replication. Of course, Swift
  supports neither one today. However, with geographic replication, one
 wants
  to have a local replica of each each object in each geographic region,
 which
  results in more copies for lower latency. With EC-storage, less data is
  stored. When they're combined, the result is a whole lot of traffic
 across
  slow, expensive WAN links.
 
  - Recombining EC-stored object chunks is going to chew up a ton more CPU
 on
  either the object or proxy servers, depending on which one does it. If
 the
  proxy, then it'll add more to an already CPU-heavy workload. If the
 object
  server, then it'll make using big storage boxes less practical (like one
 of
  the 48-drives-in-4U servers one can buy).
 
  - Can one change the EC-coding level? That is, if I'm using 10:2 coding
 (so
  each object turns into 10 data 

Re: [Openstack] Quantum OVS Agent as service

2012-10-17 Thread Srikanth Kumar Lingala
Hi Dan,
I am following Folsom Installation guide provided by Emilien Macchi. In
that document it is mentioned to create a bridge 'br-int' attached to
'eth1' through ovs. And also it suggests to assign this '*br-int*' bridge
as '*integration_bridge*' and '*br-eth1*' as '*bridge_mappings*' in
'/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini' like the
following:

*integration_bridge = br-int*
*bridge_mappings = default:br-eth1*
*
*
Please clarify me, is it necessary to enable 'bridge_mappings' field.
*
*
Regards,
Srikanth.

On Wed, Oct 17, 2012 at 10:13 PM, Dan Wendlandt d...@nicira.com wrote:

 It sounds like your config file is saying that you have an OVS bridge
 named br-eth1, but it looks like you do not.

 Did you intend to do this, or is this a config copied from somewhere else?

 If you actually intended to use VLANs that send traffic out eth1, you
 can do the following:

 # create bridge
 ovs-vsctl add-br br-eth1

 # add eth1 as a port on this bridge
 ovs-vsctl add-port br-eth1 eth1

 dan

 On Wed, Oct 17, 2012 at 1:53 AM, Srikanth Kumar Lingala
 srikanthkumar.ling...@gmail.com wrote:
  Hi all,
  If we run Quantum OVS Agent as a service ... 'service
  quantum-plugin-openvswitch-agent start', it is unable to start the
 agent. It
  is throwing the following error to '/var/log/quantum/agent-ovs.log'
 
  Stderr: 'Device br-eth1 does not exist.\n'
  2012-10-16 12:48:10ERROR
  [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Bridge br-eth1 for
  physical network default does not exist
 
  if we run the agent manually like the following command, it is able to
  start:
 
  /usr/bin/quantum-openvswitch-agent --
  --config-file=/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
  --log-file=/var/log/quantum/agent-ovs.log
  --config-file=/etc/quantum/quantum.conf
 
  Please suggest.
  --
  
  Srikanth.
 
 
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
 



 --
 ~~~
 Dan Wendlandt
 Nicira, Inc: www.nicira.com
 twitter: danwendlandt
 ~~~

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack-qa-team] Blueprint for Compute Admin API tests

2012-10-17 Thread Daryl Walleck
Hi everyone,

After our discussions earlier today about being more vocal about what's being 
worked on, I wanted to raise awareness of what I'm currently working to avoid 
duplication of efforts. This is the first blueprint I've submitted, so if 
there's any additional detail that I'm missing, I'm always glad to get feedback.

https://blueprints.launchpad.net/tempest/+spec/add-compute-admin-tests

Daryl
-- 
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-qa-team
More help   : https://help.launchpad.net/ListHelp