[Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Marc-Aurèle Brothier
Hi everyone, I was wondering how many of you are running CloudStack with a cluster of management servers. I would think most of you, but it would be nice to hear everyone voices. And do you get hosts going over their capacity limits? We discovered that during the VM allocation, if you get a lot o

Re: Need help for first time user

2017-12-18 Thread Vivek Kumar
Hello Sounder, 1- What exactly is your requirement ? Are you going to deploy public or private cloud. 2- what hypervisor do want to use, ( as per ur trail mail you are trying to configure with Xenserver, you can use either XenServer 6.5 SP1 or XenServer 7.0 ). 3- If you are planning to build Pr

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Daan Hoogland
Are you proposing to add zookeeper as an optional requirement, Marc-Aurèle? or just curator? and what is the decision mech of including it or not? On Mon, Dec 18, 2017 at 9:33 AM, Marc-Aurèle Brothier wrote: > Hi everyone, > > I was wondering how many of you are running CloudStack with a cluster

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Ivan Kudryavtsev
Hello, Marc-Aurele, I strongly believe that all mysql locks should be removed in favour of truly DLM solution like Zookeeper. The performance of 3node ZK ensemble should be enough to hold up to 1000-2000 locks per second and it helps to move to truly clustered MySQL like galera without single maste

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
How hard is it to configure Zookeeper and get everything up and running? BTW: what zookeeper would be managing? CloudStack management servers or MySQL nodes? On Mon, Dec 18, 2017 at 7:13 AM, Ivan Kudryavtsev wrote: > Hello, Marc-Aurele, I strongly believe that all mysql locks should be > removed

Re: Things to consider in replacing secondary storage

2017-12-18 Thread christian.kirmse
Hi Thura, afaik, yes. But please keep in mind that I’m not at ACS guru, so there might be a way which I’m not aware of. For us it was a quite painless migration which was the most important thing. Regards Christian > On 6. Dec 2017, at 06:07, Thura, Minn Minn wrote: > > Hi Christian, > > Appr

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Ivan Kudryavtsev
Rafael, - It's easy to configure and run ZK either in single node or cluster - zookeeper should replace mysql locking mechanism used inside ACS code (places where ACS locks tables or rows). I don't think from the other size, that moving from MySQL locks to ZK locks is easy and light and (even imp

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
so, how does that work? I mean, instead of opening a transaction with the database and executing locks, what do we need to do in the code? On Mon, Dec 18, 2017 at 7:24 AM, Ivan Kudryavtsev wrote: > Rafael, > > - It's easy to configure and run ZK either in single node or cluster > - zookeeper sho

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Marc-Aurèle Brothier
We added ZK lock for fix this issue but we will remove all current locks in ZK in favor of ZK one. The ZK lock is already encapsulated in a project with an interface, but more work should be done to have a proper interface for locks which could be implemented with the "tool" you want, either a DB l

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
Do we have framework to do this kind of looking in ZK? I mean, you said " create a new InterProcessSemaphoreMutex which handles the locking mechanism.". This feels that we would have to continue opening and closing this transaction manually, which is what causes a lot of our headaches with transact

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Marc-Aurèle Brothier
@rafael, yes there is a framework (curator), it's the link I posted in my first message: https://curator.apache.org/curator-recipes/shared-lock.html This framework helps handling all the complexity of ZK. The ZK client stays connected all the time (as the DB connection pool), and only one connecti

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
I did not check the link before. Sorry about that. Reading some of the pages there, I see curator more like a client library such as MySQL JDBC client. When I mentioned framework, I was looking for something like Spring-data. So, we could simply rely on the framework to manage connections and tra

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Marc-Aurèle Brothier
I understand your point, but there isn't any "transaction" in ZK. The transaction and commit stuff are really for DB and not part of ZK. All entries (if you start writing data in some nodes) are versioned. For example you could enforce that to overwrite a node value you must submit the node data ha

Re: PKIX path building failed

2017-12-18 Thread Makrand
Hello Jagdish, What is source URL of template you're trying to register? And when exactly this error is appearing? Immediately after you try to register a template? -- Makrand On Fri, Dec 15, 2017 at 1:01 PM, Jagdish Patil wrote: > Hey Guys, > > I am facing the following issue with this con

Re: Need help for first time user

2017-12-18 Thread soundar rajan
Please find my comments inline On Mon, Dec 18, 2017 at 2:31 PM, Vivek Kumar wrote: > Hello Sounder, > > 1- What exactly is your requirement ? Are you going to deploy public or > private cloud. > Private cloud > 2- what hypervisor do want to use, ( as per ur trail mail you are trying > t

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
So, we would need to change every piece of code that opens and uses connections and transactions to change to ZK model? I mean, to direct the flow to ZK. On Mon, Dec 18, 2017 at 8:55 AM, Marc-Aurèle Brothier wrote: > I understand your point, but there isn't any "transaction" in ZK. The > transac

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Marc-Aurèle Brothier
Sorry about the confusion. It's not going to replace the DB transactions in the DAO way. Today we can say that there are 2 types of locks in CS, either a pure transaction one, with the select for update which locks a row for any operation by other threads, or a more programmatic one with the op_loc

Re: [Discuss] Management cluster / Zookeeper holding locks

2017-12-18 Thread Rafael Weingärtner
Now, yes! Thanks for the clarification. On Mon, Dec 18, 2017 at 11:16 AM, Marc-Aurèle Brothier wrote: > Sorry about the confusion. It's not going to replace the DB transactions in > the DAO way. Today we can say that there are 2 types of locks in CS, either > a pure transaction one, with the sel

Re: Change IP of secodary Storage

2017-12-18 Thread Dag Sonstebo
Hi Benjamin, You would struggle to do this fully live (which I guess is what you mean by “active”). At a guess you would have to: - disable zone - delete your SSVM - ensure no secondary storage operations are in play (e.g.new VMs being created from new templates) - umount secondary storage on b

Re: [UPDATE] Debian 9 "stretch" systemvmtemplate for master

2017-12-18 Thread Rohit Yadav
All, Thanks for your feedback. We're reaching close to completion now. All smoketests are now passing on KVM, XenServer and VMWare now. There are however few intermittent failures on VMware being looked into. The rVR smoketests failures on VMware have been fixed as well. The systemvmtemplate

Re: [UPDATE] Debian 9 "stretch" systemvmtemplate for master

2017-12-18 Thread Rohit Yadav
Hi Wido, Thanks. I've verified, virtio-scsi seems to work for me. Qemu guest agent also works, I was able to write poc code to get rid of patchviasocket.py as well. Can you help review and test the PR? Regards. From: Wido den Hollander Sent: Saturday, December

RE: Not able to create the Secondary Storage and Console proxy Vms

2017-12-18 Thread Dickson Lam (dilam)
Hi Glenn: Do you have chance to look at the log file that I upload on pastebin? Regards Dickson -Original Message- From: Dickson Lam (dilam) Sent: Friday, December 01, 2017 11:37 AM To: users@cloudstack.apache.org Subject: RE: Not able to create the Secondary Storage and Console proxy V

Re: PKIX path building failed

2017-12-18 Thread Jagdish Patil
Thanks for responding Makrand, This error is occurring as soon as system VM gets downloaded and started and tries to download the CentOS5.5-KVM(No GUI) template. And I have also tried giving other HTTP URLs for registering but still, the PKIX path building error pops up. I have attached a part of

Re: PKIX path building failed

2017-12-18 Thread Makrand
Hi, Assuming your SSVM is up and running, have you tried to register any another template? Did it work? -- Makrand On Tue, Dec 19, 2017 at 11:38 AM, Jagdish Patil wrote: > Thanks for responding Makrand, > > This error is occurring as soon as system VM gets downloaded and started > and tries t