Re: [4.11] Testing New "Ability to disable primary storage to secondary storage backups for snapshots" Feature

[Özhan Rüzgar Karaman]

Hi Rohit;
We have tested the feature and it works great.

Thanks
Özhan

On Fri, Jan 26, 2018 at 3:08 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Simon,
>
>
> I'm not sure, we can ask the PR author - Harika Punna, and reviewers -
> Koushik, Mike, Suresh -- please advise?
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Simon Weller <swel...@ena.com.INVALID>
> Sent: Thursday, January 25, 2018 8:55:37 PM
> To: dev@cloudstack.apache.org
> Cc: Nathan Johnson
> Subject: Re: [4.11] Testing New "Ability to disable primary storage to
> secondary storage backups for snapshots" Feature
>
> I'm not sure why this was removed from public.  We haven't tested the
> feature set since the below PR was merged.
>
>
> Nathan,
>
>
> Thoughts on this?
>
>
>
>
> 
> From: Rohit Yadav <rohit.ya...@shapeblue.com>
> Sent: Thursday, January 25, 2018 8:20 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [4.11] Testing New "Ability to disable primary storage to
> secondary storage backups for snapshots" Feature
>
> Hi Ozhan,
>
>
> The global setting was removed in following PR, however you can get the
> feature/ability via the API:
>
> https://github.com/apache/cloudstack/pull/2081
>
>
> Also see:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> Separate+creation+and+backup+operations+for+a+volume+snapshot
> Separate creation and backup operations for a volume ...<
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> Separate+creation+and+backup+operations+for+a+volume+snapshot>
> cwiki.apache.org
> DB Changes. NA UI Flow. A checkbox will be added to the "Create Volume
> Snapshot" dialog box, which when checked, snapshot and copy operations will
> be separated and if ...
>
>
>
>
>
> Please test and share if the changes introduce by above are acceptable, or
> you think is blocker(ish)?
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
> [https://cloudstack.apache.org/images/monkey-144.png] ps://cloudstack.apache.org/>
>
> Apache CloudStack: Open Source Cloud Computing<https://cloudstack.
> apache.org/>
> cloudstack.apache.org
> CloudStack is open source cloud computing software for creating, managing,
> and deploying infrastructure cloud services
>
>
>
>
>
>
> 
> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> Sent: Wednesday, January 24, 2018 4:50:41 PM
> To: dev@cloudstack.apache.org
> Subject: [4.11] Testing New "Ability to disable primary storage to
> secondary storage backups for snapshots" Feature
>
> Hi;
> I plan to test "Ability to disable primary storage to secondary storage
> backups for snapshots" feature on 4.11 rc1 release. For this test i think i
> need to update "snapshot.backup.rightafter" parameter from global settings
> but i could not find the parameter on global configuration there.
>
> Is this normal?
>
> Thanks
> Özhan
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
> [http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<
> http://www.shapeblue.com/>
>
> Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com<http://www.shapeblue.com>
> Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is a
> framework developed by ShapeBlue to deliver the rapid deployment of a
> standardised ...
>
>
>
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi Rohit;
I made the test again on a fresh VR and your solution fixed the issue.

Thanks
Özhan

On Wed, Jan 24, 2018 at 11:26 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Hi Ozhan,
>
>
> With my fix, whenever dnsmasq needs to be restarted the leases file will
> be removed, and dnsmasq will be restarted whenever /etc/dhcphosts.txt or
> /etc/dnsmasq.d/cloud.conf change, otherwise it will be reloaded.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> Sent: Wednesday, January 24, 2018 7:40:18 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [4.11] VR Problem on Releasing Expunged Instance IP from
> dnsmasq.leases file
>
> Hi Rohit;
> Today i am short in time but tomorrow i will create a new network and test
> your fix over this fresh VR.
>
> I have one more question, with your current code fix do we still continue
> reloading dnsmasq on normal operations and we only flush leases on
> start/restart operations or after this fix we start to use restart the
> dnsmasq instead of reloading it on all our operations? Thanks for all your
> help.
>
> Özhan
>
> On Tue, Jan 23, 2018 at 11:45 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
> wrote:
>
> > Hi Ozhan,
> >
> >
> > During the 4.11-systemvmtemplate migration work (to debian9 based
> > template), I refactored the code to reload dnsmasq instead of restart it.
> > Based on your feedback, I've created a fix that will remove the leases
> file
> > everytime dnsmasq needs to be restarted.
> >
> >
> > Can you help test/verify it:
> >
> > https://github.com/apache/cloudstack/pull/2427/files
> >
> >
> >
> > - Rohit
> >
> > <https://cloudstack.apache.org>
> >
> >
> >
> > 
> > From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> > Sent: Tuesday, January 23, 2018 1:01:16 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [4.11] VR Problem on Releasing Expunged Instance IP from
> > dnsmasq.leases file
> >
> > Hi;
> > We made some more tests to find the root cause of the problem. The
> problem
> > occur because we crashed(power off) VR for a 4.11 HA test. After
> rebooting
> > VR, active VM's dhcp lease datas are stucked
> > in /var/lib/misc/dnsmasq.leases file and this creates problems.
> >
> > Our solution is to clear/flush dnsmasq file using dhcp_release command.
> > After dhcp_release command problem disappears for new VMs with same ip
> > address.
> >
> > So i think we need to add some code on VR startup to flush dnsmasq.leases
> > file before starting dnsmasq.
> >
> > Thanks
> > Özhan
> >
> > On Mon, Jan 22, 2018 at 1:13 PM, Özhan Rüzgar Karaman <
> > oruzgarkara...@gmail.com> wrote:
> >
> > > Hi Ivan;
> > > I am not sure PR 2393 directly points to my findings, i only tested
> this
> > > scenario on 4.11rc1.
> > >
> > > I am not a developer so i will not submit a fix, i am only testing
> 4.11rc
> > > because its a LTS release and its quality is very important.
> > >
> > > Please check the issue on your environment, all details and issue
> > > reproducing steps are written on my first email, but if you want i will
> > > create a PR only to report & record the situation, just send me message
> > if
> > > you want.
> > >
> > > Thanks
> > > Özhan
> > >
> > > On Mon, Jan 22, 2018 at 1:01 PM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com> wrote:
> > >
> > >> Reopen issue, do a PR with fix if you can, could it be that VR doesn't
> > >> have
> > >> patched code? Also, describe testing scenario, I'll try to look at it
> in
> > >> my
> > >> patched 4.10.
> > >>
> > >> 22 янв. 2018 г. 16:52 пользователь "Özhan Rüzgar Karaman" <
> > >> oruzgarkara...@gmail.com> написал:
> > >>
> > >> > Hi Ivan;
> > >> > I checked 2 PR's and they are exist on 4.11rc1 but issue still
> exists
> > >> on my
> > >> > environment. When a new vm uses IP from old expunged vm then leases
> > file
> > >> > creates problem. Please check the logs that i submitted on first
> > email,
> > >> > issue is clear there and in my opinion it still exists on 4.11rc1.
> > >> >
> > >> > By the way 2393 

[4.11] Testing New "Ability to disable primary storage to secondary storage backups for snapshots" Feature

[Özhan Rüzgar Karaman]

Hi;
I plan to test "Ability to disable primary storage to secondary storage
backups for snapshots" feature on 4.11 rc1 release. For this test i think i
need to update "snapshot.backup.rightafter" parameter from global settings
but i could not find the parameter on global configuration there.

Is this normal?

Thanks
Özhan

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi Rohit;
Today i am short in time but tomorrow i will create a new network and test
your fix over this fresh VR.

I have one more question, with your current code fix do we still continue
reloading dnsmasq on normal operations and we only flush leases on
start/restart operations or after this fix we start to use restart the
dnsmasq instead of reloading it on all our operations? Thanks for all your
help.

Özhan

On Tue, Jan 23, 2018 at 11:45 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Hi Ozhan,
>
>
> During the 4.11-systemvmtemplate migration work (to debian9 based
> template), I refactored the code to reload dnsmasq instead of restart it.
> Based on your feedback, I've created a fix that will remove the leases file
> everytime dnsmasq needs to be restarted.
>
>
> Can you help test/verify it:
>
> https://github.com/apache/cloudstack/pull/2427/files
>
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> Sent: Tuesday, January 23, 2018 1:01:16 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [4.11] VR Problem on Releasing Expunged Instance IP from
> dnsmasq.leases file
>
> Hi;
> We made some more tests to find the root cause of the problem. The problem
> occur because we crashed(power off) VR for a 4.11 HA test. After rebooting
> VR, active VM's dhcp lease datas are stucked
> in /var/lib/misc/dnsmasq.leases file and this creates problems.
>
> Our solution is to clear/flush dnsmasq file using dhcp_release command.
> After dhcp_release command problem disappears for new VMs with same ip
> address.
>
> So i think we need to add some code on VR startup to flush dnsmasq.leases
> file before starting dnsmasq.
>
> Thanks
> Özhan
>
> On Mon, Jan 22, 2018 at 1:13 PM, Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com> wrote:
>
> > Hi Ivan;
> > I am not sure PR 2393 directly points to my findings, i only tested this
> > scenario on 4.11rc1.
> >
> > I am not a developer so i will not submit a fix, i am only testing 4.11rc
> > because its a LTS release and its quality is very important.
> >
> > Please check the issue on your environment, all details and issue
> > reproducing steps are written on my first email, but if you want i will
> > create a PR only to report & record the situation, just send me message
> if
> > you want.
> >
> > Thanks
> > Özhan
> >
> > On Mon, Jan 22, 2018 at 1:01 PM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com> wrote:
> >
> >> Reopen issue, do a PR with fix if you can, could it be that VR doesn't
> >> have
> >> patched code? Also, describe testing scenario, I'll try to look at it in
> >> my
> >> patched 4.10.
> >>
> >> 22 янв. 2018 г. 16:52 пользователь "Özhan Rüzgar Karaman" <
> >> oruzgarkara...@gmail.com> написал:
> >>
> >> > Hi Ivan;
> >> > I checked 2 PR's and they are exist on 4.11rc1 but issue still exists
> >> on my
> >> > environment. When a new vm uses IP from old expunged vm then leases
> file
> >> > creates problem. Please check the logs that i submitted on first
> email,
> >> > issue is clear there and in my opinion it still exists on 4.11rc1.
> >> >
> >> > By the way 2393 is about VM's IP Changing progress, maybe it does not
> >> cover
> >> > my scenario.
> >> >
> >> > Thanks
> >> > Özhan
> >> >
> >> > On Mon, Jan 22, 2018 at 12:40 PM, Özhan Rüzgar Karaman <
> >> > oruzgarkara...@gmail.com> wrote:
> >> >
> >> > > Hi Ivan;
> >> > > I made several tests with same scenario on 4.11rc1 and got same
> >> results,
> >> > > did your 2 PR's currently exists on 4.11 rc1 in which i am testing
> or
> >> it
> >> > > will exist on future rc2? If they exists on 4.11rc1 then we have a
> >> > problem
> >> > >
> >> > > Thanks
> >> > > Özhan
> >> > >
> >> > > On Mon, Jan 22, 2018 at 12:32 PM, Ivan Kudryavtsev <
> >> > > kudryavtsev...@bw-sw.com> wrote:
> >> > >
> >> > >> Hi, Ozhan. MACs are not removed upon vm removal, but they are
> >> overriden
> >> > >> upon vm creation with same ip (or same hostname). It should work
> >> fine,
> >> > >> 4.10, 4.11 received 2 PRs to fix several possible bugs. I tested
> the
> >> > 

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi;
We made some more tests to find the root cause of the problem. The problem
occur because we crashed(power off) VR for a 4.11 HA test. After rebooting
VR, active VM's dhcp lease datas are stucked
in /var/lib/misc/dnsmasq.leases file and this creates problems.

Our solution is to clear/flush dnsmasq file using dhcp_release command.
After dhcp_release command problem disappears for new VMs with same ip
address.

So i think we need to add some code on VR startup to flush dnsmasq.leases
file before starting dnsmasq.

Thanks
Özhan

On Mon, Jan 22, 2018 at 1:13 PM, Özhan Rüzgar Karaman <
oruzgarkara...@gmail.com> wrote:

> Hi Ivan;
> I am not sure PR 2393 directly points to my findings, i only tested this
> scenario on 4.11rc1.
>
> I am not a developer so i will not submit a fix, i am only testing 4.11rc
> because its a LTS release and its quality is very important.
>
> Please check the issue on your environment, all details and issue
> reproducing steps are written on my first email, but if you want i will
> create a PR only to report & record the situation, just send me message if
> you want.
>
> Thanks
> Özhan
>
> On Mon, Jan 22, 2018 at 1:01 PM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com> wrote:
>
>> Reopen issue, do a PR with fix if you can, could it be that VR doesn't
>> have
>> patched code? Also, describe testing scenario, I'll try to look at it in
>> my
>> patched 4.10.
>>
>> 22 янв. 2018 г. 16:52 пользователь "Özhan Rüzgar Karaman" <
>> oruzgarkara...@gmail.com> написал:
>>
>> > Hi Ivan;
>> > I checked 2 PR's and they are exist on 4.11rc1 but issue still exists
>> on my
>> > environment. When a new vm uses IP from old expunged vm then leases file
>> > creates problem. Please check the logs that i submitted on first email,
>> > issue is clear there and in my opinion it still exists on 4.11rc1.
>> >
>> > By the way 2393 is about VM's IP Changing progress, maybe it does not
>> cover
>> > my scenario.
>> >
>> > Thanks
>> > Özhan
>> >
>> > On Mon, Jan 22, 2018 at 12:40 PM, Özhan Rüzgar Karaman <
>> > oruzgarkara...@gmail.com> wrote:
>> >
>> > > Hi Ivan;
>> > > I made several tests with same scenario on 4.11rc1 and got same
>> results,
>> > > did your 2 PR's currently exists on 4.11 rc1 in which i am testing or
>> it
>> > > will exist on future rc2? If they exists on 4.11rc1 then we have a
>> > problem
>> > >
>> > > Thanks
>> > > Özhan
>> > >
>> > > On Mon, Jan 22, 2018 at 12:32 PM, Ivan Kudryavtsev <
>> > > kudryavtsev...@bw-sw.com> wrote:
>> > >
>> > >> Hi, Ozhan. MACs are not removed upon vm removal, but they are
>> overriden
>> > >> upon vm creation with same ip (or same hostname). It should work
>> fine,
>> > >> 4.10, 4.11 received 2 PRs to fix several possible bugs. I tested the
>> > case
>> > >> when IP is reused.
>> > >>
>> > >> 22 янв. 2018 г. 16:07 пользователь "Özhan Rüzgar Karaman" <
>> > >> oruzgarkara...@gmail.com> написал:
>> > >>
>> > >> Hi;
>> > >> Today we noticed that one of our new provisioned instance did not
>> get IP
>> > >> from VR. When we dig into the issue we find that one different mac is
>> > >> written in dnsmasq.leases file holds new instances IP address.
>> > >>
>> > >> We checked this mac address from db and we noticed that this mac is
>> used
>> > >> for old expunged instance.
>> > >>
>> > >> So from this point we realised that when we destroy an instance its
>> mac
>> > >> did
>> > >> not removed from dnsmasq.leases file so if we use this ip for a new
>> > >> instance then we have a problem, our instance could not get IP from
>> VR.
>> > >>
>> > >> We have one host on our lab environment and its Ubuntu 16.04.3 KVM.
>> > Today
>> > >> we made a HA test and we crashed the host so VR and SystemVM's are
>> > >> rebooted
>> > >> after we boot host back. I do not think this issue is related to VR
>> > reboot
>> > >> but i like to give information about our environment.
>> > >>
>> > >> We need to manage dnsmasq.leases file when we expunge an instance.
>> > >>
>> > >> Thanks
>> > >> Özhan
>>

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi Wido & Rohit;
I tested the patch and its ok, parsing works as expected, thanks for all
help.

Özhan

On Mon, Jan 22, 2018 at 11:06 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Thanks Wido, I'll review your patch.
>
>
>
> - Rohit
> <https://cloudstack.apache.org>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> @shapeblue
>
>
>
> --
> *From:* Wido den Hollander <w...@widodh.nl>
> *Sent:* Monday, January 22, 2018 8:08:33 AM
> *To:* dev@cloudstack.apache.org
> *Cc:* Özhan Rüzgar Karaman
>
> *Subject:* Re: [4.11] KVM Advanced Networking with SG Problem
>
>
>
> On 01/22/2018 07:35 AM, Wido den Hollander wrote:
> >
> >
> > On 01/21/2018 11:23 AM, Rohit Yadav wrote:
> >> Wido - Were you able to reproduce and fix the issue? Thanks.
> >>
> >
> > Still working on it! This weekend I was short on time and wasn't able to
> > fix it yet.
> >
> > Today (Mon) and tomorrow (Tue) my time is limited as well. Trying to fix
> > it asap.
>
> During my train ride this morning I wrote this patch:
> https://github.com/apache/cloudstack/pull/2418
>
> @ Özhan, could you test this patch? It's just a matter of replacing
> security_group.py on your Hypervisor.
>
> Thanks,
>
> Wido
>
> >
> > Wido
> >
> >>
> >>
> >> - Rohit
> >>
> >> <https://cloudstack.apache.org>
> >>
> >>
> >>
> >> 
> >> From: Wido den Hollander <w...@widodh.nl>
> >> Sent: Friday, January 19, 2018 10:12:45 PM
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [4.11] KVM Advanced Networking with SG Problem
> >>
> >>
> >>
> >> On 01/19/2018 02:03 PM, Özhan Rüzgar Karaman wrote:
> >>> Hi Daan;
> >>> Wido or others will write a fix, i am not a developer, i do not have
> >>> a fix,
> >>> i just only want to report it to make it official thats all :)
> >>>
> >>
> >> I'll look into this asap. The Python script should parse these rules
> >> properly and then it should be fixed.
> >>
> >> I hope to have a fix this weekend.
> >>
> >> Wido
> >>
> >>> Thanks
> >>> Özhan
> >>>
> >>> On Fri, Jan 19, 2018 at 3:59 PM, Daan Hoogland <
> daan.hoogl...@gmail.com>
> >>> wrote:
> >>>
> >>>> This is not a PR but a ticket, Özhan. Do you plan to make a pull
> >>>> request on
> >>>> github with your solution for it?
> >>>>
> >>>> On Fri, Jan 19, 2018 at 1:53 PM, Özhan Rüzgar Karaman <
> >>>> oruzgarkara...@gmail.com> wrote:
> >>>>
> >>>>> Hi Daan;
> >>>>> Wido is the previous PR's owner, he will check it. By the way i have
> >>>>> created a PR for this problem which is below:
> >>>>>
> >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-10242
> >>>>>
> >>>>> I select its priority as blocker, if its wrong developers will
> >>>>> update its
> >>>>> priority.
> >>>>>
> >>>>> Thanks
> >>>>> Özhan
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jan 19, 2018 at 3:25 PM, Daan Hoogland
> >>>>> <daan.hoogl...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Özhan, this is sure to break ipv6. can you make it use another
> >>>> delimiter?
> >>>>>>
> >>>>>> On Fri, Jan 19, 2018 at 1:12 PM, Özhan Rüzgar Karaman <
> >>>>>> oruzgarkara...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hi Rohit;
> >>>>>>> This is a fresh install of 4.11 rc1 and we have only ipv4 setup on
> >>>> our
> >>>>>> test
> >>>>>>> environment no ipv6 addresses, our VR's are new 4.11 rc1 system
> vms.
> >>>>> Our
> >>>>>>> workaround is 4 lines of code to convert ";" character to ":" on
> >>>>>>> security_group.py
> >>>>>>> code to make it operational for ipv4 addresses but i am sure it
> will
> >>>>>> break
> >>>>>>> Wido's "Add support for ipv6

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi Ivan;
I am not sure PR 2393 directly points to my findings, i only tested this
scenario on 4.11rc1.

I am not a developer so i will not submit a fix, i am only testing 4.11rc
because its a LTS release and its quality is very important.

Please check the issue on your environment, all details and issue
reproducing steps are written on my first email, but if you want i will
create a PR only to report & record the situation, just send me message if
you want.

Thanks
Özhan

On Mon, Jan 22, 2018 at 1:01 PM, Ivan Kudryavtsev <kudryavtsev...@bw-sw.com>
wrote:

> Reopen issue, do a PR with fix if you can, could it be that VR doesn't have
> patched code? Also, describe testing scenario, I'll try to look at it in my
> patched 4.10.
>
> 22 янв. 2018 г. 16:52 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> > Hi Ivan;
> > I checked 2 PR's and they are exist on 4.11rc1 but issue still exists on
> my
> > environment. When a new vm uses IP from old expunged vm then leases file
> > creates problem. Please check the logs that i submitted on first email,
> > issue is clear there and in my opinion it still exists on 4.11rc1.
> >
> > By the way 2393 is about VM's IP Changing progress, maybe it does not
> cover
> > my scenario.
> >
> > Thanks
> > Özhan
> >
> > On Mon, Jan 22, 2018 at 12:40 PM, Özhan Rüzgar Karaman <
> > oruzgarkara...@gmail.com> wrote:
> >
> > > Hi Ivan;
> > > I made several tests with same scenario on 4.11rc1 and got same
> results,
> > > did your 2 PR's currently exists on 4.11 rc1 in which i am testing or
> it
> > > will exist on future rc2? If they exists on 4.11rc1 then we have a
> > problem
> > >
> > > Thanks
> > > Özhan
> > >
> > > On Mon, Jan 22, 2018 at 12:32 PM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com> wrote:
> > >
> > >> Hi, Ozhan. MACs are not removed upon vm removal, but they are
> overriden
> > >> upon vm creation with same ip (or same hostname). It should work fine,
> > >> 4.10, 4.11 received 2 PRs to fix several possible bugs. I tested the
> > case
> > >> when IP is reused.
> > >>
> > >> 22 янв. 2018 г. 16:07 пользователь "Özhan Rüzgar Karaman" <
> > >> oruzgarkara...@gmail.com> написал:
> > >>
> > >> Hi;
> > >> Today we noticed that one of our new provisioned instance did not get
> IP
> > >> from VR. When we dig into the issue we find that one different mac is
> > >> written in dnsmasq.leases file holds new instances IP address.
> > >>
> > >> We checked this mac address from db and we noticed that this mac is
> used
> > >> for old expunged instance.
> > >>
> > >> So from this point we realised that when we destroy an instance its
> mac
> > >> did
> > >> not removed from dnsmasq.leases file so if we use this ip for a new
> > >> instance then we have a problem, our instance could not get IP from
> VR.
> > >>
> > >> We have one host on our lab environment and its Ubuntu 16.04.3 KVM.
> > Today
> > >> we made a HA test and we crashed the host so VR and SystemVM's are
> > >> rebooted
> > >> after we boot host back. I do not think this issue is related to VR
> > reboot
> > >> but i like to give information about our environment.
> > >>
> > >> We need to manage dnsmasq.leases file when we expunge an instance.
> > >>
> > >> Thanks
> > >> Özhan
> > >>
> > >> Logs are below:
> > >>
> > >> root@r-4-VM:/var/lib/misc# tail -4 /var/log/dnsmasq.log
> > >> Jan 22 08:57:27 dnsmasq-dhcp[850]: not using configured address
> > >> 192.168.18.186 because it is leased to 1e:00:25:00:00:b9
> > >> Jan 22 08:57:27 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0)
> 1e:00:00:00:00:b9
> > no
> > >> address available
> > >> Jan 22 08:57:29 dnsmasq-dhcp[850]: not using configured address
> > >> 192.168.18.187 because it is leased to 1e:00:80:00:00:ba
> > >> Jan 22 08:57:29 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0)
> 1e:00:9f:00:00:ba
> > no
> > >> address available
> > >>
> > >> root@r-4-VM:/var/lib/misc# cat /etc/dhcphosts.txt
> > >> 1e:00:9f:00:00:ba,192.168.18.187,test411rc1mac,736h
> > >> 1e:00:00:00:00:b9,192.168.18.186,sil3sameip,733h
> > >> 1e:00:96:00:00:bf,192.168.18.192,TolgaTest02,707h
&

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi Ivan;
I checked 2 PR's and they are exist on 4.11rc1 but issue still exists on my
environment. When a new vm uses IP from old expunged vm then leases file
creates problem. Please check the logs that i submitted on first email,
issue is clear there and in my opinion it still exists on 4.11rc1.

By the way 2393 is about VM's IP Changing progress, maybe it does not cover
my scenario.

Thanks
Özhan

On Mon, Jan 22, 2018 at 12:40 PM, Özhan Rüzgar Karaman <
oruzgarkara...@gmail.com> wrote:

> Hi Ivan;
> I made several tests with same scenario on 4.11rc1 and got same results,
> did your 2 PR's currently exists on 4.11 rc1 in which i am testing or it
> will exist on future rc2? If they exists on 4.11rc1 then we have a problem
>
> Thanks
> Özhan
>
> On Mon, Jan 22, 2018 at 12:32 PM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com> wrote:
>
>> Hi, Ozhan. MACs are not removed upon vm removal, but they are overriden
>> upon vm creation with same ip (or same hostname). It should work fine,
>> 4.10, 4.11 received 2 PRs to fix several possible bugs. I tested the case
>> when IP is reused.
>>
>> 22 янв. 2018 г. 16:07 пользователь "Özhan Rüzgar Karaman" <
>> oruzgarkara...@gmail.com> написал:
>>
>> Hi;
>> Today we noticed that one of our new provisioned instance did not get IP
>> from VR. When we dig into the issue we find that one different mac is
>> written in dnsmasq.leases file holds new instances IP address.
>>
>> We checked this mac address from db and we noticed that this mac is used
>> for old expunged instance.
>>
>> So from this point we realised that when we destroy an instance its mac
>> did
>> not removed from dnsmasq.leases file so if we use this ip for a new
>> instance then we have a problem, our instance could not get IP from VR.
>>
>> We have one host on our lab environment and its Ubuntu 16.04.3 KVM. Today
>> we made a HA test and we crashed the host so VR and SystemVM's are
>> rebooted
>> after we boot host back. I do not think this issue is related to VR reboot
>> but i like to give information about our environment.
>>
>> We need to manage dnsmasq.leases file when we expunge an instance.
>>
>> Thanks
>> Özhan
>>
>> Logs are below:
>>
>> root@r-4-VM:/var/lib/misc# tail -4 /var/log/dnsmasq.log
>> Jan 22 08:57:27 dnsmasq-dhcp[850]: not using configured address
>> 192.168.18.186 because it is leased to 1e:00:25:00:00:b9
>> Jan 22 08:57:27 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:00:00:00:b9 no
>> address available
>> Jan 22 08:57:29 dnsmasq-dhcp[850]: not using configured address
>> 192.168.18.187 because it is leased to 1e:00:80:00:00:ba
>> Jan 22 08:57:29 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:9f:00:00:ba no
>> address available
>>
>> root@r-4-VM:/var/lib/misc# cat /etc/dhcphosts.txt
>> 1e:00:9f:00:00:ba,192.168.18.187,test411rc1mac,736h
>> 1e:00:00:00:00:b9,192.168.18.186,sil3sameip,733h
>> 1e:00:96:00:00:bf,192.168.18.192,TolgaTest02,707h
>> 1e:00:90:00:00:bc,192.168.18.189,TolgaTest,758h
>> 1e:00:40:00:00:bb,192.168.18.188,test411rc1,750h
>> root@r-4-VM:/var/lib/misc# cat /var/lib/misc/dnsmasq.leases
>> 1519339727 1e:00:25:00:00:b9 192.168.18.186 sil3sameip *
>> 1519331409 1e:00:40:00:00:bb 192.168.18.188 test411rc1 *
>> 1518921177 1e:00:80:00:00:ba 192.168.18.187 test411rc1mac *
>> 1518904275 1e:00:90:00:00:bc 192.168.18.189 TolgaTest *
>> 1519023297 1e:00:96:00:00:bf 192.168.18.192 TolgaTest02 *
>>
>> mysql> select name,state,private_mac_address,private_ip_address from
>> vm_instance;
>> +---+---+-++
>> | name  | state | private_mac_address | private_ip_address |
>> +---+---+-++
>> | s-1-VM| Running   | 1e:00:34:00:01:00   | 172.16.50.143  |
>> | v-2-VM| Running   | 1e:00:81:00:01:03   | 172.16.50.146  |
>> | Tolga | Expunging | 1e:00:50:00:00:bc   | 192.168.18.189 |
>> | r-4-VM| Running   | 0e:00:a9:fe:03:0e   | 169.254.3.14   |
>> | Tolga02   | Expunging | 1e:00:b4:00:00:bf   | 192.168.18.192 |
>> | Tolga03   | Expunging | 1e:00:99:00:00:bb   | 192.168.18.188 |
>> | deneme| Expunging | 1e:00:80:00:00:ba   | 192.168.18.187 |
>> | snpvmtolga02  | Expunging | 1e:00:69:00:00:b9   | 192.168.18.186 |
>> | TolgaTest | Stopped   | 1e:00:90:00:00:bc   | 192.168.18.189 |
>> | TolgaTest02   | Stopped   | 1e:00:96:00:00:bf   | 192.168.18.192 |
>> | test411rc1| Running   | 1e:00:40:00:00:bb   | 192.168.18.188 |
>> | test411rc1mac | Running   | 1e:00:9f:00:00:ba   | 192.168.18.187 |
>> | sil1  | Expunging | 1e:00:25:00:00:b9   | 192.168.18.186 |
>> | sil2sameip| Expunging | 1e:00:14:00:00:b9   | 192.168.18.186 |
>> | sil3sameip| Running   | 1e:00:00:00:00:b9   | 192.168.18.186 |
>> +---+---+-++
>> 15 rows in set (0.00 sec)
>>
>
>

Re: [4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi Ivan;
I made several tests with same scenario on 4.11rc1 and got same results,
did your 2 PR's currently exists on 4.11 rc1 in which i am testing or it
will exist on future rc2? If they exists on 4.11rc1 then we have a problem

Thanks
Özhan

On Mon, Jan 22, 2018 at 12:32 PM, Ivan Kudryavtsev <kudryavtsev...@bw-sw.com
> wrote:

> Hi, Ozhan. MACs are not removed upon vm removal, but they are overriden
> upon vm creation with same ip (or same hostname). It should work fine,
> 4.10, 4.11 received 2 PRs to fix several possible bugs. I tested the case
> when IP is reused.
>
> 22 янв. 2018 г. 16:07 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> Hi;
> Today we noticed that one of our new provisioned instance did not get IP
> from VR. When we dig into the issue we find that one different mac is
> written in dnsmasq.leases file holds new instances IP address.
>
> We checked this mac address from db and we noticed that this mac is used
> for old expunged instance.
>
> So from this point we realised that when we destroy an instance its mac did
> not removed from dnsmasq.leases file so if we use this ip for a new
> instance then we have a problem, our instance could not get IP from VR.
>
> We have one host on our lab environment and its Ubuntu 16.04.3 KVM. Today
> we made a HA test and we crashed the host so VR and SystemVM's are rebooted
> after we boot host back. I do not think this issue is related to VR reboot
> but i like to give information about our environment.
>
> We need to manage dnsmasq.leases file when we expunge an instance.
>
> Thanks
> Özhan
>
> Logs are below:
>
> root@r-4-VM:/var/lib/misc# tail -4 /var/log/dnsmasq.log
> Jan 22 08:57:27 dnsmasq-dhcp[850]: not using configured address
> 192.168.18.186 because it is leased to 1e:00:25:00:00:b9
> Jan 22 08:57:27 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:00:00:00:b9 no
> address available
> Jan 22 08:57:29 dnsmasq-dhcp[850]: not using configured address
> 192.168.18.187 because it is leased to 1e:00:80:00:00:ba
> Jan 22 08:57:29 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:9f:00:00:ba no
> address available
>
> root@r-4-VM:/var/lib/misc# cat /etc/dhcphosts.txt
> 1e:00:9f:00:00:ba,192.168.18.187,test411rc1mac,736h
> 1e:00:00:00:00:b9,192.168.18.186,sil3sameip,733h
> 1e:00:96:00:00:bf,192.168.18.192,TolgaTest02,707h
> 1e:00:90:00:00:bc,192.168.18.189,TolgaTest,758h
> 1e:00:40:00:00:bb,192.168.18.188,test411rc1,750h
> root@r-4-VM:/var/lib/misc# cat /var/lib/misc/dnsmasq.leases
> 1519339727 1e:00:25:00:00:b9 192.168.18.186 sil3sameip *
> 1519331409 1e:00:40:00:00:bb 192.168.18.188 test411rc1 *
> 1518921177 1e:00:80:00:00:ba 192.168.18.187 test411rc1mac *
> 1518904275 1e:00:90:00:00:bc 192.168.18.189 TolgaTest *
> 1519023297 1e:00:96:00:00:bf 192.168.18.192 TolgaTest02 *
>
> mysql> select name,state,private_mac_address,private_ip_address from
> vm_instance;
> +---+---+-++
> | name  | state | private_mac_address | private_ip_address |
> +---+---+-++
> | s-1-VM| Running   | 1e:00:34:00:01:00   | 172.16.50.143  |
> | v-2-VM| Running   | 1e:00:81:00:01:03   | 172.16.50.146  |
> | Tolga | Expunging | 1e:00:50:00:00:bc   | 192.168.18.189 |
> | r-4-VM| Running   | 0e:00:a9:fe:03:0e   | 169.254.3.14   |
> | Tolga02   | Expunging | 1e:00:b4:00:00:bf   | 192.168.18.192 |
> | Tolga03   | Expunging | 1e:00:99:00:00:bb   | 192.168.18.188 |
> | deneme| Expunging | 1e:00:80:00:00:ba   | 192.168.18.187 |
> | snpvmtolga02  | Expunging | 1e:00:69:00:00:b9   | 192.168.18.186 |
> | TolgaTest | Stopped   | 1e:00:90:00:00:bc   | 192.168.18.189 |
> | TolgaTest02   | Stopped   | 1e:00:96:00:00:bf   | 192.168.18.192 |
> | test411rc1| Running   | 1e:00:40:00:00:bb   | 192.168.18.188 |
> | test411rc1mac | Running   | 1e:00:9f:00:00:ba   | 192.168.18.187 |
> | sil1  | Expunging | 1e:00:25:00:00:b9   | 192.168.18.186 |
> | sil2sameip| Expunging | 1e:00:14:00:00:b9   | 192.168.18.186 |
> | sil3sameip| Running   | 1e:00:00:00:00:b9   | 192.168.18.186 |
> +---+---+-++
> 15 rows in set (0.00 sec)
>

[4.11] VR Problem on Releasing Expunged Instance IP from dnsmasq.leases file

[Özhan Rüzgar Karaman]

Hi;
Today we noticed that one of our new provisioned instance did not get IP
from VR. When we dig into the issue we find that one different mac is
written in dnsmasq.leases file holds new instances IP address.

We checked this mac address from db and we noticed that this mac is used
for old expunged instance.

So from this point we realised that when we destroy an instance its mac did
not removed from dnsmasq.leases file so if we use this ip for a new
instance then we have a problem, our instance could not get IP from VR.

We have one host on our lab environment and its Ubuntu 16.04.3 KVM. Today
we made a HA test and we crashed the host so VR and SystemVM's are rebooted
after we boot host back. I do not think this issue is related to VR reboot
but i like to give information about our environment.

We need to manage dnsmasq.leases file when we expunge an instance.

Thanks
Özhan

Logs are below:

root@r-4-VM:/var/lib/misc# tail -4 /var/log/dnsmasq.log
Jan 22 08:57:27 dnsmasq-dhcp[850]: not using configured address
192.168.18.186 because it is leased to 1e:00:25:00:00:b9
Jan 22 08:57:27 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:00:00:00:b9 no
address available
Jan 22 08:57:29 dnsmasq-dhcp[850]: not using configured address
192.168.18.187 because it is leased to 1e:00:80:00:00:ba
Jan 22 08:57:29 dnsmasq-dhcp[850]: DHCPDISCOVER(eth0) 1e:00:9f:00:00:ba no
address available

root@r-4-VM:/var/lib/misc# cat /etc/dhcphosts.txt
1e:00:9f:00:00:ba,192.168.18.187,test411rc1mac,736h
1e:00:00:00:00:b9,192.168.18.186,sil3sameip,733h
1e:00:96:00:00:bf,192.168.18.192,TolgaTest02,707h
1e:00:90:00:00:bc,192.168.18.189,TolgaTest,758h
1e:00:40:00:00:bb,192.168.18.188,test411rc1,750h
root@r-4-VM:/var/lib/misc# cat /var/lib/misc/dnsmasq.leases
1519339727 1e:00:25:00:00:b9 192.168.18.186 sil3sameip *
1519331409 1e:00:40:00:00:bb 192.168.18.188 test411rc1 *
1518921177 1e:00:80:00:00:ba 192.168.18.187 test411rc1mac *
1518904275 1e:00:90:00:00:bc 192.168.18.189 TolgaTest *
1519023297 1e:00:96:00:00:bf 192.168.18.192 TolgaTest02 *

mysql> select name,state,private_mac_address,private_ip_address from
vm_instance;
+---+---+-++
| name  | state | private_mac_address | private_ip_address |
+---+---+-++
| s-1-VM| Running   | 1e:00:34:00:01:00   | 172.16.50.143  |
| v-2-VM| Running   | 1e:00:81:00:01:03   | 172.16.50.146  |
| Tolga | Expunging | 1e:00:50:00:00:bc   | 192.168.18.189 |
| r-4-VM| Running   | 0e:00:a9:fe:03:0e   | 169.254.3.14   |
| Tolga02   | Expunging | 1e:00:b4:00:00:bf   | 192.168.18.192 |
| Tolga03   | Expunging | 1e:00:99:00:00:bb   | 192.168.18.188 |
| deneme| Expunging | 1e:00:80:00:00:ba   | 192.168.18.187 |
| snpvmtolga02  | Expunging | 1e:00:69:00:00:b9   | 192.168.18.186 |
| TolgaTest | Stopped   | 1e:00:90:00:00:bc   | 192.168.18.189 |
| TolgaTest02   | Stopped   | 1e:00:96:00:00:bf   | 192.168.18.192 |
| test411rc1| Running   | 1e:00:40:00:00:bb   | 192.168.18.188 |
| test411rc1mac | Running   | 1e:00:9f:00:00:ba   | 192.168.18.187 |
| sil1  | Expunging | 1e:00:25:00:00:b9   | 192.168.18.186 |
| sil2sameip| Expunging | 1e:00:14:00:00:b9   | 192.168.18.186 |
| sil3sameip| Running   | 1e:00:00:00:00:b9   | 192.168.18.186 |
+---+---+-++
15 rows in set (0.00 sec)

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi Wido,
I will test the patch and respond today.

Thanks
Özhan

On Mon, Jan 22, 2018 at 10:08 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
>
> On 01/22/2018 07:35 AM, Wido den Hollander wrote:
>
>>
>>
>> On 01/21/2018 11:23 AM, Rohit Yadav wrote:
>>
>>> Wido - Were you able to reproduce and fix the issue? Thanks.
>>>
>>>
>> Still working on it! This weekend I was short on time and wasn't able to
>> fix it yet.
>>
>> Today (Mon) and tomorrow (Tue) my time is limited as well. Trying to fix
>> it asap.
>>
>
> During my train ride this morning I wrote this patch:
> https://github.com/apache/cloudstack/pull/2418
>
> @ Özhan, could you test this patch? It's just a matter of replacing
> security_group.py on your Hypervisor.
>
> Thanks,
>
> Wido
>
>
>
>> Wido
>>
>>
>>>
>>> - Rohit
>>>
>>> <https://cloudstack.apache.org>
>>>
>>>
>>>
>>> ____
>>> From: Wido den Hollander <w...@widodh.nl>
>>> Sent: Friday, January 19, 2018 10:12:45 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [4.11] KVM Advanced Networking with SG Problem
>>>
>>>
>>>
>>> On 01/19/2018 02:03 PM, Özhan Rüzgar Karaman wrote:
>>>
>>>> Hi Daan;
>>>> Wido or others will write a fix, i am not a developer, i do not have a
>>>> fix,
>>>> i just only want to report it to make it official thats all :)
>>>>
>>>>
>>> I'll look into this asap. The Python script should parse these rules
>>> properly and then it should be fixed.
>>>
>>> I hope to have a fix this weekend.
>>>
>>> Wido
>>>
>>> Thanks
>>>> Özhan
>>>>
>>>> On Fri, Jan 19, 2018 at 3:59 PM, Daan Hoogland <daan.hoogl...@gmail.com
>>>> >
>>>> wrote:
>>>>
>>>> This is not a PR but a ticket, Özhan. Do you plan to make a pull
>>>>> request on
>>>>> github with your solution for it?
>>>>>
>>>>> On Fri, Jan 19, 2018 at 1:53 PM, Özhan Rüzgar Karaman <
>>>>> oruzgarkara...@gmail.com> wrote:
>>>>>
>>>>> Hi Daan;
>>>>>> Wido is the previous PR's owner, he will check it. By the way i have
>>>>>> created a PR for this problem which is below:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-10242
>>>>>>
>>>>>> I select its priority as blocker, if its wrong developers will update
>>>>>> its
>>>>>> priority.
>>>>>>
>>>>>> Thanks
>>>>>> Özhan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 19, 2018 at 3:25 PM, Daan Hoogland <
>>>>>> daan.hoogl...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Özhan, this is sure to break ipv6. can you make it use another
>>>>>>>
>>>>>> delimiter?
>>>>>
>>>>>>
>>>>>>> On Fri, Jan 19, 2018 at 1:12 PM, Özhan Rüzgar Karaman <
>>>>>>> oruzgarkara...@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Rohit;
>>>>>>>> This is a fresh install of 4.11 rc1 and we have only ipv4 setup on
>>>>>>>>
>>>>>>> our
>>>>>
>>>>>> test
>>>>>>>
>>>>>>>> environment no ipv6 addresses, our VR's are new 4.11 rc1 system vms.
>>>>>>>>
>>>>>>> Our
>>>>>>
>>>>>>> workaround is 4 lines of code to convert ";" character to ":" on
>>>>>>>> security_group.py
>>>>>>>> code to make it operational for ipv4 addresses but i am sure it will
>>>>>>>>
>>>>>>> break
>>>>>>>
>>>>>>>> Wido's "Add support for ipv6 address and subnets" PR. Workaround
>>>>>>>>
>>>>>>> works
>>>>>
>>>>>> only
>>>>>>>
>>>>>>>> for us because we have ipv4 only setup.
>>>>>>>>
>>>>>>>> If Wido c

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi Daan;
Wido or others will write a fix, i am not a developer, i do not have a fix,
i just only want to report it to make it official thats all :)

Thanks
Özhan

On Fri, Jan 19, 2018 at 3:59 PM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> This is not a PR but a ticket, Özhan. Do you plan to make a pull request on
> github with your solution for it?
>
> On Fri, Jan 19, 2018 at 1:53 PM, Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com> wrote:
>
> > Hi Daan;
> > Wido is the previous PR's owner, he will check it. By the way i have
> > created a PR for this problem which is below:
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-10242
> >
> > I select its priority as blocker, if its wrong developers will update its
> > priority.
> >
> > Thanks
> > Özhan
> >
> >
> >
> > On Fri, Jan 19, 2018 at 3:25 PM, Daan Hoogland <daan.hoogl...@gmail.com>
> > wrote:
> >
> > > Özhan, this is sure to break ipv6. can you make it use another
> delimiter?
> > >
> > > On Fri, Jan 19, 2018 at 1:12 PM, Özhan Rüzgar Karaman <
> > > oruzgarkara...@gmail.com> wrote:
> > >
> > > > Hi Rohit;
> > > > This is a fresh install of 4.11 rc1 and we have only ipv4 setup on
> our
> > > test
> > > > environment no ipv6 addresses, our VR's are new 4.11 rc1 system vms.
> > Our
> > > > workaround is 4 lines of code to convert ";" character to ":" on
> > > > security_group.py
> > > > code to make it operational for ipv4 addresses but i am sure it will
> > > break
> > > > Wido's "Add support for ipv6 address and subnets" PR. Workaround
> works
> > > only
> > > > for us because we have ipv4 only setup.
> > > >
> > > > If Wido could check parse_network_rules function on security_group.py
> > > then
> > > > that could be great. After his check and possible code fix i like to
> > make
> > > > test again on our environment.
> > > >
> > > > @Rohit i will create a JIRA ticket to follow it easily by team.
> > > >
> > > > Thanks
> > > > Özhan
> > > >
> > > > On Fri, Jan 19, 2018 at 2:51 PM, Rohit Yadav <
> > rohit.ya...@shapeblue.com>
> > > > wrote:
> > > >
> > > > > Hi Ozhan,
> > > > >
> > > > >
> > > > > Thanks for sharing.
> > > > >
> > > > >
> > > > > I traced the change to the following PR that changes the delimiter
> > > > > character to ';' than ":" to support ipv6 addresses:
> > > > >
> > > > > https://github.com/apache/cloudstack/pull/2028/files
> > > > >
> > > > >
> > > > > Can you share with the workaround, if applicable send a pull
> request?
> > > > >
> > > > >
> > > > > Were you still using old 4.9.3 VRs post upgrade, does killing old
> 4.9
> > > VRs
> > > > > help fix the issue? /cc Wido
> > > > >
> > > > >
> > > > > - Rohit
> > > > >
> > > > > <https://cloudstack.apache.org>
> > > > >
> > > > >
> > > > >
> > > > > 
> > > > > From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> > > > > Sent: Friday, January 19, 2018 3:38:19 PM
> > > > > To: dev@cloudstack.apache.org
> > > > > Subject: Re: [4.11] KVM Advanced Networking with SG Problem
> > > > >
> > > > > Hi;
> > > > > We solved the bug there and write a small workaround today, the
> > problem
> > > > is
> > > > > generally from the Java code which calls security_group.py. On
> 4.9.3
> > > > > release it was using : character but from 4.11 release delimiter
> > > changed
> > > > to
> > > > > ; character but security_group.py expects : as delimeter so
> > > > > security_group.py could not parse & send rules to the iptables.
> > > > >
> > > > > Afternoon i will create a JIRA ticket and if anyone could fix the
> > > > delimiter
> > > > > character or code in the Java code for 4.11 release that would be
> > great
> > > > > because without this code Security Groups are not operational for
> > 4.11

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi Daan;
Wido is the previous PR's owner, he will check it. By the way i have
created a PR for this problem which is below:

https://issues.apache.org/jira/browse/CLOUDSTACK-10242

I select its priority as blocker, if its wrong developers will update its
priority.

Thanks
Özhan



On Fri, Jan 19, 2018 at 3:25 PM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> Özhan, this is sure to break ipv6. can you make it use another delimiter?
>
> On Fri, Jan 19, 2018 at 1:12 PM, Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com> wrote:
>
> > Hi Rohit;
> > This is a fresh install of 4.11 rc1 and we have only ipv4 setup on our
> test
> > environment no ipv6 addresses, our VR's are new 4.11 rc1 system vms.  Our
> > workaround is 4 lines of code to convert ";" character to ":" on
> > security_group.py
> > code to make it operational for ipv4 addresses but i am sure it will
> break
> > Wido's "Add support for ipv6 address and subnets" PR. Workaround works
> only
> > for us because we have ipv4 only setup.
> >
> > If Wido could check parse_network_rules function on security_group.py
> then
> > that could be great. After his check and possible code fix i like to make
> > test again on our environment.
> >
> > @Rohit i will create a JIRA ticket to follow it easily by team.
> >
> > Thanks
> > Özhan
> >
> > On Fri, Jan 19, 2018 at 2:51 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
> > wrote:
> >
> > > Hi Ozhan,
> > >
> > >
> > > Thanks for sharing.
> > >
> > >
> > > I traced the change to the following PR that changes the delimiter
> > > character to ';' than ":" to support ipv6 addresses:
> > >
> > > https://github.com/apache/cloudstack/pull/2028/files
> > >
> > >
> > > Can you share with the workaround, if applicable send a pull request?
> > >
> > >
> > > Were you still using old 4.9.3 VRs post upgrade, does killing old 4.9
> VRs
> > > help fix the issue? /cc Wido
> > >
> > >
> > > - Rohit
> > >
> > > <https://cloudstack.apache.org>
> > >
> > >
> > >
> > > 
> > > From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> > > Sent: Friday, January 19, 2018 3:38:19 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [4.11] KVM Advanced Networking with SG Problem
> > >
> > > Hi;
> > > We solved the bug there and write a small workaround today, the problem
> > is
> > > generally from the Java code which calls security_group.py. On 4.9.3
> > > release it was using : character but from 4.11 release delimiter
> changed
> > to
> > > ; character but security_group.py expects : as delimeter so
> > > security_group.py could not parse & send rules to the iptables.
> > >
> > > Afternoon i will create a JIRA ticket and if anyone could fix the
> > delimiter
> > > character or code in the Java code for 4.11 release that would be great
> > > because without this code Security Groups are not operational for 4.11.
> > >
> > > Also @Rohit do we need to check test codes for Security Groups?
> Because i
> > > do not understand how this bug passed our testing scenarios.
> > >
> > > Thanks
> > > Özhan
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Jan 19, 2018 at 12:00 PM, Rohit Yadav <
> rohit.ya...@shapeblue.com
> > >
> > > wrote:
> > >
> > > > Can anyone help look into this issue, reproduce it and if it's a
> > genuine
> > > > bug help fix it?
> > > >
> > > > Any takers - Wido, Wei, Mike and others who may be using KVM+SG?
> > > >
> > > >
> > > > - Rohit
> > > >
> > > > <https://cloudstack.apache.org>
> > > >
> > > >
> > > >
> > > > 
> > > > From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> > > > Sent: Tuesday, January 16, 2018 9:53:59 PM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: [4.11] KVM Advanced Networking with SG Problem
> > > >
> > > > Hi;
> > > > We made a test with 4.11 rc over Ubuntu16.04 KVM hosts and we noticed
> > > that
> > > > there is a problem on setting & applying security group changes on
> KVM
>

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi Rohit;
This is a fresh install of 4.11 rc1 and we have only ipv4 setup on our test
environment no ipv6 addresses, our VR's are new 4.11 rc1 system vms.  Our
workaround is 4 lines of code to convert ";" character to ":" on
security_group.py
code to make it operational for ipv4 addresses but i am sure it will break
Wido's "Add support for ipv6 address and subnets" PR. Workaround works only
for us because we have ipv4 only setup.

If Wido could check parse_network_rules function on security_group.py then
that could be great. After his check and possible code fix i like to make
test again on our environment.

@Rohit i will create a JIRA ticket to follow it easily by team.

Thanks
Özhan

On Fri, Jan 19, 2018 at 2:51 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Hi Ozhan,
>
>
> Thanks for sharing.
>
>
> I traced the change to the following PR that changes the delimiter
> character to ';' than ":" to support ipv6 addresses:
>
> https://github.com/apache/cloudstack/pull/2028/files
>
>
> Can you share with the workaround, if applicable send a pull request?
>
>
> Were you still using old 4.9.3 VRs post upgrade, does killing old 4.9 VRs
> help fix the issue? /cc Wido
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> Sent: Friday, January 19, 2018 3:38:19 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [4.11] KVM Advanced Networking with SG Problem
>
> Hi;
> We solved the bug there and write a small workaround today, the problem is
> generally from the Java code which calls security_group.py. On 4.9.3
> release it was using : character but from 4.11 release delimiter changed to
> ; character but security_group.py expects : as delimeter so
> security_group.py could not parse & send rules to the iptables.
>
> Afternoon i will create a JIRA ticket and if anyone could fix the delimiter
> character or code in the Java code for 4.11 release that would be great
> because without this code Security Groups are not operational for 4.11.
>
> Also @Rohit do we need to check test codes for Security Groups? Because i
> do not understand how this bug passed our testing scenarios.
>
> Thanks
> Özhan
>
>
>
>
>
>
> On Fri, Jan 19, 2018 at 12:00 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
> wrote:
>
> > Can anyone help look into this issue, reproduce it and if it's a genuine
> > bug help fix it?
> >
> > Any takers - Wido, Wei, Mike and others who may be using KVM+SG?
> >
> >
> > - Rohit
> >
> > <https://cloudstack.apache.org>
> >
> >
> >
> > 
> > From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> > Sent: Tuesday, January 16, 2018 9:53:59 PM
> > To: dev@cloudstack.apache.org
> > Subject: [4.11] KVM Advanced Networking with SG Problem
> >
> > Hi;
> > We made a test with 4.11 rc over Ubuntu16.04 KVM hosts and we noticed
> that
> > there is a problem on setting & applying security group changes on KVM
> > host.
> >
> > All instances could ping vr and they could access internet but no one
> could
> > access to the instances.
> >
> > I checked iptables rules and i noticed that iptables rules for vm is in
> all
> > drop state for incoming packages while i gave access to all ingress and
> > egress tcp/udp traffic ports for that instances. Below are iptables
> output
> > for selected vm:
> >
> > Chain i-2-6-VM (1 references)
> > target prot opt source   destination
> > DROP   all  --  anywhere anywhere
> >
> > Chain i-2-6-VM-eg (1 references)
> > target prot opt source   destination
> > RETURN all  --  anywhere anywhere
> >
> > Chain i-2-6-def (2 references)
> > target prot opt source   destination
> > ACCEPT all  --  anywhere anywhere state
> > RELATED,ESTABLISHED
> > ACCEPT udp  --  anywhere anywhere PHYSDEV
> match
> > --physdev-in vnet9 --physdev-is-bridged udp spt:bootpc dpt:bootps
> > ACCEPT udp  --  anywhere anywhere PHYSDEV
> match
> > --physdev-out vnet9 --physdev-is-bridged udp spt:bootps dpt:bootpc
> > DROP   all  --  anywhere anywhere PHYSDEV
> match
> > --physdev-in vnet9 --physdev-is-bridged ! match-set i-2-6-VM src
> > RETURN udp  --  anywhere anywhere PHYSDEV
> match
> > --physdev-in v

Re: [4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi;
We solved the bug there and write a small workaround today, the problem is
generally from the Java code which calls security_group.py. On 4.9.3
release it was using : character but from 4.11 release delimiter changed to
; character but security_group.py expects : as delimeter so
security_group.py could not parse & send rules to the iptables.

Afternoon i will create a JIRA ticket and if anyone could fix the delimiter
character or code in the Java code for 4.11 release that would be great
because without this code Security Groups are not operational for 4.11.

Also @Rohit do we need to check test codes for Security Groups? Because i
do not understand how this bug passed our testing scenarios.

Thanks
Özhan






On Fri, Jan 19, 2018 at 12:00 PM, Rohit Yadav <rohit.ya...@shapeblue.com>
wrote:

> Can anyone help look into this issue, reproduce it and if it's a genuine
> bug help fix it?
>
> Any takers - Wido, Wei, Mike and others who may be using KVM+SG?
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> ____
> From: Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>
> Sent: Tuesday, January 16, 2018 9:53:59 PM
> To: dev@cloudstack.apache.org
> Subject: [4.11] KVM Advanced Networking with SG Problem
>
> Hi;
> We made a test with 4.11 rc over Ubuntu16.04 KVM hosts and we noticed that
> there is a problem on setting & applying security group changes on KVM
> host.
>
> All instances could ping vr and they could access internet but no one could
> access to the instances.
>
> I checked iptables rules and i noticed that iptables rules for vm is in all
> drop state for incoming packages while i gave access to all ingress and
> egress tcp/udp traffic ports for that instances. Below are iptables output
> for selected vm:
>
> Chain i-2-6-VM (1 references)
> target prot opt source   destination
> DROP   all  --  anywhere anywhere
>
> Chain i-2-6-VM-eg (1 references)
> target prot opt source   destination
> RETURN all  --  anywhere anywhere
>
> Chain i-2-6-def (2 references)
> target prot opt source   destination
> ACCEPT all  --  anywhere anywhere state
> RELATED,ESTABLISHED
> ACCEPT udp  --  anywhere anywhere PHYSDEV match
> --physdev-in vnet9 --physdev-is-bridged udp spt:bootpc dpt:bootps
> ACCEPT udp  --  anywhere anywhere PHYSDEV match
> --physdev-out vnet9 --physdev-is-bridged udp spt:bootps dpt:bootpc
> DROP   all  --  anywhere anywhere PHYSDEV match
> --physdev-in vnet9 --physdev-is-bridged ! match-set i-2-6-VM src
> RETURN udp  --  anywhere anywhere PHYSDEV match
> --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src udp
> dpt:domain
> RETURN tcp  --  anywhere anywhere PHYSDEV match
> --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src tcp
> dpt:domain
> i-2-6-VM-eg  all  --  anywhere anywhere PHYSDEV
> match --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src
> i-2-6-VM   all  --  anywhere anywhere PHYSDEV match
> --physdev-out vnet9 --physdev-is-bridged
>
> All management and agent logs could be accessed from:
> http://51.15.199.7/4.11r1_Test_20190116.tgz
>
> Thanks
> Özhan
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

[4.11] KVM Advanced Networking with SG Problem

[Özhan Rüzgar Karaman]

Hi;
We made a test with 4.11 rc over Ubuntu16.04 KVM hosts and we noticed that
there is a problem on setting & applying security group changes on KVM
host.

All instances could ping vr and they could access internet but no one could
access to the instances.

I checked iptables rules and i noticed that iptables rules for vm is in all
drop state for incoming packages while i gave access to all ingress and
egress tcp/udp traffic ports for that instances. Below are iptables output
for selected vm:

Chain i-2-6-VM (1 references)
target prot opt source   destination
DROP   all  --  anywhere anywhere

Chain i-2-6-VM-eg (1 references)
target prot opt source   destination
RETURN all  --  anywhere anywhere

Chain i-2-6-def (2 references)
target prot opt source   destination
ACCEPT all  --  anywhere anywhere state
RELATED,ESTABLISHED
ACCEPT udp  --  anywhere anywhere PHYSDEV match
--physdev-in vnet9 --physdev-is-bridged udp spt:bootpc dpt:bootps
ACCEPT udp  --  anywhere anywhere PHYSDEV match
--physdev-out vnet9 --physdev-is-bridged udp spt:bootps dpt:bootpc
DROP   all  --  anywhere anywhere PHYSDEV match
--physdev-in vnet9 --physdev-is-bridged ! match-set i-2-6-VM src
RETURN udp  --  anywhere anywhere PHYSDEV match
--physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src udp
dpt:domain
RETURN tcp  --  anywhere anywhere PHYSDEV match
--physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src tcp
dpt:domain
i-2-6-VM-eg  all  --  anywhere anywhere PHYSDEV
match --physdev-in vnet9 --physdev-is-bridged match-set i-2-6-VM src
i-2-6-VM   all  --  anywhere anywhere PHYSDEV match
--physdev-out vnet9 --physdev-is-bridged

All management and agent logs could be accessed from:
http://51.15.199.7/4.11r1_Test_20190116.tgz

Thanks
Özhan

Re: KVM on ubuntu

[Özhan Rüzgar Karaman]

On Tue, Jan 16, 2018 at 10:49 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
>
> On 01/16/2018 06:36 AM, Özhan Rüzgar Karaman wrote:
>
>> Hi Paul;
>> Ubuntu 16.04 has systemd installed and it directly controls libvirtd
>> process so if you add -d to /etc/default/libvirt-bin file libvirtd tries
>> to
>> control to its self and then you have problem, you could not
>> stop/start/restart libvirtd service via systemd. Only adding -l flag to
>> libvirtd_opts is enough for Cloudstack.
>>
>> We also make no change on /etc/init/libvirt-bin.conf file for 16.04
>>
>>
I recheck my previous email and my written info below is for 14.04 (i
mistakely wrote 16.04 for below info)

For Ubuntu 16.04 you need to add both -d and -l flags for
>> /etc/default/libvirt-bin
>> file.
>>
>>
> Indeed! Mainly the -l flag is important to make libvirtd listen on TCP
> sockets which are required for live migration.
>
> Wido
>
>
> Thanks
>> Özhan
>>
>> On Mon, Jan 15, 2018 at 11:34 PM, Paul Angus <paul.an...@shapeblue.com>
>> wrote:
>>
>> Hey guys,
>>>
>>> I'm no Ubuntu Guru, so I want to check that I have this right
>>>
>>> Our documentation says:
>>> --
>>> On Ubuntu: modify /etc/default/libvirt-bin
>>> Add "-l" to the following line
>>> libvirtd_opts="-d"
>>> so it looks like:
>>> libvirtd_opts="-d -l"
>>> ---
>>>
>>> I've found on Ubuntu 16.04 that you seem to need:
>>> /etc/default/libvirt-bin
>>> to have:
>>> libvirtd_opts="-l"
>>>
>>> and
>>> /etc/init/libvirt-bin.conf
>>> to have:
>>> env libvirtd_opts="-d -l"
>>>
>>> Can anyone confirm/deny this and should it also be this way for Ubuntu
>>> 14.04 ?
>>>
>>>
>>> If so I'll update the documentation...
>>>
>>>
>>> cheers
>>>
>>>
>>>
>>> Kind regards,
>>>
>>> Paul Angus
>>>
>>>
>>> paul.an...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>>
>>>
>>

Re: KVM on ubuntu

[Özhan Rüzgar Karaman]

Hi Paul;
Ubuntu 16.04 has systemd installed and it directly controls libvirtd
process so if you add -d to /etc/default/libvirt-bin file libvirtd tries to
control to its self and then you have problem, you could not
stop/start/restart libvirtd service via systemd. Only adding -l flag to
libvirtd_opts is enough for Cloudstack.

We also make no change on /etc/init/libvirt-bin.conf file for 16.04

For Ubuntu 16.04 you need to add both -d and -l flags for
/etc/default/libvirt-bin
file.

Thanks
Özhan

On Mon, Jan 15, 2018 at 11:34 PM, Paul Angus 
wrote:

> Hey guys,
>
> I'm no Ubuntu Guru, so I want to check that I have this right
>
> Our documentation says:
> --
> On Ubuntu: modify /etc/default/libvirt-bin
> Add "-l" to the following line
> libvirtd_opts="-d"
> so it looks like:
> libvirtd_opts="-d -l"
> ---
>
> I've found on Ubuntu 16.04 that you seem to need:
> /etc/default/libvirt-bin
> to have:
> libvirtd_opts="-l"
>
> and
> /etc/init/libvirt-bin.conf
> to have:
> env libvirtd_opts="-d -l"
>
> Can anyone confirm/deny this and should it also be this way for Ubuntu
> 14.04 ?
>
>
> If so I'll update the documentation...
>
>
> cheers
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan, i know that 4.9.x is lts release and 4.10 and 4.11 are not lts
releases. What will happen when new lts release comes, does this release
fork from 4.9.x branch and apply fixes on 4.10 and 4.1x over that or new
lts release will forward fork from 4.11 or 4.1x releases?

I am curious about this procedure and I also ask this question because we
are using a fix ( https://issues.apache.org/jira/browse/CLOUDSTACK-9538 )
which is only on 4.9 branch and not available for 4.1x branches, i am not
developer so i need to find a developer from community to port this fix to
new 4.1x trees, if upcoming lts release will fork from 4.1x releases :)


On Mon, Nov 13, 2017 at 12:41 PM, Ivan Kudryavtsev <kudryavtsev...@bw-sw.com
> wrote:

> I don't know if 4.10 will be improved centrally. AFAIK 4.11 WIP right
> now...
>
> 13 нояб. 2017 г. 16:37 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> > Hi Ivan;
> > I like to try virtio-scsi, i like to use its benefits like enlarging data
> > disks without rebooting vm and getting unused space in vm back to my
> > central storage, we also waited for long time, lets wait some more for
> 4.10
> > to become more stable :) Thanks for all replies
> >
> > Özhan
> >
> > On Mon, Nov 13, 2017 at 12:29 PM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > wrote:
> >
> > > Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi
> and
> > > ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
> > > waited for 4.10 for a long time...
> > >
> > > 13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
> > > oruzgarkara...@gmail.com> написал:
> > >
> > > > ok we will try to compile 4.10 from source and try it on our test
> > > > environment.
> > > >
> > > > We currently use 4.9.x on production, i make this test for deciding
> if
> > > 4.10
> > > > is suitable & stable for us, we also use very simple and generic
> setup
> > > for
> > > > this test no ipv6 just simple environment . So for production it
> looks
> > > like
> > > > we need to wait some time for upcoming releases, what do you think
> > about
> > > > that?
> > > >
> > > > There was a thread about this question one month ago and i remember
> > that
> > > > most of people still stick to 4.9 release for their production
> > > > environments...
> > > >
> > > > Thanks
> > > > Özhan
> > > >
> > > > On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU <ustcweiz...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Ivan,
> > > > >
> > > > > I would suggest you to create jira tickets for each problem you
> found
> > > in
> > > > > your testing, and create a github pull request for a jira ticket.
> > > > > It is convenient for reviewers.
> > > > >
> > > > > Kind regards,
> > > > > Wei
> > > > >
> > > > > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > >:
> > > > >
> > > > > > Hello, Ozhan
> > > > > >
> > > > > > https://github.com/apache/cloudstack/pull/2320
> > > > > >
> > > > > > fixes everything I found right now. It enables functioning of
> > > > everything
> > > > > > correctly even if no IPv6 CIDR specified for network (at least
> for
> > > > Ubuntu
> > > > > > 14.04).
> > > > > > For IPv6 configuration instruction please take a look at:
> > > > > > https://github.com/apache/cloudstack/commit/
> > > > > f10c8bfe0c99a762c2606459413a47
> > > > > > 219614e775
> > > > > > (oh my god,I spend several hours trying to find how to configure
> > IPv6
> > > > for
> > > > > > 4.10).
> > > > > >
> > > > > > Please, don't forget to recreate SSVM because there is a fix for
> > > > > templates
> > > > > > too:
> > > > > > https://github.com/apache/cloudstack/pull/2322
> > > > > >
> > > > > >
> > > > > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > > > > oruzgarkara...@gmail.com
> > > > > > >:
> > > > 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan;
I like to try virtio-scsi, i like to use its benefits like enlarging data
disks without rebooting vm and getting unused space in vm back to my
central storage, we also waited for long time, lets wait some more for 4.10
to become more stable :) Thanks for all replies

Özhan

On Mon, Nov 13, 2017 at 12:29 PM, Ivan Kudryavtsev <kudryavtsev...@bw-sw.com
> wrote:

> Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi and
> ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
> waited for 4.10 for a long time...
>
> 13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> > ok we will try to compile 4.10 from source and try it on our test
> > environment.
> >
> > We currently use 4.9.x on production, i make this test for deciding if
> 4.10
> > is suitable & stable for us, we also use very simple and generic setup
> for
> > this test no ipv6 just simple environment . So for production it looks
> like
> > we need to wait some time for upcoming releases, what do you think about
> > that?
> >
> > There was a thread about this question one month ago and i remember that
> > most of people still stick to 4.9 release for their production
> > environments...
> >
> > Thanks
> > Özhan
> >
> > On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU <ustcweiz...@gmail.com>
> wrote:
> >
> > > Hi Ivan,
> > >
> > > I would suggest you to create jira tickets for each problem you found
> in
> > > your testing, and create a github pull request for a jira ticket.
> > > It is convenient for reviewers.
> > >
> > > Kind regards,
> > > Wei
> > >
> > > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev <kudryavtsev...@bw-sw.com
> >:
> > >
> > > > Hello, Ozhan
> > > >
> > > > https://github.com/apache/cloudstack/pull/2320
> > > >
> > > > fixes everything I found right now. It enables functioning of
> > everything
> > > > correctly even if no IPv6 CIDR specified for network (at least for
> > Ubuntu
> > > > 14.04).
> > > > For IPv6 configuration instruction please take a look at:
> > > > https://github.com/apache/cloudstack/commit/
> > > f10c8bfe0c99a762c2606459413a47
> > > > 219614e775
> > > > (oh my god,I spend several hours trying to find how to configure IPv6
> > for
> > > > 4.10).
> > > >
> > > > Please, don't forget to recreate SSVM because there is a fix for
> > > templates
> > > > too:
> > > > https://github.com/apache/cloudstack/pull/2322
> > > >
> > > >
> > > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > > oruzgarkara...@gmail.com
> > > > >:
> > > >
> > > > > Hi Ivan;
> > > > > Does this hotfixes also solve qoutes and shell script interprets
> > > problem?
> > > > > We have no ipv6 setup and today we made similar test with fresh
> > install
> > > > > 4.10. We noticed that we receive similar error on security groups
> > stage
> > > > > while br_netfilter module is already active on our environment. We
> > made
> > > > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > > > >
> > > > > Logs are below:
> > > > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.
> > LibvirtComputingResource]
> > > > > (agentRequest-Handler-1:null) Executing:
> > > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > --vmip6
> > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > --rules
> > > > > I:tcp:1:65535:
> > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > 0.0.0.0/0,NEXT
> > > > ;
> > > > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.
> > LibvirtComputingResource]
> > > > > (agentRequest-Handler-1:null) Exception:
> > > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > --vmip6
> > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > 1e:00:9b:00:00:0

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

ok we will try to compile 4.10 from source and try it on our test
environment.

We currently use 4.9.x on production, i make this test for deciding if 4.10
is suitable & stable for us, we also use very simple and generic setup for
this test no ipv6 just simple environment . So for production it looks like
we need to wait some time for upcoming releases, what do you think about
that?

There was a thread about this question one month ago and i remember that
most of people still stick to 4.9 release for their production
environments...

Thanks
Özhan

On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Hi Ivan,
>
> I would suggest you to create jira tickets for each problem you found in
> your testing, and create a github pull request for a jira ticket.
> It is convenient for reviewers.
>
> Kind regards,
> Wei
>
> 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev <kudryavtsev...@bw-sw.com>:
>
> > Hello, Ozhan
> >
> > https://github.com/apache/cloudstack/pull/2320
> >
> > fixes everything I found right now. It enables functioning of everything
> > correctly even if no IPv6 CIDR specified for network (at least for Ubuntu
> > 14.04).
> > For IPv6 configuration instruction please take a look at:
> > https://github.com/apache/cloudstack/commit/
> f10c8bfe0c99a762c2606459413a47
> > 219614e775
> > (oh my god,I spend several hours trying to find how to configure IPv6 for
> > 4.10).
> >
> > Please, don't forget to recreate SSVM because there is a fix for
> templates
> > too:
> > https://github.com/apache/cloudstack/pull/2322
> >
> >
> > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com
> > >:
> >
> > > Hi Ivan;
> > > Does this hotfixes also solve qoutes and shell script interprets
> problem?
> > > We have no ipv6 setup and today we made similar test with fresh install
> > > 4.10. We noticed that we receive similar error on security groups stage
> > > while br_netfilter module is already active on our environment. We made
> > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > >
> > > Logs are below:
> > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Exception:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > java.lang.NullPointerException
> > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > addNetworkRules(LibvirtComputingResource.java:3429)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtRequestWrapper.execute(
> > > LibvirtRequestWrapper.java:75)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > executeRequest(
> > > LibvirtComputingResource.java:1369)
> > > at com.cloud.agent.Agent.processRequest(Agent.java:525)
> > > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> > > at com.cloud.utils.nio.Task.call(Task.java:83)
> > > at com.cloud.utils.nio.Task.call(Task.

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan;
Does this hotfixes also solve qoutes and shell script interprets problem?
We have no ipv6 setup and today we made similar test with fresh install
4.10. We noticed that we receive similar error on security groups stage
while br_netfilter module is already active on our environment. We made
same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts

Logs are below:
2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-1:null) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-1:null) Exception:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
at com.cloud.utils.script.Script.execute(Script.java:214)
at com.cloud.utils.script.Script.execute(Script.java:182)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.addNetworkRules(LibvirtComputingResource.java:3429)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:75)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1369)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-11-13 11:47:41,774 WARN
[resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
(agentRequest-Handler-1:null) Failed to program network rules for vm
i-2-5-VM
2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: , MgmtId:
345048635880, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"logSequenceNumber":16,"vmId":5,"reason":"PROGRAMMING_FAILED","result":false,"details":"programming
network rules failed","wait":0}}] }


When we execute command with double quotas for rules section from command
line it executes without a problem like below:
root@kvmt3:/var/log/cloudstack/agent#
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
"I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;"
root@kvmt3:/var/log/cloudstack/agent# echo $?
0
root@kvmt3:/var/log/cloudstack/agent#

Thanks
Özhan


On Sat, Nov 11, 2017 at 6:59 PM, Ivan Kudryavtsev 
wrote:

> Hello, I implemented some hotfixes for 4.10 to work
>
> https://github.com/apache/cloudstack/pull/2319 - to master (load
> br_netfilter module)
> https://github.com/apache/cloudstack/pull/2320 - to 4.10 which fixes SG
> failures related to ipv6.
>
>
> 2017-11-11 15:51 GMT+07:00 Ivan Kudryavtsev :
>
> > Following up with previous question. I managed to make it work by
> removing
> > all and heading to ubuntu 14.04 hypervisor host.
> >
> > Also, what I found more:
> >
> > 1. when setup databases (management server) if custom port is specified,
> > databases themself is not created. If create manually, import scripts
> work
> > fine.
> > 2. UI: unable to download ISO to __all__ zones. Have to specify certain
> > zone, else UI gives an error.
> > 3. Ubuntu doesn't load module *br_netfilter* but
> >
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> >
> > uses it and nothing good as a result:
> >
> > 2017-11-11 15:38:29,241 - sysctl -w net.bridge.bridge-nf-call-
> arptables=1
> > 2017-11-11 15:38:29,244 - sysctl -w net.bridge.bridge-nf-call-iptables=1
> > 2017-11-11 15:38:29,247 - 

Re: 4.10 release announcement?

[Özhan Rüzgar Karaman]

Hi Rajani;
Is the 4.10 documentation all ready? I could not see the all documentation
on web site like installation, source code compile and etc.?

Most of the docs currently only available for 4.9 release.

Thanks
Özhan

On Thu, Aug 31, 2017 at 8:18 AM, Rajani Karuturi  wrote:

> I can see the latest rn docs now. I will add apidocs and make the
> release announcement in a day.
>
> Thanks,
>
> ~ Rajani
>
> http://cloudplatform.accelerite.com/
>
> On August 30, 2017 at 5:58 PM, Rohit Yadav
> (rohit.ya...@shapeblue.com) wrote:
>
> Rajani - I've added you to the list of maintainers, please
> tag/push changes, log in and build/publish them once your changes
> pass locally (do a local build first).
>
> I've also kicked build to publish 4.10 now.
>
> - Rohit
>
> 
> From: Rajani Karuturi 
> Sent: Wednesday, August 30, 2017 1:53:04 PM
> To: dev@cloudstack.apache.org; Rohit Yadav
> Cc: Wido den Hollander
> Subject: Re: 4.10 release announcement?
>
> branch is already there and version is set to 4.10.0.0. I
> haven't tagged it yet.
> https://github.com/apache/cloudstack-docs-rn/tree/4.10
>
> I created an account RTD. username is rajani. Can you give the
> required access?
>
> ~ Rajani
>
> http://cloudplatform.accelerite.com/
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com ( http://www.shapeblue.com )
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
> On August 30, 2017 at 3:06 PM, Rohit Yadav
> (rohit.ya...@shapeblue.com)
> wrote:
>
> Rajani,
>
> After you've update the cloudstack-docs-rn, you need to create a
> branch 4.10 or tag 4.10.0.0 (see existing branches/tags and other
> previous releases for example). Once you're done, I can help with
> publishing this on read-the-docs, if you're a member of the
> maintainers of release-notes you may log in and kick the
> builds/publish yourself.
>
> I don't see you here:
> https://readthedocs.org/dashboard/cloudstack-release-notes/users
>
> You may create an account and share with us your username and I
> can add you to the cloudstack release-notes project on
> read-the-docs.
>
> - Rohit
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com ( http://www.shapeblue.com
> @shapeblue
>
> From: Rajani Karuturi
> >
> Sent: Wednesday, August 30, 2017 7:01:48 AM
> To: Rohit Yadav;
> dev@cloudstack.apache.org
> Cc: Wido den Hollander
> Subject: Re: 4.10 release announcement?
>
> readme only talks about how to create a PR, use translations
> etc. not about how to push the generated html to RTD website.
> There should be some automated jenkins job to do it.
>
> These are the contents of make.sh. It shows me the generated
> html locally. But, there is not way to push it.
>
> rm -fr build
>
> mkdir build
>
> sphinx-build -b html source build
>
> Thanks,
>
> ~ Rajani
>
> http://cloudplatform.accelerite.com/
>
> On August 28, 2017 at 6:07 PM, Will Stevens
> (williamstev...@gmail.com)
> wrote:
>
> In this case it is sphinx and not middleman, but I spent a lot
> of time on
> the README in the past to make sure the contribution details are
> clear, so
> if anything is not clear please highlight the details.
>
> Cheers,
>
> Will
>
> On Mon, Aug 28, 2017 at 8:22 AM, Rohit Yadav
> >
> wrote:
>
> Rajani,
>
> Yes middleman is used, kindly see the build/run scripts in the
> repository
> for details.
>
> - Rohit
> --
> *From:* Rajani Karuturi
> >
> *Sent:* Sunday, August 20, 2017 6:47:14 AM
> *To:* Will Stevens
> *Cc:* Rohit Yadav; Wido den Hollander;
> dev@cloudstack.apache.org
> *Subject:* Re: 4.10 release announcement?
>
> Middleman is used only for website I think. For doc sites it's
> different.
>
> ~Rajani
>
> Sent from phone.
>
> On 18 Aug 2017 8:11 pm, "Will Stevens"
> >
> wrote:
>
> The HTML is built using a build script in the repo locally by
> the person
> who suggests a change. The build script actually uses middleman
> to
>
> generate
>
> the html. The generated HTML is tracked in the pull requests so
> merges
> result in updates to the published content.
>
> The readme should be up to date and the details should be clear,
> but I
> have not looked in a little while.
>
> I am on vacation right now, but I can check the details when I
> am back.
>
> Cheers,
>
> Will
>
> On Aug 18, 2017 2:32 AM, "Rajani Karuturi"
> > wrote:
>
> Hi Wido/Will,
>
> Do you know how the html is generated for readthedocs? Is that
> through
>
> any
>
> jenkins job? I think if atleast release notes are ready, we can
> make an
> 

Re: Release packages for 4.9.3.0

[Özhan Rüzgar Karaman]

Hi Wido;
I checked http://cloudstack.apt-get.eu/ web site and 4.9.3 packages are
only under /ubuntu/dists/xenial/4.9/pool/ directory, for trusty there are
no packages available for 4.9.3 .

This directory(/ubuntu/dists/xenial/4.9/pool/) also have 4.10 packages as
well, which i think they should not be there.

When you have suitable time could you check the packages and its script.

Thanks
Özhan

On Mon, Sep 25, 2017 at 7:24 AM, Rohit Yadav 
wrote:

> Thanks Wido, can you help building and uploading of rpms as well. Maybe
> Pierre-Luc can help?
>
>
> - Rohit
>
> 
> From: Wido den Hollander 
> Sent: Thursday, September 21, 2017 1:09:49 PM
> To: Rohit Yadav; dev@cloudstack.apache.org; Pierre-Luc Dion
> Subject: Re: Release packages for 4.9.3.0
>
> Ah, sorry! The DEB packages should have been uploaded already :-)
>
> Wido
>
> > Op 21 september 2017 om 7:33 schreef Rohit Yadav <
> rohit.ya...@shapeblue.com>:
> >
> >
> > Ping - Wido/PL?
> >
> >
> > - Rohit
> >
> > 
> > From: Rohit Yadav 
> > Sent: Tuesday, September 12, 2017 5:44:36 PM
> > To: Wido den Hollander; Pierre-Luc Dion
> > Cc: dev@cloudstack.apache.org
> > Subject: Release packages for 4.9.3.0
> >
> > Wido/PL/others,
> >
> >
> > Can you please help with building and publishing of 4.9.3.0 rpms/deb
> packages on the download.cloudstack.org repository? I've built and
> published the repos on packages.shapeblue.com now (shapeblue.com/packages
> for details).
> >
> >
> > Regards.
> >
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: Repository for Ubuntu xenial (16.04) broken for CloudStack 4.10

[Özhan Rüzgar Karaman]

We are waiting for 4.10 installation and other 4.10 documentations(like
source compile) since long time there was a thread about this documentation
generation on mailing list but it looks like the problem on that step did
not solved yet, its still waiting :(

+1 for Christians & Rafaels questions

On Tue, Sep 26, 2017 at 5:23 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Folks,
>
> How is the status of the 4.10 release? I thought it was already released,
> right?
>
> Even though all of the CloudStack packages are in the right folders for
> Precise and Xenial, it seems that only the “Release” file of Ubuntu Trusty
> was updated with the new release files.
>
> Moreover, reading the latest docs found in [1], I would expect only precise
> and Trusty to be supported because of the following sentence there:
> “Please note that only packages for Ubuntu 12.04 LTS (precise) and Ubuntu
> 14.04 (trusty) are being built at this time.”
>
> So, my questions are the following:
>
>- Is Ubuntu Precise still supported?
>   - If not, we must change the docs (I can open a Jira ticket and PR
>   for that)
>- Is Ubuntu Xenial supported?
>   - If not, we do not need to do anything here. (perhaps removing the
>   files from the repo would be a good idea?)
>   - If it is supported, we must fix the repo. Does anybody know how to
>   access the “cloudstack.apt-get.eu” repository? Is it managed by the
>   PMC?
>
> One last thing, was the documentation page for ACS 4.10 generated?
>
> [1]
> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.9/
> management-server/index.html#deb-package-repository
>
> On Mon, Sep 25, 2017 at 8:14 PM, Christian Roeder <
> christian.roe...@red-hood.de> wrote:
>
> > Hi,
> >
> >
> > I just wanted to install CloudStack 4.10 on Ubuntu 16.04 using the
> > official repository at http://cloudstack.apt-get.eu/ubuntu/, but it
> > seems the .deb-files in the pool are actually not listed in the release
> > file and therefore can not be found by apt. Will there be a fix for the
> > repository?
> >
> >
> > Regards,
> > Christian
> > --
> > Christian Röder
> > Jabber: red_h...@jabber.ccc.de
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: Welcoming Wido as the new ACS VP

[Özhan Rüzgar Karaman]

Great News & Congratulations Wido :)

Thanks Will for your all effort & energy :)

On Thu, Mar 16, 2017 at 9:57 PM, Wei ZHOU  wrote:

> Congratulations Wido !
>
> Thanks you for your work as ACS VP, Will!
>
> -Wei
>
>
> 2017-03-16 18:00 GMT+01:00 Will Stevens :
>
> > Hello Everyone,
> > It has been a pleasure working with you as the ACS VP over the past year.
> > I would like to say Thank You to everyone who has supported me in this
> role
> > and have supported the project as a whole.
> >
> > It is my pleasure to announce that Wido den Hollander has been voted in
> to
> > replace me as the Apache Cloudstack VP in our annual VP rotation.  Wido
> has
> > a long history with the project and we are happy welcome him into this
> new
> > role.
> >
> > Be sure to join us at CCC in Miami [1] so we can initiate him correctly
> > over many beers.  :)
> >
> > Cheers,
> >
> > *Will Stevens*
> >
> > ​[1] http://us.cloudstackcollab.org/​
> >
>

Re: ACS 4.9.0 -> 4.9.2 upgrade and VR python file changes

[Özhan Rüzgar Karaman]

Hi Wei;
You are right, one of the kvm server which holds vr did not updated its CS
packages. After upgrading packages and rebooting vr python files are now
updated as expected.

For each CS minor or major upgrade, it looks like we need to reboot system
vm's and vr's to reread and process new systemvm.iso file. Is this correct
or CS handles this upgrade by other technique?

What is the required procedure to update files inside vr or systemvms, just
rebooting them? If yes i think we need to add this to upgrade procedure
document.

Thanks again for your fast response.
Özhan



On Mon, Jan 16, 2017 at 11:17 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Ozhan,
>
> The VR scripts are included in /usr/share/cloudstack-common/
> vms/systemvm.iso
> You have to upgrade cloudstack-common on KVM hosts.
>
>
> -Wei
>
>
>
> 2017-01-16 9:08 GMT+01:00 Özhan Rüzgar Karaman <oruzgarkara...@gmail.com>:
>
> > Hi;
> > I have successfully upgraded my test environments to 4.9.0 to 4.9.2 on
> > Ubuntu KVM env. I like to ask a question about VR update for 4.9.2.
> >
> > I am aware that there is no update for VR but i also notices that there
> are
> > some pr's which changes some files under VR like
> > https://github.com/apache/cloudstack/pull/1743 etc.
> >
> > For example this above pr updates the CsAddress.py file, i checked the
> > files md5sum at 4.9.0 release and after also upgrading to 4.9.2, i also
> > rebooted and destroy vr to repopulate the files but it looks like its
> file
> > is same like the 4.9.0 release.
> >
> > I also checked all files under /opt/cloud/bin/ are same as like in 4.9.0
> > release.
> >
> > How could i trigger VR to get new updated python files for 4.9.2 release
> > from management server.
> >
> > Thanks
> > Ozhan
> >
>

ACS 4.9.0 -> 4.9.2 upgrade and VR python file changes

[Özhan Rüzgar Karaman]

Hi;
I have successfully upgraded my test environments to 4.9.0 to 4.9.2 on
Ubuntu KVM env. I like to ask a question about VR update for 4.9.2.

I am aware that there is no update for VR but i also notices that there are
some pr's which changes some files under VR like
https://github.com/apache/cloudstack/pull/1743 etc.

For example this above pr updates the CsAddress.py file, i checked the
files md5sum at 4.9.0 release and after also upgrading to 4.9.2, i also
rebooted and destroy vr to repopulate the files but it looks like its file
is same like the 4.9.0 release.

I also checked all files under /opt/cloud/bin/ are same as like in 4.9.0
release.

How could i trigger VR to get new updated python files for 4.9.2 release
from management server.

Thanks
Ozhan

Re: Welcoming Simon Weller & Paul Angus to the PMC

[Özhan Rüzgar Karaman]

i agree with Erik, they deserved, congrats :)

On Fri, Jan 13, 2017 at 8:08 PM, Erik Weber  wrote:

> Congratulations both of you, well deserved :-)
>
> --
> Erik
> fre. 13. jan. 2017 kl. 17.30 skrev Will Stevens :
>
> > Join me in welcoming Simon Weller and Paul Angus to the Apache CloudStack
> >
> > PMC.  Both have been dedicated members of the community and it is with
> >
> > great pleasure that we welcome them to the PMC.
> >
> >
> >
> > Next time you see either of them, buy them a drink.  :)
> >
> >
> >
> > Welcome guys...
> >
> >
>

Re: 4.9 api docs went missing

[Özhan Rüzgar Karaman]

its working again, thanks Rohit

On Fri, Jan 6, 2017 at 9:58 AM, Rohit Yadav 
wrote:

> I've fixed it now, please recheck.
>
>
> Regards.
>
> 
> From: ?zhan R?zgar Karaman 
> Sent: 04 January 2017 17:59:10
> To: dev@cloudstack.apache.org
> Subject: Re: 4.9 api docs went missing
>
> Hi;
> I think they are setting up the api page for new CloudStack release, its
> not accessible till this Monday.
>
> Thanks
> ?zhan
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Wed, Jan 4, 2017 at 12:42 PM, Linas ?ilinskas  > wrote:
>
> Hello.
>
> Not sure if intentional, but 4.9 api docs at
> https://cloudstack.apache.org/api.html are missing now.
>
> It seems the page got reverted to a previous version.
>
> Linas ?ilinskas
> Head of Development
> [cid:part1.9B217719.D668B006@host1plus.com]
> website facebook com/Host1Plus> twitter linkedin<
> https://www.linkedin.com/company/digital-energy-technologies-ltd.>
>
> Host1Plus is a division of Digital Energy Technologies Ltd.
>
> 26 York Street, London W1U 6PZ, United Kingdom
>
>
>
>

Re: 4.9 api docs went missing

[Özhan Rüzgar Karaman]

Hi;
I think they are setting up the api page for new CloudStack release, its
not accessible till this Monday.

Thanks
Özhan

On Wed, Jan 4, 2017 at 12:42 PM, Linas Žilinskas 
wrote:

> Hello.
>
> Not sure if intentional, but 4.9 api docs at
> https://cloudstack.apache.org/api.html are missing now.
>
> It seems the page got reverted to a previous version.
>
> Linas Žilinskas
> Head of Development
> website  facebook
>  twitter
>  linkedin
> 
>
> Host1Plus is a division of Digital Energy Technologies Ltd.
>
> 26 York Street, London W1U 6PZ, United Kingdom
>
>
>

Re: 4.8.2.0/4.9.1.0/4.10.0.0 RC Status

[Özhan Rüzgar Karaman]

Ok John, i will ping you and Rohit again when we start to work for 4.9.2
release. Thanks for your time.

Özhan

On Thu, Nov 17, 2016 at 5:32 PM, John Burwell <john.burw...@shapeblue.com>
wrote:

> Ozhan,
>
> As I said on the PR, #1710 addresses a fairly narrow case (RBD on KVM),
> and 4.9.1.0 is extremely late.  There are a lot “small” fixes that could be
> included.  However, it would further delay delivery of significant dies
> that nearly all users.  At some point, we must draw a line, and this fix
> will have to wait until 4.9.2.0/4.10.1.0/4.11.0.0.
>
> Thanks,
> -John
>
>
> john.burw...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
> > On Nov 17, 2016, at 5:22 AM, Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com> wrote:
> >
> > Hi John & Rohit;
> > If we have time could we add PR 1710 to the upcoming RC for 4.9.1.0. Its
> a
> > small fix but its waiting for log time...
> >
> > Thanks again,
> > Özhan
> >
> > On Thu, Nov 17, 2016 at 9:23 AM, John Burwell <
> john.burw...@shapeblue.com>
> > wrote:
> >
> >> All,
> >>
> >> I apologize for being relatively radio silent.  We have mage good
> progress
> >> towards getting RCs out for 4.8.2.0, 4.9.1.0, and 4.10.0.0.  On 31
> October
> >> 2016, we 17 outstanding PRs to be merged.  As of today (17 Nov 2016), we
> >> have 9 PRs to merge with pending one potential blocker/critical defect
> fix:
> >>
> >> * 4.10.0.0 (+ all 4.8.2.0 and 4.9.1.0 PRs)
> >> * 1542 (regression tests running)
> >>  * 1545 (code review comments; regression tests pending)
> >>  * 1577 (Jenkins failures; regression tests pending)
> >>  * 1579 (needs rebase + commit squash; regression tests pending)
> >>  * 1580 (needs rebase + commit squash; regression tests pending)
> >> * 4.9.1.0 (+ all 4.8.2.0 PRs)
> >>  * 1681 (test failures to resolve)
> >>  * 1684 (working out the proper fix to an upgrade issue)
> >> * 4.8.2.0
> >>* 1745 (test failures to resolve)
> >>* 1765 (squash commits; code review feedback; regression tests
> >> pending)
> >>
> >> Once all PRs are merged, we will re-execute the tests on each of the
> >> release branches.  Rohit has opened tracking PRs for 4.8 [1], 4.9 [2],
> and
> >> master [3] to understand the state of the release branches as we
> continue
> >> to merge PRs.
> >>
> >> Thank you to everyone who has reviewed, tested, and merged PRs.  I am
> >> hopeful that we are very close to cutting these RCs.
> >>
> >> Thanks again,
> >> -John
> >>
> >> [1]: https://github.com/apache/cloudstack/pull/1752
> >> [2]: https://github.com/apache/cloudstack/pull/1753
> >> [3]: https://github.com/apache/cloudstack/pull/1754
> >>
> >> john.burw...@shapeblue.com
> >> www.shapeblue.com
> >> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >>
>
>

Re: 4.8.2.0/4.9.1.0/4.10.0.0 RC Status

[Özhan Rüzgar Karaman]

Hi John & Rohit;
If we have time could we add PR 1710 to the upcoming RC for 4.9.1.0. Its a
small fix but its waiting for log time...

Thanks again,
Özhan

On Thu, Nov 17, 2016 at 9:23 AM, John Burwell 
wrote:

> All,
>
> I apologize for being relatively radio silent.  We have mage good progress
> towards getting RCs out for 4.8.2.0, 4.9.1.0, and 4.10.0.0.  On 31 October
> 2016, we 17 outstanding PRs to be merged.  As of today (17 Nov 2016), we
> have 9 PRs to merge with pending one potential blocker/critical defect fix:
>
> * 4.10.0.0 (+ all 4.8.2.0 and 4.9.1.0 PRs)
> * 1542 (regression tests running)
>   * 1545 (code review comments; regression tests pending)
>   * 1577 (Jenkins failures; regression tests pending)
>   * 1579 (needs rebase + commit squash; regression tests pending)
>   * 1580 (needs rebase + commit squash; regression tests pending)
> * 4.9.1.0 (+ all 4.8.2.0 PRs)
>   * 1681 (test failures to resolve)
>   * 1684 (working out the proper fix to an upgrade issue)
> * 4.8.2.0
> * 1745 (test failures to resolve)
> * 1765 (squash commits; code review feedback; regression tests
> pending)
>
> Once all PRs are merged, we will re-execute the tests on each of the
> release branches.  Rohit has opened tracking PRs for 4.8 [1], 4.9 [2], and
> master [3] to understand the state of the release branches as we continue
> to merge PRs.
>
> Thank you to everyone who has reviewed, tested, and merged PRs.  I am
> hopeful that we are very close to cutting these RCs.
>
> Thanks again,
> -John
>
> [1]: https://github.com/apache/cloudstack/pull/1752
> [2]: https://github.com/apache/cloudstack/pull/1753
> [3]: https://github.com/apache/cloudstack/pull/1754
>
> john.burw...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Özhan Rüzgar Karaman]

Hi;
Whats your problematic vm's type is it Debian? Maybe you are affected from
the bug CLOUDSTACK-8326? I do not know if this bug has effected on ACS 4.2
release but i know that it effects release 4.8.x 4.9.x

Thanks
Özhan

On Mon, Nov 7, 2016 at 4:36 PM, Cloud List  wrote:

> Hi Wei,
>
> Thanks for your reply.
>
> I checked and I can confirm that the mac address is listed on
> /etc/dhcphosts.txt in VR.
>
> For example:
>
> Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
> ignored
> Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
> ignored
> Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
> ignored
>
> root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 /etc/dhcphosts.txt
> 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite
>
> YY - last two digits of the mac address
> X.X.X.X - ip address which is supposed to be allocated to the VM
>
> Any advice how can I troubleshoot this further?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
> On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU  wrote:
>
> > If the mac address is not listed in /etc/dhcphosts.txt in VR, the request
> > will be ignored.
> >
> > Can you give more details so we can reproduce it and fix it ?
> >
> > -Wei
> >
> > 2016-11-07 13:44 GMT+01:00 Cloud List :
> >
> > > Hi,
> > >
> > > After our upgrade from CloudStack 4.2 to 4.8.1.1, we noted that some
> (but
> > > not all) of our VMs are not able to get DHCP from the VR. This gives
> > > problem when the VM is restarted and cannot get up and running because
> > > unable to get IP.
> > >
> > > I logged in to the VR and found below messages showing that the DHCP
> > server
> > > is ignoring the request from the VM.
> > >
> > > Nov  7 12:19:43 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:19:52 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:05:64:01:13:d3
> > > ignored
> > > Nov  7 12:19:53 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:20:04 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:20:11 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:05:64:01:13:d3
> > > ignored
> > > Nov  7 12:20:22 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:20:30 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:20:42 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:21:02 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > > Nov  7 12:21:19 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0)
> 06:44:da:01:13:98
> > > ignored
> > >
> > > Any reason why the dnsmasq service ignored the request from the VM and
> > how
> > > can we fix that?
> > >
> > > At the moment, the only workaround we can do is to configure the IP
> > address
> > > statically to the servelet, which is not practical.
> > >
> > > Any advice is greatly appreciated.
> > >
> > > Thank you.
> > >
> >
>

Re: long waiting times on routervm for savepassword and DhcpEntryCommand's

[Özhan Rüzgar Karaman]

Hi Remi;
Does this fix approved for upcoming 4.9 release?

Thanks
Özhan

On Fri, Jun 3, 2016 at 11:29 PM, Remi Bergsma <rberg...@schubergphilis.com>
wrote:

> Hi Wido,
>
> I've fixed that recently, just need to find the time to backport it :-s
>
> https://github.com/MissionCriticalCloud/cosmic-core/pull/138
>
> Regards, Remi
>
> Sent from my iPhone
>
> > On 03 Jun 2016, at 12:10, Wido den Hollander <w...@widodh.nl> wrote:
> >
> >
> >> Op 1 juni 2016 om 11:10 schreef Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com>:
> >>
> >>
> >> Hi Developers;
> >> I could find any answer from Cloudstack User Mailing list, i am
> forwarding
> >> my mail to the development list, thanks for all answers who has
> experience
> >> on high capacity virtual router usage.
> >>
> >> Thanks
> >> Özhan
> >>
> >> Today i noticed that my vm creation commands takes a little bit long
> then
> >> before. When i check the logs i noticed that router vm takes too much
> time
> >> like 4 seconds for each savepassword and DhcpEntryCommands.
> >
> > I checked this quickly on a VR in Basic Networking and I see the same.
> It's that it is executing Python code for every Instance which has to be
> provisioned on that VR.
> >
> > I don't know of a quick way to work around this however, this probably
> requires parts of that code to be rewritten.
> >
> > Wido
> >
> >> We are using advanced networking with security groups enabled for ACS
> 4.8 .
> >> It looks like when ip address usage increases(100 ip addresess for one C
> >> Class network) for each guest network block, router vm responds slower
> than
> >> the old days.
> >>
> >> We are using default compute offering for virtual router, my plan is to
> >> change to default offering for virtual router vm to 512 mb ram & 2 core
> cpu
> >> and increase cpu shares and restart virtual router.
> >>
> >> Are there any other cleanup procedures or other ways to speed up
> >> savepassword and DhcpEntryCommands like disable logging or any other
> clues
> >> would be helpful. Because of this long waiting times on 2 commands
> total vm
> >> creation time takes longer then expected.
>

Re: long waiting times on routervm for savepassword and DhcpEntryCommand's

[Özhan Rüzgar Karaman]

Hi Wido;

On Fri, Jun 3, 2016 at 1:09 PM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 1 juni 2016 om 11:10 schreef Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com>:
> >
> >
> > Hi Developers;
> > I could find any answer from Cloudstack User Mailing list, i am
> forwarding
> > my mail to the development list, thanks for all answers who has
> experience
> > on high capacity virtual router usage.
> >
> > Thanks
> > Özhan
> >
> > Today i noticed that my vm creation commands takes a little bit long then
> > before. When i check the logs i noticed that router vm takes too much
> time
> > like 4 seconds for each savepassword and DhcpEntryCommands.
> >
>
> I checked this quickly on a VR in Basic Networking and I see the same.
> It's that it is executing Python code for every Instance which has to be
> provisioned on that VR.
>
> I don't know of a quick way to work around this however, this probably
> requires parts of that code to be rewritten.
>

This negatively effects new vm instance total creation time :( . When you
add new ip, it spends lots of time on parsing data on routervm's json cache
files so if you have more ip addresses or more cloudinit userdata, it
spends more and more time on that phase. It looks like parsing code needs
to be optimised.

@Wido do you want me to open a bug report on Jira about this issue, that a
developer who is responsible for this code could be aware & mention about
this problem and work on it.

Thanks
Özhan


>
> Wido
>
> > We are using advanced networking with security groups enabled for ACS
> 4.8 .
> > It looks like when ip address usage increases(100 ip addresess for one C
> > Class network) for each guest network block, router vm responds slower
> than
> > the old days.
> >
> > We are using default compute offering for virtual router, my plan is to
> > change to default offering for virtual router vm to 512 mb ram & 2 core
> cpu
> > and increase cpu shares and restart virtual router.
> >
> > Are there any other cleanup procedures or other ways to speed up
> > savepassword and DhcpEntryCommands like disable logging or any other
> clues
> > would be helpful. Because of this long waiting times on 2 commands total
> vm
> > creation time takes longer then expected.
>

long waiting times on routervm for savepassword and DhcpEntryCommand's

[Özhan Rüzgar Karaman]

Hi Developers;
I could find any answer from Cloudstack User Mailing list, i am forwarding
my mail to the development list, thanks for all answers who has experience
on high capacity virtual router usage.

Thanks
Özhan

Today i noticed that my vm creation commands takes a little bit long then
before. When i check the logs i noticed that router vm takes too much time
like 4 seconds for each savepassword and DhcpEntryCommands.

We are using advanced networking with security groups enabled for ACS 4.8 .
It looks like when ip address usage increases(100 ip addresess for one C
Class network) for each guest network block, router vm responds slower than
the old days.

We are using default compute offering for virtual router, my plan is to
change to default offering for virtual router vm to 512 mb ram & 2 core cpu
and increase cpu shares and restart virtual router.

Are there any other cleanup procedures or other ways to speed up
savepassword and DhcpEntryCommands like disable logging or any other clues
would be helpful. Because of this long waiting times on 2 commands total vm
creation time takes longer then expected.

Re: Problem enabling Intermediate SSL Certificate on Console VM

[Özhan Rüzgar Karaman]

Hi;
I solved the issue, somehow Alpha SSL has an extra space character in their
intermediate CA file. So i removed the space character and reinstall the
intermediate CA an now Console VM is fine. I will also remind Alpha SSL
about the extra space character in their intermediate CA file.

Thanks for your time.

Regards
Özhan

On Wed, May 18, 2016 at 1:08 PM, Özhan Rüzgar Karaman <
oruzgarkara...@gmail.com> wrote:

> Hi Developers;
> My console vm works successfully over SSL connections. Yesterday we
> realised that firefox could not validate our SSL certificate and it gives
> us certificate validation errors.
>
> We checked keystore table on database and noticed that we have not
> imported intermediate certificate and console vm works over SSL without any
> intermediate SSL certificates. We checked Alpha SSL web site and downloaded
> the certificate file(
> https://www.alphassl.com/support/install-root-certificate.html)  and
> delete the keystore table and re import all root cert + intermediate +
> server cert + private key from Cloudstack Admin interface. After that we
> have checked the console vm logs(/var/log/cloud/cloud.out) and we noticed
> that it could not successfully download the ssl certificate from ACS. The
> errors are below.
>
> I checked the keystore table and it looks okey. After that i restored the
> keystore table from backup which does not have any intermediate
> certificate, console vm started to work, but because we do not have
> intermediate certificate in console vm, windows firefox clients again could
> not connect to console sessions.
>
> Does anyone experience this kind of problem on enabling intermediate
> certificate? Also which kind of intermediate certificate format need to be
> used on ACS, are all formats valid for CloudStack 4.8 ? Alpha SSL provides
> SHA-1 and SHA-256 formats for intermediate certificates.
>
> Thanks for all responses & time.
>
> Regards
> Özhan
>
>
>
> 2016-05-18 09:45:56,205 INFO
>  [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
> (Console-Proxy-Main:null) Start initializing SSL
> 2016-05-18 09:45:56,205 INFO
>  [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
> (Console-Proxy-Main:null) No certificates passed, recheck global
> configuration and certificates
> 2016-05-18 09:45:56,205 INFO
>  [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
> (Console-Proxy-Main:null) Start initializing SSL
> 2016-05-18 09:45:56,206 INFO
>  [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
> (Console-Proxy-Main:null) No certificates passed, recheck global
> configuration and certificates
> 2016-05-18 09:45:56,227 ERROR
> [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
> (Console-Proxy-Main:null) java.lang.NullPointerException: null SSLContext
> java.lang.NullPointerException: null SSLContext
> at
> com.sun.net.httpserver.HttpsConfigurator.(HttpsConfigurator.java:81)
> at
> com.cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl$1.(ConsoleProxySecureServerFactoryImpl.java:82)
> at
> com.cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl.createHttpServerInstance(ConsoleProxySecureServerFactoryImpl.java:82)
> at
> com.cloud.consoleproxy.ConsoleProxy.startupHttpMain(ConsoleProxy.java:356)
> at com.cloud.consoleproxy.ConsoleProxy.start(ConsoleProxy.java:331)
> at
> com.cloud.consoleproxy.ConsoleProxy.startWithContext(ConsoleProxy.java:316)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.cloud.agent.resource.consoleproxy.ConsoleProxyResource$1.runInContext(ConsoleProxyResource.java:331)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.lang.Thread.run(Thread.java:745)
> 2016-05-18 09:45:56,240 ERROR [cloud.consoleproxy.ConsoleProxy]
> (Console-Proxy-Main:null) null
> java.lang.NullPointerException
> at
> com.cloud.consoleproxy.ConsoleProxy.startupHttpMain(ConsoleProxy.java:357)
> at com.cloud.consoleproxy.ConsoleProxy.start(ConsoleProxy.java:331)
> at
> com.cloud.consoleproxy.ConsoleProxy.s

Problem enabling Intermediate SSL Certificate on Console VM

[Özhan Rüzgar Karaman]

Hi Developers;
My console vm works successfully over SSL connections. Yesterday we
realised that firefox could not validate our SSL certificate and it gives
us certificate validation errors.

We checked keystore table on database and noticed that we have not imported
intermediate certificate and console vm works over SSL without any
intermediate SSL certificates. We checked Alpha SSL web site and downloaded
the certificate file(
https://www.alphassl.com/support/install-root-certificate.html)  and delete
the keystore table and re import all root cert + intermediate + server cert
+ private key from Cloudstack Admin interface. After that we have checked
the console vm logs(/var/log/cloud/cloud.out) and we noticed that it could
not successfully download the ssl certificate from ACS. The errors are
below.

I checked the keystore table and it looks okey. After that i restored the
keystore table from backup which does not have any intermediate
certificate, console vm started to work, but because we do not have
intermediate certificate in console vm, windows firefox clients again could
not connect to console sessions.

Does anyone experience this kind of problem on enabling intermediate
certificate? Also which kind of intermediate certificate format need to be
used on ACS, are all formats valid for CloudStack 4.8 ? Alpha SSL provides
SHA-1 and SHA-256 formats for intermediate certificates.

Thanks for all responses & time.

Regards
Özhan



2016-05-18 09:45:56,205 INFO
 [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
(Console-Proxy-Main:null) Start initializing SSL
2016-05-18 09:45:56,205 INFO
 [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
(Console-Proxy-Main:null) No certificates passed, recheck global
configuration and certificates
2016-05-18 09:45:56,205 INFO
 [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
(Console-Proxy-Main:null) Start initializing SSL
2016-05-18 09:45:56,206 INFO
 [cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
(Console-Proxy-Main:null) No certificates passed, recheck global
configuration and certificates
2016-05-18 09:45:56,227 ERROR
[cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl]
(Console-Proxy-Main:null) java.lang.NullPointerException: null SSLContext
java.lang.NullPointerException: null SSLContext
at
com.sun.net.httpserver.HttpsConfigurator.(HttpsConfigurator.java:81)
at
com.cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl$1.(ConsoleProxySecureServerFactoryImpl.java:82)
at
com.cloud.consoleproxy.ConsoleProxySecureServerFactoryImpl.createHttpServerInstance(ConsoleProxySecureServerFactoryImpl.java:82)
at
com.cloud.consoleproxy.ConsoleProxy.startupHttpMain(ConsoleProxy.java:356)
at com.cloud.consoleproxy.ConsoleProxy.start(ConsoleProxy.java:331)
at
com.cloud.consoleproxy.ConsoleProxy.startWithContext(ConsoleProxy.java:316)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.cloud.agent.resource.consoleproxy.ConsoleProxyResource$1.runInContext(ConsoleProxyResource.java:331)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.lang.Thread.run(Thread.java:745)
2016-05-18 09:45:56,240 ERROR [cloud.consoleproxy.ConsoleProxy]
(Console-Proxy-Main:null) null
java.lang.NullPointerException
at
com.cloud.consoleproxy.ConsoleProxy.startupHttpMain(ConsoleProxy.java:357)
at com.cloud.consoleproxy.ConsoleProxy.start(ConsoleProxy.java:331)
at
com.cloud.consoleproxy.ConsoleProxy.startWithContext(ConsoleProxy.java:316)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.cloud.agent.resource.consoleproxy.ConsoleProxyResource$1.runInContext(ConsoleProxyResource.java:331)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)

Re: Bug on delete snapshot function / CLOUDSTACK-9297

[Özhan Rüzgar Karaman]

Hi Wei;
We will follow up the ticket with you and Mike's guidance.

Thanks
Özhan

On Wed, Mar 2, 2016 at 11:35 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Hi Ozhan,
>
> please check the comments of the ticket.
>
> -Wei
>
> 2016-03-02 10:22 GMT+01:00 Özhan Rüzgar Karaman <oruzgarkara...@gmail.com
> >:
>
> > Hi;
> > 2 weeks ago we noticed a bug on delete snapshot function/api for ACS 4.8.
> > We have created a jira ticket for this bug.
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-9297
> >
> > issue looks like its similar bug which was mentioned on CLOUDSTACK-8845 .
> > But because its on other function we have created new ticket.
> >
> > Our idea is we like to store vm snapshots as disk backup even if we
> delete
> > its original(referenced) vm. To achieve this we made a test and we took
> > some snapshots of vm and after deleting its originating, same vm we
> noticed
> > that if we try to delete its snapshot ACS throws exception like "Error
> > Message : Unable to determine the storage pool of the snapshot."
> >
> > We checked details from database and we noticed that issue is like
> > CLOUDSTACK-8845 and snapshot looses its reference vm id. There are more
> > rich details and test cases on jira ticket.
> >
> > Does anyone likes to take and work on this issue, if someone could
> supply a
> > fix, we could directly make tests on our working environment.
> >
> > Kindly Regards
> > Ozhan Ruzgar Karaman
> >
>

Bug on delete snapshot function / CLOUDSTACK-9297

[Özhan Rüzgar Karaman]

Hi;
2 weeks ago we noticed a bug on delete snapshot function/api for ACS 4.8.
We have created a jira ticket for this bug.

https://issues.apache.org/jira/browse/CLOUDSTACK-9297

issue looks like its similar bug which was mentioned on CLOUDSTACK-8845 .
But because its on other function we have created new ticket.

Our idea is we like to store vm snapshots as disk backup even if we delete
its original(referenced) vm. To achieve this we made a test and we took
some snapshots of vm and after deleting its originating, same vm we noticed
that if we try to delete its snapshot ACS throws exception like "Error
Message : Unable to determine the storage pool of the snapshot."

We checked details from database and we noticed that issue is like
CLOUDSTACK-8845 and snapshot looses its reference vm id. There are more
rich details and test cases on jira ticket.

Does anyone likes to take and work on this issue, if someone could supply a
fix, we could directly make tests on our working environment.

Kindly Regards
Ozhan Ruzgar Karaman