Re: POLL: ACL default egress policy rule in VPC

[Jayapal Uradi]

Hi Rene,

Please look at my inline comments.
Let me add some context for the VPC egress/ingress rules behavior.

Pre 4.5 (subject to correction) the behavior of VPC acl is as follows.

1. Default egress is ALLOW and ingress is DROP.
   a.  When a rule is added to egress then that particular rule traffic is 
allowed and rest is blocked in egress.
   b.  When a rule is added to ingress then that particular rule traffic is 
allowed and rest is blocked in egress.

After 4.5 ACL lists and ACL items feature is introduced there we have ‘default 
allow’ and ‘default deny’ ACLs. User can also
create a custom acl. In ACL feature we can add mix of allow and deny rules and 
the ordering of rules is maintained.

1.  when ‘default allow’ is selected while creating the vpc tier
By default traffic is ALLOWED and rules can be added to ALLOW/DENY the 
traffic
   After adding the rules there will be ACCEPT at the end
2.  when ‘default deny’ is selected while creating the vpc tier
By default traffic is DENY and rules can be added to DENY/ALLOW the traffic.
  After adding the rules there will be DROP at the end
3. If no ACL selected for the ACL then Pre 4.5 behavior will be there.
4. With custom acl default ingress is DROP and egress is ALLOW. User can add 
rules for allow/deny rules.

If you see behavior other than above then there will be bug.

Currently in VPC egress behavior is controlled from the ACLs. If include  
‘egressdefaultpolicy’ then there will be confusion.

What I feel is that current VPC ACLs are flexible enough  to configure the 
required behavior.

Thanks,
Jayapal





> On Nov 13, 2017, at 11:17 PM, Rene Moser  wrote:
> 
> Hi Devs
> 
> The last days I fought with the ACL egress rule behaviour and I would
> like to make a poll in which direction the fix should go.
> 
> Short Version:
> 
> We need to define a better default behaviour for acl default egress
> rule. I see 3 different options:
> 
> 1. always add a default deny all egress rule.
> 
> This would be super easy to do (should probably also the intermediate
> fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)
> 
> 
> 2. add a deny all egress rule in case if have at least one egress allow
> rule.
> 
> A bit intransparent to the user, but doable. This seems to be the
> behaviour how it was designed and should have been implemented.
> 
Currently we can configure the ACLs to get this behavior.
> 
> 3. use the default setting in the network offering "egressdefaultpolicy"
> to specify the default behavior.
> 
> There is already a setting which specifies this behaviour but is not
> used in VPC. Why not use it?
> 
> As a consequence when using this setting, the user should get more infos
> about the policy of the network offering while choosing it for the tier.
> 
> 
> Poll:
> 
> 1. []
> 2. []
> 3. []
> 4. [] Other? What?
> 
> 
> Long Version:
> 
> First, let's have a look of the issue:
> 
> In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
> would result in a "accept egress all":
> 
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -j ACCEPT
> 
> When an egress (here deny 25 egress) rule (no mather if deny or allow)
> gets added the result is a "deny all" appended:
> 
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
> -A ACL_OUTBOUND_eth2 -j DROP
This is seen because default egress is drop and user added rule to deny port 25 
traffic.
User has choice of adding allow/deny rules with priority number.
> 
> This does not make any sense and is a bug IMHO.
> 
> 
> In 4.9 the behaviour is different:
> 
> (note there is a bug in the ordering of egress rules which is fixed by
> https://github.com/apache/cloudstack/pull/2313)
> 
> The default policy is kept accept egress all.
> 
> -A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT

In 4.9 it is a bug. After accept rules there supposed to DROP all at the end.
> 
> 
> To me it looks like the wanted behavior was "egress all as default. If
> we have allow rules, append deny all". This would make sense but is
> quite instransparent.
> 
> But let's poll
> 
> 

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business 

Re: Adoption Survey seems to be abandoned

[Ivan Kudryavtsev]

Hello, Eric, all.

I would like to follow up the topic.
The question is not about an addition of our entity to the list but the
state in general.
Eric, May be you can create the instruction, so we (or other party)
maintains the routine?

2017-10-30 17:07 GMT+07:00 Erik Weber :

> On Mon, Oct 30, 2017 at 4:51 AM, Ivan Kudryavtsev
>  wrote:
> > Hi, DevTeam.
> >
> > May be dev@ is wrong list, sorry in advance.
> >
> > It seems that adopters list is abandoned. I mean that I filled it about
> 3+
> > month ago and don't see our org in list of adopters (might be it was just
> > ignored, but worth to mention anyway). I don't know either it's intended
> or
> > not, just wanted to note that.
>
> Hey Ivan,
>
> I just want to acknowledge that your submission is received and let
> you know that it is not ignored.
>
> I'll see if I can find time tonight to update the web page with the
> latest entries.
>
> --
> Erik
>



-- 
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ 

Re: jetty simulator problem

[Rafael Weingärtner]

I removed some spring configurations that were not making sense to me.
After that, everything started working. I committed these changes at
https://github.com/rafaelweingartner/cloudstack/tree/daanTestSimulator

Could you test it? If it works I can open a PR.

On Mon, Nov 13, 2017 at 5:33 PM, Daan Hoogland 
wrote:

> trying with only the depends-on lines removed now
>
> On Mon, Nov 13, 2017 at 8:30 PM, Daan Hoogland 
> wrote:
>
> > from maven:
> > mvn -pl client jetty:run -Dsimulator
> >
> > On Mon, Nov 13, 2017 at 8:16 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> >> The error changed! That is good... It is a misconfiguration with
> >> dependencies between modules.
> >> How are you running the simulator?
> >>
> >> On Mon, Nov 13, 2017 at 5:13 PM, Daan Hoogland  >
> >> wrote:
> >>
> >> > didn't work:( trying with only some parts removed now
> >> >
> >> > Caused by: org.springframework.beans.factory.
> >> > NoSuchBeanDefinitionException:
> >> > No qualifying bean of type
> >> > 'org.apache.cloudstack.engine.subsystem.api.storage.SnapshotService'
> >> > available: expected at least 1 bean which qualifies as autowire
> >> candidate.
> >> > Dependency annotations: {@javax.inject.Inject()}
> >> > at
> >> > org.springframework.beans.factory.support.DefaultListableBeanFactory.
> >> > raiseNoMatchingBeanFound
> >> > (DefaultListableBeanFactory.java:1501)
> >> >
> >> > On Mon, Nov 13, 2017 at 7:27 PM, Rafael Weingärtner <
> >> > rafaelweingart...@gmail.com> wrote:
> >> >
> >> > > These beans declarations do not make much sense. They are declared
> >> twice,
> >> > > once in the XML (spring-engine-storage-snapshot-core-context.xml),
> >> and
> >> > the
> >> > > second declaration is executed through the use of annotations
> >> > (@Component)
> >> > > in those classes. This exception is happening because there is a
> >> > > configuration in the XML saying that one bean is depending on the
> >> other.
> >> > > You can simply remove lines 30-40 from the XML file to fix this.
> >> > >
> >> > >
> >> > >
> >> > > Side note: Spring automatically build the chain of dependencies
> >> between
> >> > > beans to execute the dependency injection properly. The
> configuration
> >> > > "depends-on" you find in the XML "bean" element is to define an
> exotic
> >> > and
> >> > > not obvious dependency.
> >> > >
> >> > >
> >> > > On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland <
> >> daan.hoogl...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > I'm looking at a problem, any one recognises this on master?
> >> simulator
> >> > > does
> >> > > > not start cleanly.
> >> > > >
> >> > > > [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@
> >> > > 51de5efa
> >> > > > {/client,file:/Users/dahn/apache/cloudstack/client/
> >> > > > target/classes/META-INF/webapp/,STARTING}{file:/Users/
> >> > > > dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
> >> > > > org.springframework.beans.factory.BeanCreationException: Error
> >> > creating
> >> > > > bean with name 'snapshotDataFactoryImpl' defined in URL
> >> > > > [jar:file:/Users/dahn/.m2/repository/org/apache/
> >> > cloudstack/cloud-engine-
> >> > > > storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
> >> > > > snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
> >> > > > spring-engine-storage-snapshot-core-context.xml]:
> >> > > > Circular depends-on relationship between 'snapshotDataFactoryImpl'
> >> and
> >> > > > 'volumeDataFactoryImpl'
> >> > > > at
> >> > > > org.springframework.beans.factory.support.
> >> > AbstractBeanFactory.doGetBean
> >> > > > (AbstractBeanFactory.java:300)
> >> > > > at
> >> > > > org.springframework.beans.factory.support.AbstractBeanFactor
> >> y.getBean
> >> > > > (AbstractBeanFactory.java:200)
> >> > > > at
> >> > > > org.springframework.beans.factory.support.
> >> > AbstractBeanFactory.doGetBean
> >> > > > (AbstractBeanFactory.java:304)
> >> > > > at
> >> > > > org.springframework.beans.factory.support.AbstractBeanFactor
> >> y.getBean
> >> > > > (AbstractBeanFactory.java:200)
> >> > > > at
> >> > > > org.springframework.beans.factory.support.DefaultListableBea
> >> nFactory.
> >> > > > preInstantiateSingletons
> >> > > > (DefaultListableBeanFactory.java:756)
> >> > > > at
> >> > > > org.springframework.context.support.AbstractApplicationContext.
> >> > > > finishBeanFactoryInitialization
> >> > > > (AbstractApplicationContext.java:868)
> >> > > > at
> >> > > > org.springframework.context.support.AbstractApplicationConte
> >> xt.refresh
> >> > > > (AbstractApplicationContext.java:549)
> >> > > > at
> >> > > > org.apache.cloudstack.spring.module.model.impl.
> >> > > DefaultModuleDefinitionSet.
> >> > > > loadContext
> >> > > > (DefaultModuleDefinitionSet.java:145)
> >> > > > at
> >> > > > org.apache.cloudstack.spring.module.model.impl.
> >> > > > DefaultModuleDefinitionSet$2.with
> 

Re: jetty simulator problem

[Daan Hoogland]

trying with only the depends-on lines removed now

On Mon, Nov 13, 2017 at 8:30 PM, Daan Hoogland 
wrote:

> from maven:
> mvn -pl client jetty:run -Dsimulator
>
> On Mon, Nov 13, 2017 at 8:16 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
>> The error changed! That is good... It is a misconfiguration with
>> dependencies between modules.
>> How are you running the simulator?
>>
>> On Mon, Nov 13, 2017 at 5:13 PM, Daan Hoogland 
>> wrote:
>>
>> > didn't work:( trying with only some parts removed now
>> >
>> > Caused by: org.springframework.beans.factory.
>> > NoSuchBeanDefinitionException:
>> > No qualifying bean of type
>> > 'org.apache.cloudstack.engine.subsystem.api.storage.SnapshotService'
>> > available: expected at least 1 bean which qualifies as autowire
>> candidate.
>> > Dependency annotations: {@javax.inject.Inject()}
>> > at
>> > org.springframework.beans.factory.support.DefaultListableBeanFactory.
>> > raiseNoMatchingBeanFound
>> > (DefaultListableBeanFactory.java:1501)
>> >
>> > On Mon, Nov 13, 2017 at 7:27 PM, Rafael Weingärtner <
>> > rafaelweingart...@gmail.com> wrote:
>> >
>> > > These beans declarations do not make much sense. They are declared
>> twice,
>> > > once in the XML (spring-engine-storage-snapshot-core-context.xml),
>> and
>> > the
>> > > second declaration is executed through the use of annotations
>> > (@Component)
>> > > in those classes. This exception is happening because there is a
>> > > configuration in the XML saying that one bean is depending on the
>> other.
>> > > You can simply remove lines 30-40 from the XML file to fix this.
>> > >
>> > >
>> > >
>> > > Side note: Spring automatically build the chain of dependencies
>> between
>> > > beans to execute the dependency injection properly. The configuration
>> > > "depends-on" you find in the XML "bean" element is to define an exotic
>> > and
>> > > not obvious dependency.
>> > >
>> > >
>> > > On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland <
>> daan.hoogl...@gmail.com>
>> > > wrote:
>> > >
>> > > > I'm looking at a problem, any one recognises this on master?
>> simulator
>> > > does
>> > > > not start cleanly.
>> > > >
>> > > > [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@
>> > > 51de5efa
>> > > > {/client,file:/Users/dahn/apache/cloudstack/client/
>> > > > target/classes/META-INF/webapp/,STARTING}{file:/Users/
>> > > > dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
>> > > > org.springframework.beans.factory.BeanCreationException: Error
>> > creating
>> > > > bean with name 'snapshotDataFactoryImpl' defined in URL
>> > > > [jar:file:/Users/dahn/.m2/repository/org/apache/
>> > cloudstack/cloud-engine-
>> > > > storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
>> > > > snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
>> > > > spring-engine-storage-snapshot-core-context.xml]:
>> > > > Circular depends-on relationship between 'snapshotDataFactoryImpl'
>> and
>> > > > 'volumeDataFactoryImpl'
>> > > > at
>> > > > org.springframework.beans.factory.support.
>> > AbstractBeanFactory.doGetBean
>> > > > (AbstractBeanFactory.java:300)
>> > > > at
>> > > > org.springframework.beans.factory.support.AbstractBeanFactor
>> y.getBean
>> > > > (AbstractBeanFactory.java:200)
>> > > > at
>> > > > org.springframework.beans.factory.support.
>> > AbstractBeanFactory.doGetBean
>> > > > (AbstractBeanFactory.java:304)
>> > > > at
>> > > > org.springframework.beans.factory.support.AbstractBeanFactor
>> y.getBean
>> > > > (AbstractBeanFactory.java:200)
>> > > > at
>> > > > org.springframework.beans.factory.support.DefaultListableBea
>> nFactory.
>> > > > preInstantiateSingletons
>> > > > (DefaultListableBeanFactory.java:756)
>> > > > at
>> > > > org.springframework.context.support.AbstractApplicationContext.
>> > > > finishBeanFactoryInitialization
>> > > > (AbstractApplicationContext.java:868)
>> > > > at
>> > > > org.springframework.context.support.AbstractApplicationConte
>> xt.refresh
>> > > > (AbstractApplicationContext.java:549)
>> > > > at
>> > > > org.apache.cloudstack.spring.module.model.impl.
>> > > DefaultModuleDefinitionSet.
>> > > > loadContext
>> > > > (DefaultModuleDefinitionSet.java:145)
>> > > > at
>> > > > org.apache.cloudstack.spring.module.model.impl.
>> > > > DefaultModuleDefinitionSet$2.with
>> > > > (DefaultModuleDefinitionSet.java:122)
>> > > > at
>> > > > org.apache.cloudstack.spring.module.model.impl.
>> > > DefaultModuleDefinitionSet.
>> > > > withModule
>> > > > (DefaultModuleDefinitionSet.java:245)
>> > > > at
>> > > > org.apache.cloudstack.spring.module.model.impl.
>> > > DefaultModuleDefinitionSet.
>> > > > withModule
>> > > > (DefaultModuleDefinitionSet.java:250)
>> > > > at
>> > > > org.apache.cloudstack.spring.module.model.impl.
>> > > DefaultModuleDefinitionSet.
>> > > > withModule
>> > > > (DefaultModuleDefinitionSet.java:250)
>> > > > at

Re: jetty simulator problem

[Daan Hoogland]

from maven:
mvn -pl client jetty:run -Dsimulator

On Mon, Nov 13, 2017 at 8:16 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> The error changed! That is good... It is a misconfiguration with
> dependencies between modules.
> How are you running the simulator?
>
> On Mon, Nov 13, 2017 at 5:13 PM, Daan Hoogland 
> wrote:
>
> > didn't work:( trying with only some parts removed now
> >
> > Caused by: org.springframework.beans.factory.
> > NoSuchBeanDefinitionException:
> > No qualifying bean of type
> > 'org.apache.cloudstack.engine.subsystem.api.storage.SnapshotService'
> > available: expected at least 1 bean which qualifies as autowire
> candidate.
> > Dependency annotations: {@javax.inject.Inject()}
> > at
> > org.springframework.beans.factory.support.DefaultListableBeanFactory.
> > raiseNoMatchingBeanFound
> > (DefaultListableBeanFactory.java:1501)
> >
> > On Mon, Nov 13, 2017 at 7:27 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > These beans declarations do not make much sense. They are declared
> twice,
> > > once in the XML (spring-engine-storage-snapshot-core-context.xml), and
> > the
> > > second declaration is executed through the use of annotations
> > (@Component)
> > > in those classes. This exception is happening because there is a
> > > configuration in the XML saying that one bean is depending on the
> other.
> > > You can simply remove lines 30-40 from the XML file to fix this.
> > >
> > >
> > >
> > > Side note: Spring automatically build the chain of dependencies between
> > > beans to execute the dependency injection properly. The configuration
> > > "depends-on" you find in the XML "bean" element is to define an exotic
> > and
> > > not obvious dependency.
> > >
> > >
> > > On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland <
> daan.hoogl...@gmail.com>
> > > wrote:
> > >
> > > > I'm looking at a problem, any one recognises this on master?
> simulator
> > > does
> > > > not start cleanly.
> > > >
> > > > [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@
> > > 51de5efa
> > > > {/client,file:/Users/dahn/apache/cloudstack/client/
> > > > target/classes/META-INF/webapp/,STARTING}{file:/Users/
> > > > dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
> > > > org.springframework.beans.factory.BeanCreationException: Error
> > creating
> > > > bean with name 'snapshotDataFactoryImpl' defined in URL
> > > > [jar:file:/Users/dahn/.m2/repository/org/apache/
> > cloudstack/cloud-engine-
> > > > storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
> > > > snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
> > > > spring-engine-storage-snapshot-core-context.xml]:
> > > > Circular depends-on relationship between 'snapshotDataFactoryImpl'
> and
> > > > 'volumeDataFactoryImpl'
> > > > at
> > > > org.springframework.beans.factory.support.
> > AbstractBeanFactory.doGetBean
> > > > (AbstractBeanFactory.java:300)
> > > > at
> > > > org.springframework.beans.factory.support.
> AbstractBeanFactory.getBean
> > > > (AbstractBeanFactory.java:200)
> > > > at
> > > > org.springframework.beans.factory.support.
> > AbstractBeanFactory.doGetBean
> > > > (AbstractBeanFactory.java:304)
> > > > at
> > > > org.springframework.beans.factory.support.
> AbstractBeanFactory.getBean
> > > > (AbstractBeanFactory.java:200)
> > > > at
> > > > org.springframework.beans.factory.support.
> DefaultListableBeanFactory.
> > > > preInstantiateSingletons
> > > > (DefaultListableBeanFactory.java:756)
> > > > at
> > > > org.springframework.context.support.AbstractApplicationContext.
> > > > finishBeanFactoryInitialization
> > > > (AbstractApplicationContext.java:868)
> > > > at
> > > > org.springframework.context.support.AbstractApplicationContext.
> refresh
> > > > (AbstractApplicationContext.java:549)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > loadContext
> > > > (DefaultModuleDefinitionSet.java:145)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > > DefaultModuleDefinitionSet$2.with
> > > > (DefaultModuleDefinitionSet.java:122)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > withModule
> > > > (DefaultModuleDefinitionSet.java:245)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > withModule
> > > > (DefaultModuleDefinitionSet.java:250)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > withModule
> > > > (DefaultModuleDefinitionSet.java:250)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > withModule
> > > > (DefaultModuleDefinitionSet.java:233)
> > > > at
> > > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet.
> > > > 

Re: jetty simulator problem

[Rafael Weingärtner]

The error changed! That is good... It is a misconfiguration with
dependencies between modules.
How are you running the simulator?

On Mon, Nov 13, 2017 at 5:13 PM, Daan Hoogland 
wrote:

> didn't work:( trying with only some parts removed now
>
> Caused by: org.springframework.beans.factory.
> NoSuchBeanDefinitionException:
> No qualifying bean of type
> 'org.apache.cloudstack.engine.subsystem.api.storage.SnapshotService'
> available: expected at least 1 bean which qualifies as autowire candidate.
> Dependency annotations: {@javax.inject.Inject()}
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.
> raiseNoMatchingBeanFound
> (DefaultListableBeanFactory.java:1501)
>
> On Mon, Nov 13, 2017 at 7:27 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > These beans declarations do not make much sense. They are declared twice,
> > once in the XML (spring-engine-storage-snapshot-core-context.xml), and
> the
> > second declaration is executed through the use of annotations
> (@Component)
> > in those classes. This exception is happening because there is a
> > configuration in the XML saying that one bean is depending on the other.
> > You can simply remove lines 30-40 from the XML file to fix this.
> >
> >
> >
> > Side note: Spring automatically build the chain of dependencies between
> > beans to execute the dependency injection properly. The configuration
> > "depends-on" you find in the XML "bean" element is to define an exotic
> and
> > not obvious dependency.
> >
> >
> > On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland 
> > wrote:
> >
> > > I'm looking at a problem, any one recognises this on master? simulator
> > does
> > > not start cleanly.
> > >
> > > [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@
> > 51de5efa
> > > {/client,file:/Users/dahn/apache/cloudstack/client/
> > > target/classes/META-INF/webapp/,STARTING}{file:/Users/
> > > dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
> > > org.springframework.beans.factory.BeanCreationException: Error
> creating
> > > bean with name 'snapshotDataFactoryImpl' defined in URL
> > > [jar:file:/Users/dahn/.m2/repository/org/apache/
> cloudstack/cloud-engine-
> > > storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
> > > snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
> > > spring-engine-storage-snapshot-core-context.xml]:
> > > Circular depends-on relationship between 'snapshotDataFactoryImpl' and
> > > 'volumeDataFactoryImpl'
> > > at
> > > org.springframework.beans.factory.support.
> AbstractBeanFactory.doGetBean
> > > (AbstractBeanFactory.java:300)
> > > at
> > > org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> > > (AbstractBeanFactory.java:200)
> > > at
> > > org.springframework.beans.factory.support.
> AbstractBeanFactory.doGetBean
> > > (AbstractBeanFactory.java:304)
> > > at
> > > org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> > > (AbstractBeanFactory.java:200)
> > > at
> > > org.springframework.beans.factory.support.DefaultListableBeanFactory.
> > > preInstantiateSingletons
> > > (DefaultListableBeanFactory.java:756)
> > > at
> > > org.springframework.context.support.AbstractApplicationContext.
> > > finishBeanFactoryInitialization
> > > (AbstractApplicationContext.java:868)
> > > at
> > > org.springframework.context.support.AbstractApplicationContext.refresh
> > > (AbstractApplicationContext.java:549)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > loadContext
> > > (DefaultModuleDefinitionSet.java:145)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > > DefaultModuleDefinitionSet$2.with
> > > (DefaultModuleDefinitionSet.java:122)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > withModule
> > > (DefaultModuleDefinitionSet.java:245)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > withModule
> > > (DefaultModuleDefinitionSet.java:250)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > withModule
> > > (DefaultModuleDefinitionSet.java:250)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > withModule
> > > (DefaultModuleDefinitionSet.java:233)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > loadContexts
> > > (DefaultModuleDefinitionSet.java:117)
> > > at
> > > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet.
> > > load
> > > (DefaultModuleDefinitionSet.java:79)
> > > at
> > > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.
> > > loadModules
> > > (ModuleBasedContextFactory.java:37)
> > > at
> > > 

Re: jetty simulator problem

[Daan Hoogland]

didn't work:( trying with only some parts removed now

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException:
No qualifying bean of type
'org.apache.cloudstack.engine.subsystem.api.storage.SnapshotService'
available: expected at least 1 bean which qualifies as autowire candidate.
Dependency annotations: {@javax.inject.Inject()}
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound
(DefaultListableBeanFactory.java:1501)

On Mon, Nov 13, 2017 at 7:27 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> These beans declarations do not make much sense. They are declared twice,
> once in the XML (spring-engine-storage-snapshot-core-context.xml), and the
> second declaration is executed through the use of annotations (@Component)
> in those classes. This exception is happening because there is a
> configuration in the XML saying that one bean is depending on the other.
> You can simply remove lines 30-40 from the XML file to fix this.
>
>
>
> Side note: Spring automatically build the chain of dependencies between
> beans to execute the dependency injection properly. The configuration
> "depends-on" you find in the XML "bean" element is to define an exotic and
> not obvious dependency.
>
>
> On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland 
> wrote:
>
> > I'm looking at a problem, any one recognises this on master? simulator
> does
> > not start cleanly.
> >
> > [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@
> 51de5efa
> > {/client,file:/Users/dahn/apache/cloudstack/client/
> > target/classes/META-INF/webapp/,STARTING}{file:/Users/
> > dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
> > org.springframework.beans.factory.BeanCreationException: Error creating
> > bean with name 'snapshotDataFactoryImpl' defined in URL
> > [jar:file:/Users/dahn/.m2/repository/org/apache/cloudstack/cloud-engine-
> > storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
> > snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
> > spring-engine-storage-snapshot-core-context.xml]:
> > Circular depends-on relationship between 'snapshotDataFactoryImpl' and
> > 'volumeDataFactoryImpl'
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
> > (AbstractBeanFactory.java:300)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> > (AbstractBeanFactory.java:200)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
> > (AbstractBeanFactory.java:304)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> > (AbstractBeanFactory.java:200)
> > at
> > org.springframework.beans.factory.support.DefaultListableBeanFactory.
> > preInstantiateSingletons
> > (DefaultListableBeanFactory.java:756)
> > at
> > org.springframework.context.support.AbstractApplicationContext.
> > finishBeanFactoryInitialization
> > (AbstractApplicationContext.java:868)
> > at
> > org.springframework.context.support.AbstractApplicationContext.refresh
> > (AbstractApplicationContext.java:549)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > loadContext
> > (DefaultModuleDefinitionSet.java:145)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> > DefaultModuleDefinitionSet$2.with
> > (DefaultModuleDefinitionSet.java:122)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > withModule
> > (DefaultModuleDefinitionSet.java:245)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > withModule
> > (DefaultModuleDefinitionSet.java:250)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > withModule
> > (DefaultModuleDefinitionSet.java:250)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > withModule
> > (DefaultModuleDefinitionSet.java:233)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > loadContexts
> > (DefaultModuleDefinitionSet.java:117)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet.
> > load
> > (DefaultModuleDefinitionSet.java:79)
> > at
> > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.
> > loadModules
> > (ModuleBasedContextFactory.java:37)
> > at
> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init
> > (CloudStackSpringContext.java:71)
> > at
> > org.apache.cloudstack.spring.module.factory.
> CloudStackSpringContext.
> > (CloudStackSpringContext.java:58)
> > at
> > org.apache.cloudstack.spring.module.factory.
> CloudStackSpringContext.
> > (CloudStackSpringContext.java:62)
> > at
> > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListene
> > r.contextInitialized
> > 

Re: POLL: ACL default egress policy rule in VPC

[Wei ZHOU]

I agree with (3).

To achieve backward compatibility, we need to add an egress rule to allow
ALL on existing ACLs which do not contain any egress rule.

For now almost every one uses integrated network offerings for VPC networks:
DefaultIsolatedNetworkOfferingForVpcNetworks
DefaultIsolatedNetworkOfferingForVpcNetworksNoLB
DefaultIsolatedNetworkOfferingForVpcNetworksWithInternalLB

The default egress policy are "Deny" in all network offerings above.

-Wei


2017-11-13 18:47 GMT+01:00 Rene Moser :

> Hi Devs
>
> The last days I fought with the ACL egress rule behaviour and I would
> like to make a poll in which direction the fix should go.
>
> Short Version:
>
> We need to define a better default behaviour for acl default egress
> rule. I see 3 different options:
>
> 1. always add a default deny all egress rule.
>
> This would be super easy to do (should probably also the intermediate
> fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)
>
>
> 2. add a deny all egress rule in case if have at least one egress allow
> rule.
>
> A bit intransparent to the user, but doable. This seems to be the
> behaviour how it was designed and should have been implemented.
>
>
> 3. use the default setting in the network offering "egressdefaultpolicy"
> to specify the default behavior.
>
> There is already a setting which specifies this behaviour but is not
> used in VPC. Why not use it?
>
> As a consequence when using this setting, the user should get more infos
> about the policy of the network offering while choosing it for the tier.
>
>
> Poll:
>
> 1. []
> 2. []
> 3. []
> 4. [] Other? What?
>
>
> Long Version:
>
> First, let's have a look of the issue:
>
> In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
> would result in a "accept egress all":
>
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -j ACCEPT
>
> When an egress (here deny 25 egress) rule (no mather if deny or allow)
> gets added the result is a "deny all" appended:
>
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
> -A ACL_OUTBOUND_eth2 -j DROP
>
> This does not make any sense and is a bug IMHO.
>
>
> In 4.9 the behaviour is different:
>
> (note there is a bug in the ordering of egress rules which is fixed by
> https://github.com/apache/cloudstack/pull/2313)
>
> The default policy is kept accept egress all.
>
> -A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT
>
>
> To me it looks like the wanted behavior was "egress all as default. If
> we have allow rules, append deny all". This would make sense but is
> quite instransparent.
>
> But let's poll
>
>
>

Re: jetty simulator problem

[Rafael Weingärtner]

These beans declarations do not make much sense. They are declared twice,
once in the XML (spring-engine-storage-snapshot-core-context.xml), and the
second declaration is executed through the use of annotations (@Component)
in those classes. This exception is happening because there is a
configuration in the XML saying that one bean is depending on the other.
You can simply remove lines 30-40 from the XML file to fix this.



Side note: Spring automatically build the chain of dependencies between
beans to execute the dependency injection properly. The configuration
"depends-on" you find in the XML "bean" element is to define an exotic and
not obvious dependency.


On Mon, Nov 13, 2017 at 3:48 PM, Daan Hoogland 
wrote:

> I'm looking at a problem, any one recognises this on master? simulator does
> not start cleanly.
>
> [WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@51de5efa
> {/client,file:/Users/dahn/apache/cloudstack/client/
> target/classes/META-INF/webapp/,STARTING}{file:/Users/
> dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean with name 'snapshotDataFactoryImpl' defined in URL
> [jar:file:/Users/dahn/.m2/repository/org/apache/cloudstack/cloud-engine-
> storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-
> snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/
> spring-engine-storage-snapshot-core-context.xml]:
> Circular depends-on relationship between 'snapshotDataFactoryImpl' and
> 'volumeDataFactoryImpl'
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
> (AbstractBeanFactory.java:300)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> (AbstractBeanFactory.java:200)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
> (AbstractBeanFactory.java:304)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean
> (AbstractBeanFactory.java:200)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.
> preInstantiateSingletons
> (DefaultListableBeanFactory.java:756)
> at
> org.springframework.context.support.AbstractApplicationContext.
> finishBeanFactoryInitialization
> (AbstractApplicationContext.java:868)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh
> (AbstractApplicationContext.java:549)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> loadContext
> (DefaultModuleDefinitionSet.java:145)
> at
> org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet$2.with
> (DefaultModuleDefinitionSet.java:122)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> withModule
> (DefaultModuleDefinitionSet.java:245)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> withModule
> (DefaultModuleDefinitionSet.java:250)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> withModule
> (DefaultModuleDefinitionSet.java:250)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> withModule
> (DefaultModuleDefinitionSet.java:233)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> loadContexts
> (DefaultModuleDefinitionSet.java:117)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> load
> (DefaultModuleDefinitionSet.java:79)
> at
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.
> loadModules
> (ModuleBasedContextFactory.java:37)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init
> (CloudStackSpringContext.java:71)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.
> (CloudStackSpringContext.java:58)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.
> (CloudStackSpringContext.java:62)
> at
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListene
> r.contextInitialized
> (CloudStackContextLoaderListener.java:52)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized
> (ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized
> (ServletContextHandler.java:444)
> at org.eclipse.jetty.server.handler.ContextHandler.startContext
> (ContextHandler.java:791)
> at org.eclipse.jetty.servlet.ServletContextHandler.startContext
> (ServletContextHandler.java:294)
> at org.eclipse.jetty.webapp.WebAppContext.startWebapp
> (WebAppContext.java:1349)
> at org.eclipse.jetty.maven.plugin.JettyWebAppContext.startWebapp
> (JettyWebAppContext.java:297)
> at org.eclipse.jetty.webapp.WebAppContext.startContext
> (WebAppContext.java:1342)
> at org.eclipse.jetty.server.handler.ContextHandler.doStart
> 

Re: POLL: ACL default egress policy rule in VPC

[Simon Weller]

3 definitely seems to make the most sense.


From: Rafael Weingärtner 
Sent: Monday, November 13, 2017 12:02 PM
To: dev@cloudstack.apache.org
Cc: u...@cloudstack.apache.org
Subject: Re: POLL: ACL default egress policy rule in VPC

3

On Mon, Nov 13, 2017 at 3:51 PM, Daan Hoogland 
wrote:

> 3 of course ;)
>
> On Mon, Nov 13, 2017 at 6:47 PM, Rene Moser  wrote:
>
> > Hi Devs
> >
> > The last days I fought with the ACL egress rule behaviour and I would
> > like to make a poll in which direction the fix should go.
> >
> > Short Version:
> >
> > We need to define a better default behaviour for acl default egress
> > rule. I see 3 different options:
> >
> > 1. always add a default deny all egress rule.
> >
> > This would be super easy to do (should probably also the intermediate
> > fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)
> >
> >
> > 2. add a deny all egress rule in case if have at least one egress allow
> > rule.
> >
> > A bit intransparent to the user, but doable. This seems to be the
> > behaviour how it was designed and should have been implemented.
> >
> >
> > 3. use the default setting in the network offering "egressdefaultpolicy"
> > to specify the default behavior.
> >
> > There is already a setting which specifies this behaviour but is not
> > used in VPC. Why not use it?
> >
> > As a consequence when using this setting, the user should get more infos
> > about the policy of the network offering while choosing it for the tier.
> >
> >
> > Poll:
> >
> > 1. []
> > 2. []
> > 3. []
> > 4. [] Other? What?
> >
> >
> > Long Version:
> >
> > First, let's have a look of the issue:
> >
> > In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
> > would result in a "accept egress all":
> >
> > -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -j ACCEPT
> >
> > When an egress (here deny 25 egress) rule (no mather if deny or allow)
> > gets added the result is a "deny all" appended:
> >
> > -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
> > -A ACL_OUTBOUND_eth2 -j DROP
> >
> > This does not make any sense and is a bug IMHO.
> >
> >
> > In 4.9 the behaviour is different:
> >
> > (note there is a bug in the ordering of egress rules which is fixed by
> > https://github.com/apache/cloudstack/pull/2313)
> >
> > The default policy is kept accept egress all.
> >
> > -A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
> > -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
> > -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT
> >
> >
> > To me it looks like the wanted behavior was "egress all as default. If
> > we have allow rules, append deny all". This would make sense but is
> > quite instransparent.
> >
> > But let's poll
> >
> >
> >
>
>
> --
> Daan
>



--
Rafael Weingärtner

Re: POLL: ACL default egress policy rule in VPC

[Rafael Weingärtner]

3

On Mon, Nov 13, 2017 at 3:51 PM, Daan Hoogland 
wrote:

> 3 of course ;)
>
> On Mon, Nov 13, 2017 at 6:47 PM, Rene Moser  wrote:
>
> > Hi Devs
> >
> > The last days I fought with the ACL egress rule behaviour and I would
> > like to make a poll in which direction the fix should go.
> >
> > Short Version:
> >
> > We need to define a better default behaviour for acl default egress
> > rule. I see 3 different options:
> >
> > 1. always add a default deny all egress rule.
> >
> > This would be super easy to do (should probably also the intermediate
> > fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)
> >
> >
> > 2. add a deny all egress rule in case if have at least one egress allow
> > rule.
> >
> > A bit intransparent to the user, but doable. This seems to be the
> > behaviour how it was designed and should have been implemented.
> >
> >
> > 3. use the default setting in the network offering "egressdefaultpolicy"
> > to specify the default behavior.
> >
> > There is already a setting which specifies this behaviour but is not
> > used in VPC. Why not use it?
> >
> > As a consequence when using this setting, the user should get more infos
> > about the policy of the network offering while choosing it for the tier.
> >
> >
> > Poll:
> >
> > 1. []
> > 2. []
> > 3. []
> > 4. [] Other? What?
> >
> >
> > Long Version:
> >
> > First, let's have a look of the issue:
> >
> > In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
> > would result in a "accept egress all":
> >
> > -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -j ACCEPT
> >
> > When an egress (here deny 25 egress) rule (no mather if deny or allow)
> > gets added the result is a "deny all" appended:
> >
> > -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
> > -A ACL_OUTBOUND_eth2 -j DROP
> >
> > This does not make any sense and is a bug IMHO.
> >
> >
> > In 4.9 the behaviour is different:
> >
> > (note there is a bug in the ordering of egress rules which is fixed by
> > https://github.com/apache/cloudstack/pull/2313)
> >
> > The default policy is kept accept egress all.
> >
> > -A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
> > NEW -j ACL_OUTBOUND_eth2
> > -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
> > -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
> > -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT
> >
> >
> > To me it looks like the wanted behavior was "egress all as default. If
> > we have allow rules, append deny all". This would make sense but is
> > quite instransparent.
> >
> > But let's poll
> >
> >
> >
>
>
> --
> Daan
>



-- 
Rafael Weingärtner

Re: POLL: ACL default egress policy rule in VPC

[Rene Moser]

Note the typo in the user mailing list email address, don't use reply
all... sry

Re: POLL: ACL default egress policy rule in VPC

[Daan Hoogland]

3 of course ;)

On Mon, Nov 13, 2017 at 6:47 PM, Rene Moser  wrote:

> Hi Devs
>
> The last days I fought with the ACL egress rule behaviour and I would
> like to make a poll in which direction the fix should go.
>
> Short Version:
>
> We need to define a better default behaviour for acl default egress
> rule. I see 3 different options:
>
> 1. always add a default deny all egress rule.
>
> This would be super easy to do (should probably also the intermediate
> fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)
>
>
> 2. add a deny all egress rule in case if have at least one egress allow
> rule.
>
> A bit intransparent to the user, but doable. This seems to be the
> behaviour how it was designed and should have been implemented.
>
>
> 3. use the default setting in the network offering "egressdefaultpolicy"
> to specify the default behavior.
>
> There is already a setting which specifies this behaviour but is not
> used in VPC. Why not use it?
>
> As a consequence when using this setting, the user should get more infos
> about the policy of the network offering while choosing it for the tier.
>
>
> Poll:
>
> 1. []
> 2. []
> 3. []
> 4. [] Other? What?
>
>
> Long Version:
>
> First, let's have a look of the issue:
>
> In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
> would result in a "accept egress all":
>
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -j ACCEPT
>
> When an egress (here deny 25 egress) rule (no mather if deny or allow)
> gets added the result is a "deny all" appended:
>
> -A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
> -A ACL_OUTBOUND_eth2 -j DROP
>
> This does not make any sense and is a bug IMHO.
>
>
> In 4.9 the behaviour is different:
>
> (note there is a bug in the ordering of egress rules which is fixed by
> https://github.com/apache/cloudstack/pull/2313)
>
> The default policy is kept accept egress all.
>
> -A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
> NEW -j ACL_OUTBOUND_eth2
> -A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
> -A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT
>
>
> To me it looks like the wanted behavior was "egress all as default. If
> we have allow rules, append deny all". This would make sense but is
> quite instransparent.
>
> But let's poll
>
>
>


-- 
Daan

jetty simulator problem

[Daan Hoogland]

I'm looking at a problem, any one recognises this on master? simulator does
not start cleanly.

[WARNING] Failed startup of context o.e.j.m.p.JettyWebAppContext@51de5efa
{/client,file:/Users/dahn/apache/cloudstack/client/target/classes/META-INF/webapp/,STARTING}{file:/Users/dahn/apache/cloudstack/client/target/classes/META-INF/webapp/}
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'snapshotDataFactoryImpl' defined in URL
[jar:file:/Users/dahn/.m2/repository/org/apache/cloudstack/cloud-engine-storage-snapshot/4.11.0.0-SNAPSHOT/cloud-engine-storage-snapshot-4.11.0.0-SNAPSHOT.jar!/META-INF/cloudstack/core/spring-engine-storage-snapshot-core-context.xml]:
Circular depends-on relationship between 'snapshotDataFactoryImpl' and
'volumeDataFactoryImpl'
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
(AbstractBeanFactory.java:300)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:200)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean
(AbstractBeanFactory.java:304)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:200)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons
(DefaultListableBeanFactory.java:756)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization
(AbstractApplicationContext.java:868)
at
org.springframework.context.support.AbstractApplicationContext.refresh
(AbstractApplicationContext.java:549)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext
(DefaultModuleDefinitionSet.java:145)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with
(DefaultModuleDefinitionSet.java:122)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule
(DefaultModuleDefinitionSet.java:245)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule
(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule
(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule
(DefaultModuleDefinitionSet.java:233)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts
(DefaultModuleDefinitionSet.java:117)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load
(DefaultModuleDefinitionSet.java:79)
at
org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules
(ModuleBasedContextFactory.java:37)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init
(CloudStackSpringContext.java:71)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.
(CloudStackSpringContext.java:58)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.
(CloudStackSpringContext.java:62)
at
org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized
(CloudStackContextLoaderListener.java:52)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized
(ContextHandler.java:800)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized
(ServletContextHandler.java:444)
at org.eclipse.jetty.server.handler.ContextHandler.startContext
(ContextHandler.java:791)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext
(ServletContextHandler.java:294)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp
(WebAppContext.java:1349)
at org.eclipse.jetty.maven.plugin.JettyWebAppContext.startWebapp
(JettyWebAppContext.java:297)
at org.eclipse.jetty.webapp.WebAppContext.startContext
(WebAppContext.java:1342)
at org.eclipse.jetty.server.handler.ContextHandler.doStart
(ContextHandler.java:741)
at org.eclipse.jetty.webapp.WebAppContext.doStart
(WebAppContext.java:505)
at org.eclipse.jetty.maven.plugin.JettyWebAppContext.doStart
(JettyWebAppContext.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start
(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start
(ContainerLifeCycle.java:132)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart
(ContainerLifeCycle.java:114)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart
(AbstractHandler.java:61)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart
(ContextHandlerCollection.java:163)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start
(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start
(ContainerLifeCycle.java:132)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart
(ContainerLifeCycle.java:114)
at 

POLL: ACL default egress policy rule in VPC

[Rene Moser]

Hi Devs

The last days I fought with the ACL egress rule behaviour and I would
like to make a poll in which direction the fix should go.

Short Version:

We need to define a better default behaviour for acl default egress
rule. I see 3 different options:

1. always add a default deny all egress rule.

This would be super easy to do (should probably also the intermediate
fix for 4.9, see https://github.com/apache/cloudstack/pull/2323)


2. add a deny all egress rule in case if have at least one egress allow
rule.

A bit intransparent to the user, but doable. This seems to be the
behaviour how it was designed and should have been implemented.


3. use the default setting in the network offering "egressdefaultpolicy"
to specify the default behavior.

There is already a setting which specifies this behaviour but is not
used in VPC. Why not use it?

As a consequence when using this setting, the user should get more infos
about the policy of the network offering while choosing it for the tier.


Poll:

1. []
2. []
3. []
4. [] Other? What?


Long Version:

First, let's have a look of the issue:

In version 4.5, creating a new acl with no egress (ACL_OUTBOUND) rule
would result in a "accept egress all":

-A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth2 -j ACCEPT

When an egress (here deny 25 egress) rule (no mather if deny or allow)
gets added the result is a "deny all" appended:

-A PREROUTING -s 10.10.0.0/24 ! -d 10.10.0.1/32 -i eth2 -m state --state
NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 25 -j DROP
-A ACL_OUTBOUND_eth2 -j DROP

This does not make any sense and is a bug IMHO.


In 4.9 the behaviour is different:

(note there is a bug in the ordering of egress rules which is fixed by
https://github.com/apache/cloudstack/pull/2313)

The default policy is kept accept egress all.

-A PREROUTING -s 10.11.1.0/24 ! -d 10.11.1.1/32 -i eth2 -m state --state
NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth2 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -p tcp -m tcp --dport 80 -j ACCEPT


To me it looks like the wanted behavior was "egress all as default. If
we have allow rules, append deny all". This would make sense but is
quite instransparent.

But let's poll

Re: Remote debugging not working anymore

[Marc-Aurèle Brothier - Exoscale]

@rhtyd - the java-opts file content (+ read in the startup script)
should be put back to allow startup customization.



On Mon, 2017-11-13 at 15:18 +0100, Sigert GOEMINNE wrote:
> Hi all,
> 
> Starting from the changes in PR2226
> , what do i need to
> specify
> to enable remote debugging on a system? Before I added
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 in
> /etc/default/cloudstack-management and restarted the cloudstack mgmt
> server
> but apparently this doesn't work anymore. If i run netstat -tlpn, It
> is not
> showing port 5005 in my case. Do I need to change something in our
> test
> infrastructure?
> 
> Thanks.
> Kind regards,
> Sigert

Re: HTTPS LB and x-forwarded-for

[Wido den Hollander]

> Op 13 november 2017 om 15:14 schreef Pierre-Luc Dion :
> 
> 
> Hi,
> 
> This is looking quite promising, I have a colleague that tested that last
> Friday, look like the proxy proto v1 work out of the box with Nginx, but
> would need an extra package for Apache 2.4 ?

It depends. You need HTTPd 2.4.28, see: 
https://httpd.apache.org/docs/trunk/mod/mod_remoteip.html#remoteipproxyprotocol

It's there upstream, but not in all packages. 

It can from this module:

- https://github.com/roadrunner2/mod-proxy-protocol
- https://roadrunner2.github.io/mod-proxy-protocol/mod_proxy_protocol.html

They donated the code to go upstream and went into mod_remoteip but landed in 
2.4.28

It will probably make it into Ubuntu 18.04 and CentOS 7.4.

Wido

> Otherwise, it seems to be a good way to do  https LB without complicated
> configuration and huge changes in CloudStack.
> 
> 
> 
> On Fri, Nov 10, 2017 at 10:36 AM, Nux!  wrote:
> 
> > Pierre-Luc,
> >
> > Haproxy docs say it should work for any kind of traffic as long as both
> > ends are PROXY-aware and it look like a majority of software is.
> > So, in short, yes.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> > > From: "Pierre-Luc Dion" 
> > > To: "Wido den Hollander" 
> > > Cc: "dev" , "Khosrow Moossavi" <
> > kmooss...@cloudops.com>, "Will Stevens"
> > > , "Nux!" , "users" <
> > us...@cloudstack.apache.org>
> > > Sent: Friday, 10 November, 2017 15:32:38
> > > Subject: Re: HTTPS LB and x-forwarded-for
> >
> > > Hi Wido, do you know if this would work for https traffic too?
> > >
> > > Le 10 nov. 2017 09 h 35, "Wido den Hollander"  a écrit :
> > >
> > >>
> > >> > Op 10 november 2017 om 14:27 schreef Pierre-Luc Dion <
> > pd...@cloudops.com
> > >> >:
> > >> >
> > >> >
> > >> > I kind of like the proxy backend type, ill check on our end if that
> > would
> > >> > work but definitely a simple and efficient approach!
> > >> >
> > >>
> > >> See: https://www.haproxy.com/blog/haproxy/proxy-protocol/
> > >>
> > >> Apache HTTPd supports PROXY since 2.4.28:
> > https://httpd.apache.org/docs/
> > >> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
> > >>
> > >> "RemoteIPProxyProtocol is only available in httpd 2.4.28 and newer"
> > >>
> > >> Wido
> > >>
> > >> >
> > >> >
> > >> > Le 10 nov. 2017 01 h 44, "Wido den Hollander"  a
> > écrit :
> > >> >
> > >> > >
> > >> > > > Op 9 november 2017 om 19:59 schreef Nux! :
> > >> > > >
> > >> > > >
> > >> > > > Wido,
> > >> > > >
> > >> > > > Excellent suggestion with the "transparent proxy", I was not
> > aware of
> > >> > > that.
> > >> > > > I think that would be a great idea and wouldn't require too many
> > >> > > modifications, especially as Haproxy comes already with the VR.
> > >> > > >
> > >> > >
> > >> > > It's indeed just a matter of a HAProxy config setting. We could
> > make it
> > >> > > configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for
> > >> example.
> > >> > >
> > >> > > That way your problem would be solved.
> > >> > >
> > >> > > Wido
> > >> > >
> > >> > > > To Paul:
> > >> > > > - imho the LB solution ACS ships now is a bit handicaped since
> > you do
> > >> > > not know the remote host ip. You're flying blind unless you use
> > google
> > >> > > analytics (and these things have gotten more and more aggressively
> > >> filtered
> > >> > > by adblocks).
> > >> > > > Enhancing Haproxy as Wido suggested would go a long way, it
> > wouldn't
> > >> > > break existing functionality and would also keep SSL processing off
> > >> the VR.
> > >> > > >
> > >> > > > --
> > >> > > > Sent from the Delta quadrant using Borg technology!
> > >> > > >
> > >> > > > Nux!
> > >> > > > www.nux.ro
> > >> > > >
> > >> > > > - Original Message -
> > >> > > > > From: "Andrija Panic" 
> > >> > > > > To: "users" 
> > >> > > > > Cc: "Khosrow Moossavi" , "Will
> > Stevens" <
> > >> > > wstev...@cloudops.com>, "dev"
> > >> > > > > , "Pierre-Luc Dion" <
> > pd...@cloudops.com
> > >> >
> > >> > > > > Sent: Thursday, 9 November, 2017 13:10:58
> > >> > > > > Subject: Re: HTTPS LB and x-forwarded-for
> > >> > > >
> > >> > > > > Wido,
> > >> > > > >
> > >> > > > > backend servers are not Linux only, for example we have a ton of
> > >> > > Windows
> > >> > > > > customers, some WEB solutions / IIS etc...
> > >> > > > >
> > >> > > > > @all - If we try to please/solve everyone's proxying
> > >> > > solution/requirement -
> > >> > > > > this is impossible IMHO - I'm thinking more about some "do it as
> > >> you
> > >> > > like"
> > >> > > > > solution, to let customer write his own haproxy config and
> > upoad it
> > >> > > (for
> > >> > > > > example, or 

Remote debugging not working anymore

[Sigert GOEMINNE]

Hi all,

Starting from the changes in PR2226
, what do i need to specify
to enable remote debugging on a system? Before I added
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 in
/etc/default/cloudstack-management and restarted the cloudstack mgmt server
but apparently this doesn't work anymore. If i run netstat -tlpn, It is not
showing port 5005 in my case. Do I need to change something in our test
infrastructure?

Thanks.
Kind regards,
Sigert

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

Hi,

This is looking quite promising, I have a colleague that tested that last
Friday, look like the proxy proto v1 work out of the box with Nginx, but
would need an extra package for Apache 2.4 ?
Otherwise, it seems to be a good way to do  https LB without complicated
configuration and huge changes in CloudStack.



On Fri, Nov 10, 2017 at 10:36 AM, Nux!  wrote:

> Pierre-Luc,
>
> Haproxy docs say it should work for any kind of traffic as long as both
> ends are PROXY-aware and it look like a majority of software is.
> So, in short, yes.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Pierre-Luc Dion" 
> > To: "Wido den Hollander" 
> > Cc: "dev" , "Khosrow Moossavi" <
> kmooss...@cloudops.com>, "Will Stevens"
> > , "Nux!" , "users" <
> us...@cloudstack.apache.org>
> > Sent: Friday, 10 November, 2017 15:32:38
> > Subject: Re: HTTPS LB and x-forwarded-for
>
> > Hi Wido, do you know if this would work for https traffic too?
> >
> > Le 10 nov. 2017 09 h 35, "Wido den Hollander"  a écrit :
> >
> >>
> >> > Op 10 november 2017 om 14:27 schreef Pierre-Luc Dion <
> pd...@cloudops.com
> >> >:
> >> >
> >> >
> >> > I kind of like the proxy backend type, ill check on our end if that
> would
> >> > work but definitely a simple and efficient approach!
> >> >
> >>
> >> See: https://www.haproxy.com/blog/haproxy/proxy-protocol/
> >>
> >> Apache HTTPd supports PROXY since 2.4.28:
> https://httpd.apache.org/docs/
> >> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
> >>
> >> "RemoteIPProxyProtocol is only available in httpd 2.4.28 and newer"
> >>
> >> Wido
> >>
> >> >
> >> >
> >> > Le 10 nov. 2017 01 h 44, "Wido den Hollander"  a
> écrit :
> >> >
> >> > >
> >> > > > Op 9 november 2017 om 19:59 schreef Nux! :
> >> > > >
> >> > > >
> >> > > > Wido,
> >> > > >
> >> > > > Excellent suggestion with the "transparent proxy", I was not
> aware of
> >> > > that.
> >> > > > I think that would be a great idea and wouldn't require too many
> >> > > modifications, especially as Haproxy comes already with the VR.
> >> > > >
> >> > >
> >> > > It's indeed just a matter of a HAProxy config setting. We could
> make it
> >> > > configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for
> >> example.
> >> > >
> >> > > That way your problem would be solved.
> >> > >
> >> > > Wido
> >> > >
> >> > > > To Paul:
> >> > > > - imho the LB solution ACS ships now is a bit handicaped since
> you do
> >> > > not know the remote host ip. You're flying blind unless you use
> google
> >> > > analytics (and these things have gotten more and more aggressively
> >> filtered
> >> > > by adblocks).
> >> > > > Enhancing Haproxy as Wido suggested would go a long way, it
> wouldn't
> >> > > break existing functionality and would also keep SSL processing off
> >> the VR.
> >> > > >
> >> > > > --
> >> > > > Sent from the Delta quadrant using Borg technology!
> >> > > >
> >> > > > Nux!
> >> > > > www.nux.ro
> >> > > >
> >> > > > - Original Message -
> >> > > > > From: "Andrija Panic" 
> >> > > > > To: "users" 
> >> > > > > Cc: "Khosrow Moossavi" , "Will
> Stevens" <
> >> > > wstev...@cloudops.com>, "dev"
> >> > > > > , "Pierre-Luc Dion" <
> pd...@cloudops.com
> >> >
> >> > > > > Sent: Thursday, 9 November, 2017 13:10:58
> >> > > > > Subject: Re: HTTPS LB and x-forwarded-for
> >> > > >
> >> > > > > Wido,
> >> > > > >
> >> > > > > backend servers are not Linux only, for example we have a ton of
> >> > > Windows
> >> > > > > customers, some WEB solutions / IIS etc...
> >> > > > >
> >> > > > > @all - If we try to please/solve everyone's proxying
> >> > > solution/requirement -
> >> > > > > this is impossible IMHO - I'm thinking more about some "do it as
> >> you
> >> > > like"
> >> > > > > solution, to let customer write his own haproxy config and
> upoad it
> >> > > (for
> >> > > > > example, or something better?).
> >> > > > >
> >> > > > > We can support newer version of haproxy (1.5+) which also
> implement
> >> > > > > "transarent proxy" (integrate with kernel so to speak)  to allow
> >> > > TCP-level
> >> > > > > connections to backend (TCP mode, not HTTP mode) but to still
> >> > > "preserve"
> >> > > > > remote IP by faking it (fake soruce IP = transarent proxy).
> >> > > > >
> >> > > > > For the rest of configuration options,  I would leave it to the
> >> > > customer
> >> > > > > how he/she wants to configure rest of haproxy configuration,
> >> inlcuding
> >> > > > > custom checks, etc. Haproxy configuration is never-ending story,
> >> and we
> >> > > > > probably should allow custom sripts/configuration instead of
> >> trying to
> >> > > > > provide GUI/API way to configure everything (which is
> >> 

Re: Strange size of template from snapshot on XenServer

[Rafael Weingärtner]

Yes, that is what is happening.
Thanks for pointing that out.


On Mon, Nov 13, 2017 at 3:14 AM, Anshul Gangwar <
anshul.gang...@accelerite.com> wrote:

> Rafael, you seem to combine two things. Optimization and my fix are two
> different commits. I mentioned about optimization (done in commit “8caf52c”
> not by me) because that was broken by commit “8a3fd10”. Commit “
> 2c4ea503f92bcf9c611f409d5cdecb” only fixed the issue for newer versions
> of XenServer. Probably you are just going through bug description and
> missing the commit “8caf52c”.
>
> Regards,
> Anshul
>
> On 10/11/17, 5:25 PM, "Rafael Weingärtner" 
> wrote:
>
> Anshul,
>
> I will not touch the merit of documentation in code, for now, this is
> another discussion.  And regarding the “optimization”, you brought it
> up,
> whenever I saw you discussing this issue, you would talk about some
> sort of
> “optimization”… see
> https://github.com/apache/cloudstack/pull/1124#issuecomment-326550060.
> At
> least for me, that was quite confusing, and I was not seeing what you
> were
> talking about.
>
> If you diff the files, what broke your “solution” was the removal of
> the
> following lines:
>
> > EndPoint ep = endPointSelector.selectHypervisorHost(new
> ZoneScope(host.
> > getDataCenterId()));
> > host = hostDao.findById(ep.getId());
> >
>
> The conditional checks were maintained. We only removed something that
> we
> seemed logicless.
>
> With PR https://github.com/apache/cloudstack/pull/2315, I did not add
> anything extra. I only re-worked the method to make it clear and more
> readable (removed the if-inception). The only point I changed is that,
> instead of using “selectOneHypervisorHostByZone”, I create a specific
> method for that, which selects a host according to the volume of the
> snapshot.
>
> On Fri, Nov 10, 2017 at 8:21 AM, Anshul Gangwar <
> anshul.gang...@accelerite.com> wrote:
>
> > Rafael, you are still misunderstanding the fix “
> > 2c4ea503f92bcf9c611f409d5cdecb”. That has only fixed the issue due
> to
> > check for specific version i.e. before “
> 2c4ea503f92bcf9c611f409d5cdecb”
> > things were fine for xenserver 6.2 but not working for 6.5. Commit
> doesn’t
> > mention anything about optimization. Other than version change it
> reverts
> > the previous commit which is introducing hypervisor specific check
> in core
> > code.
> >
> > Regarding documentation, “git history”is the best documentation. Code
> > documentation may not make sense after some changes as it is often
> ignored.
> >
> > See the PR for more comments.
> >
> > Regards,
> > Anshul
> >
> > On 10/11/17, 3:14 AM, "Rafael Weingärtner" <
> rafaelweingart...@gmail.com>
> > wrote:
> >
> > Anshul and Lotic, after debugging and inspecting code for quite a
> > while I
> > understood the complete picture and created a solution for it.
> The PR
> > with
> > the solution is found athttps://github.com/apache/
> cloudstack/pull/2315
> > .
> > This was a clear example of what I constantly tell people here,
> code
> > needs
> > to be clear, concise, well tested and documented. Here follows
> some
> > explanation regarding the removal of some code introduced with
> commit
> > “2c4ea503f92bcf9c611f409d5cdecb”.
> >
> > With commit “8caf52c” Daan and I removed a small piece of code
> > introduced
> > by “2c4ea503f92bcf9c611f409d5cdecb” because it did not seem to
> make
> > much
> > sense. We did not remove everything; we only removed the part
> that was
> > looking for a random host in the zone to execute the command. As
> we
> > (Anshul) exchanged some messages in some other PR, the code
> introduced
> > in
> > “2c4ea503f92bcf9c611f409d5cdecb” would only work for zones that
> do
> > not have
> > other hypervisor types if they are deployed with XenServer
> clusters.It
> > was
> > being created a limitation in ACS that should not exist. I
> stress again
> > that this only happened for the lack of documentation and clear
> coding.
> > For instance, when I read the commit “
> 2c4ea503f92bcf9c611f409d5cdecb”,
> > it
> > says it introduced an “optimization”, then I assumed that the
> process
> > executed with our code base before commit “
> > 2c4ea503f92bcf9c611f409d5cdecb”
> > is working, but not as fast or with as much quality as the code
> with
> > “2c4ea503f92bcf9c611f409d5cdecb”. However, that is not the
> case; the
> > code
> > “2c4ea503f92bcf9c611f409d5cdecb” is not optimizing anything; it
> is, in
> > fact, fixing/creating a workflow to create templates from
> snapshots in
> > XenServer deployments.
> >
> > The first PR(#1176) 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan, i know that 4.9.x is lts release and 4.10 and 4.11 are not lts
releases. What will happen when new lts release comes, does this release
fork from 4.9.x branch and apply fixes on 4.10 and 4.1x over that or new
lts release will forward fork from 4.11 or 4.1x releases?

I am curious about this procedure and I also ask this question because we
are using a fix ( https://issues.apache.org/jira/browse/CLOUDSTACK-9538 )
which is only on 4.9 branch and not available for 4.1x branches, i am not
developer so i need to find a developer from community to port this fix to
new 4.1x trees, if upcoming lts release will fork from 4.1x releases :)


On Mon, Nov 13, 2017 at 12:41 PM, Ivan Kudryavtsev  wrote:

> I don't know if 4.10 will be improved centrally. AFAIK 4.11 WIP right
> now...
>
> 13 нояб. 2017 г. 16:37 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> > Hi Ivan;
> > I like to try virtio-scsi, i like to use its benefits like enlarging data
> > disks without rebooting vm and getting unused space in vm back to my
> > central storage, we also waited for long time, lets wait some more for
> 4.10
> > to become more stable :) Thanks for all replies
> >
> > Özhan
> >
> > On Mon, Nov 13, 2017 at 12:29 PM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > wrote:
> >
> > > Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi
> and
> > > ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
> > > waited for 4.10 for a long time...
> > >
> > > 13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
> > > oruzgarkara...@gmail.com> написал:
> > >
> > > > ok we will try to compile 4.10 from source and try it on our test
> > > > environment.
> > > >
> > > > We currently use 4.9.x on production, i make this test for deciding
> if
> > > 4.10
> > > > is suitable & stable for us, we also use very simple and generic
> setup
> > > for
> > > > this test no ipv6 just simple environment . So for production it
> looks
> > > like
> > > > we need to wait some time for upcoming releases, what do you think
> > about
> > > > that?
> > > >
> > > > There was a thread about this question one month ago and i remember
> > that
> > > > most of people still stick to 4.9 release for their production
> > > > environments...
> > > >
> > > > Thanks
> > > > Özhan
> > > >
> > > > On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU 
> > > wrote:
> > > >
> > > > > Hi Ivan,
> > > > >
> > > > > I would suggest you to create jira tickets for each problem you
> found
> > > in
> > > > > your testing, and create a github pull request for a jira ticket.
> > > > > It is convenient for reviewers.
> > > > >
> > > > > Kind regards,
> > > > > Wei
> > > > >
> > > > > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > >:
> > > > >
> > > > > > Hello, Ozhan
> > > > > >
> > > > > > https://github.com/apache/cloudstack/pull/2320
> > > > > >
> > > > > > fixes everything I found right now. It enables functioning of
> > > > everything
> > > > > > correctly even if no IPv6 CIDR specified for network (at least
> for
> > > > Ubuntu
> > > > > > 14.04).
> > > > > > For IPv6 configuration instruction please take a look at:
> > > > > > https://github.com/apache/cloudstack/commit/
> > > > > f10c8bfe0c99a762c2606459413a47
> > > > > > 219614e775
> > > > > > (oh my god,I spend several hours trying to find how to configure
> > IPv6
> > > > for
> > > > > > 4.10).
> > > > > >
> > > > > > Please, don't forget to recreate SSVM because there is a fix for
> > > > > templates
> > > > > > too:
> > > > > > https://github.com/apache/cloudstack/pull/2322
> > > > > >
> > > > > >
> > > > > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > > > > oruzgarkara...@gmail.com
> > > > > > >:
> > > > > >
> > > > > > > Hi Ivan;
> > > > > > > Does this hotfixes also solve qoutes and shell script
> interprets
> > > > > problem?
> > > > > > > We have no ipv6 setup and today we made similar test with fresh
> > > > install
> > > > > > > 4.10. We noticed that we receive similar error on security
> groups
> > > > stage
> > > > > > > while br_netfilter module is already active on our environment.
> > We
> > > > made
> > > > > > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > > > > > >
> > > > > > > Logs are below:
> > > > > > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.
> > > > LibvirtComputingResource]
> > > > > > > (agentRequest-Handler-1:null) Executing:
> > > > > > > /usr/share/cloudstack-common/scripts/vm/network/security_
> > group.py
> > > > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip
> 192.168.18.6
> > > > > --vmip6
> > > > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > > > --rules
> > > > > > > I:tcp:1:65535:
> > > > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > > > 0.0.0.0/0,NEXT
> > > > > > ;
> > > > > > > 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Ivan Kudryavtsev]

I don't know if 4.10 will be improved centrally. AFAIK 4.11 WIP right now...

13 нояб. 2017 г. 16:37 пользователь "Özhan Rüzgar Karaman" <
oruzgarkara...@gmail.com> написал:

> Hi Ivan;
> I like to try virtio-scsi, i like to use its benefits like enlarging data
> disks without rebooting vm and getting unused space in vm back to my
> central storage, we also waited for long time, lets wait some more for 4.10
> to become more stable :) Thanks for all replies
>
> Özhan
>
> On Mon, Nov 13, 2017 at 12:29 PM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com
> > wrote:
>
> > Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi and
> > ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
> > waited for 4.10 for a long time...
> >
> > 13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
> > oruzgarkara...@gmail.com> написал:
> >
> > > ok we will try to compile 4.10 from source and try it on our test
> > > environment.
> > >
> > > We currently use 4.9.x on production, i make this test for deciding if
> > 4.10
> > > is suitable & stable for us, we also use very simple and generic setup
> > for
> > > this test no ipv6 just simple environment . So for production it looks
> > like
> > > we need to wait some time for upcoming releases, what do you think
> about
> > > that?
> > >
> > > There was a thread about this question one month ago and i remember
> that
> > > most of people still stick to 4.9 release for their production
> > > environments...
> > >
> > > Thanks
> > > Özhan
> > >
> > > On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU 
> > wrote:
> > >
> > > > Hi Ivan,
> > > >
> > > > I would suggest you to create jira tickets for each problem you found
> > in
> > > > your testing, and create a github pull request for a jira ticket.
> > > > It is convenient for reviewers.
> > > >
> > > > Kind regards,
> > > > Wei
> > > >
> > > > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com
> > >:
> > > >
> > > > > Hello, Ozhan
> > > > >
> > > > > https://github.com/apache/cloudstack/pull/2320
> > > > >
> > > > > fixes everything I found right now. It enables functioning of
> > > everything
> > > > > correctly even if no IPv6 CIDR specified for network (at least for
> > > Ubuntu
> > > > > 14.04).
> > > > > For IPv6 configuration instruction please take a look at:
> > > > > https://github.com/apache/cloudstack/commit/
> > > > f10c8bfe0c99a762c2606459413a47
> > > > > 219614e775
> > > > > (oh my god,I spend several hours trying to find how to configure
> IPv6
> > > for
> > > > > 4.10).
> > > > >
> > > > > Please, don't forget to recreate SSVM because there is a fix for
> > > > templates
> > > > > too:
> > > > > https://github.com/apache/cloudstack/pull/2322
> > > > >
> > > > >
> > > > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > > > oruzgarkara...@gmail.com
> > > > > >:
> > > > >
> > > > > > Hi Ivan;
> > > > > > Does this hotfixes also solve qoutes and shell script interprets
> > > > problem?
> > > > > > We have no ipv6 setup and today we made similar test with fresh
> > > install
> > > > > > 4.10. We noticed that we receive similar error on security groups
> > > stage
> > > > > > while br_netfilter module is already active on our environment.
> We
> > > made
> > > > > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > > > > >
> > > > > > Logs are below:
> > > > > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.
> > > LibvirtComputingResource]
> > > > > > (agentRequest-Handler-1:null) Executing:
> > > > > > /usr/share/cloudstack-common/scripts/vm/network/security_
> group.py
> > > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > > --vmip6
> > > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > > --rules
> > > > > > I:tcp:1:65535:
> > > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > > 0.0.0.0/0,NEXT
> > > > > ;
> > > > > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.
> > > LibvirtComputingResource]
> > > > > > (agentRequest-Handler-1:null) Exception:
> > > > > > /usr/share/cloudstack-common/scripts/vm/network/security_
> group.py
> > > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > > --vmip6
> > > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > > --rules
> > > > > > I:tcp:1:65535:
> > > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > > 0.0.0.0/0,NEXT
> > > > > ;
> > > > > > java.lang.NullPointerException
> > > > > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > > > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > > > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > > > > at
> > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > > > > 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan;
I like to try virtio-scsi, i like to use its benefits like enlarging data
disks without rebooting vm and getting unused space in vm back to my
central storage, we also waited for long time, lets wait some more for 4.10
to become more stable :) Thanks for all replies

Özhan

On Mon, Nov 13, 2017 at 12:29 PM, Ivan Kudryavtsev  wrote:

> Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi and
> ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
> waited for 4.10 for a long time...
>
> 13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
> oruzgarkara...@gmail.com> написал:
>
> > ok we will try to compile 4.10 from source and try it on our test
> > environment.
> >
> > We currently use 4.9.x on production, i make this test for deciding if
> 4.10
> > is suitable & stable for us, we also use very simple and generic setup
> for
> > this test no ipv6 just simple environment . So for production it looks
> like
> > we need to wait some time for upcoming releases, what do you think about
> > that?
> >
> > There was a thread about this question one month ago and i remember that
> > most of people still stick to 4.9 release for their production
> > environments...
> >
> > Thanks
> > Özhan
> >
> > On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU 
> wrote:
> >
> > > Hi Ivan,
> > >
> > > I would suggest you to create jira tickets for each problem you found
> in
> > > your testing, and create a github pull request for a jira ticket.
> > > It is convenient for reviewers.
> > >
> > > Kind regards,
> > > Wei
> > >
> > > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev  >:
> > >
> > > > Hello, Ozhan
> > > >
> > > > https://github.com/apache/cloudstack/pull/2320
> > > >
> > > > fixes everything I found right now. It enables functioning of
> > everything
> > > > correctly even if no IPv6 CIDR specified for network (at least for
> > Ubuntu
> > > > 14.04).
> > > > For IPv6 configuration instruction please take a look at:
> > > > https://github.com/apache/cloudstack/commit/
> > > f10c8bfe0c99a762c2606459413a47
> > > > 219614e775
> > > > (oh my god,I spend several hours trying to find how to configure IPv6
> > for
> > > > 4.10).
> > > >
> > > > Please, don't forget to recreate SSVM because there is a fix for
> > > templates
> > > > too:
> > > > https://github.com/apache/cloudstack/pull/2322
> > > >
> > > >
> > > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > > oruzgarkara...@gmail.com
> > > > >:
> > > >
> > > > > Hi Ivan;
> > > > > Does this hotfixes also solve qoutes and shell script interprets
> > > problem?
> > > > > We have no ipv6 setup and today we made similar test with fresh
> > install
> > > > > 4.10. We noticed that we receive similar error on security groups
> > stage
> > > > > while br_netfilter module is already active on our environment. We
> > made
> > > > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > > > >
> > > > > Logs are below:
> > > > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.
> > LibvirtComputingResource]
> > > > > (agentRequest-Handler-1:null) Executing:
> > > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > --vmip6
> > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > --rules
> > > > > I:tcp:1:65535:
> > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > 0.0.0.0/0,NEXT
> > > > ;
> > > > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.
> > LibvirtComputingResource]
> > > > > (agentRequest-Handler-1:null) Exception:
> > > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > > --vmip6
> > > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> > --rules
> > > > > I:tcp:1:65535:
> > > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > > 0.0.0.0/0,NEXT
> > > > ;
> > > > > java.lang.NullPointerException
> > > > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > > > at
> > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > > > addNetworkRules(LibvirtComputingResource.java:3429)
> > > > > at
> > > > > com.cloud.hypervisor.kvm.resource.wrapper.
> > > LibvirtSecurityGroupRulesComma
> > > > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > > > > at
> > > > > com.cloud.hypervisor.kvm.resource.wrapper.
> > > LibvirtSecurityGroupRulesComma
> > > > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > > > > at
> > > > > com.cloud.hypervisor.kvm.resource.wrapper.

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Ivan Kudryavtsev]

Ozhan, I suggest 4.10 only if you need KVM VM snapshots, virtio-scsi and
ipv6. Don't see other reasons to go from 4.9 (at least for kvm). But I
waited for 4.10 for a long time...

13 нояб. 2017 г. 16:21 пользователь "Özhan Rüzgar Karaman" <
oruzgarkara...@gmail.com> написал:

> ok we will try to compile 4.10 from source and try it on our test
> environment.
>
> We currently use 4.9.x on production, i make this test for deciding if 4.10
> is suitable & stable for us, we also use very simple and generic setup for
> this test no ipv6 just simple environment . So for production it looks like
> we need to wait some time for upcoming releases, what do you think about
> that?
>
> There was a thread about this question one month ago and i remember that
> most of people still stick to 4.9 release for their production
> environments...
>
> Thanks
> Özhan
>
> On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU  wrote:
>
> > Hi Ivan,
> >
> > I would suggest you to create jira tickets for each problem you found in
> > your testing, and create a github pull request for a jira ticket.
> > It is convenient for reviewers.
> >
> > Kind regards,
> > Wei
> >
> > 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev :
> >
> > > Hello, Ozhan
> > >
> > > https://github.com/apache/cloudstack/pull/2320
> > >
> > > fixes everything I found right now. It enables functioning of
> everything
> > > correctly even if no IPv6 CIDR specified for network (at least for
> Ubuntu
> > > 14.04).
> > > For IPv6 configuration instruction please take a look at:
> > > https://github.com/apache/cloudstack/commit/
> > f10c8bfe0c99a762c2606459413a47
> > > 219614e775
> > > (oh my god,I spend several hours trying to find how to configure IPv6
> for
> > > 4.10).
> > >
> > > Please, don't forget to recreate SSVM because there is a fix for
> > templates
> > > too:
> > > https://github.com/apache/cloudstack/pull/2322
> > >
> > >
> > > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> > oruzgarkara...@gmail.com
> > > >:
> > >
> > > > Hi Ivan;
> > > > Does this hotfixes also solve qoutes and shell script interprets
> > problem?
> > > > We have no ipv6 setup and today we made similar test with fresh
> install
> > > > 4.10. We noticed that we receive similar error on security groups
> stage
> > > > while br_netfilter module is already active on our environment. We
> made
> > > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > > >
> > > > Logs are below:
> > > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.
> LibvirtComputingResource]
> > > > (agentRequest-Handler-1:null) Executing:
> > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > --vmip6
> > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> --rules
> > > > I:tcp:1:65535:
> > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > 0.0.0.0/0,NEXT
> > > ;
> > > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.
> LibvirtComputingResource]
> > > > (agentRequest-Handler-1:null) Exception:
> > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> > --vmip6
> > > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0:
> --rules
> > > > I:tcp:1:65535:
> > > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> > 0.0.0.0/0,NEXT
> > > ;
> > > > java.lang.NullPointerException
> > > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > > at
> > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > > addNetworkRules(LibvirtComputingResource.java:3429)
> > > > at
> > > > com.cloud.hypervisor.kvm.resource.wrapper.
> > LibvirtSecurityGroupRulesComma
> > > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > > > at
> > > > com.cloud.hypervisor.kvm.resource.wrapper.
> > LibvirtSecurityGroupRulesComma
> > > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > > > at
> > > > com.cloud.hypervisor.kvm.resource.wrapper.
> > LibvirtRequestWrapper.execute(
> > > > LibvirtRequestWrapper.java:75)
> > > > at
> > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > executeRequest(
> > > > LibvirtComputingResource.java:1369)
> > > > at com.cloud.agent.Agent.processRequest(Agent.java:525)
> > > > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> > > > at com.cloud.utils.nio.Task.call(Task.java:83)
> > > > at com.cloud.utils.nio.Task.call(Task.java:29)
> > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > at
> > > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > > 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Ivan Kudryavtsev]

Hi, I did separate Jira issues and PRs to master, but everything in one
related to to 4.10, because 4.10 is already out and I don't want to do
duplicate PRs to different branches.

13 нояб. 2017 г. 16:18 пользователь "Wei ZHOU" 
написал:

> Hi Ivan,
>
> I would suggest you to create jira tickets for each problem you found in
> your testing, and create a github pull request for a jira ticket.
> It is convenient for reviewers.
>
> Kind regards,
> Wei
>
> 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev :
>
> > Hello, Ozhan
> >
> > https://github.com/apache/cloudstack/pull/2320
> >
> > fixes everything I found right now. It enables functioning of everything
> > correctly even if no IPv6 CIDR specified for network (at least for Ubuntu
> > 14.04).
> > For IPv6 configuration instruction please take a look at:
> > https://github.com/apache/cloudstack/commit/
> f10c8bfe0c99a762c2606459413a47
> > 219614e775
> > (oh my god,I spend several hours trying to find how to configure IPv6 for
> > 4.10).
> >
> > Please, don't forget to recreate SSVM because there is a fix for
> templates
> > too:
> > https://github.com/apache/cloudstack/pull/2322
> >
> >
> > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com
> > >:
> >
> > > Hi Ivan;
> > > Does this hotfixes also solve qoutes and shell script interprets
> problem?
> > > We have no ipv6 setup and today we made similar test with fresh install
> > > 4.10. We noticed that we receive similar error on security groups stage
> > > while br_netfilter module is already active on our environment. We made
> > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > >
> > > Logs are below:
> > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Exception:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > java.lang.NullPointerException
> > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > addNetworkRules(LibvirtComputingResource.java:3429)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtRequestWrapper.execute(
> > > LibvirtRequestWrapper.java:75)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > executeRequest(
> > > LibvirtComputingResource.java:1369)
> > > at com.cloud.agent.Agent.processRequest(Agent.java:525)
> > > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> > > at com.cloud.utils.nio.Task.call(Task.java:83)
> > > at com.cloud.utils.nio.Task.call(Task.java:29)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1149)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:624)
> > > at java.lang.Thread.run(Thread.java:748)
> > > 2017-11-13 11:47:41,774 WARN
> > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
> > > (agentRequest-Handler-1:null) Failed to program network rules for vm
> > > i-2-5-VM
> > > 2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: ,
> > MgmtId:
> > > 345048635880, via: 1, Ver: v1, Flags: 110,
> > > [{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"
> > > logSequenceNumber":16,"vmId":5,"reason":"PROGRAMMING_
> > > FAILED","result":false,"details":"programming
> > > network rules failed","wait":0}}] }
> > >
> > >
> > > When we execute command with double quotas for rules section from
> command
> > > line it executes without 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

ok we will try to compile 4.10 from source and try it on our test
environment.

We currently use 4.9.x on production, i make this test for deciding if 4.10
is suitable & stable for us, we also use very simple and generic setup for
this test no ipv6 just simple environment . So for production it looks like
we need to wait some time for upcoming releases, what do you think about
that?

There was a thread about this question one month ago and i remember that
most of people still stick to 4.9 release for their production
environments...

Thanks
Özhan

On Mon, Nov 13, 2017 at 12:18 PM, Wei ZHOU  wrote:

> Hi Ivan,
>
> I would suggest you to create jira tickets for each problem you found in
> your testing, and create a github pull request for a jira ticket.
> It is convenient for reviewers.
>
> Kind regards,
> Wei
>
> 2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev :
>
> > Hello, Ozhan
> >
> > https://github.com/apache/cloudstack/pull/2320
> >
> > fixes everything I found right now. It enables functioning of everything
> > correctly even if no IPv6 CIDR specified for network (at least for Ubuntu
> > 14.04).
> > For IPv6 configuration instruction please take a look at:
> > https://github.com/apache/cloudstack/commit/
> f10c8bfe0c99a762c2606459413a47
> > 219614e775
> > (oh my god,I spend several hours trying to find how to configure IPv6 for
> > 4.10).
> >
> > Please, don't forget to recreate SSVM because there is a fix for
> templates
> > too:
> > https://github.com/apache/cloudstack/pull/2322
> >
> >
> > 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com
> > >:
> >
> > > Hi Ivan;
> > > Does this hotfixes also solve qoutes and shell script interprets
> problem?
> > > We have no ipv6 setup and today we made similar test with fresh install
> > > 4.10. We noticed that we receive similar error on security groups stage
> > > while br_netfilter module is already active on our environment. We made
> > > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> > >
> > > Logs are below:
> > > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Executing:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > 2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
> > > (agentRequest-Handler-1:null) Exception:
> > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6
> --vmip6
> > > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > > I:tcp:1:65535:
> > > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:
> 0.0.0.0/0,NEXT
> > ;
> > > java.lang.NullPointerException
> > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > > at com.cloud.utils.script.Script.execute(Script.java:214)
> > > at com.cloud.utils.script.Script.execute(Script.java:182)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > > addNetworkRules(LibvirtComputingResource.java:3429)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtSecurityGroupRulesComma
> > > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > > at
> > > com.cloud.hypervisor.kvm.resource.wrapper.
> LibvirtRequestWrapper.execute(
> > > LibvirtRequestWrapper.java:75)
> > > at
> > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > executeRequest(
> > > LibvirtComputingResource.java:1369)
> > > at com.cloud.agent.Agent.processRequest(Agent.java:525)
> > > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> > > at com.cloud.utils.nio.Task.call(Task.java:83)
> > > at com.cloud.utils.nio.Task.call(Task.java:29)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > > ThreadPoolExecutor.java:1149)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > > ThreadPoolExecutor.java:624)
> > > at java.lang.Thread.run(Thread.java:748)
> > > 2017-11-13 11:47:41,774 WARN
> > > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
> > > (agentRequest-Handler-1:null) Failed to program network rules for vm
> > > i-2-5-VM
> > > 2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
> > > (agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: ,
> > MgmtId:
> > > 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Wei ZHOU]

Hi Ivan,

I would suggest you to create jira tickets for each problem you found in
your testing, and create a github pull request for a jira ticket.
It is convenient for reviewers.

Kind regards,
Wei

2017-11-13 10:01 GMT+01:00 Ivan Kudryavtsev :

> Hello, Ozhan
>
> https://github.com/apache/cloudstack/pull/2320
>
> fixes everything I found right now. It enables functioning of everything
> correctly even if no IPv6 CIDR specified for network (at least for Ubuntu
> 14.04).
> For IPv6 configuration instruction please take a look at:
> https://github.com/apache/cloudstack/commit/f10c8bfe0c99a762c2606459413a47
> 219614e775
> (oh my god,I spend several hours trying to find how to configure IPv6 for
> 4.10).
>
> Please, don't forget to recreate SSVM because there is a fix for templates
> too:
> https://github.com/apache/cloudstack/pull/2322
>
>
> 2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman  >:
>
> > Hi Ivan;
> > Does this hotfixes also solve qoutes and shell script interprets problem?
> > We have no ipv6 setup and today we made similar test with fresh install
> > 4.10. We noticed that we receive similar error on security groups stage
> > while br_netfilter module is already active on our environment. We made
> > same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
> >
> > Logs are below:
> > 2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-1:null) Executing:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > I:tcp:1:65535:
> > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT
> ;
> > 2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-1:null) Exception:
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > I:tcp:1:65535:
> > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT
> ;
> > java.lang.NullPointerException
> > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> > at com.cloud.utils.script.Script.execute(Script.java:214)
> > at com.cloud.utils.script.Script.execute(Script.java:182)
> > at
> > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> > addNetworkRules(LibvirtComputingResource.java:3429)
> > at
> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesComma
> > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> > at
> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesComma
> > ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> > at
> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(
> > LibvirtRequestWrapper.java:75)
> > at
> > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> executeRequest(
> > LibvirtComputingResource.java:1369)
> > at com.cloud.agent.Agent.processRequest(Agent.java:525)
> > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> > at com.cloud.utils.nio.Task.call(Task.java:83)
> > at com.cloud.utils.nio.Task.call(Task.java:29)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> > ThreadPoolExecutor.java:1149)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> > ThreadPoolExecutor.java:624)
> > at java.lang.Thread.run(Thread.java:748)
> > 2017-11-13 11:47:41,774 WARN
> > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
> > (agentRequest-Handler-1:null) Failed to program network rules for vm
> > i-2-5-VM
> > 2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
> > (agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: ,
> MgmtId:
> > 345048635880, via: 1, Ver: v1, Flags: 110,
> > [{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"
> > logSequenceNumber":16,"vmId":5,"reason":"PROGRAMMING_
> > FAILED","result":false,"details":"programming
> > network rules failed","wait":0}}] }
> >
> >
> > When we execute command with double quotas for rules section from command
> > line it executes without a problem like below:
> > root@kvmt3:/var/log/cloudstack/agent#
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> > add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> > null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> > 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> > "I:tcp:1:65535:
> > 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT
> ;"
> > root@kvmt3:/var/log/cloudstack/agent# echo 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Ivan Kudryavtsev]

Hello, Ozhan

https://github.com/apache/cloudstack/pull/2320

fixes everything I found right now. It enables functioning of everything
correctly even if no IPv6 CIDR specified for network (at least for Ubuntu
14.04).
For IPv6 configuration instruction please take a look at:
https://github.com/apache/cloudstack/commit/f10c8bfe0c99a762c2606459413a47219614e775
(oh my god,I spend several hours trying to find how to configure IPv6 for
4.10).

Please, don't forget to recreate SSVM because there is a fix for templates
too:
https://github.com/apache/cloudstack/pull/2322


2017-11-13 15:51 GMT+07:00 Özhan Rüzgar Karaman :

> Hi Ivan;
> Does this hotfixes also solve qoutes and shell script interprets problem?
> We have no ipv6 setup and today we made similar test with fresh install
> 4.10. We noticed that we receive similar error on security groups stage
> while br_netfilter module is already active on our environment. We made
> same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts
>
> Logs are below:
> 2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-1:null) Executing:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> I:tcp:1:65535:
> 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
> 2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-1:null) Exception:
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> I:tcp:1:65535:
> 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
> java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
> at com.cloud.utils.script.Script.execute(Script.java:214)
> at com.cloud.utils.script.Script.execute(Script.java:182)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.
> addNetworkRules(LibvirtComputingResource.java:3429)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesComma
> ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesComma
> ndWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(
> LibvirtRequestWrapper.java:75)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(
> LibvirtComputingResource.java:1369)
> at com.cloud.agent.Agent.processRequest(Agent.java:525)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
> at com.cloud.utils.nio.Task.call(Task.java:83)
> at com.cloud.utils.nio.Task.call(Task.java:29)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2017-11-13 11:47:41,774 WARN
> [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
> (agentRequest-Handler-1:null) Failed to program network rules for vm
> i-2-5-VM
> 2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: , MgmtId:
> 345048635880, via: 1, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"
> logSequenceNumber":16,"vmId":5,"reason":"PROGRAMMING_
> FAILED","result":false,"details":"programming
> network rules failed","wait":0}}] }
>
>
> When we execute command with double quotas for rules section from command
> line it executes without a problem like below:
> root@kvmt3:/var/log/cloudstack/agent#
> /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
> null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
> 1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
> "I:tcp:1:65535:
> 0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;"
> root@kvmt3:/var/log/cloudstack/agent# echo $?
> 0
> root@kvmt3:/var/log/cloudstack/agent#
>
> Thanks
> Özhan
>
>
> On Sat, Nov 11, 2017 at 6:59 PM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Hello, I implemented some hotfixes for 4.10 to work
> >
> > https://github.com/apache/cloudstack/pull/2319 - to master (load
> > br_netfilter module)
> > https://github.com/apache/cloudstack/pull/2320 - to 4.10 which fixes SG
> > failures related to ipv6.
> >
> >
> > 2017-11-11 15:51 GMT+07:00 Ivan Kudryavtsev 

Re: Apache CloudStack 4.10 VR/BasicZone/KVM Problem

[Özhan Rüzgar Karaman]

Hi Ivan;
Does this hotfixes also solve qoutes and shell script interprets problem?
We have no ipv6 setup and today we made similar test with fresh install
4.10. We noticed that we receive similar error on security groups stage
while br_netfilter module is already active on our environment. We made
same tests for Ubuntu 16.04.3 and 14.04.5 kvm hosts

Logs are below:
2017-11-13 11:47:41,773 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-1:null) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
2017-11-13 11:47:41,773 WARN  [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-1:null) Exception:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;
java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
at com.cloud.utils.script.Script.execute(Script.java:214)
at com.cloud.utils.script.Script.execute(Script.java:182)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.addNetworkRules(LibvirtComputingResource.java:3429)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:57)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper.execute(LibvirtSecurityGroupRulesCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:75)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1369)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:833)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-11-13 11:47:41,774 WARN
[resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
(agentRequest-Handler-1:null) Failed to program network rules for vm
i-2-5-VM
2017-11-13 11:47:41,775 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) Seq 1-6412562919422165093:  { Ans: , MgmtId:
345048635880, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"logSequenceNumber":16,"vmId":5,"reason":"PROGRAMMING_FAILED","result":false,"details":"programming
network rules failed","wait":0}}] }


When we execute command with double quotas for rules section from command
line it executes without a problem like below:
root@kvmt3:/var/log/cloudstack/agent#
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
add_network_rules --vmname i-2-5-VM --vmid 5 --vmip 192.168.18.6 --vmip6
null --sig 74a6d8c403af9c3c7b89ecf206e4ac26 --seq 16 --vmmac
1e:00:9b:00:00:05 --vif vnet8 --brname breth0-23 --nicsecips 0: --rules
"I:tcp:1:65535:
0.0.0.0/0,NEXT;I:udp:1:65535:0.0.0.0/0,NEXT;E:tcp:1:65535:0.0.0.0/0,NEXT;"
root@kvmt3:/var/log/cloudstack/agent# echo $?
0
root@kvmt3:/var/log/cloudstack/agent#

Thanks
Özhan


On Sat, Nov 11, 2017 at 6:59 PM, Ivan Kudryavtsev 
wrote:

> Hello, I implemented some hotfixes for 4.10 to work
>
> https://github.com/apache/cloudstack/pull/2319 - to master (load
> br_netfilter module)
> https://github.com/apache/cloudstack/pull/2320 - to 4.10 which fixes SG
> failures related to ipv6.
>
>
> 2017-11-11 15:51 GMT+07:00 Ivan Kudryavtsev :
>
> > Following up with previous question. I managed to make it work by
> removing
> > all and heading to ubuntu 14.04 hypervisor host.
> >
> > Also, what I found more:
> >
> > 1. when setup databases (management server) if custom port is specified,
> > databases themself is not created. If create manually, import scripts
> work
> > fine.
> > 2. UI: unable to download ISO to __all__ zones. Have to specify certain
> > zone, else UI gives an error.
> > 3. Ubuntu doesn't load module *br_netfilter* but
> >
> > /usr/share/cloudstack-common/scripts/vm/network/security_group.py
> >
> > uses it and nothing good as a result:
> >
> > 2017-11-11 15:38:29,241 - sysctl -w net.bridge.bridge-nf-call-
> arptables=1
> > 2017-11-11 15:38:29,244 - sysctl -w net.bridge.bridge-nf-call-iptables=1
> > 2017-11-11 15:38:29,247 - 

Apache CloudStack 4.9.3 Simulator with Kafka Logging Integrated

[Ivan Kudryavtsev]

Hello, comunity.

We recently developed simulator image for ACS 4.9.3 integrated with Apache
Kafka event bus logging system. This is for development purposes. We use
Kafka as integration point, so we need it. It's published in:
GitHub: https://github.com/bwsw/cs-simulator-kafka
DockerHub: https://hub.docker.com/r/bwsw/cs-simulator-kafka/

Basic container is René Moser image:
https://github.com/resmo/docker-cloudstack-simulator-for-ansible

René, many thanks for high quality source code. Please, all feel free
PR-ing and commenging if some improvements are required or bugs found.

-- 
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/