shwstppr opened a new issue, #7174:
URL: https://github.com/apache/cloudstack/issues/7174
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete
the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
API
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main
branch.
-->
~~~
4.16.1+ , maybe earlier as well
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network,
advanced networking, etc. N/A otherwise
-->
N/A
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
N/A
##### SUMMARY
<!-- Explain the problem/feature briefly -->
listLoadBalancerRuleInstances API fails when VM has a removed NIC in the
same network. This could happen in the cases when a VM is removed from the VPC
tier and then added to it again. Later when a LB is created using this VM for
the VPC, listLoadBalancerRuleInstances would fail.
Possibly due to this method which list instances with JOIN and and may
return removed NIC entries
https://github.com/apache/cloudstack/blob/main/engine/schema/src/main/java/com/cloud/vm/dao/UserVmDaoImpl.java#L345-L352
It doesn't affect LB rule as such but just affect list API.
##### STEPS TO REPRODUCE
Steps mentioned below
DB state
```
MariaDB [cloud]> SELECT * FROM nics WHERE instance_id=(SELECT id FROM
vm_instance WHERE uuid='620d7d83-533d-45e6-a8d6-88abcc149b6e');
+----+--------------------------------------+-------------+-------------------+-------------+---------------+----------+---------+---------------+------------+------+--------------+----------+--------------------------+----------------+-----------+---------------------+---------------+-------------+-------------+---------+---------------------+---------------------+-------------+----------+--------------+-------------+
| id | uuid | instance_id | mac_address
| ip4_address | netmask | gateway | ip_type | broadcast_uri |
network_id | mode | state | strategy | reserver_name |
reservation_id | device_id | update_time | isolation_uri | ip6_address
| default_nic | vm_type | created | removed |
ip6_gateway | ip6_cidr | secondary_ip | display_nic |
+----+--------------------------------------+-------------+-------------------+-------------+---------------+----------+---------+---------------+------------+------+--------------+----------+--------------------------+----------------+-----------+---------------------+---------------+-------------+-------------+---------+---------------------+---------------------+-------------+----------+--------------+-------------+
| 12 | bd0183b1-0604-499d-9ef5-4bd3ffc70f2f | 6 |
02:00:33:77:00:03 | 10.1.1.169 | 255.255.255.0 | 10.1.1.1 | Ip4 | NULL
| 205 | Dhcp | Deallocating | Start | ExternalGuestNetworkGuru |
NULL | 1 | 2023-02-06 06:09:40 | NULL | NULL
| 0 | User | 2023-02-06 06:09:40 | 2023-02-06 11:35:36 | NULL
| NULL | 0 | 1 |
| 21 | 5a627371-3cbf-48dd-86f9-4bf972fb0d3e | 6 |
02:00:50:f1:00:02 | NULL | NULL | NULL | Ip4 |
vlan://1368 | 204 | Dhcp | Reserved | Start |
ExternalGuestNetworkGuru | NULL | 0 | 2023-02-06 11:34:04 |
vlan://1368 | NULL | 1 | User | 2023-02-06 11:34:04 |
NULL | NULL | NULL | 0 | 1 |
| 22 | 736371b1-f5e4-4327-af7d-238e70ae5ebf | 6 |
02:00:21:c5:00:04 | 10.1.1.59 | 255.255.255.0 | 10.1.1.1 | Ip4 |
vlan://1367 | 205 | Dhcp | Reserved | Start |
ExternalGuestNetworkGuru | NULL | 1 | 2023-02-06 11:35:54 |
vlan://1367 | NULL | 0 | User | 2023-02-06 11:35:54 |
NULL | NULL | NULL | 0 | 1 |
+----+--------------------------------------+-------------+-------------------+-------------+---------------+----------+---------+---------------+------------+------+--------------+----------+--------------------------+----------------+-----------+---------------------+---------------+-------------+-------------+---------+---------------------+---------------------+-------------+----------+--------------+-------------+
3 rows in set (0.00 sec)
```
2023-02-06 11:38:42,135 DEBUG [c.c.a.ApiServlet]
(qtp396883763-17:ctx-08f485df) (logid:bd584e95) ===START=== 10.0.3.251 -- GET
id=12625eed-7b69-443a-b755-a9e2fad6940e&vmidipmap[0].vmid=620d7d83-533d-45e6-a8d6-88abcc149b6e&vmidipmap[0].vmip=10.1.1.59&command=assignToLoadBalancerRule&response=json
2023-02-06 11:38:42,140 DEBUG [c.c.a.ApiServer]
(qtp396883763-17:ctx-08f485df ctx-1e844f80) (logid:bd584e95) CIDRs from which
account 'Acct[17fbff81-a395-11ed-804d-1e0009000324-admin] -- Account {"id": 2,
"name": "admin", "uuid": "17fbff81-a395-11ed-804d-1e0009000324"}' is allowed to
perform API calls: 0.0.0.0/0,::/0
2023-02-06 11:38:42,152 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-29:ctx-a25d3b90 job-89) (logid:6ca0570b) Add job-89 into job
monitoring
2023-02-06 11:38:42,155 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(qtp396883763-17:ctx-08f485df ctx-1e844f80) (logid:bd584e95) submit async
job-89, details: AsyncJobVO: {id:89, userId: 2, accountId: 2, instanceType:
None, instanceId: null, cmd:
org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd,
cmdInfo:
{"vmidipmap[0].vmid":"620d7d83-533d-45e6-a8d6-88abcc149b6e","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"247","id":"12625eed-7b69-443a-b755-a9e2fad6940e","vmidipmap[0].vmip":"10.1.1.59","ctxDetails":"{\"interface
com.cloud.network.rules.FirewallRule\":\"12625eed-7b69-443a-b755-a9e2fad6940e\"}","ctxAccountId":"2","uuid":"12625eed-7b69-443a-b755-a9e2fad6940e","cmdEventType":"LB.ASSIGN.TO.RULE"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 32985499829028, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null, removed: null}
2023-02-06 11:38:42,156 DEBUG [c.c.a.ApiServlet]
(qtp396883763-17:ctx-08f485df ctx-1e844f80) (logid:bd584e95) ===END===
10.0.3.251 -- GET
id=12625eed-7b69-443a-b755-a9e2fad6940e&vmidipmap[0].vmid=620d7d83-533d-45e6-a8d6-88abcc149b6e&vmidipmap[0].vmip=10.1.1.59&command=assignToLoadBalancerRule&response=json
2023-02-06 11:38:42,157 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-29:ctx-a25d3b90 job-89) (logid:dabb8445) Executing
AsyncJobVO: {id:89, userId: 2, accountId: 2, instanceType: None, instanceId:
null, cmd:
org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd,
cmdInfo:
{"vmidipmap[0].vmid":"620d7d83-533d-45e6-a8d6-88abcc149b6e","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"247","id":"12625eed-7b69-443a-b755-a9e2fad6940e","vmidipmap[0].vmip":"10.1.1.59","ctxDetails":"{\"interface
com.cloud.network.rules.FirewallRule\":\"12625eed-7b69-443a-b755-a9e2fad6940e\"}","ctxAccountId":"2","uuid":"12625eed-7b69-443a-b755-a9e2fad6940e","cmdEventType":"LB.ASSIGN.TO.RULE"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 32985499829028, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null, removed: null}
2023-02-06 11:38:42,173 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(API-Job-Executor-29:ctx-a25d3b90 job-89 ctx-d195db5c) (logid:dabb8445) Adding
VM instance {id: "6", name: "i-2-6-VM", uuid:
"620d7d83-533d-45e6-a8d6-88abcc149b6e", type="User"} to the load balancer pool
2023-02-06 11:38:42,194 DEBUG [o.a.c.n.t.AdvancedNetworkTopology]
(API-Job-Executor-29:ctx-a25d3b90 job-89 ctx-d195db5c) (logid:dabb8445)
APPLYING VPC IP RULES
2023-02-06 11:38:42,197 DEBUG [o.a.c.n.t.BasicNetworkTopology]
(API-Job-Executor-29:ctx-a25d3b90 job-89 ctx-d195db5c) (logid:dabb8445)
Applying vpc ip association in network Ntwk[205|Guest|11]
2023-02-06 11:38:42,215 DEBUG [c.c.a.t.Request]
(API-Job-Executor-29:ctx-a25d3b90 job-89 ctx-d195db5c) (logid:dabb8445) Seq
1-158751886864811237: Sending { Cmd , MgmtId: 32985499829028, via:
1(ref-trl-4355-k-M7-abhishek-kumar-kvm1), Ver: v1, Flags: 100001,
[{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":"2","publicIp":"10.0.53.224","sourceNat":"false","add":"true","oneToOneNat":"false","firstIP":"true","broadcastUri":"vlan://51","vlanGateway":"10.0.48.1","vlanNetmask":"255.255.240.0","vifMacAddress":"1e:00:c4:00:00:03","networkRate":"200","trafficType":"Public","networkName":"cloudbr1","newNic":"false","isPrivateGateway":"false","nicTO":{"deviceId":"0","networkRateMbps":"-1","defaultNic":"false","pxeDisable":"false","details":{"MacAddressChanges":"true","PromiscuousMode":"false","ForgedTransmits":"true","MacLearning":"false"},"dpdkEnabled":"false","gateway":"10.0.48.1","mac":"1e:00:c4:00:00:03","broadcastType":"Vlan","type":"Public","broadcastUri":"
vlan://51","isSecurityGroupEnabled":"false","name":"cloudbr1"}}],"accessDetails":{"router.name":"r-3-VM","router.guest.ip":"10.0.53.223","network.public.last.ip":"false","router.ip":"169.254.128.211","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}]
}
2023-02-06 11:38:42,336 DEBUG [c.c.a.ApiServlet]
(qtp396883763-23:ctx-65287e60) (logid:5510388f) ===START=== 10.0.3.251 -- GET
jobId=dabb8445-540d-432e-ab0d-7044b7b0e4ed&command=queryAsyncJobResult&response=json
2023-02-06 11:38:42,340 DEBUG [c.c.a.ApiServer]
(qtp396883763-23:ctx-65287e60 ctx-67a69ba8) (logid:5510388f) CIDRs from which
account 'Acct[17fbff81-a395-11ed-804d-1e0009000324-admin] -- Account {"id": 2,
"name": "admin", "uuid": "17fbff81-a395-11ed-804d-1e0009000324"}' is allowed to
perform API calls: 0.0.0.0/0,::/0
2023-02-06 11:38:42,351 DEBUG [c.c.a.ApiServlet]
(qtp396883763-23:ctx-65287e60 ctx-67a69ba8) (logid:5510388f) ===END===
10.0.3.251 -- GET
jobId=dabb8445-540d-432e-ab0d-7044b7b0e4ed&command=queryAsyncJobResult&response=json
...
```
2023-02-06 11:38:46,509 ERROR [c.c.a.ApiServer]
(qtp396883763-17:ctx-e72603b7 ctx-eca3fb69) (logid:191355ef) unhandled
exception executing api command: [Ljava.lang.String;@72b7f33b
java.lang.IndexOutOfBoundsException: Index 1 out of bounds for length 1
at
java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at
java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at
java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.get(ArrayList.java:459)
at
org.apache.cloudstack.api.command.user.loadbalancer.ListLoadBalancerRuleInstancesCmd.execute(ListLoadBalancerRuleInstancesCmd.java:133)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:772)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:596)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:645)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at
org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
```
<!--
For bugs, show exactly how to reproduce the problem, using a minimal
test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
- Deploy a VPC with its tier
- Deploy a VM into this tier and with one additional network
- Remove VM's nic for the VPC tier
- Add VPC tier network again. In DB, VM will have 3 entries now in the
cloud.nics table
- Acquire a new public IP for the VPC
- Add a new LB rule using the VM for the acquired public IP. Observe failure
in the UI and logs
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
No error seen in the UI and logs. listLoadBalancerRuleInstances returns
result.
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
Error seen as shown in the above screenshot/logs
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
