--- Begin Message ---
Sonali, sure. It will have to be improved. Please be very explicit in
the ticket, better change it will be taken by someone.
On Fri, Mar 20, 2015 at 9:01 AM, Sonali Jadhav
<[email protected]> wrote:
> @Somesh: added ticket https://issues.apache.org/jira/browse/CLOUDSTACK-8328
> @Daan: should I add ticket for overzealous exception handling ?
>
> /Sonali
>
> -----Original Message-----
> From: Somesh Naidu [mailto:[email protected]]
> Sent: Tuesday, March 10, 2015 8:00 PM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> It could be a bug in that case. I guess you could file it on the tracker for
> further investigation.
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Sonali Jadhav [mailto:[email protected]]
> Sent: Tuesday, March 10, 2015 8:21 AM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> Hi somesh,
>
> I just cross checked that, service_offering where id=14, is nothing but
> "Custom" compute offering I created. That’s why ram_size value is set to
> NULL. So basically I cant/don’t want to change value of ram_size service
> offering, Is this bug?
>
> /Sonali
>
> -----Original Message-----
> From: Sonali Jadhav [mailto:[email protected]]
> Sent: Tuesday, March 10, 2015 5:38 PM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> Hi Somesh,
>
> I have very weird values in Service offering table,
>
> mysql> select * from service_offering;
> +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+
> | id | cpu | speed | ram_size | nw_rate | mc_rate | ha_enabled |
> limit_cpu_use | host_tag | default_use | vm_type | sort_key |
> is_volatile | deployment_planner |
> +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+
> | 1 | 1 | 500 | 512 | NULL | NULL | 0 |
> 0 | NULL | 0 | NULL | 0 |
> 0 | NULL |
> | 2 | 1 | 1000 | 1024 | NULL | NULL | 0 |
> 0 | NULL | 0 | NULL | 0 |
> 0 | NULL |
> | 7 | 1 | 500 | 128 | NULL | NULL | 1 |
> 0 | NULL | 1 | domainrouter | 0 |
> 0 | NULL |
> | 8 | 1 | 500 | 1024 | 0 | 0 | 0 |
> 0 | NULL | 1 | consoleproxy | 0 |
> 0 | NULL |
> | 9 | 1 | 256 | 128 | NULL | NULL | 1 |
> 0 | NULL | 1 | internalloadbalancervm | 0 |
> 0 | NULL |
> | 10 | 1 | 500 | 512 | NULL | NULL | 0 |
> 0 | NULL | 1 | secondarystoragevm | 0 |
> 0 | NULL |
> | 11 | 1 | 128 | 128 | 0 | 0 | 1 |
> 0 | NULL | 1 | elasticloadbalancervm | 0 |
> 0 | NULL |
> | 12 | 4 | 1000 | 4096 | NULL | NULL | 0 |
> 0 | NULL | 0 | secondarystoragevm | 0 |
> 0 | NULL |
> | 13 | 4 | 1000 | 4096 | NULL | NULL | 0 |
> 0 | NULL | 0 | secondarystoragevm | 0 |
> 0 | NULL |
> | 14 | NULL | NULL | NULL | NULL | NULL | 0 |
> 0 | NULL | 0 | NULL | 0 | 0
> | NULL |
> | 16 | 2 | 2000 | 4096 | NULL | NULL | 0 |
> 0 | NULL | 0 | NULL | 0 |
> 0 | NULL |
> | 17 | NULL | NULL | NULL | NULL | NULL | 0 |
> 0 | NULL | 0 | NULL | 0 | 0
> | NULL |
> | 18 | NULL | NULL | NULL | NULL | NULL | 1 |
> 0 | NULL | 0 | NULL | 0 | 0
> | NULL |
> | 20 | NULL | NULL | NULL | NULL | NULL | 1 |
> 0 | NULL | 0 | NULL | 0 | 0
> | NULL |
> +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+
> 14 rows in set (0.00 sec)
>
> mysql>
>
> any clue? There are 4 rows, which are having NULL values set, I have no clue
> from where did that came.
>
> /Sonali
>
> -----Original Message-----
> From: Somesh Naidu [mailto:[email protected]]
> Sent: Thursday, March 5, 2015 8:49 PM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> That's causing the NPE. When you fix that, ram_size should have some value,
> you should be able to remove the NIC.
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Sonali Jadhav [mailto:[email protected]]
> Sent: Thursday, March 05, 2015 12:20 AM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> Hi Somesh,
>
> Here is output,
>
> mysql> select ram_size from service_offering where id=14;
> +----------+
> | ram_size |
> +----------+
> | NULL |
> +----------+
> 1 row in set (0.00 sec)
>
> Thanks,
> /Sonali
>
> -----Original Message-----
> From: Somesh Naidu [mailto:[email protected]]
> Sent: Thursday, March 5, 2015 2:19 AM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> Yes, I did. Looks strange, an NPE when calculating minimum memory required
> for this VM.
>
> Could you send me the output of the following SQL:
> select ram_size from service_offering where id=14;
>
> Somesh
> CloudPlatform Escalations
> Citrix Systems, Inc.
>
> -----Original Message-----
> From: Sonali Jadhav [mailto:[email protected]]
> Sent: Tuesday, March 03, 2015 6:39 AM
> To: [email protected]
> Subject: RE: unable to remove NIC from instance
>
> Hi Somesh,
>
> Have you looked at the logs and query results I sent, what do you think?
>
> /Sonali
>
> -----Original Message-----
> From: Daan Hoogland [mailto:[email protected]]
> Sent: Thursday, February 26, 2015 1:42 PM
> To: [email protected]
> Subject: Re: unable to remove NIC from instance
>
> As far as I can tell it tries to delete a secondary ip from a nic instead of
> a second (or first) nic. This is going to cost me some diving into. I hope
> Somesh has a simple answer for you. The initial error we talked about is in
> your log between lines 97 and 133. I will have a look when I find the time
> but I can't promise.
>
> If you are familiar with git you could look at the history of the 4.4 branch
> since the release to see if a similar problem has been fixed.
>
>
> On Thu, Feb 26, 2015 at 8:35 AM, Sonali Jadhav <[email protected]>
> wrote:
>> Hi Somesh n Daan,
>>
>> Thanks for reply.
>>
>> Here are logs from point I logged to CloudStack till the error
>> occurred http://pastebin.com/j1rfEq5s
>>
>> And here is sql query output,
>>
>> mysql> select * from vm_instance where instance_name="i-9-24-VM" \G
>> *************************** 1. row ***************************
>> id: 24
>> name: linuxvm
>> uuid: b5c535b4-10c4-44a9-b117-6df07537434c
>> instance_name: i-9-24-VM
>> state: Stopped
>> vm_template_id: 202
>> guest_os_id: 201
>> private_mac_address: 02:00:3e:c4:00:01
>> private_ip_address: 10.1.1.14
>> pod_id: 1
>> data_center_id: 1
>> host_id: NULL
>> last_host_id: 1
>> proxy_id: 1
>> proxy_assign_time: 2015-02-20 11:25:18
>> vnc_password: 7fJhsb8PVskfWRFgiQw02qDERH4PYxUS7xHy+2OU/HI=
>> ha_enabled: 0
>> limit_cpu_use: 0
>> update_count: 25
>> update_time: 2015-02-23 06:26:46
>> created: 2015-02-20 11:24:26
>> removed: NULL
>> type: User
>> vm_type: User
>> account_id: 9
>> domain_id: 5
>> service_offering_id: 14
>> reservation_id: c59c1ead-fd98-4822-b7bb-71124fbf0018
>> hypervisor_type: XenServer
>> disk_offering_id: 6
>> owner: 9
>> host_name: linuxvm
>> display_name: linuxvm
>> desired_state: NULL
>> dynamically_scalable: 0
>> display_vm: 1
>> power_state: NULL
>> power_state_update_time: NULL
>> power_state_update_count: 0
>> power_host: NULL
>> 1 row in set (0.00 sec)
>>
>> mysql>
>>
>> mysql> select * from nics where instance_id=24 \G
>> *************************** 1. row ***************************
>> id: 60
>> uuid: 99378e9f-ba64-4c76-ac16-d9ac977d979f
>> instance_id: 24
>> mac_address: 06:by:bg:00:01:33
>> ip4_address: 1.1.1.1
>> netmask: 255.255.0.0
>> gateway: 1.1.1.1
>> ip_type: Ip4
>> broadcast_uri: vlan://201
>> network_id: 207
>> mode: Dhcp
>> state: Allocated
>> strategy: Create
>> reserver_name: DirectNetworkGuru
>> reservation_id: NULL
>> device_id: 1
>> update_time: 2015-02-23 07:25:40
>> isolation_uri: vlan://201
>> ip6_address: NULL
>> default_nic: 0
>> vm_type: User
>> created: 2015-02-20 11:24:26
>> removed: NULL
>> ip6_gateway: NULL
>> ip6_cidr: NULL
>> secondary_ip: 0
>> display_nic: 1
>> *************************** 2. row ***************************
>> id: 61
>> uuid: 42178ae9-1c4c-43be-8a05-5d6259ee9ee4
>> instance_id: 24
>> mac_address: 02:00:3f:a4:00:01
>> ip4_address: 10.1.1.14
>> netmask: 255.255.255.0
>> gateway: 10.1.1.1
>> ip_type: Ip4
>> broadcast_uri: NULL
>> network_id: 211
>> mode: Dhcp
>> state: Allocated
>> strategy: Start
>> reserver_name: ExternalGuestNetworkGuru
>> reservation_id: c59c1ead-fd98-4822-b7bb-71124fbf0018
>> device_id: 0
>> update_time: 2015-02-23 07:25:41
>> isolation_uri: NULL
>> ip6_address: NULL
>> default_nic: 1
>> vm_type: User
>> created: 2015-02-20 13:21:07
>> removed: NULL
>> ip6_gateway: NULL
>> ip6_cidr: NULL
>> secondary_ip: 0
>> display_nic: 1
>> 2 rows in set (0.00 sec)
>>
>> mysql>
>>
>> Let me know if you find something from this.
>>
>> /Sonali
>>
>> -----Original Message-----
>> From: Somesh Naidu [mailto:[email protected]]
>> Sent: Wednesday, February 25, 2015 11:07 PM
>> To: [email protected]
>> Subject: RE: unable to remove NIC from instance
>>
>> Sonali,
>>
>> While this could be fixed in the DB, as Daan mentioned, it is not
>> recommended. The primary table that needs to be update is the nics table but
>> in addition, relevant tables like user_ip_address, etc need to be updated.
>> There may also be impact on usage. Moreover, updating the DB won’t take care
>> of removing the nic on the hypervisor/VM.
>>
>> A better way to deal with this is to resolve the original error that is
>> preventing CCP from removing the nic.
>>
>> I have looked at the information (log snippet) you have shared. As Daan
>> pointed out, these logs contain consequential errors where
>> queryAsyncJobResult API is failing to get the result of a job (happens to be
>> the RemoveNicFromVMCmdByAdmin API call that you are interested in) that
>> relate to some trouble with the job framework. But this is not why the
>> original remove nic command failed.
>>
>> What you can do is,
>> 1. Note the time of the last line of log in mgmt. server (just do a tail on
>> mgmt. log) 2. From CS UI, remove the nic from the VM 3. Send us mgmt. logs
>> starting from time noted in step #1 4. In addition provide details of the VM,
>> a. select * from vm_instance where id=<VM_ID>;
>> b. select * from nics where instance_id=<VM_ID>;
>>
>> If I have this information I can try and identify why the original operation
>> is failing.
>>
>> Somesh
>> CloudPlatform Escalations
>> Citrix Systems, Inc.
>>
>> -----Original Message-----
>> From: Daan Hoogland [mailto:[email protected]]
>> Sent: Wednesday, February 25, 2015 7:47 AM
>> To: [email protected]
>> Subject: Re: unable to remove NIC from instance
>>
>> I have no experience with this. It could be fixed in the db probably
>> but that is always a bad advice
>> 1 - coming from a developer
>> 2 - for application in a production system
>>
>> If you are very familiar with the datamodel you might try to, I would only
>> try to do it in a test environment (or at least first) Also I do not have
>> the db change that would help you by head.
>>
>>
>> On Wed, Feb 25, 2015 at 12:25 PM, Sonali Jadhav <[email protected]>
>> wrote:
>>> Hi Daan,
>>>
>>> Is there any manual way to fix it temporary ? I have production customer
>>> who is facing this issue.
>>>
>>> /Sonali
>>>
>>> -----Original Message-----
>>> From: Sonali Jadhav [mailto:[email protected]]
>>> Sent: Tuesday, February 24, 2015 1:27 PM
>>> To: [email protected]
>>> Subject: RE: unable to remove NIC from instance
>>>
>>> I am running xenserver 6.2, I have 1 cluster with 4 Hosts.
>>>
>>> /Sonali
>>>
>>> -----Original Message-----
>>> From: Daan Hoogland [mailto:[email protected]]
>>> Sent: Tuesday, February 24, 2015 1:22 PM
>>> To: [email protected]
>>> Subject: Re: unable to remove NIC from instance
>>>
>>> Ok, thanks, we are running a custom 4.4.2 version we had to make due to
>>> problems with the stock 4.4.2. I will look into making another release
>>> candidate for 4.4.3 and have a look at the code to see if I can find a
>>> cause to your problem. The last 4.4.3 release candidate I made was largely
>>> ignored and therefore never released. It also has a problem with creating
>>> new volumes on vmware which is not solved yet.
>>> What type of hypervisors are you running?
>>>
>>> On Tue, Feb 24, 2015 at 8:45 AM, Sonali Jadhav <[email protected]>
>>> wrote:
>>>> Hi Daan,
>>>>
>>>> Super thanks for reply. I am running version 4.4.2
>>>>
>>>> /Sonali
>>>>
>>>> -----Original Message-----
>>>> From: Daan Hoogland [mailto:[email protected]]
>>>> Sent: Tuesday, February 24, 2015 1:05 PM
>>>> To: [email protected]
>>>> Subject: Re: unable to remove NIC from instance
>>>>
>>>> It seems to me you are a victim of overzealous exception handling.
>>>> I've seen that happen in the async job handling. The job dispatching or
>>>> construction has an exception that is not handled but caught and ignored,
>>>> it end and the async job is not removed from the queue. I'd like to look
>>>> at the code near the stack trace but for that I need the exact version of
>>>> Cloudstack you are running.
>>>>
>>>> On Tue, Feb 24, 2015 at 5:44 AM, Sonali Jadhav <[email protected]>
>>>> wrote:
>>>>> Hi Daan,
>>>>>
>>>>> I have checked logs form starting of api call, and I have not found
>>>>> anything special, But apart from error, I found one warning,
>>>>>
>>>>> 2015-02-23 13:11:46,226 WARN [c.c.u.d.Merovingian2]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable
>>>>> to find lock for the key vm_instance24 and thread id 137380473
>>>>>
>>>>> Don’t know meaning of this.
>>>>>
>>>>> Here are my logs,
>>>>>
>>>>> 2015-02-23 13:11:45,457 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgentCronJob-296:ctx-11de89f6) Seq 5-8090716730571096067: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:45,458 DEBUG [c.c.a.t.Request]
>>>>> (DirectAgentCronJob-296:ctx-11de89f6) Seq 5-8090716730571096067:
>>>>> Processing: { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags:
>>>>> 10,
>>>>> [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1,"
>>>>> _
>>>>> vmMetaDatum":{"i-2-11-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:
>>>>> true","s-15-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true","
>>>>> r
>>>>> -
>>>>> 6
>>>>> -VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true","i-2-5-VM":"
>>>>> v
>>>>> i
>>>>> r
>>>>> idian:true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-13-VM":"
>>>>> viridian:true;acpi:1;apic:true;pae:true;nx:true","i-5-12-VM":"virid
>>>>> i
>>>>> a
>>>>> n
>>>>> :true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-14-VM":"vi
>>>>> r
>>>>> i
>>>>> d
>>>>> ian:true;acpi:1;apic:true;pae:true;nx:true","v-1-VM":"viridian:true
>>>>> ;
>>>>> a
>>>>> c
>>>>> pi:1;apic:true;pae:true;nx:true","i-9-23-VM":"viridian:true;acpi:1;
>>>>> a
>>>>> p
>>>>> i
>>>>> c:true;pae:true;nx:true;timeoffset:-28800"},"_isExecuted":false,"re
>>>>> s
>>>>> u
>>>>> l
>>>>> t":true,"wait":0}}] }
>>>>> 2015-02-23 13:11:45,572 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: Executing
>>>>> request
>>>>> 2015-02-23 13:11:45,628 DEBUG [c.c.h.x.r.CitrixResourceBase]
>>>>> (DirectAgent-48:ctx-9e4c61a9) Vm cpu utilization
>>>>> 0.11499999999999999
>>>>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: MgmtId
>>>>> 249373720971888: Resp: Routing to peer
>>>>> 2015-02-23 13:11:45,682 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: Executing
>>>>> request
>>>>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase]
>>>>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.1152941176470588
>>>>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase]
>>>>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.2599999999999999
>>>>> 2015-02-23 13:11:45,766 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: Response Received:
>>>>> 2015-02-23 13:11:45,767 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: MgmtId
>>>>> 249373720971888: Resp: Routing to peer
>>>>> 2015-02-23 13:11:45,791 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: Executing
>>>>> request
>>>>> 2015-02-23 13:11:45,835 DEBUG [c.c.h.x.r.CitrixResourceBase]
>>>>> (DirectAgent-192:ctx-61e81d25) Vm cpu utilization
>>>>> 0.13343749999999996
>>>>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: MgmtId
>>>>> 249373720971888: Resp: Routing to peer
>>>>> 2015-02-23 13:11:45,921 DEBUG [c.c.a.ApiServlet]
>>>>> (http-6443-exec-4:ctx-5a612122) ===START=== 192.168.45.32 -- GET
>>>>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4i
>>>>> E
>>>>> l
>>>>> w
>>>>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6d
>>>>> f
>>>>> 0
>>>>> 7
>>>>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955
>>>>> 2015-02-23 13:11:46,095 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) submit async job-526,
>>>>> details: AsyncJobVO {id:526, userId: 2, accountId: 2, instanceType:
>>>>> None, instanceId: null, cmd:
>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi
>>>>> n
>>>>> ,
>>>>> cmdInfo:
>>>>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d","
>>>>> c
>>>>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977
>>>>> d
>>>>> 9
>>>>> 7
>>>>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0
>>>>> 7
>>>>> 5
>>>>> 3
>>>>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c","
>>>>> c
>>>>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d
>>>>> 9
>>>>> 7
>>>>> 9
>>>>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccou
>>>>> n t I d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status:
>>>>> IN_PROGRESS,
>>>>> processStatus: 0, resultCode: 0, result: null, initMsid:
>>>>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled:
>>>>> null, created: null}
>>>>> 2015-02-23 13:11:46,096 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Add job-526 into job
>>>>> monitoring
>>>>> 2015-02-23 13:11:46,096 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Executing AsyncJobVO
>>>>> {id:526, userId: 2, accountId: 2, instanceType: None, instanceId:
>>>>> null, cmd:
>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi
>>>>> n
>>>>> ,
>>>>> cmdInfo:
>>>>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d","
>>>>> c
>>>>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977
>>>>> d
>>>>> 9
>>>>> 7
>>>>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0
>>>>> 7
>>>>> 5
>>>>> 3
>>>>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c","
>>>>> c
>>>>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d
>>>>> 9
>>>>> 7
>>>>> 9
>>>>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccou
>>>>> n t I d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status:
>>>>> IN_PROGRESS,
>>>>> processStatus: 0, resultCode: 0, result: null, initMsid:
>>>>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled:
>>>>> null, created: null}
>>>>> 2015-02-23 13:11:46,097 DEBUG [c.c.a.ApiServlet]
>>>>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) ===END===
>>>>> 192.168.45.32
>>>>> -- GET
>>>>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4i
>>>>> E
>>>>> l
>>>>> w
>>>>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6d
>>>>> f
>>>>> 0
>>>>> 7
>>>>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955
>>>>> 2015-02-23 13:11:46,219 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>>>>> (consoleproxy-1:ctx-a5fe3580) Zone 1 is ready to launch console
>>>>> proxy
>>>>> 2015-02-23 13:11:46,221 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Sync
>>>>> job-527 execution on object VmWorkJobQueue.24
>>>>> 2015-02-23 13:11:46,226 WARN [c.c.u.d.Merovingian2]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable
>>>>> to find lock for the key vm_instance24 and thread id 137380473
>>>>> 2015-02-23 13:11:46,534 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>>>>> (secstorage-1:ctx-be45d820) Zone 1 is ready to launch secondary
>>>>> storage VM
>>>>> 2015-02-23 13:11:49,095 DEBUG [c.c.s.StatsCollector]
>>>>> (StatsCollector-4:ctx-4a027738) HostStatsCollector is running...
>>>>> 2015-02-23 13:11:49,115 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-93:ctx-3810dbca) Seq 1-9127107594819755362: Executing
>>>>> request
>>>>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-93:ctx-3810dbca) Seq 1-9127107594819755362: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.t.Request]
>>>>> (StatsCollector-4:ctx-4a027738) Seq 1-9127107594819755362: Received:
>>>>> { Ans: , MgmtId: 59778234354585, via: 1, Ver: v1, Flags: 10, {
>>>>> GetHostStatsAnswer } }
>>>>> 2015-02-23 13:11:49,182 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-4:ctx-5d8fe3d8) Seq 4-677791743919277703: Executing
>>>>> request
>>>>> 2015-02-23 13:11:49,226 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-4:ctx-5d8fe3d8) Seq 4-677791743919277703: Response Received:
>>>>> 2015-02-23 13:11:49,227 DEBUG [c.c.a.t.Request]
>>>>> (StatsCollector-4:ctx-4a027738) Seq 4-677791743919277703: Received:
>>>>> {
>>>>> Ans: , MgmtId: 59778234354585, via: 4, Ver: v1, Flags: 10, {
>>>>> GetHostStatsAnswer } }
>>>>> 2015-02-23 13:11:49,242 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-194:ctx-9a8ead36) Seq 5-8090716730571121376: Executing
>>>>> request
>>>>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-194:ctx-9a8ead36) Seq 5-8090716730571121376: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.t.Request]
>>>>> (StatsCollector-4:ctx-4a027738) Seq 5-8090716730571121376: Received:
>>>>> { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: 10, {
>>>>> GetHostStatsAnswer } }
>>>>> 2015-02-23 13:11:49,288 DEBUG [c.c.a.ApiServlet]
>>>>> (http-6443-exec-3:ctx-c1a661f7) ===START=== 192.168.45.32 -- GET
>>>>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadb
>>>>> b
>>>>> 8
>>>>> a
>>>>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469
>>>>> 3
>>>>> 4
>>>>> 0
>>>>> 6327
>>>>> 2015-02-23 13:11:49,302 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-383:ctx-5f562eca) Seq 6-892275676172801479: Executing
>>>>> request
>>>>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.m.DirectAgentAttache]
>>>>> (DirectAgent-383:ctx-5f562eca) Seq 6-892275676172801479: Response
>>>>> Received:
>>>>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.t.Request]
>>>>> (StatsCollector-4:ctx-4a027738) Seq 6-892275676172801479: Received:
>>>>> {
>>>>> Ans: , MgmtId: 59778234354585, via: 6, Ver: v1, Flags: 10, {
>>>>> GetHostStatsAnswer } }
>>>>> 2015-02-23 13:11:49,626 DEBUG [c.c.a.ApiServlet]
>>>>> (http-6443-exec-3:ctx-c1a661f7 ctx-62507b94) ===END===
>>>>> 192.168.45.32
>>>>> -- GET
>>>>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadb
>>>>> b
>>>>> 8
>>>>> a
>>>>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469
>>>>> 3
>>>>> 4
>>>>> 0
>>>>> 6327
>>>>> 2015-02-23 13:11:49,672 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Unexpected exception
>>>>> while executing
>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi
>>>>> n
>>>>> java.lang.RuntimeException: Job failed due to exception null
>>>>> at
>>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>>>> at
>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>>> at
>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>> at
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>> at
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>> at
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>> at
>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>> at
>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>>> at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>> 2015-02-23 13:11:49,673 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Complete async job-526,
>>>>> jobStatus: FAILED, resultCode: 530, result:
>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>>>>> [],"errorcode":530,"errortext":"Job failed due to exception null"}
>>>>> 2015-02-23 13:11:49,730 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Done executing
>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi
>>>>> n
>>>>> for job-526
>>>>> 2015-02-23 13:11:49,757 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Remove job-526 from job
>>>>> monitoring
>>>>>
>>>>>
>>>>> /Sonali
>>>>>
>>>>> -----Original Message-----
>>>>> From: Daan Hoogland [mailto:[email protected]]
>>>>> Sent: Monday, February 23, 2015 2:42 PM
>>>>> To: [email protected]
>>>>> Subject: Re: unable to remove NIC from instance
>>>>>
>>>>> Sonali,
>>>>>
>>>>> this looks like a consequential error. You need to look further back into
>>>>> the log to find the root cause. The start job is the a query for results,
>>>>> please find the start for the actual api command.
>>>>>
>>>>> On Mon, Feb 23, 2015 at 6:55 AM, Sonali Jadhav
>>>>> <[email protected]> wrote:
>>>>>> I am trying to remove "Shared guest network". These are new logs,
>>>>>>
>>>>>> 2015-02-23 06:53:59,737 DEBUG
>>>>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>>>>> (RouterStatusMonitor-1:ctx-28b1e134) Found 0 networks to update RvR
>>>>>> status.
>>>>>> 2015-02-23 06:54:01,726 DEBUG [c.c.a.ApiServlet]
>>>>>> (http-6443-exec-3:ctx-6eb9497d) ===START=== 115.114.134.114 --
>>>>>> GET
>>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b
>>>>>> 9
>>>>>> 4
>>>>>> c
>>>>>> f
>>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246
>>>>>> 7
>>>>>> 0
>>>>>> 7
>>>>>> 3
>>>>>> 8796
>>>>>> 2015-02-23 06:54:01,818 DEBUG [c.c.a.ApiServlet]
>>>>>> (http-6443-exec-3:ctx-6eb9497d ctx-007c94be) ===END===
>>>>>> 115.114.134.114 -- GET
>>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b
>>>>>> 9
>>>>>> 4
>>>>>> c
>>>>>> f
>>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246
>>>>>> 7
>>>>>> 0
>>>>>> 7
>>>>>> 3
>>>>>> 8796
>>>>>> 2015-02-23 06:54:01,993 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Unexpected exception
>>>>>> while executing
>>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdm
>>>>>> i
>>>>>> n
>>>>>> java.lang.RuntimeException: Job failed due to exception null
>>>>>> at
>>>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>>>>> at
>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>>>> at
>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>>> at
>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>>> at
>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>>> at
>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>>> at
>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>>> at
>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>>>> at
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>> 2015-02-23 06:54:01,994 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Complete async
>>>>>> job-499,
>>>>>> jobStatus: FAILED, resultCode: 530, result:
>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>>>>>> [],"errorcode":530,"errortext":"Job failed due to exception null"}
>>>>>> 2015-02-23 06:54:02,051 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Done executing
>>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdm
>>>>>> i
>>>>>> n
>>>>>> for job-499
>>>>>> 2015-02-23 06:54:02,077 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Remove job-499 from
>>>>>> job monitoring
>>>>>> 2015-02-23 06:54:03,942 DEBUG [c.c.s.StatsCollector]
>>>>>> (StatsCollector-2:ctx-ce679066) AutoScaling Monitor is running...
>>>>>> 2015-02-23 06:54:04,727 DEBUG [c.c.a.ApiServlet]
>>>>>> (http-6443-exec-4:ctx-e51e68c3) ===START=== 115.114.134.114 --
>>>>>> GET
>>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b
>>>>>> 9
>>>>>> 4
>>>>>> c
>>>>>> f
>>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246
>>>>>> 7
>>>>>> 0
>>>>>> 7
>>>>>> 4
>>>>>> 1797
>>>>>> 2015-02-23 06:54:04,802 DEBUG [c.c.a.ApiServlet]
>>>>>> (http-6443-exec-4:ctx-e51e68c3 ctx-a70a6fff) ===END===
>>>>>> 115.114.134.114 -- GET
>>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b
>>>>>> 9
>>>>>> 4
>>>>>> c
>>>>>> f
>>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246
>>>>>> 7
>>>>>> 0
>>>>>> 7
>>>>>> 4
>>>>>> 1797
>>>>>>
>>>>>>
>>>>>> /Sonali
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Sonali Jadhav [mailto:[email protected]]
>>>>>> Sent: Monday, February 23, 2015 10:25 AM
>>>>>> To: [email protected]
>>>>>> Subject: RE: unable to remove NIC from instance
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Yes I have tried that, and issue is still same. Same logs.
>>>>>>
>>>>>> /Sonali
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Daan Hoogland [mailto:[email protected]]
>>>>>> Sent: Sunday, February 22, 2015 10:57 PM
>>>>>> To: [email protected]
>>>>>> Subject: Re: unable to remove NIC from instance
>>>>>>
>>>>>> Sonil, Did you try what Gopalakrishnan suggested?
>>>>>>
>>>>>> Also can you expand on the version of cloudstack you are running?
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> On Sun, Feb 22, 2015 at 12:24 PM, Sonali Jadhav
>>>>>> <[email protected]> wrote:
>>>>>>> Anyone help?
>>>>>>> /Sonali
>>>>>>>
>>>>>>>
>>>>>>> ---- Sonali Jadhav wrote ----
>>>>>>>
>>>>>>> Anyone?
>>>>>>>
>>>>>>> Sent from my Sony Xperia™ smartphone
>>>>>>>
>>>>>>>
>>>>>>> ---- Sonali Jadhav wrote ----
>>>>>>>
>>>>>>> Here are logs,
>>>>>>>
>>>>>>> 2015-02-20 14:26:34,122 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>>>> (AgentManager-Handler-12:null) SeqA 2-260495: Sending Seq 2-260495:
>>>>>>> {
>>>>>>> Ans: , MgmtId: 59778234354585, via: 2, Ver: v1, Flags: 100010,
>>>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":
>>>>>>> 0
>>>>>>> }
>>>>>>> }
>>>>>>> ]
>>>>>>> }
>>>>>>> 2015-02-20 14:26:34,638 DEBUG [o.a.c.f.j.d.VmWorkJobDaoImpl]
>>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Expunge completed work
>>>>>>> job-483
>>>>>>> 2015-02-20 14:26:34,640 DEBUG [c.c.u.d.T.Transaction]
>>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Rolling back the transaction:
>>>>>>> Time = 2 Name = Vm-Operations-Cleanup-1; called by
>>>>>>> -TransactionLegacy.rollback:900-TransactionLegacy.removeUpTo:843-
>>>>>>> T
>>>>>>> r
>>>>>>> a
>>>>>>> n
>>>>>>> s
>>>>>>> actionLegacy.close:667-TransactionContextInterceptor.invoke:36-Re
>>>>>>> f
>>>>>>> l
>>>>>>> e
>>>>>>> c
>>>>>>> t
>>>>>>> iveMethodInvocation.proceed:161-ExposeInvocationInterceptor.invoke:
>>>>>>> 9
>>>>>>> 1
>>>>>>> -
>>>>>>> ReflectiveMethodInvocation.proceed:172-JdkDynamicAopProxy.invoke:
>>>>>>> 2
>>>>>>> 0
>>>>>>> 4
>>>>>>> -
>>>>>>> $
>>>>>>> Proxy182.expungeCompletedWorkJobs:-1-VirtualMachineManagerImpl$Cl
>>>>>>> e
>>>>>>> a
>>>>>>> n
>>>>>>> u
>>>>>>> p
>>>>>>> Task.runInContext:2389-ManagedContextRunnable$1.run:49-DefaultMan
>>>>>>> a
>>>>>>> g
>>>>>>> e
>>>>>>> d
>>>>>>> C
>>>>>>> ontext$1.call:56
>>>>>>> 2015-02-20 14:26:34,658 ERROR [c.c.v.VirtualMachineManagerImpl]
>>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) VM Operations failed due
>>>>>>> to
>>>>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
>>>>>>> com.mysql.jdbc.PreparedStatement@e7a49c5: DELETE FROM async_job WHERE
>>>>>>> async_job.id= 483
>>>>>>> at
>>>>>>> com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1178)
>>>>>>> at
>>>>>>> org.apache.cloudstack.framework.jobs.dao.VmWorkJobDaoImpl.expungeCompletedWorkJobs(VmWorkJobDaoImpl.java:149)
>>>>>>> at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>> at
>>>>>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>>>>>>> at
>>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>>>>>>> at
>>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>>>>>>> at
>>>>>>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>>>>>>> at
>>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>>>>>>> at
>>>>>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>>>>>>> at
>>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>>>>>>> at
>>>>>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>>>>>>> at com.sun.proxy.$Proxy182.expungeCompletedWorkJobs(Unknown
>>>>>>> Source)
>>>>>>> at
>>>>>>> com.cloud.vm.VirtualMachineManagerImpl$CleanupTask.runInContext(VirtualMachineManagerImpl.java:2389)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>>>> at
>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>>>> at
>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>>> Caused by:
>>>>>>> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException:
>>>>>>> Cannot delete or update a parent row: a foreign key constraint fails
>>>>>>> (`cloud`.`async_job_join_map`, CONSTRAINT
>>>>>>> `fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`)
>>>>>>> REFERENCES `async_job` (`id`))
>>>>>>> at
>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>> at
>>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>>>>>> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>>>>>> at com.mysql.jdbc.Util.getInstance(Util.java:386)
>>>>>>> at
>>>>>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1040)
>>>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)
>>>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)
>>>>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)
>>>>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
>>>>>>> at
>>>>>>> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)
>>>>>>> at
>>>>>>> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
>>>>>>> at
>>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415)
>>>>>>> at
>>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2333)
>>>>>>> at
>>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2318)
>>>>>>> at
>>>>>>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
>>>>>>> at
>>>>>>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
>>>>>>> at
>>>>>>> com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1169)
>>>>>>> ... 26 more
>>>>>>> 2015-02-20 14:26:34,770 DEBUG [c.c.a.ApiServlet]
>>>>>>> (http-6443-exec-4:ctx-46b96aa1) ===START=== 192.168.34.1 -- GET
>>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0
>>>>>>> a
>>>>>>> b
>>>>>>> 3
>>>>>>> f
>>>>>>> 1
>>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443
>>>>>>> 8
>>>>>>> 6
>>>>>>> 9
>>>>>>> 6
>>>>>>> 3
>>>>>>> 84
>>>>>>> 2015-02-20 14:26:34,797 DEBUG [c.c.u.AccountManagerImpl]
>>>>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) Access to
>>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to
>>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker
>>>>>>> 2015-02-20 14:26:34,858 DEBUG [c.c.a.ApiServlet]
>>>>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) ===END===
>>>>>>> 192.168.34.1
>>>>>>> -- GET
>>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0
>>>>>>> a
>>>>>>> b
>>>>>>> 3
>>>>>>> f
>>>>>>> 1
>>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443
>>>>>>> 8
>>>>>>> 6
>>>>>>> 9
>>>>>>> 6
>>>>>>> 3
>>>>>>> 84
>>>>>>> 2015-02-20 14:26:35,024 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Unexpected exception
>>>>>>> while executing
>>>>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd
>>>>>>> java.lang.RuntimeException: Job failed due to exception null
>>>>>>> at
>>>>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>>>>>> at
>>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>>>> at
>>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>>>> at
>>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>>>>> at
>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>>> 2015-02-20 14:26:35,025 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Complete async
>>>>>>> job-482,
>>>>>>> jobStatus: FAILED, resultCode: 530, result:
>>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>>>>>>> [],"errorcode":530,"errortext":"Job failed due to exception
>>>>>>> null"}
>>>>>>> 2015-02-20 14:26:35,082 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Done executing
>>>>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd for
>>>>>>> job-482
>>>>>>> 2015-02-20 14:26:35,111 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Resetting hosts
>>>>>>> suitable for reconnect
>>>>>>> 2015-02-20 14:26:35,113 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Remove job-482 from
>>>>>>> job monitoring
>>>>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed resetting
>>>>>>> hosts suitable for reconnect
>>>>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for
>>>>>>> clusters already owned by this management server
>>>>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring
>>>>>>> hosts for clusters already owned by this management server
>>>>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for
>>>>>>> clusters not owned by any management server
>>>>>>> 2015-02-20 14:26:35,117 DEBUG [c.c.h.d.HostDaoImpl]
>>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring
>>>>>>> hosts for clusters not owned by any management server
>>>>>>> 2015-02-20 14:26:37,771 DEBUG [c.c.a.ApiServlet]
>>>>>>> (http-6443-exec-2:ctx-1053809f) ===START=== 192.168.34.1 -- GET
>>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0
>>>>>>> a
>>>>>>> b
>>>>>>> 3
>>>>>>> f
>>>>>>> 1
>>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443
>>>>>>> 8
>>>>>>> 6
>>>>>>> 9
>>>>>>> 9
>>>>>>> 3
>>>>>>> 85
>>>>>>> 2015-02-20 14:26:37,798 DEBUG [c.c.u.AccountManagerImpl]
>>>>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) Access to
>>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to
>>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker
>>>>>>> 2015-02-20 14:26:37,857 DEBUG [c.c.a.ApiServlet]
>>>>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) ===END===
>>>>>>> 192.168.34.1
>>>>>>> -- GET
>>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0
>>>>>>> a
>>>>>>> b
>>>>>>> 3
>>>>>>> f
>>>>>>> 1
>>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443
>>>>>>> 8
>>>>>>> 6
>>>>>>> 9
>>>>>>> 9
>>>>>>> 3
>>>>>>> 85
>>>>>>> 2015-02-20 14:26:37,975 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>>>> (AgentManager-Handler-2:null) Ping from 2
>>>>>>> 2015-02-20 14:26:38,048 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>>>> (AgentManager-Handler-7:null) Ping from 9
>>>>>>> 2015-02-20 14:26:40,509 WARN [c.c.a.m.DirectAgentAttache]
>>>>>>> (DirectAgentCronJob-261:ctx-f3e99173) Unable to get current
>>>>>>> status on
>>>>>>> 4(SeSolXS02)
>>>>>>> 2015-02-20 14:26:40,687 WARN [c.c.a.m.DirectAgentAttache]
>>>>>>> (DirectAgentCronJob-130:ctx-8b85b559) Unable to get current
>>>>>>> status on
>>>>>>> 1(SeSolXS01)
>>>>>>> 2015-02-20 14:26:40,747 WARN [c.c.a.m.DirectAgentAttache]
>>>>>>> (DirectAgentCronJob-131:ctx-e6c7b284) Unable to get current
>>>>>>> status on
>>>>>>> 5(SeSolXS03)
>>>>>>> 2015-02-20 14:26:40,981 WARN [c.c.a.m.DirectAgentAttache]
>>>>>>> (DirectAgentCronJob-33:ctx-c6748244) Unable to get current status
>>>>>>> on
>>>>>>> 6(SeSolXS04)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> /Sonali
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Gopalakrishnan S [mailto:[email protected]]
>>>>>>> Sent: Friday, February 20, 2015 7:12 PM
>>>>>>> To: [email protected]
>>>>>>> Subject: Re: unable to remove NIC from instance
>>>>>>>
>>>>>>> Hi Sonali,
>>>>>>>
>>>>>>> It may be still routing that shared guest network. Try to restart your
>>>>>>> virutal router and virtual machine or shutdown your VM before removing
>>>>>>> that NIC. let me know the logs when you remove that NIC.
>>>>>>>
>>>>>>> Thank You.
>>>>>>> Gopalakrishnan.S
>>>>>>> Cloud Engineer - Fogpanel
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: Sonali Jadhav
>>>>>>> To: [email protected]
>>>>>>> Sent: Friday, February 20, 2015 7:06 PM
>>>>>>> Subject: unable to remove NIC from instance
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I have one linux instance to which I had added "Shared guest network"
>>>>>>> NIC. Then I added "isolated guest network" nic to it. Made "isolated
>>>>>>> guest network" NIC as default and tried to remove "Shared guest
>>>>>>> network" NIC, but its failing with null exception. I have attached logs.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> /sonali
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Daan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Daan
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Daan
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Daan
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Daan
>>
>>
>
>
>
> --
> Daan
>
>
>
>
>
>
>
>
>
>
--
Daan
--- End Message ---
