Sonali, sure. It will have to be improved. Please be very explicit in the ticket, better change it will be taken by someone.
On Fri, Mar 20, 2015 at 9:01 AM, Sonali Jadhav <[email protected]> wrote: > @Somesh: added ticket https://issues.apache.org/jira/browse/CLOUDSTACK-8328 > @Daan: should I add ticket for overzealous exception handling ? > > /Sonali > > -----Original Message----- > From: Somesh Naidu [mailto:[email protected]] > Sent: Tuesday, March 10, 2015 8:00 PM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > It could be a bug in that case. I guess you could file it on the tracker for > further investigation. > > Regards, > Somesh > > -----Original Message----- > From: Sonali Jadhav [mailto:[email protected]] > Sent: Tuesday, March 10, 2015 8:21 AM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > Hi somesh, > > I just cross checked that, service_offering where id=14, is nothing but > "Custom" compute offering I created. That’s why ram_size value is set to > NULL. So basically I cant/don’t want to change value of ram_size service > offering, Is this bug? > > /Sonali > > -----Original Message----- > From: Sonali Jadhav [mailto:[email protected]] > Sent: Tuesday, March 10, 2015 5:38 PM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > Hi Somesh, > > I have very weird values in Service offering table, > > mysql> select * from service_offering; > +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+ > | id | cpu | speed | ram_size | nw_rate | mc_rate | ha_enabled | > limit_cpu_use | host_tag | default_use | vm_type | sort_key | > is_volatile | deployment_planner | > +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+ > | 1 | 1 | 500 | 512 | NULL | NULL | 0 | > 0 | NULL | 0 | NULL | 0 | > 0 | NULL | > | 2 | 1 | 1000 | 1024 | NULL | NULL | 0 | > 0 | NULL | 0 | NULL | 0 | > 0 | NULL | > | 7 | 1 | 500 | 128 | NULL | NULL | 1 | > 0 | NULL | 1 | domainrouter | 0 | > 0 | NULL | > | 8 | 1 | 500 | 1024 | 0 | 0 | 0 | > 0 | NULL | 1 | consoleproxy | 0 | > 0 | NULL | > | 9 | 1 | 256 | 128 | NULL | NULL | 1 | > 0 | NULL | 1 | internalloadbalancervm | 0 | > 0 | NULL | > | 10 | 1 | 500 | 512 | NULL | NULL | 0 | > 0 | NULL | 1 | secondarystoragevm | 0 | > 0 | NULL | > | 11 | 1 | 128 | 128 | 0 | 0 | 1 | > 0 | NULL | 1 | elasticloadbalancervm | 0 | > 0 | NULL | > | 12 | 4 | 1000 | 4096 | NULL | NULL | 0 | > 0 | NULL | 0 | secondarystoragevm | 0 | > 0 | NULL | > | 13 | 4 | 1000 | 4096 | NULL | NULL | 0 | > 0 | NULL | 0 | secondarystoragevm | 0 | > 0 | NULL | > | 14 | NULL | NULL | NULL | NULL | NULL | 0 | > 0 | NULL | 0 | NULL | 0 | 0 > | NULL | > | 16 | 2 | 2000 | 4096 | NULL | NULL | 0 | > 0 | NULL | 0 | NULL | 0 | > 0 | NULL | > | 17 | NULL | NULL | NULL | NULL | NULL | 0 | > 0 | NULL | 0 | NULL | 0 | 0 > | NULL | > | 18 | NULL | NULL | NULL | NULL | NULL | 1 | > 0 | NULL | 0 | NULL | 0 | 0 > | NULL | > | 20 | NULL | NULL | NULL | NULL | NULL | 1 | > 0 | NULL | 0 | NULL | 0 | 0 > | NULL | > +----+------+-------+----------+---------+---------+------------+---------------+----------+-------------+------------------------+----------+-------------+--------------------+ > 14 rows in set (0.00 sec) > > mysql> > > any clue? There are 4 rows, which are having NULL values set, I have no clue > from where did that came. > > /Sonali > > -----Original Message----- > From: Somesh Naidu [mailto:[email protected]] > Sent: Thursday, March 5, 2015 8:49 PM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > That's causing the NPE. When you fix that, ram_size should have some value, > you should be able to remove the NIC. > > Regards, > Somesh > > -----Original Message----- > From: Sonali Jadhav [mailto:[email protected]] > Sent: Thursday, March 05, 2015 12:20 AM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > Hi Somesh, > > Here is output, > > mysql> select ram_size from service_offering where id=14; > +----------+ > | ram_size | > +----------+ > | NULL | > +----------+ > 1 row in set (0.00 sec) > > Thanks, > /Sonali > > -----Original Message----- > From: Somesh Naidu [mailto:[email protected]] > Sent: Thursday, March 5, 2015 2:19 AM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > Yes, I did. Looks strange, an NPE when calculating minimum memory required > for this VM. > > Could you send me the output of the following SQL: > select ram_size from service_offering where id=14; > > Somesh > CloudPlatform Escalations > Citrix Systems, Inc. > > -----Original Message----- > From: Sonali Jadhav [mailto:[email protected]] > Sent: Tuesday, March 03, 2015 6:39 AM > To: [email protected] > Subject: RE: unable to remove NIC from instance > > Hi Somesh, > > Have you looked at the logs and query results I sent, what do you think? > > /Sonali > > -----Original Message----- > From: Daan Hoogland [mailto:[email protected]] > Sent: Thursday, February 26, 2015 1:42 PM > To: [email protected] > Subject: Re: unable to remove NIC from instance > > As far as I can tell it tries to delete a secondary ip from a nic instead of > a second (or first) nic. This is going to cost me some diving into. I hope > Somesh has a simple answer for you. The initial error we talked about is in > your log between lines 97 and 133. I will have a look when I find the time > but I can't promise. > > If you are familiar with git you could look at the history of the 4.4 branch > since the release to see if a similar problem has been fixed. > > > On Thu, Feb 26, 2015 at 8:35 AM, Sonali Jadhav <[email protected]> > wrote: >> Hi Somesh n Daan, >> >> Thanks for reply. >> >> Here are logs from point I logged to CloudStack till the error >> occurred http://pastebin.com/j1rfEq5s >> >> And here is sql query output, >> >> mysql> select * from vm_instance where instance_name="i-9-24-VM" \G >> *************************** 1. row *************************** >> id: 24 >> name: linuxvm >> uuid: b5c535b4-10c4-44a9-b117-6df07537434c >> instance_name: i-9-24-VM >> state: Stopped >> vm_template_id: 202 >> guest_os_id: 201 >> private_mac_address: 02:00:3e:c4:00:01 >> private_ip_address: 10.1.1.14 >> pod_id: 1 >> data_center_id: 1 >> host_id: NULL >> last_host_id: 1 >> proxy_id: 1 >> proxy_assign_time: 2015-02-20 11:25:18 >> vnc_password: 7fJhsb8PVskfWRFgiQw02qDERH4PYxUS7xHy+2OU/HI= >> ha_enabled: 0 >> limit_cpu_use: 0 >> update_count: 25 >> update_time: 2015-02-23 06:26:46 >> created: 2015-02-20 11:24:26 >> removed: NULL >> type: User >> vm_type: User >> account_id: 9 >> domain_id: 5 >> service_offering_id: 14 >> reservation_id: c59c1ead-fd98-4822-b7bb-71124fbf0018 >> hypervisor_type: XenServer >> disk_offering_id: 6 >> owner: 9 >> host_name: linuxvm >> display_name: linuxvm >> desired_state: NULL >> dynamically_scalable: 0 >> display_vm: 1 >> power_state: NULL >> power_state_update_time: NULL >> power_state_update_count: 0 >> power_host: NULL >> 1 row in set (0.00 sec) >> >> mysql> >> >> mysql> select * from nics where instance_id=24 \G >> *************************** 1. row *************************** >> id: 60 >> uuid: 99378e9f-ba64-4c76-ac16-d9ac977d979f >> instance_id: 24 >> mac_address: 06:by:bg:00:01:33 >> ip4_address: 1.1.1.1 >> netmask: 255.255.0.0 >> gateway: 1.1.1.1 >> ip_type: Ip4 >> broadcast_uri: vlan://201 >> network_id: 207 >> mode: Dhcp >> state: Allocated >> strategy: Create >> reserver_name: DirectNetworkGuru >> reservation_id: NULL >> device_id: 1 >> update_time: 2015-02-23 07:25:40 >> isolation_uri: vlan://201 >> ip6_address: NULL >> default_nic: 0 >> vm_type: User >> created: 2015-02-20 11:24:26 >> removed: NULL >> ip6_gateway: NULL >> ip6_cidr: NULL >> secondary_ip: 0 >> display_nic: 1 >> *************************** 2. row *************************** >> id: 61 >> uuid: 42178ae9-1c4c-43be-8a05-5d6259ee9ee4 >> instance_id: 24 >> mac_address: 02:00:3f:a4:00:01 >> ip4_address: 10.1.1.14 >> netmask: 255.255.255.0 >> gateway: 10.1.1.1 >> ip_type: Ip4 >> broadcast_uri: NULL >> network_id: 211 >> mode: Dhcp >> state: Allocated >> strategy: Start >> reserver_name: ExternalGuestNetworkGuru >> reservation_id: c59c1ead-fd98-4822-b7bb-71124fbf0018 >> device_id: 0 >> update_time: 2015-02-23 07:25:41 >> isolation_uri: NULL >> ip6_address: NULL >> default_nic: 1 >> vm_type: User >> created: 2015-02-20 13:21:07 >> removed: NULL >> ip6_gateway: NULL >> ip6_cidr: NULL >> secondary_ip: 0 >> display_nic: 1 >> 2 rows in set (0.00 sec) >> >> mysql> >> >> Let me know if you find something from this. >> >> /Sonali >> >> -----Original Message----- >> From: Somesh Naidu [mailto:[email protected]] >> Sent: Wednesday, February 25, 2015 11:07 PM >> To: [email protected] >> Subject: RE: unable to remove NIC from instance >> >> Sonali, >> >> While this could be fixed in the DB, as Daan mentioned, it is not >> recommended. The primary table that needs to be update is the nics table but >> in addition, relevant tables like user_ip_address, etc need to be updated. >> There may also be impact on usage. Moreover, updating the DB won’t take care >> of removing the nic on the hypervisor/VM. >> >> A better way to deal with this is to resolve the original error that is >> preventing CCP from removing the nic. >> >> I have looked at the information (log snippet) you have shared. As Daan >> pointed out, these logs contain consequential errors where >> queryAsyncJobResult API is failing to get the result of a job (happens to be >> the RemoveNicFromVMCmdByAdmin API call that you are interested in) that >> relate to some trouble with the job framework. But this is not why the >> original remove nic command failed. >> >> What you can do is, >> 1. Note the time of the last line of log in mgmt. server (just do a tail on >> mgmt. log) 2. From CS UI, remove the nic from the VM 3. Send us mgmt. logs >> starting from time noted in step #1 4. In addition provide details of the VM, >> a. select * from vm_instance where id=<VM_ID>; >> b. select * from nics where instance_id=<VM_ID>; >> >> If I have this information I can try and identify why the original operation >> is failing. >> >> Somesh >> CloudPlatform Escalations >> Citrix Systems, Inc. >> >> -----Original Message----- >> From: Daan Hoogland [mailto:[email protected]] >> Sent: Wednesday, February 25, 2015 7:47 AM >> To: [email protected] >> Subject: Re: unable to remove NIC from instance >> >> I have no experience with this. It could be fixed in the db probably >> but that is always a bad advice >> 1 - coming from a developer >> 2 - for application in a production system >> >> If you are very familiar with the datamodel you might try to, I would only >> try to do it in a test environment (or at least first) Also I do not have >> the db change that would help you by head. >> >> >> On Wed, Feb 25, 2015 at 12:25 PM, Sonali Jadhav <[email protected]> >> wrote: >>> Hi Daan, >>> >>> Is there any manual way to fix it temporary ? I have production customer >>> who is facing this issue. >>> >>> /Sonali >>> >>> -----Original Message----- >>> From: Sonali Jadhav [mailto:[email protected]] >>> Sent: Tuesday, February 24, 2015 1:27 PM >>> To: [email protected] >>> Subject: RE: unable to remove NIC from instance >>> >>> I am running xenserver 6.2, I have 1 cluster with 4 Hosts. >>> >>> /Sonali >>> >>> -----Original Message----- >>> From: Daan Hoogland [mailto:[email protected]] >>> Sent: Tuesday, February 24, 2015 1:22 PM >>> To: [email protected] >>> Subject: Re: unable to remove NIC from instance >>> >>> Ok, thanks, we are running a custom 4.4.2 version we had to make due to >>> problems with the stock 4.4.2. I will look into making another release >>> candidate for 4.4.3 and have a look at the code to see if I can find a >>> cause to your problem. The last 4.4.3 release candidate I made was largely >>> ignored and therefore never released. It also has a problem with creating >>> new volumes on vmware which is not solved yet. >>> What type of hypervisors are you running? >>> >>> On Tue, Feb 24, 2015 at 8:45 AM, Sonali Jadhav <[email protected]> >>> wrote: >>>> Hi Daan, >>>> >>>> Super thanks for reply. I am running version 4.4.2 >>>> >>>> /Sonali >>>> >>>> -----Original Message----- >>>> From: Daan Hoogland [mailto:[email protected]] >>>> Sent: Tuesday, February 24, 2015 1:05 PM >>>> To: [email protected] >>>> Subject: Re: unable to remove NIC from instance >>>> >>>> It seems to me you are a victim of overzealous exception handling. >>>> I've seen that happen in the async job handling. The job dispatching or >>>> construction has an exception that is not handled but caught and ignored, >>>> it end and the async job is not removed from the queue. I'd like to look >>>> at the code near the stack trace but for that I need the exact version of >>>> Cloudstack you are running. >>>> >>>> On Tue, Feb 24, 2015 at 5:44 AM, Sonali Jadhav <[email protected]> >>>> wrote: >>>>> Hi Daan, >>>>> >>>>> I have checked logs form starting of api call, and I have not found >>>>> anything special, But apart from error, I found one warning, >>>>> >>>>> 2015-02-23 13:11:46,226 WARN [c.c.u.d.Merovingian2] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable >>>>> to find lock for the key vm_instance24 and thread id 137380473 >>>>> >>>>> Don’t know meaning of this. >>>>> >>>>> Here are my logs, >>>>> >>>>> 2015-02-23 13:11:45,457 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgentCronJob-296:ctx-11de89f6) Seq 5-8090716730571096067: Response >>>>> Received: >>>>> 2015-02-23 13:11:45,458 DEBUG [c.c.a.t.Request] >>>>> (DirectAgentCronJob-296:ctx-11de89f6) Seq 5-8090716730571096067: >>>>> Processing: { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: >>>>> 10, >>>>> [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1," >>>>> _ >>>>> vmMetaDatum":{"i-2-11-VM":"viridian:true;acpi:1;apic:true;pae:true;nx: >>>>> true","s-15-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true"," >>>>> r >>>>> - >>>>> 6 >>>>> -VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true","i-2-5-VM":" >>>>> v >>>>> i >>>>> r >>>>> idian:true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-13-VM":" >>>>> viridian:true;acpi:1;apic:true;pae:true;nx:true","i-5-12-VM":"virid >>>>> i >>>>> a >>>>> n >>>>> :true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-14-VM":"vi >>>>> r >>>>> i >>>>> d >>>>> ian:true;acpi:1;apic:true;pae:true;nx:true","v-1-VM":"viridian:true >>>>> ; >>>>> a >>>>> c >>>>> pi:1;apic:true;pae:true;nx:true","i-9-23-VM":"viridian:true;acpi:1; >>>>> a >>>>> p >>>>> i >>>>> c:true;pae:true;nx:true;timeoffset:-28800"},"_isExecuted":false,"re >>>>> s >>>>> u >>>>> l >>>>> t":true,"wait":0}}] } >>>>> 2015-02-23 13:11:45,572 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: Executing >>>>> request >>>>> 2015-02-23 13:11:45,628 DEBUG [c.c.h.x.r.CitrixResourceBase] >>>>> (DirectAgent-48:ctx-9e4c61a9) Vm cpu utilization >>>>> 0.11499999999999999 >>>>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: Response >>>>> Received: >>>>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] >>>>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: MgmtId >>>>> 249373720971888: Resp: Routing to peer >>>>> 2015-02-23 13:11:45,682 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: Executing >>>>> request >>>>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase] >>>>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.1152941176470588 >>>>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase] >>>>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.2599999999999999 >>>>> 2015-02-23 13:11:45,766 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: Response Received: >>>>> 2015-02-23 13:11:45,767 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] >>>>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: MgmtId >>>>> 249373720971888: Resp: Routing to peer >>>>> 2015-02-23 13:11:45,791 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: Executing >>>>> request >>>>> 2015-02-23 13:11:45,835 DEBUG [c.c.h.x.r.CitrixResourceBase] >>>>> (DirectAgent-192:ctx-61e81d25) Vm cpu utilization >>>>> 0.13343749999999996 >>>>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: Response >>>>> Received: >>>>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] >>>>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: MgmtId >>>>> 249373720971888: Resp: Routing to peer >>>>> 2015-02-23 13:11:45,921 DEBUG [c.c.a.ApiServlet] >>>>> (http-6443-exec-4:ctx-5a612122) ===START=== 192.168.45.32 -- GET >>>>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4i >>>>> E >>>>> l >>>>> w >>>>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6d >>>>> f >>>>> 0 >>>>> 7 >>>>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955 >>>>> 2015-02-23 13:11:46,095 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) submit async job-526, >>>>> details: AsyncJobVO {id:526, userId: 2, accountId: 2, instanceType: >>>>> None, instanceId: null, cmd: >>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi >>>>> n >>>>> , >>>>> cmdInfo: >>>>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d"," >>>>> c >>>>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977 >>>>> d >>>>> 9 >>>>> 7 >>>>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0 >>>>> 7 >>>>> 5 >>>>> 3 >>>>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c"," >>>>> c >>>>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d >>>>> 9 >>>>> 7 >>>>> 9 >>>>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccou >>>>> n t I d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status: >>>>> IN_PROGRESS, >>>>> processStatus: 0, resultCode: 0, result: null, initMsid: >>>>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled: >>>>> null, created: null} >>>>> 2015-02-23 13:11:46,096 INFO [o.a.c.f.j.i.AsyncJobMonitor] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Add job-526 into job >>>>> monitoring >>>>> 2015-02-23 13:11:46,096 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Executing AsyncJobVO >>>>> {id:526, userId: 2, accountId: 2, instanceType: None, instanceId: >>>>> null, cmd: >>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi >>>>> n >>>>> , >>>>> cmdInfo: >>>>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d"," >>>>> c >>>>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977 >>>>> d >>>>> 9 >>>>> 7 >>>>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0 >>>>> 7 >>>>> 5 >>>>> 3 >>>>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c"," >>>>> c >>>>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d >>>>> 9 >>>>> 7 >>>>> 9 >>>>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccou >>>>> n t I d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status: >>>>> IN_PROGRESS, >>>>> processStatus: 0, resultCode: 0, result: null, initMsid: >>>>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled: >>>>> null, created: null} >>>>> 2015-02-23 13:11:46,097 DEBUG [c.c.a.ApiServlet] >>>>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) ===END=== >>>>> 192.168.45.32 >>>>> -- GET >>>>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4i >>>>> E >>>>> l >>>>> w >>>>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6d >>>>> f >>>>> 0 >>>>> 7 >>>>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955 >>>>> 2015-02-23 13:11:46,219 DEBUG [c.c.c.ConsoleProxyManagerImpl] >>>>> (consoleproxy-1:ctx-a5fe3580) Zone 1 is ready to launch console >>>>> proxy >>>>> 2015-02-23 13:11:46,221 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Sync >>>>> job-527 execution on object VmWorkJobQueue.24 >>>>> 2015-02-23 13:11:46,226 WARN [c.c.u.d.Merovingian2] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable >>>>> to find lock for the key vm_instance24 and thread id 137380473 >>>>> 2015-02-23 13:11:46,534 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] >>>>> (secstorage-1:ctx-be45d820) Zone 1 is ready to launch secondary >>>>> storage VM >>>>> 2015-02-23 13:11:49,095 DEBUG [c.c.s.StatsCollector] >>>>> (StatsCollector-4:ctx-4a027738) HostStatsCollector is running... >>>>> 2015-02-23 13:11:49,115 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-93:ctx-3810dbca) Seq 1-9127107594819755362: Executing >>>>> request >>>>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-93:ctx-3810dbca) Seq 1-9127107594819755362: Response >>>>> Received: >>>>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.t.Request] >>>>> (StatsCollector-4:ctx-4a027738) Seq 1-9127107594819755362: Received: >>>>> { Ans: , MgmtId: 59778234354585, via: 1, Ver: v1, Flags: 10, { >>>>> GetHostStatsAnswer } } >>>>> 2015-02-23 13:11:49,182 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-4:ctx-5d8fe3d8) Seq 4-677791743919277703: Executing >>>>> request >>>>> 2015-02-23 13:11:49,226 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-4:ctx-5d8fe3d8) Seq 4-677791743919277703: Response Received: >>>>> 2015-02-23 13:11:49,227 DEBUG [c.c.a.t.Request] >>>>> (StatsCollector-4:ctx-4a027738) Seq 4-677791743919277703: Received: >>>>> { >>>>> Ans: , MgmtId: 59778234354585, via: 4, Ver: v1, Flags: 10, { >>>>> GetHostStatsAnswer } } >>>>> 2015-02-23 13:11:49,242 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-194:ctx-9a8ead36) Seq 5-8090716730571121376: Executing >>>>> request >>>>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-194:ctx-9a8ead36) Seq 5-8090716730571121376: Response >>>>> Received: >>>>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.t.Request] >>>>> (StatsCollector-4:ctx-4a027738) Seq 5-8090716730571121376: Received: >>>>> { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: 10, { >>>>> GetHostStatsAnswer } } >>>>> 2015-02-23 13:11:49,288 DEBUG [c.c.a.ApiServlet] >>>>> (http-6443-exec-3:ctx-c1a661f7) ===START=== 192.168.45.32 -- GET >>>>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadb >>>>> b >>>>> 8 >>>>> a >>>>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469 >>>>> 3 >>>>> 4 >>>>> 0 >>>>> 6327 >>>>> 2015-02-23 13:11:49,302 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-383:ctx-5f562eca) Seq 6-892275676172801479: Executing >>>>> request >>>>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.m.DirectAgentAttache] >>>>> (DirectAgent-383:ctx-5f562eca) Seq 6-892275676172801479: Response >>>>> Received: >>>>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.t.Request] >>>>> (StatsCollector-4:ctx-4a027738) Seq 6-892275676172801479: Received: >>>>> { >>>>> Ans: , MgmtId: 59778234354585, via: 6, Ver: v1, Flags: 10, { >>>>> GetHostStatsAnswer } } >>>>> 2015-02-23 13:11:49,626 DEBUG [c.c.a.ApiServlet] >>>>> (http-6443-exec-3:ctx-c1a661f7 ctx-62507b94) ===END=== >>>>> 192.168.45.32 >>>>> -- GET >>>>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadb >>>>> b >>>>> 8 >>>>> a >>>>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469 >>>>> 3 >>>>> 4 >>>>> 0 >>>>> 6327 >>>>> 2015-02-23 13:11:49,672 ERROR [c.c.a.ApiAsyncJobDispatcher] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Unexpected exception >>>>> while executing >>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi >>>>> n >>>>> java.lang.RuntimeException: Job failed due to exception null >>>>> at >>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) >>>>> at >>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) >>>>> at >>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >>>>> at >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >>>>> at >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >>>>> at >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >>>>> at >>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >>>>> at >>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) >>>>> at >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> 2015-02-23 13:11:49,673 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Complete async job-526, >>>>> jobStatus: FAILED, resultCode: 530, result: >>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": >>>>> [],"errorcode":530,"errortext":"Job failed due to exception null"} >>>>> 2015-02-23 13:11:49,730 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Done executing >>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmi >>>>> n >>>>> for job-526 >>>>> 2015-02-23 13:11:49,757 INFO [o.a.c.f.j.i.AsyncJobMonitor] >>>>> (API-Job-Executor-118:ctx-adab6031 job-526) Remove job-526 from job >>>>> monitoring >>>>> >>>>> >>>>> /Sonali >>>>> >>>>> -----Original Message----- >>>>> From: Daan Hoogland [mailto:[email protected]] >>>>> Sent: Monday, February 23, 2015 2:42 PM >>>>> To: [email protected] >>>>> Subject: Re: unable to remove NIC from instance >>>>> >>>>> Sonali, >>>>> >>>>> this looks like a consequential error. You need to look further back into >>>>> the log to find the root cause. The start job is the a query for results, >>>>> please find the start for the actual api command. >>>>> >>>>> On Mon, Feb 23, 2015 at 6:55 AM, Sonali Jadhav >>>>> <[email protected]> wrote: >>>>>> I am trying to remove "Shared guest network". These are new logs, >>>>>> >>>>>> 2015-02-23 06:53:59,737 DEBUG >>>>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >>>>>> (RouterStatusMonitor-1:ctx-28b1e134) Found 0 networks to update RvR >>>>>> status. >>>>>> 2015-02-23 06:54:01,726 DEBUG [c.c.a.ApiServlet] >>>>>> (http-6443-exec-3:ctx-6eb9497d) ===START=== 115.114.134.114 -- >>>>>> GET >>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b >>>>>> 9 >>>>>> 4 >>>>>> c >>>>>> f >>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246 >>>>>> 7 >>>>>> 0 >>>>>> 7 >>>>>> 3 >>>>>> 8796 >>>>>> 2015-02-23 06:54:01,818 DEBUG [c.c.a.ApiServlet] >>>>>> (http-6443-exec-3:ctx-6eb9497d ctx-007c94be) ===END=== >>>>>> 115.114.134.114 -- GET >>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b >>>>>> 9 >>>>>> 4 >>>>>> c >>>>>> f >>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246 >>>>>> 7 >>>>>> 0 >>>>>> 7 >>>>>> 3 >>>>>> 8796 >>>>>> 2015-02-23 06:54:01,993 ERROR [c.c.a.ApiAsyncJobDispatcher] >>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Unexpected exception >>>>>> while executing >>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdm >>>>>> i >>>>>> n >>>>>> java.lang.RuntimeException: Job failed due to exception null >>>>>> at >>>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) >>>>>> at >>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) >>>>>> at >>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >>>>>> at >>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >>>>>> at >>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >>>>>> at >>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >>>>>> at >>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >>>>>> at >>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> 2015-02-23 06:54:01,994 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Complete async >>>>>> job-499, >>>>>> jobStatus: FAILED, resultCode: 530, result: >>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": >>>>>> [],"errorcode":530,"errortext":"Job failed due to exception null"} >>>>>> 2015-02-23 06:54:02,051 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Done executing >>>>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdm >>>>>> i >>>>>> n >>>>>> for job-499 >>>>>> 2015-02-23 06:54:02,077 INFO [o.a.c.f.j.i.AsyncJobMonitor] >>>>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Remove job-499 from >>>>>> job monitoring >>>>>> 2015-02-23 06:54:03,942 DEBUG [c.c.s.StatsCollector] >>>>>> (StatsCollector-2:ctx-ce679066) AutoScaling Monitor is running... >>>>>> 2015-02-23 06:54:04,727 DEBUG [c.c.a.ApiServlet] >>>>>> (http-6443-exec-4:ctx-e51e68c3) ===START=== 115.114.134.114 -- >>>>>> GET >>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b >>>>>> 9 >>>>>> 4 >>>>>> c >>>>>> f >>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246 >>>>>> 7 >>>>>> 0 >>>>>> 7 >>>>>> 4 >>>>>> 1797 >>>>>> 2015-02-23 06:54:04,802 DEBUG [c.c.a.ApiServlet] >>>>>> (http-6443-exec-4:ctx-e51e68c3 ctx-a70a6fff) ===END=== >>>>>> 115.114.134.114 -- GET >>>>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b >>>>>> 9 >>>>>> 4 >>>>>> c >>>>>> f >>>>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246 >>>>>> 7 >>>>>> 0 >>>>>> 7 >>>>>> 4 >>>>>> 1797 >>>>>> >>>>>> >>>>>> /Sonali >>>>>> >>>>>> -----Original Message----- >>>>>> From: Sonali Jadhav [mailto:[email protected]] >>>>>> Sent: Monday, February 23, 2015 10:25 AM >>>>>> To: [email protected] >>>>>> Subject: RE: unable to remove NIC from instance >>>>>> >>>>>> Hi, >>>>>> >>>>>> Yes I have tried that, and issue is still same. Same logs. >>>>>> >>>>>> /Sonali >>>>>> >>>>>> -----Original Message----- >>>>>> From: Daan Hoogland [mailto:[email protected]] >>>>>> Sent: Sunday, February 22, 2015 10:57 PM >>>>>> To: [email protected] >>>>>> Subject: Re: unable to remove NIC from instance >>>>>> >>>>>> Sonil, Did you try what Gopalakrishnan suggested? >>>>>> >>>>>> Also can you expand on the version of cloudstack you are running? >>>>>> >>>>>> thanks, >>>>>> >>>>>> On Sun, Feb 22, 2015 at 12:24 PM, Sonali Jadhav >>>>>> <[email protected]> wrote: >>>>>>> Anyone help? >>>>>>> /Sonali >>>>>>> >>>>>>> >>>>>>> ---- Sonali Jadhav wrote ---- >>>>>>> >>>>>>> Anyone? >>>>>>> >>>>>>> Sent from my Sony Xperia™ smartphone >>>>>>> >>>>>>> >>>>>>> ---- Sonali Jadhav wrote ---- >>>>>>> >>>>>>> Here are logs, >>>>>>> >>>>>>> 2015-02-20 14:26:34,122 DEBUG [c.c.a.m.AgentManagerImpl] >>>>>>> (AgentManager-Handler-12:null) SeqA 2-260495: Sending Seq 2-260495: >>>>>>> { >>>>>>> Ans: , MgmtId: 59778234354585, via: 2, Ver: v1, Flags: 100010, >>>>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait": >>>>>>> 0 >>>>>>> } >>>>>>> } >>>>>>> ] >>>>>>> } >>>>>>> 2015-02-20 14:26:34,638 DEBUG [o.a.c.f.j.d.VmWorkJobDaoImpl] >>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Expunge completed work >>>>>>> job-483 >>>>>>> 2015-02-20 14:26:34,640 DEBUG [c.c.u.d.T.Transaction] >>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Rolling back the transaction: >>>>>>> Time = 2 Name = Vm-Operations-Cleanup-1; called by >>>>>>> -TransactionLegacy.rollback:900-TransactionLegacy.removeUpTo:843- >>>>>>> T >>>>>>> r >>>>>>> a >>>>>>> n >>>>>>> s >>>>>>> actionLegacy.close:667-TransactionContextInterceptor.invoke:36-Re >>>>>>> f >>>>>>> l >>>>>>> e >>>>>>> c >>>>>>> t >>>>>>> iveMethodInvocation.proceed:161-ExposeInvocationInterceptor.invoke: >>>>>>> 9 >>>>>>> 1 >>>>>>> - >>>>>>> ReflectiveMethodInvocation.proceed:172-JdkDynamicAopProxy.invoke: >>>>>>> 2 >>>>>>> 0 >>>>>>> 4 >>>>>>> - >>>>>>> $ >>>>>>> Proxy182.expungeCompletedWorkJobs:-1-VirtualMachineManagerImpl$Cl >>>>>>> e >>>>>>> a >>>>>>> n >>>>>>> u >>>>>>> p >>>>>>> Task.runInContext:2389-ManagedContextRunnable$1.run:49-DefaultMan >>>>>>> a >>>>>>> g >>>>>>> e >>>>>>> d >>>>>>> C >>>>>>> ontext$1.call:56 >>>>>>> 2015-02-20 14:26:34,658 ERROR [c.c.v.VirtualMachineManagerImpl] >>>>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) VM Operations failed due >>>>>>> to >>>>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on: >>>>>>> com.mysql.jdbc.PreparedStatement@e7a49c5: DELETE FROM async_job WHERE >>>>>>> async_job.id= 483 >>>>>>> at >>>>>>> com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1178) >>>>>>> at >>>>>>> org.apache.cloudstack.framework.jobs.dao.VmWorkJobDaoImpl.expungeCompletedWorkJobs(VmWorkJobDaoImpl.java:149) >>>>>>> at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source) >>>>>>> at >>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>>>> at >>>>>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) >>>>>>> at >>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) >>>>>>> at >>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) >>>>>>> at >>>>>>> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34) >>>>>>> at >>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) >>>>>>> at >>>>>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) >>>>>>> at >>>>>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) >>>>>>> at >>>>>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) >>>>>>> at com.sun.proxy.$Proxy182.expungeCompletedWorkJobs(Unknown >>>>>>> Source) >>>>>>> at >>>>>>> com.cloud.vm.VirtualMachineManagerImpl$CleanupTask.runInContext(VirtualMachineManagerImpl.java:2389) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>> Caused by: >>>>>>> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: >>>>>>> Cannot delete or update a parent row: a foreign key constraint fails >>>>>>> (`cloud`.`async_job_join_map`, CONSTRAINT >>>>>>> `fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`) >>>>>>> REFERENCES `async_job` (`id`)) >>>>>>> at >>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>>>>>> at >>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) >>>>>>> at >>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>>>>>> at >>>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:526) >>>>>>> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) >>>>>>> at com.mysql.jdbc.Util.getInstance(Util.java:386) >>>>>>> at >>>>>>> com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1040) >>>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) >>>>>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) >>>>>>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990) >>>>>>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) >>>>>>> at >>>>>>> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625) >>>>>>> at >>>>>>> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119) >>>>>>> at >>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415) >>>>>>> at >>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2333) >>>>>>> at >>>>>>> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2318) >>>>>>> at >>>>>>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) >>>>>>> at >>>>>>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) >>>>>>> at >>>>>>> com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1169) >>>>>>> ... 26 more >>>>>>> 2015-02-20 14:26:34,770 DEBUG [c.c.a.ApiServlet] >>>>>>> (http-6443-exec-4:ctx-46b96aa1) ===START=== 192.168.34.1 -- GET >>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0 >>>>>>> a >>>>>>> b >>>>>>> 3 >>>>>>> f >>>>>>> 1 >>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443 >>>>>>> 8 >>>>>>> 6 >>>>>>> 9 >>>>>>> 6 >>>>>>> 3 >>>>>>> 84 >>>>>>> 2015-02-20 14:26:34,797 DEBUG [c.c.u.AccountManagerImpl] >>>>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) Access to >>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to >>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker >>>>>>> 2015-02-20 14:26:34,858 DEBUG [c.c.a.ApiServlet] >>>>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) ===END=== >>>>>>> 192.168.34.1 >>>>>>> -- GET >>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0 >>>>>>> a >>>>>>> b >>>>>>> 3 >>>>>>> f >>>>>>> 1 >>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443 >>>>>>> 8 >>>>>>> 6 >>>>>>> 9 >>>>>>> 6 >>>>>>> 3 >>>>>>> 84 >>>>>>> 2015-02-20 14:26:35,024 ERROR [c.c.a.ApiAsyncJobDispatcher] >>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Unexpected exception >>>>>>> while executing >>>>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd >>>>>>> java.lang.RuntimeException: Job failed due to exception null >>>>>>> at >>>>>>> com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) >>>>>>> at >>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >>>>>>> at >>>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >>>>>>> at >>>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>> 2015-02-20 14:26:35,025 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Complete async >>>>>>> job-482, >>>>>>> jobStatus: FAILED, resultCode: 530, result: >>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": >>>>>>> [],"errorcode":530,"errortext":"Job failed due to exception >>>>>>> null"} >>>>>>> 2015-02-20 14:26:35,082 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Done executing >>>>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd for >>>>>>> job-482 >>>>>>> 2015-02-20 14:26:35,111 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Resetting hosts >>>>>>> suitable for reconnect >>>>>>> 2015-02-20 14:26:35,113 INFO [o.a.c.f.j.i.AsyncJobMonitor] >>>>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Remove job-482 from >>>>>>> job monitoring >>>>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed resetting >>>>>>> hosts suitable for reconnect >>>>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for >>>>>>> clusters already owned by this management server >>>>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring >>>>>>> hosts for clusters already owned by this management server >>>>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for >>>>>>> clusters not owned by any management server >>>>>>> 2015-02-20 14:26:35,117 DEBUG [c.c.h.d.HostDaoImpl] >>>>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring >>>>>>> hosts for clusters not owned by any management server >>>>>>> 2015-02-20 14:26:37,771 DEBUG [c.c.a.ApiServlet] >>>>>>> (http-6443-exec-2:ctx-1053809f) ===START=== 192.168.34.1 -- GET >>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0 >>>>>>> a >>>>>>> b >>>>>>> 3 >>>>>>> f >>>>>>> 1 >>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443 >>>>>>> 8 >>>>>>> 6 >>>>>>> 9 >>>>>>> 9 >>>>>>> 3 >>>>>>> 85 >>>>>>> 2015-02-20 14:26:37,798 DEBUG [c.c.u.AccountManagerImpl] >>>>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) Access to >>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to >>>>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker >>>>>>> 2015-02-20 14:26:37,857 DEBUG [c.c.a.ApiServlet] >>>>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) ===END=== >>>>>>> 192.168.34.1 >>>>>>> -- GET >>>>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0 >>>>>>> a >>>>>>> b >>>>>>> 3 >>>>>>> f >>>>>>> 1 >>>>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443 >>>>>>> 8 >>>>>>> 6 >>>>>>> 9 >>>>>>> 9 >>>>>>> 3 >>>>>>> 85 >>>>>>> 2015-02-20 14:26:37,975 DEBUG [c.c.a.m.AgentManagerImpl] >>>>>>> (AgentManager-Handler-2:null) Ping from 2 >>>>>>> 2015-02-20 14:26:38,048 DEBUG [c.c.a.m.AgentManagerImpl] >>>>>>> (AgentManager-Handler-7:null) Ping from 9 >>>>>>> 2015-02-20 14:26:40,509 WARN [c.c.a.m.DirectAgentAttache] >>>>>>> (DirectAgentCronJob-261:ctx-f3e99173) Unable to get current >>>>>>> status on >>>>>>> 4(SeSolXS02) >>>>>>> 2015-02-20 14:26:40,687 WARN [c.c.a.m.DirectAgentAttache] >>>>>>> (DirectAgentCronJob-130:ctx-8b85b559) Unable to get current >>>>>>> status on >>>>>>> 1(SeSolXS01) >>>>>>> 2015-02-20 14:26:40,747 WARN [c.c.a.m.DirectAgentAttache] >>>>>>> (DirectAgentCronJob-131:ctx-e6c7b284) Unable to get current >>>>>>> status on >>>>>>> 5(SeSolXS03) >>>>>>> 2015-02-20 14:26:40,981 WARN [c.c.a.m.DirectAgentAttache] >>>>>>> (DirectAgentCronJob-33:ctx-c6748244) Unable to get current status >>>>>>> on >>>>>>> 6(SeSolXS04) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> /Sonali >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Gopalakrishnan S [mailto:[email protected]] >>>>>>> Sent: Friday, February 20, 2015 7:12 PM >>>>>>> To: [email protected] >>>>>>> Subject: Re: unable to remove NIC from instance >>>>>>> >>>>>>> Hi Sonali, >>>>>>> >>>>>>> It may be still routing that shared guest network. Try to restart your >>>>>>> virutal router and virtual machine or shutdown your VM before removing >>>>>>> that NIC. let me know the logs when you remove that NIC. >>>>>>> >>>>>>> Thank You. >>>>>>> Gopalakrishnan.S >>>>>>> Cloud Engineer - Fogpanel >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>> From: Sonali Jadhav >>>>>>> To: [email protected] >>>>>>> Sent: Friday, February 20, 2015 7:06 PM >>>>>>> Subject: unable to remove NIC from instance >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I have one linux instance to which I had added "Shared guest network" >>>>>>> NIC. Then I added "isolated guest network" nic to it. Made "isolated >>>>>>> guest network" NIC as default and tried to remove "Shared guest >>>>>>> network" NIC, but its failing with null exception. I have attached logs. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> /sonali >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Daan >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Daan >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Daan >>>> >>>> >>> >>> >>> >>> -- >>> Daan >>> >>> >>> >>> >> >> >> >> -- >> Daan >> >> > > > > -- > Daan > > > > > > > > > > -- Daan
