if you look at the hypervisors, do you see traffic data for the nics there?
On Fri, Jun 25, 2021 at 4:32 PM Hean Seng <heans...@gmail.com> wrote: > Yes, thats right, I am Advance zone with Security Group, and Shared > Network with Public IP range, However, I had change value of > vm.network.stats.interval. > so that calculate the Lan traffic of the Network Interface . So it should > work for calculating all traffic incur in the network interface . But > seems not, about 50% of it not calculating. > > > > On Fri, Jun 25, 2021 at 9:51 PM Daan Hoogland <daan.hoogl...@gmail.com> > wrote: > > > The only thing I can think of is that the VM is not on a network that > > provides routing. for instance in a basic zone or on a shared network > with > > a public ip range. > > > > On Fri, Jun 25, 2021 at 12:23 PM Hean Seng <heans...@gmail.com> wrote: > > > > > HI Daan > > > > > > > > > My usage record have this , this is one of the example that do not > have. > > > record of type 4 and 5, but have type 1, > > > > > > select * from usage_event where account_id=3391; > > > > > > > > > > > > -------+-------------------------+------------+---------------------+---------+-------------+--------------------+-------------+-------------+-------------+----------------+-----------+--------------+ > > > > > > | id | type | account_id | created | > > > zone_id | resource_id | resource_name | offering_id | template_id > | > > > size | resource_type | processed | virtual_size | > > > > > > > > > > > > +-------+-------------------------+------------+---------------------+---------+-------------+--------------------+-------------+-------------+-------------+----------------+-----------+--------------+ > > > > > > | 75910 | NET.IPASSIGN | 3391 | 2021-05-06 10:16:55 | > > > 1 | 2396 | 11.11.1.11 | NULL | 0 | > > > 0 > > > | DirectAttached | 1 | NULL | > > > > > > | 75911 | VOLUME.CREATE | 3391 | 2021-05-06 10:16:55 | > > > 1 | 6472 | ROOT-3649 | NULL | 210 | > > > 21474836480 | NULL | 1 | NULL | > > > > > > | 75912 | VM.CREATE | 3391 | 2021-05-06 10:16:55 | > > > 1 | 3649 | VM-38629-vasuflor | 51 | 210 | > > > NULL > > > | KVM | 1 | NULL | > > > > > > | 75913 | NETWORK.OFFERING.ASSIGN | 3391 | 2021-05-06 10:17:06 | > > > 1 | 3649 | 3753 | 6 | NULL | > > > 1 | NULL | 1 | NULL | > > > > > > | 75914 | SG.ASSIGN | 3391 | 2021-05-06 10:17:06 | > > > 1 | 3649 | NULL | 3664 | NULL | > > > NULL > > > | NULL | 1 | NULL | > > > > > > | 75915 | VM.START | 3391 | 2021-05-06 10:17:06 | > > > 1 | 3649 | VM-38629-vasuflor | 51 | 210 | > > > NULL > > > | KVM | 1 | NULL | > > > > > > > > > > > > +-------+-------------------------+------------+---------------------+---------+-------------+--------------------+-------------+-------------+-------------+----------------+-----------+--------------+ > > > > > > 6 row > > > > > > mysql> select * from usage_network where account_id=3391; > > > > > > Empty set (0.00 sec) > > > > > > > > > > > > mysql> select * from usage_network_offering where account_id=3391 > > > > > > -> ; > > > > > > > > > > > > +---------+------------+-----------+----------------+---------------------+------------+---------------------+---------+--------+ > > > > > > | zone_id | account_id | domain_id | vm_instance_id | > > network_offering_id | > > > is_default | created | deleted | nic_id | > > > > > > > > > > > > +---------+------------+-----------+----------------+---------------------+------------+---------------------+---------+--------+ > > > > > > | 1 | 3391 | 3383 | 3649 | > > 6 | > > > 1 | 2021-05-06 10:17:06 | NULL | 3753 | > > > > > > > > > > > > +---------+------------+-----------+----------------+---------------------+------------+---------------------+---------+--------+ > > > > > > 1 row in set (0.01 sec) > > > > > > > > > > > > > > > > > > On Fri, Jun 25, 2021 at 3:40 PM Daan Hoogland <daan.hoogl...@gmail.com > > > > > wrote: > > > > > > > Hean, > > > > To be honest, no i don't. You can start looking at any log records > > > related > > > > to the VMs and than track to their nics. Also look at the > > > > usage_event-table, to see if any records are there. First in the > cloud > > > DB, > > > > and if these are there in the cloud_usage DB. > > > > Sorry to not have an obviously useful clue for you. > > > > > > > > On Thu, Jun 24, 2021 at 7:49 PM Hean Seng <heans...@gmail.com> > wrote: > > > > > > > > > Hi Daan, > > > > > > > > > > Thanks for reply. I deleted the null record, and manage to make it > > > work > > > > > again. The it will auto generate back the record with correct ID . > > > > > > > > > > > > > > > However, I am facing the issue is the Usage Type 4, and 5 > (Network > > > sent > > > > > and received) are not populated . Other Type like Type1 is always > > > > working. > > > > > > > > > > Fyi, I am using share network, and set the parameter > > > > > vm.network.stats.interval=300 > > > > > so that the VM statistic is updated . > > > > > > > > > > This issue not happen on all VM, but part of it. ( about 50%) . I > > > > > tried look at the DB, and seem cannot find the clue what actually > > diff > > > , > > > > in > > > > > term of those updated and those is not. > > > > > > > > > > The record is appearing in usage_network_offering and seems good. > Do > > > you > > > > > have any idea of this ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 21, 2021 at 3:19 PM Daan Hoogland < > > daan.hoogl...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > Sorry Hean, wasn't on line most of the weekend, Is it running > now? > > > > > > as you wrote this two days ago, it should already have run twice > > > > > (assuming > > > > > > you didn't change a lot) > > > > > > > > > > > > On Sat, Jun 19, 2021 at 8:41 PM Hean Seng <heans...@gmail.com> > > > wrote: > > > > > > > > > > > > > And it will insert another success is NULL to usage_job table , > > > > right ? > > > > > > > > > > > > > > i deleted the null record and restart usage servoce and see > > > > following > > > > > > in > > > > > > > usage log : > > > > > > > > > > > > > > 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] > > > > > (main:null) > > > > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] > > > > > (main:null) > > > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] > > > > > (main:null) > > > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] > > > > > (main:null) > > > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > > > > > It seem execution time before current time ? Is that normal ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng <heans...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete all > > > > record > > > > > > > where > > > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > > > daan.hoogl...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Heanm, > > > > > > > >> > > > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng < > heans...@gmail.com > > > > > > > > wrote: > > > > > > > >> > > > > > > > >> > i have no unprocess job , but still have this issue: > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,657 INFO > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, > > > skipping... > > > > > > > >> > > > > > > > > >> this line ^^ means there is a unprocessed job owned by > another > > > > > > proccess > > > > > > > >> ... > > > > > > > >> > > > > > > > >> > > > > > > > >> > 2021-06-19 10:09:57,661 INFO > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > > > > heartbeat > > > > > > from > > > > > > > >> > usage_job where success=0 order by start_date desc limit > > 100; > > > > > > > >> > > > > > > > > >> you should search for `where success is null` > > > > > > > >> > > > > > > > >> > > > > > > > >> > Empty set (0.00 sec) > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng < > > > heans...@gmail.com> > > > > > > > wrote: > > > > > > > >> > > > > > > > > >> > > yes, I restarted the mgmt server, wait till can login > > web > > > > > > > interface > > > > > > > >> . > > > > > > > >> > > then only restart usage . > > > > > > > >> > > > > > > > > > >> > > I did follwoig also : > > > > > > > >> > > > > > > > > > >> > > * stop usage server > > > > > > > >> > > > > > > > > > >> > > * remove the record from usage_job for the unprocessed > job > > > > > > > >> > > > > > > > > > >> > > * start usage server again > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > Still same thing. Not sure where the Cloudstack keep > the > > > > Usage > > > > > > > server > > > > > > > >> > PID > > > > > > > >> > > id , and keep getting wrong id. > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > > > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > > > > > >> > > > > > > > > > >> > >> Hi Hean Seng, > > > > > > > >> > >> > > > > > > > >> > >> Have you waited the management service to come up and > > then > > > > > > > restarted > > > > > > > >> > >> usage service? It might be some race condition? > > > > > > > >> > >> > > > > > > > >> > >> Bobby. > > > > > > > >> > >> > > > > > > > >> > >> From: Hean Seng <heans...@gmail.com> > > > > > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > > > > > >> > >> To: users@cloudstack.apache.org < > > > users@cloudstack.apache.org > > > > > > > > > > > > >> > >> Subject: Cloudstack Usage --- not owner > > > > > > > >> > >> Hi > > > > > > > >> > >> > > > > > > > >> > >> I have mysql hang, and restarted mysql, restarted > > > cloudstack > > > > > > mgmt, > > > > > > > >> and > > > > > > > >> > >> usage. > > > > > > > >> > >> > > > > > > > >> > >> After that facing issue of following: > > > > > > > >> > >> > > > > > > > >> > >> duration is 120 minutes) > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,656 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,657 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,669 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, > > > > skipping... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,669 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> Seems usage pid is not updated to db . > > > > > > > >> > >> > > > > > > > >> > >> Andybody know how to fix this ? > > > > > > > >> > >> > > > > > > > >> > >> -- > > > > > > > >> > >> Regards, > > > > > > > >> > >> Hean Seng > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > > -- > > > > > > > >> > > Regards, > > > > > > > >> > > Hean Seng > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > -- > > > > > > > >> > Regards, > > > > > > > >> > Hean Seng > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> Daan > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Regards, > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Hean Seng > > > > > > > > > > > > > > > > > -- > > > > Daan > > > > > > > > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > > > > -- > > Daan > > > > > -- > Regards, > Hean Seng > -- Daan
