Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...
On May 23, 2018, at 10:57 AM, Chuck Mariotti wrote: > > We've run into a data overage situation at a datacenter... We get charged a > premium per GB over 500GB (yes I know, stupid). Their reporting system seems > to indicate significantly less data usages vs pfSense's RRD reporting... > their billing system seems to be indicating overage similar to their > reporting... Uploads seem to be growing significantly. Any idea why the > pfSense box seems to be counting differently than the datacenter's metrics? > We need to track down where this usage is happened, but I know users have > only grown ~5% over that same period of time. > > Here are stats for each month: > > JanuaryFebruary > March April >May (to 23rd) > Datacenter (Upload/Download): 618.95GB/76.01GB > 365.25/47.15GB799.92/79.81GB801.67/105.01GB >581.57/76.26GB > pfSense RRD (Upload/Download):1372.41GiB/148.91GiB > 1388.65/149.60GiB 1697.71/152.24GiB > 1706.53/200.86GiB 1177.95/139.55GiB > > > Any suggestions how or why there is a mismatch? > What version of pfSense? I recall there was an issue with counting double. With the exception of your Feb numbers those are all very close to double. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...
This is certainly possible, but the RRD GUI has a choice to display stats for WAN (Default) and LAN... selectin LAN essentially swaps the In/Out columns +/- a few gigs... We are running ntopng but it only has data for the last 12 days... the one webserver that is likely causing a lot of usage is reporting ~300GB used via ntopng... I assume that's total in/out. -Original Message- From: List On Behalf Of Melvin Backus Sent: May 23, 2018 7:47 PM To: 'pfSense Support and Discussion Mailing List' Subject: Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider... Is it possible these numbers are for both interfaces on the pfSense box? If so, do they include both inbound and outbound traffic for both? That would effectively double the true data transfer if traffic isn't being routed between other subnets / interfaces on the firewall. I don't have RRD loaded so this is strictly speculation on a possible cause. -Original Message- From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck Mariotti Sent: Wednesday, May 23, 2018 1:57 PM To: list@lists.pfsense.org Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider... We've run into a data overage situation at a datacenter... We get charged a premium per GB over 500GB (yes I know, stupid). Their reporting system seems to indicate significantly less data usages vs pfSense's RRD reporting... their billing system seems to be indicating overage similar to their reporting... Uploads seem to be growing significantly. Any idea why the pfSense box seems to be counting differently than the datacenter's metrics? We need to track down where this usage is happened, but I know users have only grown ~5% over that same period of time. Here are stats for each month: JanuaryFebruary March April May (to 23rd) Datacenter (Upload/Download): 618.95GB/76.01GB 365.25/47.15GB799.92/79.81GB801.67/105.01GB 581.57/76.26GB pfSense RRD (Upload/Download):1372.41GiB/148.91GiB 1388.65/149.60GiB 1697.71/152.24GiB 1706.53/200.86GiB 1177.95/139.55GiB Any suggestions how or why there is a mismatch? Regards, Chuck ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...
Is it possible these numbers are for both interfaces on the pfSense box? If so, do they include both inbound and outbound traffic for both? That would effectively double the true data transfer if traffic isn't being routed between other subnets / interfaces on the firewall. I don't have RRD loaded so this is strictly speculation on a possible cause. -Original Message- From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck Mariotti Sent: Wednesday, May 23, 2018 1:57 PM To: list@lists.pfsense.org Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider... We've run into a data overage situation at a datacenter... We get charged a premium per GB over 500GB (yes I know, stupid). Their reporting system seems to indicate significantly less data usages vs pfSense's RRD reporting... their billing system seems to be indicating overage similar to their reporting... Uploads seem to be growing significantly. Any idea why the pfSense box seems to be counting differently than the datacenter's metrics? We need to track down where this usage is happened, but I know users have only grown ~5% over that same period of time. Here are stats for each month: JanuaryFebruary March April May (to 23rd) Datacenter (Upload/Download): 618.95GB/76.01GB 365.25/47.15GB799.92/79.81GB801.67/105.01GB 581.57/76.26GB pfSense RRD (Upload/Download):1372.41GiB/148.91GiB 1388.65/149.60GiB 1697.71/152.24GiB 1706.53/200.86GiB 1177.95/139.55GiB Any suggestions how or why there is a mismatch? Regards, Chuck ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...
I don't have a straight answer for you, but are you sure the DC is counting all traffic and not just HTTP/SMTP/etc? I would think they are, but... Something that may help...the firewall/rules page tracks data usage in the States column. I'm assuming from when it was last booted. Perhaps make an allow rule for each server and/or service and see what is tracked? -- Steve Yates ITS, Inc. -Original Message- From: List On Behalf Of Chuck Mariotti Sent: Wednesday, May 23, 2018 12:57 PM To: list@lists.pfsense.org Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider... We've run into a data overage situation at a datacenter... We get charged a premium per GB over 500GB (yes I know, stupid). Their reporting system seems to indicate significantly less data usages vs pfSense's RRD reporting... their billing system seems to be indicating overage similar to their reporting... Uploads seem to be growing significantly. Any idea why the pfSense box seems to be counting differently than the datacenter's metrics? We need to track down where this usage is happened, but I know users have only grown ~5% over that same period of time. Here are stats for each month: JanuaryFebruary March April May (to 23rd) Datacenter (Upload/Download): 618.95GB/76.01GB 365.25/47.15GB799.92/79.81GB801.67/105.01GB 581.57/76.26GB pfSense RRD (Upload/Download):1372.41GiB/148.91GiB 1388.65/149.60GiB 1697.71/152.24GiB1706.53/200.86GiB 1177.95/139.55GiB Any suggestions how or why there is a mismatch? Regards, Chuck ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
[pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...
We've run into a data overage situation at a datacenter... We get charged a premium per GB over 500GB (yes I know, stupid). Their reporting system seems to indicate significantly less data usages vs pfSense's RRD reporting... their billing system seems to be indicating overage similar to their reporting... Uploads seem to be growing significantly. Any idea why the pfSense box seems to be counting differently than the datacenter's metrics? We need to track down where this usage is happened, but I know users have only grown ~5% over that same period of time. Here are stats for each month: JanuaryFebruary March April May (to 23rd) Datacenter (Upload/Download): 618.95GB/76.01GB 365.25/47.15GB799.92/79.81GB801.67/105.01GB 581.57/76.26GB pfSense RRD (Upload/Download):1372.41GiB/148.91GiB 1388.65/149.60GiB 1697.71/152.24GiB1706.53/200.86GiB 1177.95/139.55GiB Any suggestions how or why there is a mismatch? Regards, Chuck ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold