Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-24 Thread Chris L 
On May 23, 2018, at 10:57 AM, Chuck Mariotti  wrote:
> 
> We've run into a data overage situation at a datacenter... We get charged a 
> premium per GB over 500GB (yes I know, stupid). Their reporting system seems 
> to indicate significantly less data usages vs pfSense's RRD reporting... 
> their billing system seems to be indicating overage similar to their 
> reporting... Uploads seem to be growing significantly. Any idea why the 
> pfSense box seems to be counting differently than the datacenter's metrics? 
> We need to track down where this usage is happened, but I know users have 
> only grown ~5% over that same period of time.
> 
> Here are stats for each month:
> 
> JanuaryFebruary  
> March   April 
>May (to 23rd)
> Datacenter (Upload/Download):   618.95GB/76.01GB  
> 365.25/47.15GB799.92/79.81GB801.67/105.01GB   
>581.57/76.26GB
> pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
> 1388.65/149.60GiB   1697.71/152.24GiB
> 1706.53/200.86GiB   1177.95/139.55GiB
> 
> 
> Any suggestions how or why there is a mismatch?
> 

What version of pfSense? I recall there was an issue with counting double. With 
the exception of your Feb numbers those are all very close to double.


___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Chuck Mariotti 
This is certainly possible, but the RRD GUI has a choice to display stats for 
WAN (Default) and LAN... selectin LAN essentially swaps the In/Out columns +/- 
a few gigs... 

We are running ntopng but it only has data for the last 12 days... the one 
webserver that is likely causing a lot of usage is reporting ~300GB used via 
ntopng... I assume that's total in/out.

-Original Message-
From: List <list-boun...@lists.pfsense.org> On Behalf Of Melvin Backus
Sent: May 23, 2018 7:47 PM
To: 'pfSense Support and Discussion Mailing List' <list@lists.pfsense.org>
Subject: Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

Is it possible these numbers are for both interfaces on the pfSense box? If so, 
do they include both inbound and outbound traffic for both? That would 
effectively double the true data transfer if traffic isn't being routed between 
other subnets / interfaces on the firewall.  I don't have RRD loaded so this is 
strictly speculation on a possible cause.

-Original Message-
From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck Mariotti
Sent: Wednesday, May 23, 2018 1:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

We've run into a data overage situation at a datacenter... We get charged a 
premium per GB over 500GB (yes I know, stupid). Their reporting system seems to 
indicate significantly less data usages vs pfSense's RRD reporting...
their billing system seems to be indicating overage similar to their 
reporting... Uploads seem to be growing significantly. Any idea why the pfSense 
box seems to be counting differently than the datacenter's metrics?
We need to track down where this usage is happened, but I know users have only 
grown ~5% over that same period of time.

Here are stats for each month:

JanuaryFebruary
March   April
May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB
365.25/47.15GB799.92/79.81GB801.67/105.01GB
581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB
1706.53/200.86GiB   1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Melvin Backus 
Is it possible these numbers are for both interfaces on the pfSense box? If
so, do they include both inbound and outbound traffic for both? That would
effectively double the true data transfer if traffic isn't being routed
between other subnets / interfaces on the firewall.  I don't have RRD loaded
so this is strictly speculation on a possible cause.

-Original Message-
From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck
Mariotti
Sent: Wednesday, May 23, 2018 1:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center
Provider...

We've run into a data overage situation at a datacenter... We get charged a
premium per GB over 500GB (yes I know, stupid). Their reporting system seems
to indicate significantly less data usages vs pfSense's RRD reporting...
their billing system seems to be indicating overage similar to their
reporting... Uploads seem to be growing significantly. Any idea why the
pfSense box seems to be counting differently than the datacenter's metrics?
We need to track down where this usage is happened, but I know users have
only grown ~5% over that same period of time.

Here are stats for each month:

JanuaryFebruary
March   April
May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB
365.25/47.15GB799.92/79.81GB801.67/105.01GB
581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB
1706.53/200.86GiB   1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Steve Yates 
I don't have a straight answer for you, but are you sure the DC is counting all 
traffic and not just HTTP/SMTP/etc?  I would think they are, but...

Something that may help...the firewall/rules page tracks data usage in the 
States column.  I'm assuming from when it was last booted.  Perhaps make an 
allow rule for each server and/or service and see what is tracked?

--

Steve Yates
ITS, Inc.

-Original Message-
From: List  On Behalf Of Chuck Mariotti
Sent: Wednesday, May 23, 2018 12:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

We've run into a data overage situation at a datacenter... We get charged a 
premium per GB over 500GB (yes I know, stupid). Their reporting system seems to 
indicate significantly less data usages vs pfSense's RRD reporting... their 
billing system seems to be indicating overage similar to their reporting... 
Uploads seem to be growing significantly. Any idea why the pfSense box seems to 
be counting differently than the datacenter's metrics? We need to track down 
where this usage is happened, but I know users have only grown ~5% over that 
same period of time.

Here are stats for each month:

JanuaryFebruary  
March   April   
 May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB  
365.25/47.15GB799.92/79.81GB801.67/105.01GB 
 581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB1706.53/200.86GiB  
 1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold