Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Chuck Mariotti
This is certainly possible, but the RRD GUI has a choice to display stats for 
WAN (Default) and LAN... selectin LAN essentially swaps the In/Out columns +/- 
a few gigs... 

We are running ntopng but it only has data for the last 12 days... the one 
webserver that is likely causing a lot of usage is reporting ~300GB used via 
ntopng... I assume that's total in/out.

-Original Message-
From: List  On Behalf Of Melvin Backus
Sent: May 23, 2018 7:47 PM
To: 'pfSense Support and Discussion Mailing List' 
Subject: Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

Is it possible these numbers are for both interfaces on the pfSense box? If so, 
do they include both inbound and outbound traffic for both? That would 
effectively double the true data transfer if traffic isn't being routed between 
other subnets / interfaces on the firewall.  I don't have RRD loaded so this is 
strictly speculation on a possible cause.

-Original Message-
From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck Mariotti
Sent: Wednesday, May 23, 2018 1:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

We've run into a data overage situation at a datacenter... We get charged a 
premium per GB over 500GB (yes I know, stupid). Their reporting system seems to 
indicate significantly less data usages vs pfSense's RRD reporting...
their billing system seems to be indicating overage similar to their 
reporting... Uploads seem to be growing significantly. Any idea why the pfSense 
box seems to be counting differently than the datacenter's metrics?
We need to track down where this usage is happened, but I know users have only 
grown ~5% over that same period of time.

Here are stats for each month:

JanuaryFebruary
March   April
May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB
365.25/47.15GB799.92/79.81GB801.67/105.01GB
581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB
1706.53/200.86GiB   1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


Re: [pfSense] Syntax error in rules.debug for lagg0 (WAN) after upgrade to 2.4.3_1

2018-05-23 Thread Steve Yates
I found Suricata won't start, and I'm guessing the error Suricata is 
logging when it terminates (leaving its .pid file behind), "23/5/2018 -- 
22:42:18 -  -- [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - alert-pf: Could 
not validate pf table: snort2c, module init failed." ...is related to this...?

--

Steve Yates
ITS, Inc.

-Original Message-
From: Steve Yates 
Sent: Wednesday, May 23, 2018 10:34 PM
To: 'pfSense Support and Discussion Mailing List' 
Subject: Syntax error in rules.debug for lagg0 (WAN) after upgrade to 2.4.3_1

After upgrading our HA routers from 2.4.2_1 to 2.4.3_1, every few minutes they 
are logging:

There were error(s) loading the rules: /tmp/rules.debug:242: syntax error - The 
line in question reads [242]: pass out  route-to ( lagg0 64.79.96.145 ) from  
to !/ tracker 105913 keep state allow-opts label "let out anything from 
firewall host itself"

64.79.96.145 is our WAN gateway.  We have the WAN configured to use a 
one-interface LAGG to allow sharing CARP states if we ever use a different 
router with a different interface name.

Searching /tmp/rules.debug for "lagg0" I see three lines at the top of the 
output:

pass out  route-to ( lagg0 64.79.96.145 ) from 64.79.96.149 to !64.79.96.144/29 
tracker 105911 keep state allow-opts label "let out anything from firewall 
host itself"
pass out  route-to ( lagg0 64.79.96.145 ) from 64.79.96.150 to !64.79.96.144/29 
tracker 105912 keep state allow-opts label "let out anything from firewall 
host itself"
pass out  route-to ( lagg0 64.79.96.145 ) from  to !/ tracker 105913 keep 
state allow-opts label "let out anything from firewall host itself"

.149 is the WAN IP, .150 the CARP shared IP.  Given the first two are there, 
I'm not sure what the third is supposed to be?

Re-applying the firewall rules does not clear it, though does appear to trigger 
it (presumably due to the rules reload).

Suggestions?

Steve Yates
ITS, Inc.

___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


[pfSense] Syntax error in rules.debug for lagg0 (WAN) after upgrade to 2.4.3_1

2018-05-23 Thread Steve Yates
After upgrading our HA routers from 2.4.2_1 to 2.4.3_1, every few minutes they 
are logging:

There were error(s) loading the rules: /tmp/rules.debug:242: syntax error - The 
line in question reads [242]: pass out  route-to ( lagg0 64.79.96.145 ) from  
to !/ tracker 105913 keep state allow-opts label "let out anything from 
firewall host itself"

64.79.96.145 is our WAN gateway.  We have the WAN configured to use a 
one-interface LAGG to allow sharing CARP states if we ever use a different 
router with a different interface name.

Searching /tmp/rules.debug for "lagg0" I see three lines at the top of the 
output:

pass out  route-to ( lagg0 64.79.96.145 ) from 64.79.96.149 to !64.79.96.144/29 
tracker 105911 keep state allow-opts label "let out anything from firewall 
host itself"
pass out  route-to ( lagg0 64.79.96.145 ) from 64.79.96.150 to !64.79.96.144/29 
tracker 105912 keep state allow-opts label "let out anything from firewall 
host itself"
pass out  route-to ( lagg0 64.79.96.145 ) from  to !/ tracker 105913 keep 
state allow-opts label "let out anything from firewall host itself"

.149 is the WAN IP, .150 the CARP shared IP.  Given the first two are there, 
I'm not sure what the third is supposed to be?

Re-applying the firewall rules does not clear it, though does appear to trigger 
it (presumably due to the rules reload).

Suggestions?

Steve Yates
ITS, Inc.

___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


Re: [pfSense] Upgrades to 2.4.3.x failing after updating metadata

2018-05-23 Thread Steve Yates
FWIW I upgraded our SG-4860 pair and saw the same behavior, fails after 
the metadata update.  I waited 5 minutes and it did not restart and saw no 
indication in system log it was going to, or upgrading.

--

Steve Yates
ITS, Inc.

-Original Message-
From: Steve Yates 
Sent: Wednesday, May 16, 2018 9:14 AM
To: pfSense Support and Discussion Mailing List 
Subject: RE: [pfSense] Upgrades to 2.4.3.x failing after updating metadata

Huh, I should remember to look there first.  So used to this list. 

The "sort of scary" part is comments like "Same thing here.  The page reported 
the upgrade had failed.  We waited about two minutes and the page refreshed and 
we logged in.  The upgrade had worked after all."  Like it's running in the 
background despite the failure?  And I ran it a second time during this?  
That's what "KPA" posted last night: "The WebGUI upgrade still seems to suffer 
from the same problem as it did a while ago which is that it gets disconnected 
from the real upgrade run and reports a failure when the upgrade is actually 
running successfully in the background."

--

Steve Yates
ITS, Inc.

-Original Message-
From: List  On Behalf Of John Kline
Sent: Tuesday, May 15, 2018 10:29 PM
To: pfSense Support and Discussion Mailing List 
Subject: Re: [pfSense] Upgrades to 2.4.3.x failing after updating metadata

Many of us a e seeing this.
See:https://forum.pfsense.org/index.php?topic=147853.0 




On Tuesday, May 15, 2018, 7:53 PM, Steve Yates  wrote:

I upgraded two routers from 2.4.2 to 2.4.3 and today to 2.4.3_1.  One is an 
SG-3100 and one is a PC.  On both, both times, the upgrade almost immediately 
fails, but if I try again it works.  I click the pending-update icon on the 
dashboard to go to System Update and it detects the update.  I start and I get:

">>> Updating repositories metadata... done.
2.4.3_1 version of pfSense is available"

Then a red bar at the top of the page, "System update failed!"

If I click the already-highlighted System Update tab again, confirm the update, 
it then immediately installs.

Is anyone else seeing this?

--

Steve Yates
ITS, Inc.

___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold



___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Melvin Backus
Is it possible these numbers are for both interfaces on the pfSense box? If
so, do they include both inbound and outbound traffic for both? That would
effectively double the true data transfer if traffic isn't being routed
between other subnets / interfaces on the firewall.  I don't have RRD loaded
so this is strictly speculation on a possible cause.

-Original Message-
From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Chuck
Mariotti
Sent: Wednesday, May 23, 2018 1:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center
Provider...

We've run into a data overage situation at a datacenter... We get charged a
premium per GB over 500GB (yes I know, stupid). Their reporting system seems
to indicate significantly less data usages vs pfSense's RRD reporting...
their billing system seems to be indicating overage similar to their
reporting... Uploads seem to be growing significantly. Any idea why the
pfSense box seems to be counting differently than the datacenter's metrics?
We need to track down where this usage is happened, but I know users have
only grown ~5% over that same period of time.

Here are stats for each month:

JanuaryFebruary
March   April
May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB
365.25/47.15GB799.92/79.81GB801.67/105.01GB
581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB
1706.53/200.86GiB   1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


Re: [pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Steve Yates
I don't have a straight answer for you, but are you sure the DC is counting all 
traffic and not just HTTP/SMTP/etc?  I would think they are, but...

Something that may help...the firewall/rules page tracks data usage in the 
States column.  I'm assuming from when it was last booted.  Perhaps make an 
allow rule for each server and/or service and see what is tracked?

--

Steve Yates
ITS, Inc.

-Original Message-
From: List  On Behalf Of Chuck Mariotti
Sent: Wednesday, May 23, 2018 12:57 PM
To: list@lists.pfsense.org
Subject: [pfSense] Bandwidth Mismatch between pfSense and Data Center 
Provider...

We've run into a data overage situation at a datacenter... We get charged a 
premium per GB over 500GB (yes I know, stupid). Their reporting system seems to 
indicate significantly less data usages vs pfSense's RRD reporting... their 
billing system seems to be indicating overage similar to their reporting... 
Uploads seem to be growing significantly. Any idea why the pfSense box seems to 
be counting differently than the datacenter's metrics? We need to track down 
where this usage is happened, but I know users have only grown ~5% over that 
same period of time.

Here are stats for each month:

JanuaryFebruary  
March   April   
 May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB  
365.25/47.15GB799.92/79.81GB801.67/105.01GB 
 581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB1706.53/200.86GiB  
 1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


[pfSense] memstick-2.4.3-RELEASE-amd64.img debugflags needed for ZFS

2018-05-23 Thread Jason Hellenthal
Sorry for the long subject but has anyone experienced in the ZFS install for a 
mirrored setup of two disks that you need to set kern.geom.debugflags=16 to 
allow shooting yourself in the foot just to get the kernel to stop denying you 
access to the disks ?


The UFS install works as intended.


image used in the install pfSense-CE-memstick-2.4.3-RELEASE-amd64.img located 
here:
https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-2.4.3-RELEASE-amd64.img.gz




-- 

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.





___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


[pfSense] Bandwidth Mismatch between pfSense and Data Center Provider...

2018-05-23 Thread Chuck Mariotti
We've run into a data overage situation at a datacenter... We get charged a 
premium per GB over 500GB (yes I know, stupid). Their reporting system seems to 
indicate significantly less data usages vs pfSense's RRD reporting... their 
billing system seems to be indicating overage similar to their reporting... 
Uploads seem to be growing significantly. Any idea why the pfSense box seems to 
be counting differently than the datacenter's metrics? We need to track down 
where this usage is happened, but I know users have only grown ~5% over that 
same period of time.

Here are stats for each month:

JanuaryFebruary  
March   April   
 May (to 23rd)
Datacenter (Upload/Download):   618.95GB/76.01GB  
365.25/47.15GB799.92/79.81GB801.67/105.01GB 
 581.57/76.26GB
pfSense RRD (Upload/Download):1372.41GiB/148.91GiB
1388.65/149.60GiB   1697.71/152.24GiB1706.53/200.86GiB  
 1177.95/139.55GiB


Any suggestions how or why there is a mismatch?

Regards,

Chuck
___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold


Re: [pfSense] How could I block messages trying to pass as from my net?

2018-05-23 Thread Alberto José García Fumero
El mar, 22-05-2018 a las 20:54 -0400, John Johnstone escribió:
> On 5/18/2018 10:42 AM, Alberto José García Fumero wrote:
> 
> > Im trying to block spam (for instance, from 185.234.217.232).
> > As far as I know, it's trying to pass as a message from my very
> > net:
> > 
> > Transcript of session follows.
> > De: Mail Delivery System  > .co.
> > cu>
> > Para:   Postmaster 
> > Asunto: Postfix SMTP server: errors from
> > unknown[185.234.217.232]
> > Fecha:  Fri, 18 May 2018 10:10:39 -0400 (CDT)
> >   Out: 220 partagas.ettpartagas.co.cu ESMTP Partagas
> >   In:  EHLO 190.6.79.98
> >   Out: 250-partagas.ettpartagas.co.cu
> >   Out: 250-PIPELINING
> >   Out: 250-SIZE 1524
> >   Out: 250-ETRN
> >   Out: 250-STARTTLS
> >   Out: 250-ENHANCEDSTATUSCODES
> >   Out: 250-8BITMIME
> >   Out: 250 DSN
> >   In:  AUTH LOGIN
> >   Out: 503 5.5.1 Error: authentication not enabled
> > 
> > Session aborted, reason: lost connection
> > 
> > For other details, see the local mail logfile
> > 
> > but the MTA correctly rejects it as a fake.
> 
> It might not be good to describe what happened here as your MTA
> rejected 
> the connection as a fake.  If your MTA is configured to reject a 
> connection because the EHLO contains your IP address (which is 
> unlikely), that isn't what happened here.
> 
> Your MTA returned a 503 error to the sending server because your MTA
> is 
> not configured to accept an AUTH login.  250-AUTH is not part of its 
> response to the EHLO.  Most mail servers accept AUTH only on port 465
> or 
> port 587.

Right.

> 
> > I have created an alias list (rechaza) in the menu
> > Firewall/Aliases,
> > where I put all the addresses known to be spammers, and tried to
> > reject
> > them with the rule in Firewall/Rules/WAN
> > 
> > Action: Block
> > Interface: WAN
> > TCP/IP version: IPV4
> > Protocol: TCP
> > Source: (single hots or alias) rechaza
> > Destination: 190.6.79.98
> > Destination port range: any
> > 
> > but I can not stop the spam right in the WAN interface.
> 
> If you take a look at Status > System Logs > Firewall and notice
> what 
> you see for Source and Destination this can help you understand
> better 
> how filtering and NAT works.  For your WAN interface, Source will be
> the 
> public IP of the origin of the packet.  If there is no port
> forwarding 
> configured for the destination port, no NAT occurs so the
> destination 
> address will be your public IP.  If port forwarding is configured
> for 
> the destination port, then NAT does apply and the destination
> address 
> will be your LAN IP.  It helps to keep this in mind when developing
> rules.

That did the trick. 

My first idea was to take a better look at the order of the
instructions, that is, to see if the NATing occurred **before** the
blocking. But it seems I should consider then as working in different
"contextes".


Thanks a lot to all!
-- 
M.Sc. Alberto García Fumero
Usuario Linux 97 138, registrado 10/12/1998
http://interese.cubava.cu
No son las horas que pones en tu trabajo lo que cuenta, sino el trabajo
que pones en esas horas.




___
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold