Re: disk i/o test
Den sön 6 mars 2022 kl 16:41 skrev Mihai Popescu : > > Since this thread is moving slowly in another direction, let me True > reiterate my situation again: I am running a browser (mostly chromium) > and the computer slows down on downloads. Since I've checked the > downloads rates, I observed they are slow than my maximum 500Mbps for > the line. > I can reach 320Mbps maximum, but mostly it stays at 280Mbps and the > Chromium has 30 seconds delays in everything i do. I would make sure it is not some kind of DNS thing, 30 second delays sounds A LOT like trying a "dead" resolver 3 times with 10 secs in between, before moving to a "working" one. -- May the most significant bit of your life be positive.
Re: PF pass not working (on complex "firewall")
Dear @misc
We found the error!
This is not PF problem.
I found this:
http://undeadly.org/cgi?action=article=20090127205841
If i modify an ipsec config *from:*
ike active esp from 172.20.123.0/24 to 172.20.122.0/24 \
*to:*
ike active esp from 172.20.123.0/24 *(192.168.123.0/24)* to
172.20.122.0/24 \
PF rules working correctly.
--
Regards
Gábor Szél
email:gabor.s...@wantax.hu
2022. 03. 05. 23:08 keltezéssel, Szél Gábor írta:
Dear @misc
We have an stupid problem.
On a complex firewall (currently PF rules 1200 row), one PASS rule not
working.
I do not know why.
There are many VLANs, WAN, LAN interfaces, many ipsec VPNs, CARP
(master-backup), pfsync, etc ...
PF main rules:
# set
#.
set block-policy drop
set loginterface $ext_wan1_if
set skip on { lo $pfsync_if }
set reassemble no
set timeout { tcp.established 600, tcp.closing 60 }
set optimization aggressive
set ruleset-optimization none
set limit { states 10, src-nodes 10, tables 10,
table-entries 10 }
# scrub
# -
match on $ext_wan1_if all scrub ( no-df max-mss 1440 random-id )
#. antispof
#.
antispoof quick for { $ext_wan1_if } inet
# anchors
# -
anchor "ftp-proxy/*"
# Block(s)
#.
block quick proto udp to port { 1985 8116 } # neighbours
HSRP & ...
block quick log on $ext_wan1_if from { }
label IPBlackList
block log inet6 all
block log all
So all interface traffic are basically forbidden (block).
Each traffic is allowed separately
We have one ipsec VPN, where there are NAT on both sides. (on both
sides have 192.168.x.x subnets, there is a subnet collision)
we want to solve a simple thing:
* comes in the packet on VPN tunnel to "virtual" IP address -
172.20.123.54 (bind to oBSD vlan interface)
* from this address PF redirect packet to destination server -
192.168.123.54
* destination server make return package, and send back
* the response packet comes in oBSD VLAN interface (vlan141)
* PF NAT-ed this packate to 172.20.123.54
* NAT-ed package return to source address in VPN
rules:
match in log on enc0 proto tcp from 172.20.122.0/24 to
172.20.123.54 port 5240 rdr-to 192.168.123.54 port 5240
pass in log on enc0 proto tcp from 172.20.122.0/24 to
192.168.123.54
pass out log on vlan141 from 172.20.122.0/24 to 192.168.123.54
match in log on vlan141 from 192.168.123.54 to
172.20.122.0/24 nat-to 172.20.123.54
pass in log on vlan141 from 172.20.123.54 to 172.20.122.0/24
pass in log on vlan141 from 192.168.123.54 to
172.20.122.0/24 (not needed, but ... :)
return package tcpdump:
nat-to, okay:
Mar 05 23:01:09.418806 rule 410/(match) [uid 0, pid 32543] match in on
vlan141: [orig src 192.168.123.54:5240, dst 172.20.122.10:39322]
172.20.123.54.51958 > 172.20.122.10.39322: S [bad tcp cksum 5166! ->
af7b] 966412712:966412712(0) ack 437277320 win 65160 1460,sackOK,timestamp 452766647 201794907,nop,wscale 7> (DF) (ttl 64,
id 0, len 60, bad ip cksum d8be! -> ed52)
and, PF block this packet:
Mar 05 23:01:09.418820 rule 9/(match) [uid 0, pid 32543]*block in on
vlan141:* [orig src 192.168.123.54:5240, dst 172.20.122.10:39322]
172.20.123.54.51958 > 172.20.122.10.39322: S [bad tcp cksum 5166! ->
af7b] 966412712:966412712(0) ack 437277320 win 65160 1460,sackOK,timestamp 452766647 201794907,nop,wscale 7> (DF) (ttl 64,
id 0, len 60, bad ip cksum d8be! -> ed52)
If i modify pass rule, to match rule:
match in log on vlan141 from 172.20.123.54
i see, match it works, but pass rule not works!
I've tried a lot of things already, without match rules, without nat
(okay, no route, but ...), it is always blocked.
Why can't i override the block rule?
Everywhere else goes ...
--
Regards
Gábor Szél
email:gabor.s...@wantax.hu
Re: disk i/o test
> On Mar 6, 2022, at 7:41 AM, Mihai Popescu wrote: > > Since this thread is moving slowly in another direction, let me > reiterate my situation again: I am running a browser (mostly chromium) > and the computer slows down on downloads. Since I've checked the > downloads rates, I observed they are slow than my maximum 500Mbps for > the line. > I can reach 320Mbps maximum, but mostly it stays at 280Mbps and the > Chromium has 30 seconds delays in everything i do. > > As a suggestion from Stuart, I was trying to separate tests for > downloading and disk write. The disk looks slow. Is the disk brand new? If I missed this somewhere, apologies. If it’s not new, how confident are you that the region of disk where chromium is writing data to disk has not suffered from any reallocations at the physical layer? I find read and write performance to spinning disks is highly regulated by physical layout more than anything else. For linear access, of course. Getting 41 MB/sec on an old disk depending on the region you are accessing is not out of my expectations, if the disk has reallocations in the region accessed. Reallocations occur when the physical media is no longer usable within thresholds so a new sector/area is allocated elsewhere on the disk and mapped. This causes seeks for what you consider a linear access. The hardware does this for you and you can’t stop it nor should you want to. Solution: Get SSD’s. > I tried both Debian 11 and Ubuntu and the download and disk write > jumps to 500Mbps without problems. And no, I cannot tolerate Linux > enough to use it as a daily OS, so don't bother to recommend it. I > cannot attain this in OpenBSD. Maybe that is the maximum possible for > my hardware. Just asking, for the moment i can live with this delays. > I was curious if someone with similar hardware can do better. > > OpenBSD 7.1-beta (GENERIC.MP) #401: Thu Mar 3 12:48:28 MST 2022 >dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 7711543296 (7354MB) > avail mem = 7460630528 (7115MB) > random: good seed from bootblocks > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe86ed (64 entries) > bios0: vendor Hewlett-Packard version "K06 v02.77" date 03/22/2018 > bios0: Hewlett-Packard HP Compaq Pro 6305 SFF > acpi0 at bios0: ACPI 5.0 > acpi0: sleep states S0 S3 S4 S5 > acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT MSDM TCPA IVRS SSDT SSDT CRAT > acpi0: wakeup devices SBAZ(S4) PS2K(S3) PS2M(S3) P0PC(S4) PE20(S4) > PE21(S4) PE22(S4) BNIC(S4) PE23(S4) BR12(S4) BR14(S4) OHC1(S3) > EHC1(S3) OHC2(S3) EHC2(S3) OHC3(S3) [...] > acpitimer0 at acpi0: 3579545 Hz, 32 bits > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat > cpu0 at mainbus0: apid 16 (boot processor) > cpu0: AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.47 MHz, 15-10-01 > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,TCE,NODEID,TBM,TOPEXT,CPCTR,ITSC,BMI1,IBPB > cpu0: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB
Re: disk i/o test
Since this thread is moving slowly in another direction, let me reiterate my situation again: I am running a browser (mostly chromium) and the computer slows down on downloads. Since I've checked the downloads rates, I observed they are slow than my maximum 500Mbps for the line. I can reach 320Mbps maximum, but mostly it stays at 280Mbps and the Chromium has 30 seconds delays in everything i do. As a suggestion from Stuart, I was trying to separate tests for downloading and disk write. The disk looks slow. I tried both Debian 11 and Ubuntu and the download and disk write jumps to 500Mbps without problems. And no, I cannot tolerate Linux enough to use it as a daily OS, so don't bother to recommend it. I cannot attain this in OpenBSD. Maybe that is the maximum possible for my hardware. Just asking, for the moment i can live with this delays. I was curious if someone with similar hardware can do better. OpenBSD 7.1-beta (GENERIC.MP) #401: Thu Mar 3 12:48:28 MST 2022 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 7711543296 (7354MB) avail mem = 7460630528 (7115MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe86ed (64 entries) bios0: vendor Hewlett-Packard version "K06 v02.77" date 03/22/2018 bios0: Hewlett-Packard HP Compaq Pro 6305 SFF acpi0 at bios0: ACPI 5.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT MSDM TCPA IVRS SSDT SSDT CRAT acpi0: wakeup devices SBAZ(S4) PS2K(S3) PS2M(S3) P0PC(S4) PE20(S4) PE21(S4) PE22(S4) BNIC(S4) PE23(S4) BR12(S4) BR14(S4) OHC1(S3) EHC1(S3) OHC2(S3) EHC2(S3) OHC3(S3) [...] acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 16 (boot processor) cpu0: AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.47 MHz, 15-10-01 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,TCE,NODEID,TBM,TOPEXT,CPCTR,ITSC,BMI1,IBPB cpu0: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu0: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, IBE cpu1 at mainbus0: apid 17 (application processor) cpu1: AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.06 MHz, 15-10-01 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,TCE,NODEID,TBM,TOPEXT,CPCTR,ITSC,BMI1,IBPB cpu1: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu1: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu1: smt 1, core 0, package 0 cpu2 at mainbus0: apid 18 (application processor) cpu2: AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.06 MHz, 15-10-01 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,TCE,NODEID,TBM,TOPEXT,CPCTR,ITSC,BMI1,IBPB cpu2: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu2: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu2: disabling user TSC (skew=206) cpu2: smt 0, core 1, package 0 cpu3 at mainbus0: apid 19 (application processor) cpu3: AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.06 MHz, 15-10-01 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,TCE,NODEID,TBM,TOPEXT,CPCTR,ITSC,BMI1,IBPB cpu3: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu3: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu3: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative cpu3: smt 1, core 1, package 0 ioapic0 at mainbus0: apid 5 pa 0xfec0, version 21, 24 pins acpimcfg0
Re: disk i/o test
On 2022-03-06, Alceu Rodrigues de Freitas Junior wrote: > > > Em 05/03/2022 15:29, Janne Johansson escreveu: > >> It can work the other way around also, using free RAM on the >> hypervisor to create >> a larger write cache than the VM itself can have. > > That would improve performance, but at the cost of losing data. > > Not sure if already suggested, but depending on the nature of data (ETL, > for example, would be acceptable), using MFS as file system would have > much better performance. Don't over-estimate the capabilities of MFS, it is not particularly fast. Ignoring VM (and I don't know how things behave there) but on physical hardware I often see faster writes to even just plain SATA SSDs than to MFS. -- Please keep replies on the mailing list.
PF pass not working (on complex "firewall")
Dear @misc
We have an stupid problem.
On a complex firewall (currently PF rules 1200 row), one PASS rule not
working.
I do not know why.
There are many VLANs, WAN, LAN interfaces, many ipsec VPNs, CARP
(master-backup), pfsync, etc ...
PF main rules:
# set
#.
set block-policy drop
set loginterface $ext_wan1_if
set skip on { lo $pfsync_if }
set reassemble no
set timeout { tcp.established 600, tcp.closing 60 }
set optimization aggressive
set ruleset-optimization none
set limit { states 10, src-nodes 10, tables 10,
table-entries 10 }
# scrub
# -
match on $ext_wan1_if all scrub ( no-df max-mss 1440 random-id )
#. antispof
#.
antispoof quick for { $ext_wan1_if } inet
# anchors
# -
anchor "ftp-proxy/*"
# Block(s)
#.
block quick proto udp to port { 1985 8116 } # neighbours
HSRP & ...
block quick log on $ext_wan1_if from { }
label IPBlackList
block log inet6 all
block log all
So all interface traffic are basically forbidden (block).
Each traffic is allowed separately
We have one ipsec VPN, where there are NAT on both sides. (on both sides
have 192.168.x.x subnets, there is a subnet collision)
we want to solve a simple thing:
* comes in the packet on VPN tunnel to "virtual" IP address -
172.20.123.54 (bind to oBSD vlan interface)
* from this address PF redirect packet to destination server -
192.168.123.54
* destination server make return package, and send back
* the response packet comes in oBSD VLAN interface (vlan141)
* PF NAT-ed this packate to 172.20.123.54
* NAT-ed package return to source address in VPN
rules:
match in log on enc0 proto tcp from 172.20.122.0/24 to
172.20.123.54 port 5240 rdr-to 192.168.123.54 port 5240
pass in log on enc0 proto tcp from 172.20.122.0/24 to
192.168.123.54
pass out log on vlan141 from 172.20.122.0/24 to
192.168.123.54
match in log on vlan141 from 192.168.123.54 to
172.20.122.0/24 nat-to 172.20.123.54
pass in log on vlan141 from 172.20.123.54 to 172.20.122.0/24
pass in log on vlan141 from 192.168.123.54 to
172.20.122.0/24 (not needed, but ... :)
return package tcpdump:
nat-to, okay:
Mar 05 23:01:09.418806 rule 410/(match) [uid 0, pid 32543] match in on
vlan141: [orig src 192.168.123.54:5240, dst 172.20.122.10:39322]
172.20.123.54.51958 > 172.20.122.10.39322: S [bad tcp cksum 5166! ->
af7b] 966412712:966412712(0) ack 437277320 win 65160 1460,sackOK,timestamp 452766647 201794907,nop,wscale 7> (DF) (ttl 64, id
0, len 60, bad ip cksum d8be! -> ed52)
and, PF block this packet:
Mar 05 23:01:09.418820 rule 9/(match) [uid 0, pid 32543]*block in on
vlan141:* [orig src 192.168.123.54:5240, dst 172.20.122.10:39322]
172.20.123.54.51958 > 172.20.122.10.39322: S [bad tcp cksum 5166! ->
af7b] 966412712:966412712(0) ack 437277320 win 65160 1460,sackOK,timestamp 452766647 201794907,nop,wscale 7> (DF) (ttl 64, id
0, len 60, bad ip cksum d8be! -> ed52)
If i modify pass rule, to match rule:
match in log on vlan141 from 172.20.123.54
i see, match it works, but pass rule not works!
I've tried a lot of things already, without match rules, without nat
(okay, no route, but ...), it is always blocked.
Why can't i override the block rule?
Everywhere else goes ...
--
Regards
Gábor Szél
email:gabor.s...@wantax.hu
