Re: native recording of all network connections on freebsd

[Sami Halabi]

using firewall ike ipfw with rule to log any to any would be a start.. for
advanced use, stateful fw so You can log start of connections

בתאריך יום ד׳, 28 בדצמ׳ 2022, 16:21, מאת Dan Mack ‏:

>
> I'm wondering if anyone can help point me at a good way to continously
> capture every inbound and outbound connection made to a freebsd system.
> I'd prefer a way that is native in base if possible.   I don't really want
> to record all the packets, just the src:dest:rport:dport stats.
>
> Happy to RTFM as well,
>
> Dan
>
>

Re: recover deleted file

[Sami Halabi]

Hi warner,
Thanks for trying :)

Actually my use case was (if you read later replies, i gave up since the
downtime was too long and couldn't wait more) a VM in ESXi.
so all the underlying stuff of the disks/TRIM.. is hidden an
inaccessible for me (hosting provider).

In my case I tried to recover MariaDB database files, and some tar.gz file
of backups of the db that I accidentally deleted all together (mysql*
instead of mysql/*  , my bad, deleted mysql/ & mysql_backups!!). I stopped
all services immediately, so maybe I would succeed if I wasn't time limited.

I understand its hard to undelete since no one designed UFS/ZFS to do so..
that why I asked in later replies to see if someone would step in and
implement such a "feature" and I suggested some directions/thoughts.

As soren@ suggested in later reply it maybe would be easier to implement
custom rm script that moves files to "Recycle bin" directory (and empty it
after some period) but as a programmer I know that perfection is needed :)
so It might start as a simple task and end in many what-if's
(unfortunattly I did my last C programming in late 2003!).

What amzes me is that this "feature" was asked too much in the last decade
or two and no one ever implemented it, maybe it's not needed in daily
usage, but in disasters it would be super userful, save admins many time
and nerves..

For now I did some backup tools locally and used chflags to mark them
undeletable so I wouldn't do that mistake again, plus I rsync them to my
home storage.. so probably I would be more resilent to such mistakes in the
future.. but the same problem remains.. accidently deleted
file(s)/directory(s) are the nightmare of all admins in earth!!

Sami

On Sun, Apr 17, 2022 at 12:42 AM Warner Losh  wrote:

>
>
> On Sat, Apr 16, 2022 at 5:24 AM Sami Halabi  wrote:
>
>> Hi,
>> is there anyway easy to restore deleted file by accident in UFS
>>
>
> Do you know what the contents of the file is? At least the first, say,
> ~32k?
>
> The problem with unrm for ufs is that the directory entry has the inode
> number stored in it.
> Without the inode number, you won't get very far.
> With the inode number, you can get the first 12 filesystem blocks of the
> file and the
> first three indirect blocks. Once you have those, you can reconstruct the
> file.
>
> But only if the inode hasn't been zero'd out (which it likely has, another
> thing that makes
> UFS undelete harder). But all hope isn't lost...  UFS has a predictable
> allocation algorithm
> that lets you get much of the file back (which is why I asked if you know
> how it should start:
> you can find where it starts in the data blocks and maybe get lucky with
> the rest if the
> data spills into indirect blocks).
>
> However, that's only if you don't have TRIM enabled on the filesystem. If
> you do,
> then UFS will do a BIO_DELETE of the blocks, which means their contents are
> likely gone at the drive level. I say likely because there's weasel words
> in the ATA
> spec that allows a drive to return the prior contents of the blocks, or
> all zeros or
> the drive's initialization pattern (usually all 1's) when the blocks are
> later read. Same
> goes for NVMe drives (with the additional constraint it must be
> deterministic). So there's
> may still be a chance you can read the old contents, but drives that do
> that are rare
> in my experience (which is admittedly quite narrow).
>
> But, if you want to use fsdb to try to recover this data, or write your
> own tools,
> then you should likely have a copy of the daemon book (The Design and
> Implementation
> of the FreeBSD Operating System). It explains a lot of the finer details
> of UFS and
> reference to it likely will catch me where my memory isn't quite right in
> the above
> descriptions.
>
> So, it's for all these reasons you can't find somebody with a unrm command
> for ufs
> like you can for DOS or other filesystems. I wish I had a better answer
> for you.
>
> Warner
>
>
>> Sami
>>
>> --
>> Sami Halabi
>> Information Systems Engineer
>> NMS Projects Expert, FreeBSD SysAdmin Expert
>> Asterisk Expert
>>
>

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: recover deleted file

[Sami Halabi]

Hi,
thanks for your response.

Would someone from the foundation step in and put it in GSOC ideas?

kirk@ - would it be possible for you to do it ? :)

Sami


On Sat, Apr 16, 2022 at 7:26 PM Julian H. Stacey  wrote:

> > okay...
> > all seems very time consuming operations!!
>
> Yes
>
> > There should be an os "undelete" as happens in NTFS for example.. which
> is
> > very fast and can be done also with extra tools without a hassle.
>
> A WIBNI (Wouldnt It Be Nice If) for Unix FS's for as long as I can remember
> (decades) but no one's ever done it.
>
> Ways to get it done:
>   Get it listed as a Google Summer Of Code project for FreeBSD, Or
>
>   Get your employer to help pay for it, eg chip in with other BSD
>   user companies to pay some money to FreeBSD Foundation, & get
>   them to pay for it to be developed.
>
>   Or hire are an individual freelance BSD Consultant to do it,
> There's a global index here http://berklix.com/consultants/
>
>   & a mail list that's moderated jobs@freebsd
>
>   Some author(s) of BSD FFS are on list fs@, Kirk is one name springs
>   to mind ? Some freelancers on fs@ I recall.
>
>   IMO Would be a fun job if funded :-)
>
> >
> > for now I got backup from last day .. caused me a lot of troubles, not
> say
> > legal ones, but I passed the point to hold the machine down.
> >
> > any advice?
> >
> > Maybe UFS developer would do a rework so latest deleted inodes would put
> in
> > a "recycle bin" (maybe with a sysctl or whatever) for say one day (or any
> > other configurable sysctl) and allow to recover quickly or "force delete
> /
> > empty recycle bin" , rather than delete and give back space immediately
> for
> > use and destroy possibility to restore.
> >
> > my 2 cents.
> >
> > Sami
>
> Cheers,
> --
> Julian Stacey  http://berklix.com/jhs/ http://StolenVotes.UK
> Kill / remove Putin to stop him killing & provoking world war.
>


-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: recover deleted file

[Sami Halabi]

okay...
all seems very time consuming operations!!

There should be an os "undelete" as happens in NTFS for example.. which is
very fast and can be done also with extra tools without a hassle.

for now I got backup from last day .. caused me a lot of troubles, not say
legal ones, but I passed the point to hold the machine down.

any advice?

Maybe UFS developer would do a rework so latest deleted inodes would put in
a "recycle bin" (maybe with a sysctl or whatever) for say one day (or any
other configurable sysctl) and allow to recover quickly or "force delete /
empty recycle bin" , rather than delete and give back space immediately for
use and destroy possibility to restore.

my 2 cents.

Sami



On Sat, Apr 16, 2022 at 5:23 PM Julian H. Stacey  wrote:

> > Then I would reboot single user,
> > fsck & mount only the partitions the data was Not on.,
> > dd the partition to recover,
> > then fsck the partition & mount it, & go multi user,
> > then I'd make a 2nd copy of the partition with data to recover
>
> Oops. I meant:
>
> .. I'd make a 2nd copy (with cp) from the 1st image file,
>not of course Not a copy of raw decice partition after fsck
>has discarded blocks.
>
> The spare 2nd. copy because I've zapped data too often, trying to rescue
> it, while fumbling with unfamiliar resue tools: its easier to
> have a play image one can experimentaly try to recover from, &
> periodicaly while one learns, & that gets in a mess,  one can refresh
> copy from master to experimental copy.
>
> If any recovery tools want to run on devices, & refuse images in files, use
> mdconfig -a -t vnode -f imagefile
>
> I recall FS has journals etc,
> Specalists on list fs@
>
> Cheers,
> --
> Julian Stacey  http://berklix.com/jhs/ http://StolenVotes.UK
> Kill / remove Putin to stop him killing & provoking world war.
>


-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: recover deleted file

[Sami Halabi]

how to do step 3 /?

On Sat, Apr 16, 2022 at 2:59 PM Michael Gmelin  wrote:

> Depends on the kind of file.
>
> You can always:
> 1. reboot the system into single user mode, mount the fs readonly
> (important to not overwrite data you want to recover)
> 2. dd the partition and into a file
> 3. find the content of the deleted file in the dump
>
> I was able to recover a complete codebase i deleted accidentally that way
> a long time ago.
>
> Good luck
> Michael
>
> On 16. Apr 2022, at 13:52, Sami Halabi  wrote:
>
> 
> well.. thats the trivial answer.. the problem is backups is a day
> before... if i can undelete it would save me loss of 1 day offset..
>
> anyone?
>
> On Sat, Apr 16, 2022 at 2:49 PM Matthias Apitz  wrote:
>
>> El día sábado, abril 16, 2022 a las 02:23:25 +0300, Sami Halabi escribió:
>>
>> > Hi,
>> > is there anyway easy to restore deleted file by accident in UFS
>>
>> Yes, restore it from a backup media.
>>
>> matthias
>>
>>
>> --
>> Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/
>> +49-176-38902045
>> Public GnuPG key: http://www.unixarea.de/key.pub
>>
>> Peace instead of NATO!  Мир вместо НАТО!  Frieden statt NATO! ¡Paz en vez
>> de OTAN!
>>
>>
>
> --
> Sami Halabi
> Information Systems Engineer
> NMS Projects Expert, FreeBSD SysAdmin Expert
> Asterisk Expert
>
>

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: recover deleted file

[Sami Halabi]

well.. thats the trivial answer.. the problem is backups is a day before...
if i can undelete it would save me loss of 1 day offset..

anyone?

On Sat, Apr 16, 2022 at 2:49 PM Matthias Apitz  wrote:

> El día sábado, abril 16, 2022 a las 02:23:25 +0300, Sami Halabi escribió:
>
> > Hi,
> > is there anyway easy to restore deleted file by accident in UFS
>
> Yes, restore it from a backup media.
>
> matthias
>
>
> --
> Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/
> +49-176-38902045
> Public GnuPG key: http://www.unixarea.de/key.pub
>
> Peace instead of NATO!  Мир вместо НАТО!  Frieden statt NATO! ¡Paz en vez
> de OTAN!
>
>

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

recover deleted file

[Sami Halabi]

Hi,
is there anyway easy to restore deleted file by accident in UFS

Sami

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: running cron jobs setpriority permission denied

[Sami Halabi]

Hi,
Thank You!! indeed that helped!

Sami

On Wed, Mar 9, 2022 at 11:03 AM Ronald Klop  wrote:

> It sounds similar to this issue.
>
> https://github.com/cbsd/cbsd/issues/437 "default nice 1 prevents cron in
> jail #437"
>
> Does that help?
>
> Regards,
> Ronald.
>
>
>
> *Van:* Sami Halabi 
> *Datum:* dinsdag, 8 maart 2022 22:00
> *Aan:* freebsd-sta...@freebsd.org, FreeBSD Current <
> freebsd-current@freebsd.org>, freebsd-j...@freebsd.org,
> freebsd-...@freebsd.org, Oleg Ginzburg 
> *Onderwerp:* running cron jobs setpriority permission denied
>
> Hi,
>
> I have a jail ran by cbsd which has a cronjob like this:
> * * * * * root /usr/local/directadmin/dataskq
>
> I see every minute this error logged in /var/log/messages:
> cron[71002]: setpriority 'root' (daemon): Permission denied
>
> I see in ps xau that it runs but at nobody user
>
> even when loggin to the jail I have:
> cron[68825]: setpriority 'root' (daemon): Permission denied
> login[68900]: setpriority 'root' (root): Permission denied
> jexec[69404]: setpriority 'root' (root): Permission denied
>
> # uname -a
> FreeBSD j5.sody.com 12.3-RELEASE-p1 FreeBSD 12.3-RELEASE-p1 GENERIC  amd64
>
> what am I missing?
>
> Sami
>
> --
> Sami Halabi
> Information Systems Engineer
> NMS Projects Expert, FreeBSD SysAdmin Expert
> Asterisk Expert
>
>

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

running cron jobs setpriority permission denied

[Sami Halabi]

Hi,

I have a jail ran by cbsd which has a cronjob like this:
* * * * * root /usr/local/directadmin/dataskq

I see every minute this error logged in /var/log/messages:
cron[71002]: setpriority 'root' (daemon): Permission denied

I see in ps xau that it runs but at nobody user

even when loggin to the jail I have:
cron[68825]: setpriority 'root' (daemon): Permission denied
login[68900]: setpriority 'root' (root): Permission denied
jexec[69404]: setpriority 'root' (root): Permission denied

# uname -a
FreeBSD j5.sody.com 12.3-RELEASE-p1 FreeBSD 12.3-RELEASE-p1 GENERIC  amd64

what am I missing?

Sami

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert, FreeBSD SysAdmin Expert
Asterisk Expert

Re: linux debian jail - network problems

[Sami Halabi]

Hi,
Thank you for your response.. I wonder if Is it really only netlink problem?
Their are fee problems in the logs.. I dont kbow if they all related only
to netlink (prctl immutable for example).. I also saw oncompatibilities in
socket.c 

Btw: I tried to enter the link you sent and it asked for username and
password.. its not public review?

Sami

בתאריך יום ו׳, 25 בפבר׳ 2022, 04:18, מאת Zhenlei Huang ‏<
zlei.hu...@gmail.com>:

> Hi,
> You can also track the WIP netlink feature,
> https://reviews.freebsd.org/D33975
>
> On Feb 25, 2022, at 4:05 AM, Sami Halabi  wrote:
>
> Hi,
> Added Current, maybe will be lucky ;)
>
> Anyone have idea how approach and fix this?
>
> Sami
>
> בתאריך יום ג׳, 22 בפבר׳ 2022, 23:30, מאת Sami Halabi ‏ >:
>
>> Hi all,
>> sorry for the cross post but I need help and I'm not sure where it hangs.
>>
>> I create linux jail (debian bullseye) via cbsd.
>> the jail is being populated with the debian userland..
>> so far so good... services running (sshd) and I can login to the jail, I
>> also can update packages and I can install apache httpd and all works fine
>> (apt install or make from src).
>> I also manage to install packages even if their scripts depend on "ip"
>> command that fails:
>> cbsd@j2> ip
>> Cannot open netlink socket: Address family not supported by protocol
>>
>> ifconfig show empty interfaces:
>> cbsd@j2> ifconfig
>> eth0: flags=4163  mtu 1500
>> ether 00:50:56:0a:b3:a0  (Ethernet)
>> RX packets 139798314  bytes 12029597009 (11.2 GiB)
>> RX errors 0  dropped 0  overruns 0  frame 0
>> TX packets 26879143  bytes 34400160833 (32.0 GiB)
>> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>>
>> lo0: flags=4169  mtu 16384
>> loop  (Local Loopback)
>> RX packets 28548  bytes 160312960 (152.8 MiB)
>> RX errors 0  dropped 0  overruns 0  frame 0
>> TX packets 28548  bytes 160312960 (152.8 MiB)
>> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>>
>> I know linux emulation doesn't implement netlink.. so what I do is fake
>> the response by replacing /bin/ip by a bash script that prints the correct
>> IP and fakes some other (needed by packages i Installed):
>> #!/bin/bash
>> if [ "$1" = "-o" ]; then
>> echo "1: eth0 inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0"
>> elif [ "$1" = "route" ]; then
>> if [ "$2" = "get" ]; then
>> echo "8.8.8.8 via  192.168.1.2   dev eth0  src
>> 192.168.1.2  "
>> else
>> echo "default via  192.168.1.2   dev eth0"
>> fi
>> else
>> echo "1: eth0:  mtu 1500 qdisc mq state
>> UP qlen 1000"
>> echo "  inet  192.168.1.2  /24 brd  192.168.1.255 scope global eth0"
>>
>>
>> still ifconfig shows no IP... its time to say it a regular jail and *NOT*
>> VNET.
>>
>> *however* package that pull ips via libraries fail..
>> eg: installed bind916 (name) in the logs I see these errors (relevant
>> only):
>> cbsd@j2> service named start
>> Starting domain name service...: namednamed: prctl(PR_SET_DUMPABLE)
>> failed: Invalid argument
>> cbsd@j2>
>>
>>
>> log file shows:
>> 22-Feb-2022 23:11:58.705 general: notice: BIND 9 is maintained by
>> Internet Systems Consortium,
>> 22-Feb-2022 23:11:58.705 general: notice: Inc. (ISC), a non-profit
>> 501(c)(3) public-benefit
>> 22-Feb-2022 23:11:58.705 general: notice: corporation.  Support and
>> training for BIND 9 are
>> 22-Feb-2022 23:11:58.705 general: notice: available at
>> https://www.isc.org/support
>> 22-Feb-2022 23:11:58.705 general: notice:
>> 
>> 22-Feb-2022 23:11:58.705 general: info: found 6 CPUs, using 6 worker
>> threads
>> 22-Feb-2022 23:11:58.705 general: info: using 6 UDP listeners per
>> interface
>> 22-Feb-2022 23:11:58.705 general: info: using up to 21000 sockets
>> 22-Feb-2022 23:11:58.715 general: info: loading configuration from
>> '/etc/bind/named.conf'
>> 22-Feb-2022 23:11:58.715 general: info: reading built-in trust anchors
>> from file '/etc/bind/bind.keys'
>> 22-Feb-2022 23:11:58.715 general: info: looking for GeoIP2 databases in
>> '/usr/share/GeoIP'
>> 22-Feb-2022 23:11:58.715 general: info: using default UDP/IPv4 port
>> range: [1024, 65535]
>> 22-Feb-2022 23:11:58.715 general: i

Re: linux debian jail - network problems

[Sami Halabi]

Hi,
Added Current, maybe will be lucky ;)

Anyone have idea how approach and fix this?

Sami

בתאריך יום ג׳, 22 בפבר׳ 2022, 23:30, מאת Sami Halabi ‏:

> Hi all,
> sorry for the cross post but I need help and I'm not sure where it hangs.
>
> I create linux jail (debian bullseye) via cbsd.
> the jail is being populated with the debian userland..
> so far so good... services running (sshd) and I can login to the jail, I
> also can update packages and I can install apache httpd and all works fine
> (apt install or make from src).
> I also manage to install packages even if their scripts depend on "ip"
> command that fails:
> cbsd@j2> ip
> Cannot open netlink socket: Address family not supported by protocol
>
> ifconfig show empty interfaces:
> cbsd@j2> ifconfig
> eth0: flags=4163  mtu 1500
> ether 00:50:56:0a:b3:a0  (Ethernet)
> RX packets 139798314  bytes 12029597009 (11.2 GiB)
> RX errors 0  dropped 0  overruns 0  frame 0
> TX packets 26879143  bytes 34400160833 (32.0 GiB)
> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> lo0: flags=4169  mtu 16384
> loop  (Local Loopback)
> RX packets 28548  bytes 160312960 (152.8 MiB)
> RX errors 0  dropped 0  overruns 0  frame 0
> TX packets 28548  bytes 160312960 (152.8 MiB)
> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> I know linux emulation doesn't implement netlink.. so what I do is fake
> the response by replacing /bin/ip by a bash script that prints the correct
> IP and fakes some other (needed by packages i Installed):
> #!/bin/bash
> if [ "$1" = "-o" ]; then
> echo "1: eth0 inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0"
> elif [ "$1" = "route" ]; then
> if [ "$2" = "get" ]; then
> echo "8.8.8.8 via  192.168.1.2   dev eth0  src
> 192.168.1.2  "
> else
> echo "default via  192.168.1.2   dev eth0"
> fi
> else
> echo "1: eth0:  mtu 1500 qdisc mq state
> UP qlen 1000"
> echo "  inet  192.168.1.2  /24 brd  192.168.1.255 scope global eth0"
>
>
> still ifconfig shows no IP... its time to say it a regular jail and *NOT*
> VNET.
>
> *however* package that pull ips via libraries fail..
> eg: installed bind916 (name) in the logs I see these errors (relevant
> only):
> cbsd@j2> service named start
> Starting domain name service...: namednamed: prctl(PR_SET_DUMPABLE)
> failed: Invalid argument
> cbsd@j2>
>
>
> log file shows:
> 22-Feb-2022 23:11:58.705 general: notice: BIND 9 is maintained by Internet
> Systems Consortium,
> 22-Feb-2022 23:11:58.705 general: notice: Inc. (ISC), a non-profit
> 501(c)(3) public-benefit
> 22-Feb-2022 23:11:58.705 general: notice: corporation.  Support and
> training for BIND 9 are
> 22-Feb-2022 23:11:58.705 general: notice: available at
> https://www.isc.org/support
> 22-Feb-2022 23:11:58.705 general: notice:
> 
> 22-Feb-2022 23:11:58.705 general: info: found 6 CPUs, using 6 worker
> threads
> 22-Feb-2022 23:11:58.705 general: info: using 6 UDP listeners per interface
> 22-Feb-2022 23:11:58.705 general: info: using up to 21000 sockets
> 22-Feb-2022 23:11:58.715 general: info: loading configuration from
> '/etc/bind/named.conf'
> 22-Feb-2022 23:11:58.715 general: info: reading built-in trust anchors
> from file '/etc/bind/bind.keys'
> 22-Feb-2022 23:11:58.715 general: info: looking for GeoIP2 databases in
> '/usr/share/GeoIP'
> 22-Feb-2022 23:11:58.715 general: info: using default UDP/IPv4 port range:
> [1024, 65535]
> 22-Feb-2022 23:11:58.715 general: info: using default UDP/IPv6 port range:
> [1024, 65535]
> 22-Feb-2022 23:11:58.715 network: info: no IPv6 interfaces found
> 22-Feb-2022 23:11:58.715 general: error: ifiter_getifaddrs.c:79:
> unexpected error:
> 22-Feb-2022 23:11:58.715 general: error: getting interface addresses:
> getifaddrs: Address family not supported by protocol
> 22-Feb-2022 23:11:58.715 network: warning: not listening on any interfaces
> *snip*
> *snip*
> 22-Feb-2022 23:11:58.735 general: error: socket.c:2405: unexpected error:
> 22-Feb-2022 23:11:58.735 general: error: setsockopt(50, IP_RECVTOS)
> failed: Protocol not available
> 22-Feb-2022 23:11:58.735 general: notice: couldn't add command channel
> 127.0.0.1#953: permission denied
> 22-Feb-2022 23:11:58.735 general: error: socket.c:2405: unexpected error:
> 22-Feb-2022 23:11:58.735 general: error: setsockopt(50, IP_RECVTOS)
> failed: Protocol not available
> 22-Feb-2022 23:11:58.735 general: notice: couldn't add 

Re: Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux

[Sami Halabi]

Hi,
I see these claims over and over.
So I must ask.
Is there any tunibg guide(s) to make the default not conservative in a
regrding to several use cases like using as web server? How to Utilize gpu
maybe?
I know there are few network (aka routing / forwarding) guides.. but maybe
instead of that superior feeling "oh they are linuxish and knoe shit" maybe
better supply the tuning needed to get better results?
And I'm not talking to get an engineer to analyze the tests case..
Maybe the linux defaults fit better for most use cases rather than being
conservative??

Just to be clear I almost not used linux and always freebsd for simplicity
usage..  but I must say it makes me wonder

Sami

בתאריך שבת, 11 בדצמ׳ 2021, 11:52, מאת beepc.ch ‏:

> > I am surprised to see that the BSD cluster today has much worse
> performance
> > than Linux.
> > What do you think of this?
>
> "Default" FreeBSD install setting are quite conservative.
> The Linux common distros are high tuned, those benchmark is in my
> opinion comparison of apples and oranges.
>
> Comparing "default" FreeBSD install with "default" Slackware install
> would be more interesting, because Slackware builds are at most vanilla.
>
>

Re: FreeBSD 12.0-RC2 Now Available

[Sami Halabi]

HI,
I went over the release notes and honestly I don't see what 12 brings new..
I remember older versions were big change each in different aspect.. So
what is 12-Rel unique aspect VS 11 let's say?

Thanks in advance,
Sami

בתאריך יום א׳, 25 בנוב׳ 2018, 4:04, מאת Glen Barber :

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> The second RC build of the 12.0-RELEASE release cycle is now available.
>
> Installation images are available for:
>
> o 12.0-RC2 amd64 GENERIC
> o 12.0-RC2 i386 GENERIC
> o 12.0-RC2 powerpc GENERIC
> o 12.0-RC2 powerpc64 GENERIC64
> o 12.0-RC2 powerpcspe MPC85XXSPE
> o 12.0-RC2 sparc64 GENERIC
> o 12.0-RC2 armv6 RPI-B
> o 12.0-RC2 armv7 BANANAPI
> o 12.0-RC2 armv7 BEAGLEBONE
> o 12.0-RC2 armv7 CUBIEBOARD
> o 12.0-RC2 armv7 CUBIEBOARD2
> o 12.0-RC2 armv7 CUBOX-HUMMINGBOARD
> o 12.0-RC2 armv7 PANDABOARD
> o 12.0-RC2 armv7 WANDBOARD
> o 12.0-RC2 armv7 GENERICSD
> o 12.0-RC2 aarch64 GENERIC
> o 12.0-RC2 aarch64 RPI3
> o 12.0-RC2 aarch64 PINE64
> o 12.0-RC2 aarch64 PINE64-LTS
>
> Note: The 12.0-RC2 armv7 RPI2 build failed, and the cause is being
> investigated.
>
> Also note, at present, freebsd-update(8) patch builds are still in
> progress.  A followup email will be sent in reply to this announcement
> when they are available.
>
> Note regarding arm SD card images: For convenience for those without
> console access to the system, a freebsd user with a password of
> freebsd is available by default for ssh(1) access.  Additionally,
> the root user password is set to root.  It is strongly recommended
> to change the password for both users after gaining access to the
> system.
>
> Installer images and memory stick images are available here:
>
> https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/
>
> The image checksums follow at the end of this e-mail.
>
> If you notice problems you can report them through the Bugzilla PR
> system or on the -stable mailing list.
>
> If you would like to use SVN to do a source based update of an existing
> system, use the "releng/12.0" branch.
>
> A summary of changes since 12.0-RC1 includes:
>
> o Kernel debugging support in various kernel configurations has been
>   disabled, which was missed when branching releng/12.0 from stable/12.
>
> o Allow set ether/vlan PCP operation from the VNET jails.
>
> o Align IA32_ARCH_CAP MSR definitions and use with SDM rev. 068.
>
> o Several IFLIB-related fixes.
>
> o Regressions when using 'pciconf -l' were fixed.
>
> o Handle kernel superpage mappings in pmap_remove_l2().  (PR 233088)
>
> o Fix /etc/ntp permissions.
>
> o OpenSSL has been updated to version 1.1.1a.
>
> o Various fixes to libbe(3) and bectl(8).
>
> o A src.conf knob to build userland with retpoline was added (off by
>   default).
>
> o Various other miscellaneous fixes.
>
> A list of changes since 11.2-RELEASE is available in the releng/12.0
> release notes:
>
> https://www.freebsd.org/releases/12.0R/relnotes.html
>
> Please note, the release notes page is not yet complete, and will be
> updated on an ongoing basis as the 12.0-RELEASE cycle progresses.
>
> === Virtual Machine Disk Images ===
>
> VM disk images are available for the amd64 and i386 architectures.
> Disk images may be downloaded from the following URL (or any of the
> FreeBSD FTP mirrors):
>
> https://download.freebsd.org/ftp/releases/VM-IMAGES/12.0-RC2/
>
> The partition layout is:
>
> ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label)
> ~ 1 GB  - freebsd-swap GPT partition type (swapfs GPT label)
> ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label)
>
> The disk images are available in QCOW2, VHD, VMDK, and raw disk image
> formats.  The image download size is approximately 135 MB and 165 MB
> respectively (amd64/i386), decompressing to a 21 GB sparse image.
>
> Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI
> loader file is needed for qemu-system-aarch64 to be able to boot the
> virtual machine images.  See this page for more information:
>
> https://wiki.freebsd.org/arm64/QEMU
>
> To boot the VM image, run:
>
> % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt  \
> -bios QEMU_EFI.fd -serial telnet::,server -nographic \
> -drive if=none,file=VMDISK,id=hd0 \
> -device virtio-blk-device,drive=hd0 \
> -device virtio-net-device,netdev=net0 \
> -netdev user,id=net0
>
> Be sure to replace "VMDISK" with the path to the virtual machine image.
>
> === Amazon EC2 AMI Images ===
>
> FreeBSD/amd64 EC2 AMIs are available in the following regions:
>
>  ap-south-1 region: ami-0285a4b0c311d9e5e
>  eu-west-3 region: ami-01989f54cc5fc3425
>  eu-west-2 region: ami-0058f626d39ade7dc
>  eu-west-1 region: ami-07cca4933d62d5d22
>  ap-northeast-2 region: ami-084b8fc685e73d718
>  ap-northeast-1 region: ami-0fd072608bc5cc041
>  sa-east-1 region: ami-0df9e331ad6b563cd
>  ca-central-1 region: ami-01360ca27677e8deb
>  ap-southeast-1 region: ami-0dc6b473d0770bd29
>  

Re: need help using ng_patch to modify src/dst packets or alternative way

[Sami Halabi]

Hi Eugene,
I'm looking for a solution for IP traffic. in linux iptables its possible
but I couldn't find freebsd way yet.
bkuncr soulution works for tcp only.

Thanks for the hint though,

Sami

בתאריך 17 בדצמ׳ 2017 11:29 AM,‏ "Eugene Grosbein" <eu...@grosbein.net> כתב:

> 17.12.2017 14:52, Sami Halabi пишет:
> > hi,
> >
> > Can you help in my situation? My goal is so Box in my lan 10.1.1.2 to
> talk
> > to 10.1.1.1 and actually it would be talking to X.X.X.X outside ip using
> > one of my public IPs say 1.1.1.1.
>
> If you need this just for single or several tcp ports, easiest way
> is to use any of port forwarders/bouncers like this:
>
> pkg install bounce
> bounce -a 10.1.1.1 -b 1.1.1.1 -p 443 X.X.X.X 443
>
>
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

need help using ng_patch to modify src/dst packets or alternative way

[Sami Halabi]

hi,

Can you help in my situation? My goal is so Box in my lan 10.1.1.2 to talk
to 10.1.1.1 and actually it would be talking to X.X.X.X outside ip using
one of my public IPs say 1.1.1.1.

I'm trying to modify packets to passthrough to a local IP.
I have a box that a specific IP is routed to it.. say 1.1.1.1
in my bce0 i don't have that ip configured but i have my public IP that say
2.2.2.2 that 1.1.1.1 is routed to it.
i configured 10.1.1.1/24 in bce0, my target box is 10.1.1.2/24.
i tried the following inside ngctl:

mkpeer ipfw: patch 300 in
name ipfw:300 src_dst_chg
msg src_dst_chg: setconfig { count=2 csum_flags=1 ops=[  { mode=1
value=0x0a010101 length=4 offset=3 }  { mode=1 value=0x0a010102 length=4
offset=4 } ] }

in my box(10.1.1.1) i did:
sysctl net.inet.ip.fw.one_pass=0
/sbin/ipfw add 50 netgraph 300 ip from any to any to 1.1.1.1

then i do simple ping from outside box
i see the packets arrive on my 160 rule
but never leaves the box..

I would at least see packeta flow one direction to 10.1.1.2 and then that
need another ipfw and netgraph opposite rule.

If you have alternative way I'm happy to try...


Help much appreciated...
Sami
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Update to 11.0-RELEASE Schedule

[Sami Halabi]

Hi,
Great news!
usually I read your emails going to freebad-net/jails but seems i missed
this.
I remmember Roman(?) went over and tried to patch current viamge against
open PRs but somewhere the work stopped (or maybe i mssed this too :).
is there some patch to test?

Thanks for your hard work.
Sami
בתאריך 15 באפר׳ 2016 18:35,‏ "Bjoern A. Zeeb" <
bzeeb-li...@lists.zabbadoz.net> כתב:

>
> > On 15 Apr 2016, at 15:32 , Sami Halabi <sodyn...@gmail.com> wrote:
> >
> > Hi,
> > Myabe i missed something... what isbthe work about? is it about
> stabilizimg
> > current viamge or something else?
>
> Yes more stable top-down-teardown and reducing the possible memory leaks.
>
> /bz
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Update to 11.0-RELEASE Schedule

[Sami Halabi]

Hi,
Myabe i missed something... what isbthe work about? is it about stabilizimg
current viamge or something else?

Sami
בתאריך 15 באפר׳ 2016 18:24,‏ "Bjoern A. Zeeb" <
bzeeb-li...@lists.zabbadoz.net> כתב:

>
> > On 15 Apr 2016, at 13:49 , Ernie Luzar  wrote:
> >
> > Is the VIMAGE revamp by "Bjoern A. Zeeb” completed and is it going to be
> included in 11.0?
>
> It’s not completed yet but I’ll try to make sure as much as possible will
> be in HEAD before the code slush date.
>
> Bjoern
>
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: forwarding/ipfw/pf evolution (in pps) on -current

[Sami Halabi]

Oliver,
Great and impressive job.
If I interpret the plot as is the result say (approximatly of course):
1. Forwarding using ipfw with single rule degrades ~25% the pps.
2. Forwarding with pf however gets ~50%+ of degredation if performance pps.
3. there some point of improved performance (without fw) that went down
again somewhere before Clang got prod.
4. I think that the results don't necessarly can be translated to SMP
versions because of scheduler, affinity issues.

For now i would continue using ipfw :-)

Sami
On Apr 24, 2013 1:45 PM, Olivier Cochard-Labbé oliv...@cochard.me wrote:

 Hi all,

 here is the result of my simple-and-dummy bench script regarding
 forwarding/ipfw/pf performance evolution on -current on a single-core
 server with one flow only.
 It's the result of more than 810 bench tests (including reboot between
 each) done twice for validating my methodology.

 # Disclaimer #

 1. It's not a max performance bench: The purpose is to graph the
 variation of the performance only.
 2. I know that using a single-core server in 2013 is a stupid idea but
 it's all I've got on my lab :-(

 # Why all these benchs ? #

 I've found performance regression regarding packet forwarding/ipfw/pf
 speed on -current comparing to 9.1 on my old server.
 glebius@ ask me to do some bisection hunting on different -current
 revision for spotting the culprit commit.
 But as a lazy guy, in place of doing bisection, I've choose about 50
 svn revision and graph them all: It's a lot's more easy to script this
 than a bisection algorithm :-)
 And the result is interesting…

 # The results #

 The gnuplot diagram in png format with some confirmed specifics spots
 is available here:
 http://gugus69.free.fr/freebsd/benchs/current/current-pps.png

 A confirmed spot is a measurable change between revision N-1 and revision
 N.

 = Remember that I'm used a single-core before reading the result!
 The regression of the new SMP pf is not really a regression: The
 system is now usable during this high PPS bench and it was not the
 case before this improvement.

 ## gnuplot data ##

 Available here: http://gugus69.free.fr/freebsd/benchs/current/plot/
 It's the data and plot file used for generating the graph: You can use
 them for zooming on it.

 ## ministat data ##

 Available here: http://gugus69.free.fr/freebsd/benchs/current/ministat/

 You can use it for comparing result between 2 revision, like as example:
 ministat -s 242160.ipfw 242161.ipfw

 ## raw data ##

 Outpout of pkg-gen during all tests:
 http://gugus69.free.fr/freebsd/benchs/current/raw/

 ## nanobsd images #

 All binary mages used for these benchs are here:
 http://gugus69.free.fr/freebsd/benchs/current/nanobsd-images/

 There is only one full image to be used for the first installation,
 and all other are upgrade image.
 They use the serial port as default console too.

 # Methodology used #

 ## First step: building a small lab ##

 I've used 3 old unused servers and a good switch:
 - One server as netmap pkt-gen packet generator (1.38Mpps of minimum
 size packet);
 - One server as netmap pkt-gen receiver;
 - One server with 2 NIC in the middle as a router/firewall, serial
 connection, and nanobsd image on it (very easy to upgrade): IBM
 eServer xSeries 306m with one core (Intel Pentium4 3.00GHz,
 hyper-threading disabled) and a dual NIC 82546GB connected to the
 PCI-X Bus;
 - a Cisco Catalyst switch for connecting all (its own statistics can
 be used as a tie breaker if I've got a doubt regarding the result
 given by netmap pkt-gen).

 All servers have another NIC for the admin network (bench script send
 SSH commands and nanobsd image upgrade over this dedicated NIC).

 I've used netmap pkt-gen for generating smallest packet size from the
 generator to the receiver like that:
 pkt-gen -i em0 -t 0 -l 42 -d 1.1.1.1 -D 00:0e:0c:de:45:df -s 2.2.2.2 -w 10
 Results was collected on the pkt-gen receiver.

 ## Second step: building small nanobsd images ##

 Now we need lot's of small nanobsd images generated from the svn
 revision number selected for the bench: cf script [1].
 About 50 revisions were selected between 236884 to 249506: Candidate
 chosen by reading the svn commit log.

 ## Third step: auto-bench script ##

 This auto-bench script [2] do these tasks:
 1. Upgrading the server to the release to be tested;
 2.   Uploading configuration set to be tested (forwarding-only, ipfw
 or pf)  reboot;
 3. Start the bench test, collecting the result, and reboot: 5
 times for each configuration-set;
 4Loop to next configuration set;
 5. Loop to next release.

 ## Last step: converting result for ministat and gnuplot ##

 I've used a last script for interpreting the output of pkt-gen
 receiver for ministat and gnuplot [3].

 Because I'm not sure if I've used the good method for preparing my
 data, here is how I've generated the ministat and gnuplot graph:

 For just one test, the output of pkt-gen in receive mode is lot's of
 lines like that:
 main [1085] 

Re: ZFS on HEAD

[Sami Halabi]

/usr/src/sys/amd64/confSAMI: unknown option KVA_PAGES

On Fri, Sep 28, 2012 at 6:33 PM, Glen Barber g...@freebsd.org wrote:

 On Fri, Sep 28, 2012 at 06:08:41PM +0200, Sami Halabi wrote:
  I tried to follow:
 
 http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html
 
  to recompile the kernel with KVA_PAGES
  and i couldn't compile.
 
  any ideas why this?
 

 What was the error?

 Glen




-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert
FreeBSD SysAdmin Expert
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: ZFS on HEAD

[Sami Halabi]

got it, I thought amd64 is i386 with 64 bit, seems i was wrong in termenlogy
Thanks a lot

On Fri, Sep 28, 2012 at 9:36 PM, Glen Barber g...@freebsd.org wrote:

 On Fri, Sep 28, 2012 at 09:31:41PM +0200, Sami Halabi wrote:
  On Fri, Sep 28, 2012 at 6:33 PM, Glen Barber g...@freebsd.org wrote:
   On Fri, Sep 28, 2012 at 06:08:41PM +0200, Sami Halabi wrote:
I tried to follow:
   
  
 http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html
   
to recompile the kernel with KVA_PAGES
and i couldn't compile.
   
any ideas why this?
   
  
   What was the error?
  
 
  /usr/src/sys/amd64/confSAMI: unknown option KVA_PAGES
 

 KVA_PAGES is not a valid option for amd64 kernel configurations.

 It is only needed/recommended for i386 and pc98 architectures.

 Glen




-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert
FreeBSD SysAdmin Expert
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: ZFS on HEAD

[Sami Halabi]

Hi,

what count for little, and what count for huge.
is there any documented tunings needed for both cases? if not I'd
appreciate it much if you explain the tunungs needed and what they do.

Sami

On Fri, Sep 28, 2012 at 9:37 PM, Matthew D. Fuller fulle...@over-yonder.net
 wrote:

 On Fri, Sep 28, 2012 at 09:31:41PM +0200 I heard the voice of
 Sami Halabi, and lo! it spake thus:
  /usr/src/sys/amd64/confSAMI: unknown option KVA_PAGES

 You're using amd64, not i386; you don't need to mess with KVA_PAGES.

 In fact, you probably don't need to tune anything on amd64, unless
 you've got either very little or very huge physical memory.


 --
 Matthew Fuller (MF4839)   |  fulle...@over-yonder.net
 Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
On the Internet, nobody can hear you scream.




-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert
FreeBSD SysAdmin Expert
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

MPLS in freebsd

[Sami Halabi]

Hi,
is there any on-job work on MPLS support in FreeBSD?

what are the plan to integrate this in production use?

Thanks in advance,

-- 
Sami Halabi
Information Systems Engineer
NMS Projects Expert
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org