Re: CARP under Hyper-V: weird things happen

[Daniel Kalchev]

Hi Eugene,

Might it be the Hyper-V doesn’t properly implement multicast? Or there is 
perhaps some setting in there to let it work. From memory CARP is not trivial 
on vmware as well, unless you make special settings. Some ideas here: 
https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-high-availability-clusters.html#hypervisor-users-especially-vmware-esx-esxi
 


Daniel

> On 31 May 2020, at 19:07, Eugene M. Zheganin  wrote:
> 
> Hello,
> 
> I'm Running 12.0-REL in a VM under W2016S with CARP enabled and paired to a 
> baremetal FreeBSD server.
> 
> All of a sudden I realized that thjis machine is unable to become a CARP 
> MASTER - because it sees it's own ACRP announces, but instead of seeing them 
> from a CARP synthetic MAC address only, it sees additional extra packets with 
> several MACs derived from the original one (I'm well awared about the 
> -MacAddressSpoof on SetVmNetworkAdapterVlan switch, and it's running with 
> this thingg on, but still). These packets always almost (but not 100%) 
> accompany each valid CARP advertisement.
> 
> Say, we have a CARP-enabled interface:
> 
> vlan2: flags=8943 metric 0 
> mtu 1500
> description: AS WAN
> options=8
> ether 00:15:5d:0a:79:12
> inet 91.206.242.9/28 broadcast 91.206.242.15
> inet 91.206.242.12/28 broadcast 91.206.242.15 vhid 3
> groups: vlan
> carp: BACKUP vhid 3 advbase 1 advskew 250
> vlan: 2 vlanpcp: 0 parent interface: hn1
> media: Ethernet autoselect (10Gbase-T )
> status: active
> nd6 options=29
> 
> Notice the MAC and now look at this:
> 
> ===Cut===
> 
> [root@gw1:~]# tcpdump -T carp -nepi vlan2 carp
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on vlan2, link-type EN10MB (Ethernet), capture size 262144 bytes
> 20:45:54.152619 00:00:5e:00:01:03 > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227035
> 
> ^^^ this is the ordinary and valid CARP advertisement, notice the synthetic 
> MAC which is requiring setting mac address spoofing.
> 
> 20:45:54.152880 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227035
> 
> ^^^ this is some insanity happening
> 
> 20:45:54.153234 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227035
> 
> ^^^ and again
> 
> 20:45:54.153401 9c:8e:99:0f:79:42 > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227035
> 
> ^^^ and again
> 
> 20:45:57.562470 00:00:5e:00:01:03 > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227036
> 
> ^^^ valid CARP advertisement, next one-second advbase cycle
> 
> 20:45:57.562874 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227036
> 
> ^^^ more insane stuff, notice the NEW (sic !) MAC-address
> 
> 20:45:57.562955 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227036
> 20:45:57.562989 9c:8e:99:0f:79:3c > 01:00:5e:00:00:12, ethertype IPv4 
> (0x0800), length 70: 91.206.242.9 > 224.0.0.18: CARPv2-advertise 36: vhid=3 
> advbase=1 advskew=100 authlen=7 counter=13769798250643227036
> ^C
> 8 packets captured
> 3195 packets received by filter
> 
> ===Cut===
> 
> 
> Does anyone has, by any chance, some idea about what's happening ? As soon as 
> I stop CARP stack on this VM these "mad" MACs aren't received anymore, so I'm 
> pretty confident these are somehow procuced on the Hyper-V side.
> 
> Another weird this is that vlan1  is refusing to work (seems like packets are 
> never received on the VM side) unless its configured on another adapter in 
> the -Untagged (once again powershell term for SetVmNetworkAdapterVlan).
> 
> 
> Thanks.
> 
> Eugene.
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list

Re: Running FreeBSD on M.2 SSD

[Daniel Kalchev]

Is TRIM still on?

I understand the quirks patch indicates the drive has some trouble with 4K 
aligned writes. If memory serves it I also indicated broken TRIM so to be safe 
you need both.

Daniel


> On 29 Feb 2020, at 2:46, Mario Olofo  wrote:
> 
> Hello guys, a little update that let me more confused
> 
> I reinstalled the FreeBSD with 4k pages using the sysctl
> vfs.zfs.min_auto_ashift = 12 and no errors after a lot of stress I put on
> it.
> One thing that I noticed is that with the pool as 4k, the disk fill up very
> fast, recompiling the kernel used my 8GB space and didn't even completed.
> But now I don't know if the 4k is the correct answer or if this just delays
> the problem as the pages are bigger.
> 
> Mario
> 
>> Em sex., 28 de fev. de 2020 às 13:18, Mario Olofo 
>> escreveu:
>> 
>> Yes, tried 4k quirk but not on install because don't know how to, I did a
>> clean install then patch and rebuild the kernel, but
>> the volume was already configured for 512bytes, I think I would need to
>> create manually the volume, but don't remember how to anymore xD
>> But I'll search some tutorials and try. From what I saw, the patch
>> suggested on bugzilla got merged into the stable branch, so the quirk will
>> be
>> detected to use 4k in the installer in a near future.
>> 
>> Mario
>> 
>> Em sex., 28 de fev. de 2020 às 12:52, Theron 
>> escreveu:
>> 
>>> On 2020-02-28 09:14, Mario Olofo wrote:
 Thanks!
 
 The only thing that I didn't checked was the questions of Theron, about
 misaligned data.
 The layout of the disk is as follows:
 
 Disco /dev/sdb: 447,1 GiB, 480113590272 bytes, 937721856 setores
 Unidades: setor de 1 * 512 = 512 bytes
 Tamanho de setor (lógico/físico): 512 bytes / 512 bytes
 Tamanho E/S (mínimo/ótimo): 512 bytes / 512 bytes
 Tipo de rótulo do disco: gpt
 Identificador do disco: D1725E60-D734-4461-90F8-E9EB2376A65A
 
 DispositivoInício   Fim   Setores Tamanho Tipo
 /dev/sdb12048   1023999   1021952499M Windows ambiente de
 recuperação
 /dev/sdb2 1024000   1228799204800100M Sistema EFI
 /dev/sdb3 1228800   1261567 32768 16M Microsoft reservado
 /dev/sdb4 1261568 532482047 531220480  253,3G Microsoft dados básico
 /dev/sdb5   532482048 549257215  16775168  8G FreeBSD ZFS
 /dev/sdb6   549257216 937719807 388462592  185,2G Linux sistema de
>>> arquivos
 
 The zfsroot was configured automatically by the installer, so I think
>>> that
 it align the volume automaticaly right?
 
 Mario
>>> 
>>> Yes, I don't see any potential alignment issue here.  I would wonder if
>>> this drive is misrepresenting its physical sector size, deceiving ZFS
>>> and the SATA driver into making small writes that the drive does not
>>> actually support, but it looks like you may have already tried the
>>> relevant workaround:
>>> 
>>> On 2020-02-27 23:44, Mario Olofo wrote:
 Maybe the problem really is a combination of factors, for the person
>>> that
 filed a bug on bugzilla the fix was setting the quirks 4k and
>>> broken_trim,
 but for me the real block size is 512bytes and only setting the flag
 broken_trim didn't help...
 
 Mario
>>> Did you try 4k quirk ?
>>> 
>>> Theron
>>> 
>> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Running FreeBSD on M.2 SSD

[Daniel Kalchev]

If my memory serves well, TRIM was originally not enabled by default on 
FreeBSD, because there were many drives that claimed to support it, but didn’t, 
or didn’t support it properly. That sort of is resolved today and WD Green is 
supposedly relatively recent drive.

I am not aware of disabling TRIM creating any kind of problem for data 
consistency or drive longevity, except that it leads to slower writes, as the 
drive runs out of pre-erased blocks to write to. This is typically more 
pronounced on “cheaper” drives like this one where there is not much 
over-provisioning but much less pronounced on “server” drives that keep certain 
buffer capacity on the driv eunused for precisely that purpose.

But why should a drive without TRIM wear more quickly?

Daniel

> On 25 Feb 2020, at 16:07, Pete French  wrote:
> 
> 
> 
> On 25/Feb/2020 13:28, Mario Olofo wrote:
>> Good morning all,
>> @Pete French, you have trim activated on your SSDs right? I heard that if 
>> its not activated, the SSD disc can stop working very quickly.
> 
> On the curent dfives, yes, but I have run with trim disabled in the past, It 
> kinds of depends on the drive - so were horirbly slow on trim, and trim could 
> be a big bottleneck. Havent seen that on a recent drive though, and they all 
> now have trim enabled on them.
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Running FreeBSD on M.2 SSD

[Daniel Kalchev]

FreeBSD does not technically have driver for different disks. People asked 
whether it is an NVMe device or SATA device, because those interfaces have 
different drivers.

But for FreeBSD, an mechanical SATA, hybrid SATA or SSD SATA will use exactly 
the same SATA driver. It depends on the chipset.

It is possible however, that the timing between the drive and the SATA 
controller might be different and that is causing the problem.

Did you experiment with different settings of the SATA controller in BIOS?

If the problem is related to the size of journal, that might mean for some 
reason the SSD is slow. About th eonly thing an SSD might be slow for is TRIM. 
Therefore, TRIM might be your problem if weirdly implemented in that drive … so 
you might try to disable it and see if the problem goes away. As it’s not a 
server, I doubt you will notice much of performance drop.

You can disable TRIM for ZFS with

sysctl vfs.zfs.trim.enabled=0

You can put it in /boot/loader.conf. Do this before writing any data to the 
pool or even creating the pool.

Speaking of that, the output of 

sysctl kstat.zfs.misc.zio_trim

might tell us something.

I would advise doing all such tests with ZFS, because it will spot any flaky 
hardware/setup easily.

Daniel

> On 25 Feb 2020, at 15:28, Mario Olofo  wrote:
> 
> Good morning all,
> 
> @Pete French, you have trim activated on your SSDs right? I heard that if
> its not activated, the SSD disc can stop working very quickly.
> @Daniel Kalchev, I used UFS2 with SU+J as suggested on the forums for me,
> and in this case the filesystem didn't "corrupted", it justs kernel panic
> from time to time so I gave up.
> I think that the problem was related to the size of the journal, that
> become full when I put so many files at once on the system, or was
> deadlocks in the version of the OS that I was using.
> @Alexander Leidinger I have the original HDD 1TB Hybrid that came with the
> notebook will try to reinstall FreeBSD on it to see if it works correctly.
> 
> Besides my notebook been a 2019 model Dell G3 with no customizations other
> than the m.2 SSD, I never trust that the system is 100%, so I'll try all
> possibilities.
> 1- The BIOS received an update last month but I'll look if there's
> something newer.
> 2- Reinstall the FreeBSD on the Hybrid HDD, but if the problem is the
> FreeBSD driver, it'll work correctly on that HD.
> 3- Will try with other RAM. This I really don't think that is the problem
> because is a brand new notebook, but... who knows =).
> 
> Thank you,
> 
> Mario
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Running FreeBSD on M.2 SSD

[Daniel Kalchev]

I have had disks, that work “perfectly" under UFS and various RAID controllers 
(and DOS and Windows), but always reported checksum errors when running under 
ZFS. It would happen on any motherboard or controller. That made me never use 
anything but ZFS on data that I cannot recreate 100%, fast… but that is 
separate story. I labeled those disks bad and they sit in my “museum”. Needless 
to say some were brand new. Not saying you have this issue, but sharing 
anecdotal evidence.

But I wonder how you discovered you had corruption with UFS? What is observed?

It might well be, that FreeBSD is more agressive with your motherboard/chipset 
or does not implement known quirk of that — which might trigger some edge cases 
for the SSD. Ultimately, if you can move that SSD to another motherboard and 
test it, it would confirm where the issue is.

Daniel

> On 25 Feb 2020, at 3:35, Mario Olofo  wrote:
> 
> Hi Mike, thanks for the insight.
> 
> I tried both, but not at the same time.
> When I found that the ZFS was corrupting the filesystem, I reinstalled the
> FreeBSD using UFS but no luck.
> Ulf told me that he had the same problem and it turned out the problem was
> a defective RAM, but here I just ran the test 2 times,
> one from Dell BIOS Diagnostics Tool and other from mdsched.exe from Windos
> 10, but here the RAM is ok...
> 
> Thank you again,
> 
> Mario
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ZFS...

[Daniel Kalchev]

> On 30 Apr 2019, at 16:11, Karl Denninger  wrote:
> 
> 
> My experience is that ZFS is materially more-resilient but there is no
> such thing as "can never be corrupted by any set of events."  Backup
> strategies for moderately large (e.g. many Terabytes) to very large
> (e.g. Petabytes and beyond) get quite complex but they're also very
> necessary.
> 

I can only second that statement. Being paranoid with your data (keep many 
copies, have many backups) is never enough.

A colleague just complained the other day, that they lost a zpool and that ZFS 
didn’t save their data…. by not making a redundant pool and the hard drive  
trashing heads. And no backups. The unreadable part of the drive happened in 
metadata and the pool can not be imported.

I keep an HDD around, that since it was brand new, runs perfectly under any OS. 
Rock solid, that is… and only ZFS complains that it reads things back it didn’t 
write. Before that, I would think UFS was ok… since then, I don’t build a 
single installation, that does not have at least a mirrored ZFS pool. And 
“archive servers” (stands for backup) have become the central focus of my work. 
These are never enough..

Daniel

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: OpenSSH changes between 10.2 and 10.3 ...

[Daniel Kalchev]

host *
KexAlgorithms diffie-hellman-group1-sha1

in ~/.ssh/config works for me.

Daniel 

> On 14.04.2016 г., at 12:44, Patrick M. Hausen  wrote:
> 
> Hi, all,
> 
> minor problem/annoyance here:
> 
> root@noc:/etc/ssh # ssh admin@10.4.0.62
> Unable to negotiate with 10.4.0.62 port 22: no matching key exchange method 
> found. Their offer: diffie-hellman-group1-sha1,none
> root@noc:/etc/ssh # uname -a
> FreeBSD noc.pluspunkthosting.de 10.3-RELEASE FreeBSD 10.3-RELEASE #3: Wed Apr 
> 13 14:46:57 CEST 2016 
> r...@noc.pluspunkthosting.de:/usr/obj/usr/src/sys/GENERIC  amd64
> 
> Of course I was able to find http://www.openssh.com/legacy.html myself.
> 
> FreeBSD 10.2 uses OpenSSH 6.6.x while 10.3 imported 7.2.
> So far so good.
> 
> The recommended method from the document above works on the
> command line:
> 
>   ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 admin@10.4.0.62
> 
> But if I add
> 
>   KexAlgorithms +diffie-hellman-group1-sha1
> 
> to /etc/ssh/ssh_config, that does not change anything. Oddly enough,
> checking which algorithms are supported gives the same result
> regardless of any configuration options:
> 
> root@noc:/etc/ssh # ssh -Q kex
> diffie-hellman-group1-sha1
> diffie-hellman-group14-sha1
> diffie-hellman-group-exchange-sha1
> diffie-hellman-group-exchange-sha256
> ecdh-sha2-nistp256
> ecdh-sha2-nistp384
> ecdh-sha2-nistp521
> curve25519-sha...@libssh.org
> 
> So, diffie-hellman-group1-sha1 is supported but not used unless
> specified on the command line? And there is no way to override that
> *globally*? This is an isolated management network with IPMI
> interfaces - we won't be getting updates for all of these machines'
> IPMI firmware ...
> 
> Am I stuck with writing shell aliases or putting the config in each and
> every user's private ~/.ssh/config?
> 
> Thanks for any hints,
> Patrick
> -- 
> punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe
> Tel. 0721 9109 0 * Fax 0721 9109 100
> i...@punkt.de   http://www.punkt.de
> Gf: Jürgen Egeling  AG Mannheim 108285
> 
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: FreeBSD 9.1 ix driver vlan problem

[Daniel Kalchev]

On 25.09.2013, at 02:16, Rumen Telbizov telbi...@gmail.com wrote:

 
 
 Example:
 ifconfig vlan200 create  # this is OK
 ifconfig vlan200 inet 1.2.3.1/30  vlan 200 vlandev ix1 description DEBUG #
 this second line makes the rest of the vlans freeze for 6-7 seconds
 
 On the switch side (Juniper) the physical interface flaps. There's nothing
 in logs/dmesg.
 

Might be, you have spanning tree enabled on the switch port and when the 
interface flaps it needs time to converge. Disable spanning tree on the port 
and the reset will be very short.

Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 31.07.13 09:38, Shane Ambler wrote:

On 31/07/2013 01:31, Daniel Kalchev wrote:


But here is an idea: Remove BIND from HEAD overnight and see how many
 will complain ;-) If nobody complains, don't put it back in.


Or change the default to off. If you want bind add WITH_BIND=yes to 
src.conf


That is just as good solution as removing BIND from base. It is also 
easier and faster to ass it as package/point, instead of recompiling the 
whole base system.




It's hard to say FreeBSD is a safe and secure OS when part of the base
install is always being shown to have security flaws. New features need
to prove they are reliable before they are accepted into a release yet
we allow something that has a long proven history of being a source of
security concerns.


Stop right here! There is plenty of other software that is in base and 
is just as buggy or even more than BIND.
BIND, by the way benefits from the fact that it runs on many other 
platforms and that those bugs are typically found there, not on FreeBSD.
In contrast to that the perfect FreeBSD only code has bugs discovered 
only when someone stumbles on them in FreeBSD.




For something that needs to be constantly updated in between system
updates then ports is the place to install it from.


You don't have to update BIND constantly, especially if you are not 
using it. If you are using it, you will want it updated, no matter what.




I think it is less about whether bind is useful and needs to be in base
and more about should every user of FreeBSD be open to security issues
or should a user have the option to say yes I want potentially insecure
software on my machine. The ports system allows messages that make it
obvious to the user about security concerns.


You are reading too much into that messages. FreeBSD is not bug free, 
nor is any other piece of code.




How many people setup and use a FreeBSD machine without adding something
from ports or packages?


Anyone who can, does prefer to not install any ports. I have over a 
dozens servers (and a gazillion jailed instances) that don't have one 
single port installed. I find this feature of FreeBSD especially 
appealing and something we should keep.
By the way, for those inclined to ask me for statistics: this is my 
personal experience. It works for me. If you don't do that, it tells me 
nothing I care about. We might have different reasons to make different 
choices.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 31.07.13 15:22, Mark Felder wrote:

On Wed, Jul 31, 2013, at 6:15, Daniel Kalchev wrote:

On 31.07.13 09:38, Shane Ambler wrote:

For something that needs to be constantly updated in between system
updates then ports is the place to install it from.

You don't have to update BIND constantly, especially if you are not
using it. If you are using it, you will want it updated, no matter what.


Let's take a moment and consider the state of the internet and DNS
attacks. The RRL and RPZ2 patchsets[1] are newer developments that
successfully add additional security and features to BIND. It was also
recently announced that due to the success of this work the RRL[2] patch
will be accepted by ISC into BIND mainline.

How many users of BIND on FreeBSD are going to realize they need to run
a copy of BIND from ports to get this extremely important protection? It
certainly isn't going to get backported to 8-STABLE or 9-STABLE;


There is one solution to this, which I proposed earlier. Just don't 
ship/build the BIND binary by default. You will end up with only the 
resolver available and not be concerned with things like DDoS 
amplification. If you want an authoritative name server, just install it 
from ports.


Another solution is to include the appropriate warning in named.conf for 
anyone setting up name server on FreeBSD to read. In fact, text like 
this is already present in say, 6-stable's version (I know, that version 
is very outdated already):


/*
*
*   _  _ _ _ _   _ _ ___ ___  _ _ *
*  / \|_   _|_   _| | \ | |_   _|_ _/ _ \| \ | |*
* / _ \ | |   | | |  _| |  \| | | |  | | | | |  \| |*
*/ ___ \| |   | | | |___| |\  | | |  | | |_| | |\ |*
*   /_/   \_\_|   |_| |_|_| \_| |_| |___\___/|_| \_|*
* *
*

The version of BIND in the RELENG_6 branch (FreeBSD 6.x) is NOT suitable
for use with DNSSEC, either as a validating resolver or an authoritative
name server.  If you plan to use DNSSEC for any purpose you should use a
newer version of BIND, preferably version 9.6.x or higher.

Additionally, this version of BIND (9.3.x) is beyond its End Of Life (EOL)
date and is no longer supported by ISC.

Newer versions are available in the ports tree (e.g., /usr/ports/dns/bind96)
or by upgrading your FreeBSD installation to version 8.0 or higher.
*/

A better solution would be to apply the RRL patch to BIND in 8-stable 
and 9-stable. FreeBSD does ship a very controlled version of BIND in 
base and keeping it patched is trivial, in comparison with someone 
applying the patches themselves on original BIND sources that were 
just released (in a port). FreeBSD does apply patches to other software 
in base: for example ssh and the HPN patches.


Even if you personally prefer some other DNS resolver/server that won't 
replace BIND In 8-stable or 9-stable (which will live in the coming 
years and result in the same problems).
Every FreeBSD installation does benefit from an mature and full feature 
recursive resolver being available in the base system. What else than 
BIND you propose? Why is it better and ... most importantly, considering 
the topic of this thread: why you think it will not be subject to many 
new SAs over time?

For.. if we don't have anything better at hand, BIND will apparently stay.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 30.07.13 15:21, Mark Felder wrote:

People don't seem upset about not having a webserver, IMAP/POP daemon,
or LDAP server in base, so I don't understand what the big deal is about
removing BIND.


I believe the primary reason these things are not in the base system is 
that they have plenty of dependencies, with possibly conflicting 
licenses etc.



If the concern is over the rare case when you absolutely
need a DNS recursor and there are none you can reach I suppose we should
just import Unbound.


There are many and good reasons to include an fully featured name 
server, or at least full recursive resolver. For example, for properly 
supporting DNSSEC.
We could in theory remove the BIND's authoritative name server 
executable... if that is attracting the SAs.


The justification reduce the number of SA's, that is, the bad PR is 
probably not enough. Going that direction, we should consider Comrade 
Stalin's maxim FreeBSD exists, there are problems, here is the solution 
-- no FreeBSD, no problems! :-)


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 30.07.13 16:13, Mehmet Erol Sanliturk wrote:




On Tue, Jul 30, 2013 at 8:47 AM, Daniel Kalchev dan...@digsys.bg 
mailto:dan...@digsys.bg wrote:



Going that direction, we should consider Comrade Stalin's maxim
FreeBSD exists, there are problems, here is the solution -- no
FreeBSD, no problems! :-)

Daniel




Then , there exists a new problem :


There is no FreeBSD ...


We already know Comrade Stalin's solution had... bugs. Not before 
millions parted with their lives...


When/if we remove BIND from FreeBSD, we might find out whether that 
solution has bugs, or not. Not until then, though.


Back to the topic :)

My take on this is that removing BIND from the base today is.. 
irresponsible. First, most who use FreeBSD expect an DNS server to be 
readily available. Some people would just avoid to use any ports etc.
BIND in base is well tested and known evil. If we are ever to replace it 
with something else, that something else has to prove itself - 
demonstrate that it is at least as good as BIND -- in the base system. 
In practice, not in theory.


This is very much an situation like replacing gcc with clang/llvm. 
However, in the case of BIND we have no licensing problems, stability 
problems, performance problems etc --- just concerns that BIND generates 
many SAs -- which might be actually good indicator, as it demonstrates 
that BIND is worked on.


I personally see no reason to remove BIND from base. If someone does not 
want BIND in their system, they could always use the WITHOUT_BIND build 
switch.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 30.07.13 18:26, Peter Maxwell wrote:

On 30 July 2013 14:42, sth...@nethelp.no wrote:



Yes, I know everything can be installed from packages/ports. Two of
*my* main reasons for using FreeBSD is that:

1. It's an integrated *system*, not just a kernel.


That's not an argument for retaining something that is non-essential for
most people and can easily be installed from ports.  There is very little
that is actually essential in base... having to turn sendmail off on every
new installation already does my nut in but having mail facilities is
essential, so it has to be there.


I am surprised why so many people insist having an MTA is necessary, but 
having well testes recursive DNS resolver is not.
Even on a typical client installation, it is more likely the resolver 
will be useful, than the MTA.


By the way, both sendmail and BIND are off by default...


Having bind in base does have one advantage in that it is more carefully
scrutinised that it would likely be in ports.


This too..

I have always viewed FreeBSD not as an product, but instead as an 
toolkit. A toolkit, from which to build the OS you need.
So far, FreeBSD has worked better for that purpose than any other 
toolkit around (plus, I am biased).


There are a number of knobs, that let you customize FreeBSD to your 
heart's content.


In theory, everything but the absolute minimum of the base system might 
be removed.. and have everything depend on ports. However, the base 
system is just that -- one collection of code that gets built and tested 
together. This brings quality.


Having said this, it is perfectly ok to replace BIND with any other 
resolver + name server as long as there is suitable candidate that 
has passed enough testing. Is there one? Do we know enough of their quirks?


Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 30.07.13 16:44, Ronald Klop wrote:
On Tue, 30 Jul 2013 15:32:44 +0200, Daniel Kalchev dan...@digsys.bg 
wrote:




Back to the topic :)

My take on this is that removing BIND from the base today is.. 
irresponsible. First, most who use FreeBSD expect an DNS server to be 
readily available.


Interesting. What are your statistics of 'most' based on?


Unfortunately, not much objective statistics. The bsdstats sample is 
rather small and obviously biased (towards people who would share their 
config, mostly).


I was hoping for some usable data from the Open Resolver Project 
(http://openresolverproject.org/)but there is not much useful 
information for this purpose there either. It is also very unlikely a 
pool would result in any meaningful data...


But here is an idea: Remove BIND from HEAD overnight and see how many 
will complain ;-)

If nobody complains, don't put it back in.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Bind in FreeBSD, security advisories

[Daniel Kalchev]

On 30.07.2013, at 19:49, Peter Maxwell pe...@allicient.co.uk wrote:

 I personally prefer qmail over sendmail
 but I wouldn't suggest qmail should be in base for the reason that sendmail
 is the de facto standard on *nix shaped systems.
 

One can argue that BIND is the de facto standard on *nix shaped systems too.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: zpool labelclear destroys GPT data

[Daniel Kalchev]

On 14.06.2013, at 19:16, Tom Evans tevans...@googlemail.com wrote:

 
 I suppose if labelclear was made to check for the existence of a
 pre-existing ZFS label, the force flag could be used to force the
 change… I still don't like it, the command is not called
 labelclear-only-if-the-label-area-already-has-sane-data.

It apparently does, because zpool labelclear would refuse to wipe a label that 
it believes belongs to an active zpool. So it apparently does check data. Just 
applies some weird logic.

The logic is also flawed and annoyed me few times. For example, you want to 
remove a drive from an raidz. You do this by marking the drive offline and 
wiping it's ZFS labels, so that you can repurpose it. However, as long as the 
zpool is not exported, zpool labelclear  refuses to wipe labels and you re 
forced to resort to dd.
Why would it refuse to clear labels on an offlined drive? Is there better 
method?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: recommended memory for zfs

[Daniel Kalchev]

On May 10, 2013, at 1:08 PM, Ronald Klop ronald-freeb...@klop.yi.org wrote:

 On Fri, 10 May 2013 04:18:42 +0200, Benjamin Adams benjamindad...@gmail.com 
 wrote:
 
 On 05/09/2013 10:06 PM, Jeremy Chadwick wrote:
 On Thu, May 09, 2013 at 09:47:27PM -0400, Benjamin Adams wrote:
 On 05/09/2013 08:53 PM, Shane Ambler wrote:
 On 09/05/2013 22:48, Benjamin Adams wrote:
 Hello zfs question about memory.
 I heard zfs is very ram hungry.
 Service looking to run:
 - nginx
 - postgres
 - php-fpm
 - python
 
 I have a machine with two quad core cpus but only 4 G Memory
 
 I'm looking to buy more ram now.
 What would be the recommend amount of memory for zfs across 6 drives on
 this setup?
 
 I believe I heard a calculation of 1GB cache per 1TB of disk. But
 basically zfs will use all free ram available if you access that
 much data from disk. You will want to set vfs.zfs.arc_max to allow
 enough ram for your apps to work in.
 
 If you consider the files for your website and the data you store
 you may find that you would never fill more than 500MB of cache.
 
 If you will be serving large media files that will easily use up
 the cache you could give them their own filesystem that only
 caches metadata - zfs set primarycache=metadata zroot/mediafiles
 
 
 Thanks for all the replies  Size of DB and HD's are:
 
 Current DB Size = 23 GB
 HD sizes = (6) 500 GB drives
 Nobody is going to be able to give you a precise/accurate recommendation
 given the lack of detail provided, I'm sorry to say.  What's the RES
 size of nginx (all processes combined)?  What's the RES size of
 postgres (same)?  Do you have PHP scripts that run amok for long
 periods of time and take up lots of RAM?  Same with python?  How many
 concurrent visitors and what sort of content are you hosting?  Do you
 maintain/write your own PHP/Python code or are you using some crap like
 Wordpress?
 
 This is just a **small** list of questions -- and what may come as a
 shock is that I do not expect you to provide answers to any of them.
 They are questions that you should, for yourself, attempt to answer and
 work out what you need from there (teach a man to fish and all that).
 
 The advice of 1GB of RAM per 1TB of disk space is absolute nonsense on
 numerous levels -- whoever gave this advice to Shane either has no
 understanding of how filesystems/ZFS works, or does but chose to
 simplify to the point where they're providing half-ass information.
 There is no direct, or even indirect, correlation between disk capacity
 and ZFS ARC size -- what matter is your working set (to quote Tom).
 You need to have some idea of how much disk I/O you're doing, and what
 type of I/O (sequential or random).
 
 If you want my general advice, Benjamin, it's this: get yourself a
 system with *minimum* 8GB of RAM but has the physical possibility of
 supporting more (and only add more RAM when/if you know you need it); do
 not bother with ZFS on a system with 4GB.  Run amd64, not i386 (I don't
 recommend bothering with ZFS on i386 -- I am not going to get into a
 discussion about this either).  Run stable/9, not 9.1-RELEASE.  Avoid
 compression and dedup.  And test disk failures as well (don't get caught
 with your pants down later).
 
 The above advice comes from someone who did hosting (web/ssh/etc.) for
 almost 20 years with KISS principle applied at all levels.  YMMV though,
 depending on what all you're doing/what you truly need.
 
 Good luck.
 
 Jeremy,
 
 Was just see if I should just get raid controller and more ram down the road.
 List of priorities.
 
 Main thing is I move from BSD when 9.0 came out.  Was looking to see if zfs 
 is included in the installer.  Now.
 
 Sum up:
 upgrade ram to 16GB (not 64 like plained)
 and  raid controller that supports level 5.
 
 
 Let ZFS do the RAID stuff. Do not use a RAID controller, but give the plain 
 disks to ZFS. Some of the nice features come from ZFS doing the RAID stuff.

To paraphrase this.

Get yourself a nice HBA. non-RAID! For example, something based on LSI2008 with 
IT firmware. With few enough discs you can avoid using SAS expanders as well. 
RAID controllers, in addition to causing all kinds of troubles are unlikely to 
have sufficient bandwidth and might turn out to be the bottleneck (unless you 
are prepared to spend an unholy amount of money -- which are better spent for 
RAM and CPU).

If you want performance and low latency, avoid using compression and dedup in 
ZFS. Set your record size appropriately for postgresql (8k) *before* you run 
initdb. It is best to create an separate filesystem for the database and set 
that property only there. If your database is heavy on updates, you might be 
interested to use an SSD for ZIL. In general, if you can afford it, an cheap 
SSD for L2ARC might do wonders -- if your data set can fit there. If you intend 
to use SSDs and want the best performance, use different SSDs for ZIL and 
L2ARC. The first needs to be fast at writing and optimised for his (for 
example, the 

Re: ZFS stalls -- and maybe we should be talking about defaults?

[Daniel Kalchev]

On 06.03.13 02:42, Steven Hartland wrote:


- Original Message - From: Daniel Kalchev

On Mar 6, 2013, at 12:09 AM, Jeremy Chadwick j...@koitsu.org wrote:


I say that knowing lots of people use ZFS-on-root, which is great -- I
just wonder how many of them have tested all the crazy scenarios and
then tried to boot from things.


I have verified that ZFS-on-root works reliably in all of the following
scenarios: single disk, one mirror vdev, many mirror vdevs, raidz.
Haven't found the time to test many raidz vdevs, I admit. :)


One thing to watch out for is the available BIOS boot disks. If you try
to do a large RAIDZ with lots of disk as you root pool your likely to
run into problems not because of any ZFS issue but simply because the
disks the BIOS sees and hence tries to boot may not be what you expect.


A prudent system administrator should understand this issue and verify 
that whatever (boot) architecture they come up with, is supported by 
their particular hardware and firmware. This is no different for ZFS 
than for any other case.


The 2nd stage boot from ZFS loader in FreeBSD could in fact end up with 
it's own drive detection code one day, which will eliminate it's 
dependence on BIOS at all. For relatively small systems, where the 
administrator might be careless enough to not consider all scenarios, 
today's BIOSes already provide support for enough devices (e.n. most 
motherboards provide 4-6 SATA ports etc).


Using separate boot pools of just few devices is what I do for large 
storage boxes too. Mostly because I want to be able to fiddle with data 
disks without caring that might impact the OS. Just make sure the BIOS 
does see these in the drives list it creates. That is, don't put the 
boot disks at the last positions in your chassis :) -- use the on-board 
SATA slots that are scanned first -- sadly, almost every vendor provides 
for such drives placed inside the chassis, which makes it very 
inconvenient if one of the drives dies.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS stalls -- and maybe we should be talking about defaults?

[Daniel Kalchev]

On Mar 6, 2013, at 12:09 AM, Jeremy Chadwick j...@koitsu.org wrote:

 I say that knowing lots of people use ZFS-on-root, which is great -- I
 just wonder how many of them have tested all the crazy scenarios and
 then tried to boot from things.

I have verified that ZFS-on-root works reliably in all of the following 
scenarios: single disk, one mirror vdev, many mirror vdevs, raidz. Haven't 
found the time to test many raidz vdevs, I admit. :)

Combined with boot environments (that can be served many different ways), ZFS 
on root is short of a miracle.

ZFS on FreeBSD has some issues, mostly with huge installations and 
defaults/tuning, but not really with ZFS-on-root.

Of course, if for example, you follow stable, you should be prepared with 
alternative boot media that supports the current zpool/zfs versions. But this 
is small cost to pay.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS stalls -- and maybe we should be talking about defaults?

[Daniel Kalchev]

On Mar 5, 2013, at 8:17 PM, Freddie Cash fjwc...@gmail.com wrote:

 
 ZFS send/recv would eventually complete, but what used to take 15-20
 minutes would take 6-8 hours to complete.
 
 I've reduced the ARC to only 32 GB, with arc_meta set to 28 GB, and things
 are running much smoother now (50-200 MB/s writes for 3-5 seconds every
 10s), and send/recv is back down to 10-15 minutes.
 
 Who would have thought too much RAM would be an issue?
 
 Will play with this over the next couple of days with different ARC max
 settings to see where the problems start.  All of our ZFS boxes until this
 one had under 64 GB of RAM.  (And we had issues with dedupe enabled on
 boxes with too little RAM, as in under 32 GB.)

I have an archive box running very similar setup as yours, but with 72GB of 
RAM. I have set both arc_max and arc_meta_limit to 64GB, with no issues. I am 
still doing a very complex snapshot reordering between two pools. One of the 
pools has dedup enabled (which prompted me to add RAM), with dedup ratio f over 
10x and there are still no issues or any stalling. The other pool has both 
dedup and compression for some filesystems. 

My only issue is that replacing a drive in either pool takes few days (6-drive 
vdevs of 3TB drives).

Perhaps the memory indexing/search algorithms are inefficient?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: RFC: Suggesting ZFS best practices in FreeBSD

[Daniel Kalchev]

On 25.02.13 18:44, Karl Denninger wrote:

On 2/25/2013 8:31 AM, Tom Evans wrote:

Using GPT labels is easy to do, and provides a cast iron guarantee
that your disk will not EVER be mistaken for a different drive.

I put a GPT label on the drive, and then write it in permanent marker
on the top of the drive and on a sticky label that is stuck on the
front of the chassis. The disk label never changes in its lifetime, so
you only get issues if you insert a drive without labelling it first.

Cheers

Tom


Listen to this man.  Either do it in gpart or do it with glabel
(depending on where you want it to show up); once done the drive will
always have the same name in the device tree no matter what slot it
shows up in.

Then stick that label on the front of the drive carrier, and you never
have this problem.

Do that or suffer.  Your choice.



I can only second this advice.

Learn to make your setups as generic as possible. Wiring down device 
names using CAM etc, only complicates matters, at least because that 
knowledge is contained within the system you boot, not the drive. If you 
boot from an recovery media, all of your fine crafted CAM wire-down 
system will be gone and you will suffer. You will suffer exactly at the 
moment when you need those labels most. Not wise.

Yes, Jeremy, there is always the first time.

Both glabel and GPT labels do the trick. Both behave differently and 
depending on your habits and intended usage one or the other is good. 
Just don't be inclined to use both at the same time. My personal 
preference as of later is GPT labels.


By the way, glabel is a generic term in FreeBSD. It refers to all 
label types, including those created with glabel, gpart and 
newfs/tunefs. The later are not really geom labels, these are volume 
labels, but glabel is smart enough to detect/report them, even if it 
can't manipulate these labels. I don't find the glabel documentation all 
that confusing.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why can't gcc-4.2.1 build usable libreoffice?

[Daniel Kalchev]

On 21.02.13 04:23, Greg Miller wrote:

On 2/20/13, Matthias Andreematthias.and...@gmx.de  wrote:

What is your point, besides getting software from the museum to build
stuff from the relative future?

I can't speak for the OP, but I tried it because clang, gcc46, and
gcc47 wouldn't produce a working executable at all for a long time
(and continue to fail) on my 9.0 and 9.1 systems. There's been so much
libreoffice breakage that I don't even bother reporting it or making
much effort to fix it. I just reboot to Windows for the cases where I
need a working libreoffice. I don't much care whether gcc 4.2 produces
a working libreoffice; I just wish something did.


Did you build the Windows version yourself from source?

If not, why you just don't get the binary WhateverOffice for FreeBSD and 
be done with this problem? That will surely save you the reboots. At 
least.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why can't gcc-4.2.1 build usable libreoffice?

[Daniel Kalchev]

On 19.02.13 20:54, Mikhail T. wrote:
My complaint is that, though the port works out of the box, the 
office@ maintainers have given up on the base compiler too easily -- 
comments in the makefile make no mention of any bug-reports filed with 
anyone, for example. It sure seems, no attempts were made to analyze 
the failures... I don't think, such going with the flow is 
responsible and am afraid, the inglorious days of building a special 
compiler just for the office will return...


Neither of these best open source office suites is supposed to be 
built from source, by the normal user. As already mentioned, normal 
users are guided to use the pre-compiled binaries. The reasons for this 
are many and different. Only one of the reasons is that those ports are 
rather complex and let's not forget it - buggy. They more or less 
require special build environments, which are easier provided, as you 
guessed it, by an purposely configured compiler. Since the ports 
themselves are huge, compiling an relatively small compiler for the 
purpose to build the rest is ok. Count it as 'bootstrap' process. I for 
one, don't buy your argument that the makefile lacks enough evidence 
of why certain choices were made - it is an file with instructions for 
the computer, after all. Humans discuss these things at other places.




Am I really the only one here disturbed by the fact, that the 
compilers shipped as cc(1) and/or c++(1) in our favorite operating 
system's most recent stable versions (9.1 and 8.3) are considered buggy?


As already mentioned, the compilers in the base exist in order to 
compile FreeBSD and bootstrap other compilers. For that purpose, even 
the ancient gcc does the job. It even does the job for many, many ports 
as well. Nobody has ever made the promise that the base cc will compile 
any source code thrown at it.


Because it is buggy and because newer versions have different license, 
that doesn't fit well with FreeBSD, gcc is being phased out from FreeBSD 
and replaced by llvm/clang. Still a work in progress and might not be 
complete for 10.0.



On 19.02.2013 13:05, Adrian Chadd wrote:

.. I think the compiler people just use the port as compiled with the
compiler that is known to work with it, and move on.


Such people would, perhaps, be even better served by an RPM-based 
system, don't you think? But I don't think so -- the amount of OPTIONS 
in the port is large, and a lot of people are likely to build their 
own. Not because they like  it, but because they want a PostgreSQL 
driver or KDE4 (or GTK3) interface or...




This is why it exists as source code and FreeBSD port. I myself build 
all software from source, whatever it takes. And if it requires that I 
have dozen of special-purpose gcc versions built in the process, I don't 
care.


For people with less resources and patience, there is the precompiled 
binary package. An RPM-like technology.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: svn - but smaller?

[Daniel Kalchev]

On 23.01.13 21:09, Peter Wemm wrote:

On Wed, Jan 23, 2013 at 9:05 AM, Isaac (.ike) Levy
i...@blackskyresearch.net  wrote:


1) License.  Many of SVN's dependencies will never be available in the FreeBSD 
source.
While this is totally OK for development, SVN is 3rd party software, this is 
unacceptable to force as 'the' respected path for OS source builds.

Don't confuse the excessive ports default settings as dependencies.
You can make a quite mean and lean svn client.  I did a 100%
BSD-license-compatible src/contrib/svn style proof-of-concept back
when we were planning what to do.  Things like gdbm and bdb are not
required and are license contamination that we don't need.  But that's
the fault of the port, not a fundamental property of using svn.



The logical question is then: Why is this slimmed down, fully BSD 
license compatible svn not in the base system by now?


It is absurd to require the installation of any port, if your only 
intention is to update the base system sources.


Portsnap is an entirely different mess.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Spontaneous reboots on Intel i5 and FreeBSD 9.0

[Daniel Kalchev]

On 25.01.13 21:44, Marin Atanasov Nikolov wrote:
Btw, after removing all ZFS snapshots today (more than a 1k) the 
system is still running (not something I can say for the past few days 
where I've seen multiple reboots a day).. But it's still early to say 
that the snapshots might be causing this :)


Might be. If you have something in your daily scripts that traverses 
each and every snapshot (filesystem). I believe this issue with ZFS was 
fixed some time ago. But you might have leftover config files that still 
trigger it.


You could have tried disabling some of the daily scripts to see which 
one triggers the problem.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ULE Scheduler

[Daniel Kalchev]

On 12.06.12 16:08, Momchil Ivanov wrote:
So the L2 cache is shared between both cores and hence it's size does 
not matter at all?


If the cache is shared between both cores then it does not matter on 
which core the process runs, as long as data is in teh case. The cache 
size is irrelevant.


Some CPUs have shared cache between cores, some don't. The ULE scheduler 
takes this into account, the 4BSD does not. Even if the ULE scheduler 
takes the CPU topology into consideration, if you only have two cores, 
it is almost guaranteed that processes will be switched between both, 
because the OS is running way more than two processes at the same time.


Even with more cores... it is not guaranteed an computational process 
won't be 'bouncing'. Here is an example.
Suppose you have an 8 core (or threads) CPU. If you happen to have an 
modern Ethernet controller, like the Intel 82576 (the igb driver in 
FreeBSD), then it will use up to 8 interrupt lines, by default routing 
them each to a different core. Then, if you have heavier network 
traffic, chances are that at any given moment all 8 interrupts might be 
fired and all 8 cores switched to service network traffic -- removing 
your computational process from the running queue. The next time it 
runs, it might run on any other core, especially if the cache is not shared.


Of course, if you have sufficiently large number of CPUs, you can 
configure your system so that such things do not happen, like by 
limiting the number of cores the igb driver attaches to, and have some 
of the cores dedicated to 'only' running an computational task.


There is however, very little sense doing so.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Documenting ports options (was Re: Why Are You NOT Using FreeBSD ?)

[Daniel Kalchev]

On Jun 12, 2012, at 2:41 AM, grenville armitage wrote:

 
 
 Rainer Duffnerrai...@ultra-secure.de  writes:
   [..]
 Personally, I don't need more frequent FreeBSD-releases but two or
 maybe three ports-tree freezes per year would be good.
 
 Perhaps not so much freezes per se, but if there are particular
 dates at which the ports tree is known to compile properly (for
 some preferred definition of 'properly') those dates could be
 kept in a list somewhere, for people to use with the cvsup
 date= option?


I believe the reason this is not happening is that there is no date, when the 
ports tree does build all ports just fine. Some of the ports are not 
compilable if you compile other ports, or select certain options in other ports 
as well.

For example, you might have a date, when KDE4 compiles and runs, just fine. But 
at the same date, you cannot say the same for say Gnome, or science/meep 
(random pick).

It is of course doable. The reason nobody is doing it is because by the time 
you have stable ports tree lots of software in there and more importantly 
most of the mainstream software in there that sees active development is 
already out of date and sometimes with unattached security problems.

There is a fundamental misconception of what the ports tree is. This is not the 
ready made software, where you just user portmaster/portinstall to add new 
software or you go to the port's directory and type make install.

The ports tree is a collection of instructions how to build foreign to FreeBSD 
software, plus the necessary infrastructure and few common sense options and 
that is it. If you view it any other way, you are in trouble.

The pick and install software functionality does not really exist in FreeBSD. 
The closest is to use packages and yet closer is the packaging system found in 
PC-BSD that uses the Apple style fat app approach.

It is more appropriate to view FreeBSD and ports as the tools to build your own 
OS (in the sense that most people understand it) with functionality, tuning 
and packages you need.

Of course, the ports system can be improved and is in fact, all the time.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Netflix's New Peering Appliance Uses FreeBSD

[Daniel Kalchev]

On 06.06.12 03:16, Scott Long wrote:

[...]

Each disk has its own UFS+J filesystem, except for
the SSDs that are mirrored together with gmirror.  The SSDs hold the OS image
and cache some of the busiest content.  The other disks hold nothing but the
audio and video files for our content streams.


Could you please explain the rationale of using UFS+J for this large 
storage. Your published documentation states that you have reasonable 
redundancy in case of multiple disk failure and I wonder how you handle 
this with plain UFS. Things like avoiding hangs and panics when an 
disk is going to die.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Documenting 'make config' options

[Daniel Kalchev]

On 07.06.12 08:27, Dave Hayes wrote:

Personally, a 'pkg-options-descr' text file would suit me just fine.


I have considered this lack of information about port options myself. 
Sometimes, when installing new (to your understanding) software 
finding out what those options actually do and what are the real 
implications is very hard. The same situation, even worse happens when 
you update an port and the new version has introduced yet new options.

Sometimes, hints on what those do end up in /usr/ports/UPDATING.

It is clear, that something has to be done. Unfortunately, you can't 
just force port maintainers to document it, because of many reasons. One 
not very obvious reason is that many ports use pretty much 'generic' 
options, like WITHOUT_CUPS. For most such ports, this means I don't 
want cups on my system, but in few it might mean something different. 
Finding adequate way to document all this is a bit tricky. Good ideas 
are welcome.


The idea Warren Block has is very useful and I fully agree that there 
should be some option to reset to default. It would be also extremely 
helpful if the dialog UI indicates somehow which options are set to 
something different from the default. Lacking this information is a big 
waste of time, especially when upgrading older ports. Or having the 
saved options lying around from an previous install of that port.


At least, some way of mandatory describing of what setting particular 
option for a port does, outside of what is found in the Makefile and in 
plain English will be very, very useful.


[off topic]
By the way, on the waste of time. I view it a bit differently, when it 
comes to internet mailing lists. Suppose I waste 30 minutes of my time 
researching and preparing an response to a stupid question. It is 
apparently my decision to do so and nobody is blaming anyone. But 
imagine, this mailing list is read by tens, hundreds, thousands or even 
tens of thousands people (Google searches not accounted for). All those 
people are going to read the stupid question and the trivial answer. 
This is huge waste of time. Doesn't help that all those people are 
willingly wasting their time.
Of course, we need discussions like this as a lot of people learn new 
things. Just the proper balance sometimes is difficult to achieve. But 
those who ask, will do better for everyone to try to comprehend what 
they have been told, before continuing their jihad.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ULE Scheduler

[Daniel Kalchev]

On 07.06.12 11:16, Momchil Ivanov wrote:
Though, it was strange seeing both processes hopping around... I will 
probably go back to the 4BSD scheduler if my laptop does another 
self-shutdown in the next few days as Doug suggested.


You never run just two processes on FreeBSD, ever. The kernel too runs 
multiple threads.


However small the CPU usage of the other processes is, they must run 
from time to time, kicking out at least one of your CPU intensive 
processes, possibly kicking them out both, as well. When that happens, 
and they are queued to run again it does not matter much on which core 
they ran before, because chances are it's cache will be invalidated 
anyway. Also, different CPUs have different cache affinity. ULE is 
supposed to be aware of this, while the 4BSD scheduler is not.


In any case, on an older single/dual core CPU there is rarely any 
difference between both schedulers. Differences might appear in modern 
multi-core CPUs..


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD?

[Daniel Kalchev]

On 06.06.12 05:28, Erich wrote:
Why should a normal user continue to search for a tag when the 
handbook is so clear on this? Erich


I continue to wonder, why are you searching for tags on the ports tree, 
when you were told on a number of occasions that those who depend on 
particular state of the ports tree use DATE.


There is not much point in tagging the ports tree, because it is never 
'released' as such. You will end up with millions of tags and sorting 
out which one you need will become difficult. Further, you are not 
advised to use an not-current ports tree, unless you know exactly what 
you are doing. If you know what you are doing, you are not likely to ask 
questions like these. (*)


The ports tree is a collection of instructions how to compile and 
install particular software on FreeBSD. Don't think of the ports tree in 
any other way.


Daniel

(*) I gave earlier the example of how BSDRP builds. It's build script 
pulls a version of the ports tree at certain date. Then compiles and 
installs a number of ports from there. The project uses a bunch of 
networking tools and nobody cares if the version of KDE, LibreOffice or 
the PNG library is broken in that particular version of the ports tree. 
They do care, great deal, if the version of net/quagga for example, in 
that particular ports tree version is broken. In any case, when pulling 
the ports tree, they do not care about any particular tag, but specify 
an date. The date, when the ports were tested to be ok.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 04.06.12 22:32, Dave Hayes wrote:

Chris Nehrenapeiron+freebsd-sta...@isuckatdomains.net  writes:

The descriptions of the options assume the admin is familiar with the
software they're installing. I do not think it is the FreeBSD Project's
purview to document every option for every port. At the very least it'd
take quite a lot of time and effort to document all of that.

That's a fair position. Perhaps it would not be too much trouble to add
this one idea to optionsng: a more info field on each option knob
which may be filled in by a port maintainer.


The pkg-descr file in the port already contains link to the software's 
origin. The various options the software has are or should be described 
there. We definitely don't want the ports cluttered with extraneous and 
sometimes out of date (and thus misleading) information.



Beyond this, such explanations would duplicate each port's own
documentation.

Not necessarily. I don't have an example offhand, but I suspect there
are a number of FreeBSD specific option knobs applied to ports.


There are in a way, and all of them are pretty much generic. Like 
WITHOUT_X11, WITH_CUPS etc. The purpose of these options is to more or 
less define the environment in which the port is intended to be used. 
For example, on a head-less server you most definitely want to build 
(say) php5 with WITHOUT_X11 in order to not pull unnecessary X11 related 
pieces. The intent of such options is to go to make.conf.


These options are for convenience however. You can set each port's 
options individually. In all case, compiling from source is not for 
those having no clue what they do. The ports infrastructure in FreeBSD 
is already doing the hard work to port the software to your OS, you need 
to make informed decisions on options yourself.
If this is beyond you (and not you personally), then by all means use 
pre-packaged software in binary form.


Since it is very likely that you interpret this as yet another elitist 
comment, let's make it clear: anyone is welcome to ask for help with 
FreeBSD and ports (in the proper mailing list as to not create much 
noise and get negative response). Nobody is obliged to provide any help 
on anything. Nevertheless, the FreeBSD users are great community and you 
are often getting help even for the most stupid questions. Except when 
you start with name calling, or insist if you don't help me, I will go 
elsewhere or apparently, you don't want the number of FreeBSD users to 
grow. Then you waste everyone's time -- that could be spent on 
answering other people's stupid questions.



Daniel
Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 05.06.12 07:33, Zane C. B-H. wrote:

[on Exchange wiping devices]

 From a enterprise perspective, it makes sense. Lets say a device goes
missing, it allows one to wipe it the next time it calls home.


This is supposed to be handled by the device management software. Not by 
your e-mail server. Because it does not belong to the mail server, this 
is why you will not find this functionality implemented in any open 
mail server or calendaring or groupware software.


As you involved enterprise and your previous statement on Apple, they 
surprisingly do have such device deployment and management solution. 
You can either use their own Apple Configurator, or any third party MDM 
as described in http://www.apple.com/ipad/business/integration/mdm/. I 
would not call this technique proprietary. Ok, it only works on iOS ;)



The usefulness of such a feature is better disconnected from the
debate of proprietary v. non-proprietary though, given the different
nature of both issues.


With this I fully agree. I hope you too agree, than when you disconnect 
the e-mail handling features of Exchange, from the lock-in technology 
Microsoft integrated there (ActiveSync), Exchange is no different than 
any other non-proprietary mail server.


The point here is that in order to have more freedom, one has to set 
expectations and setups right, from the begining. Separate the MDM and 
e-mail functionality and you are no longer locked with Microsoft or 
anyone. You can easily replace each component of your system and use the 
best software for the task. Not make compromises.


Daniel

PS: Yes, I don't like Microsoft's offerings. I understand why they do 
this, but this doesn't mean I agree and since I have plenty of other 
choices... I prefer the best. My enterprise(s) have been running on BSD 
UNIX for good over 20 years now. No Microsoft stuff. Not needed, not missed.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD?

[Daniel Kalchev]

On 03.06.12 07:24, Erich wrote:


isn't this what I just suggested to be done by the team? Give the ports tree a 
new version number and people can fall back to this then.

Isn't this solution too simple to be done?


As was mentioned earlier in this discussion, by virtue of the ports tree 
being hosted on CVS, you are able to get a version of the ports there at 
any date you chose. Just set


PORTS_DATE=date=2012.06.01.00.00.00

to get the ports tree as it was on midnight 1st of June 2012. You can 
specify hours, minutes, seconds if you need. Way more powerful than any 
version number thing.


As you can see, this is already available with FreeBSD. A lot more 
hidden gems are available with FreeBSD. People are just lazy and for 
the most part, refuse to learn. This is one aspect FreeBSD could benefit 
greatly from more education of the 'users' -- just because it has way 
more hidden gems than anything else around.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 04.06.12 05:24, Dave Hayes wrote:
Anyway, given my workload, it will probably take me a man week to get 
two virtualized test servers. Someone I know with a vmware gui and 
windows is doing this in 15 minutes (and that's being careful). Just 
my $0.02. 


You are unfortunately comparing apples with oranges here.

If you want true comparison, compare how fast you will have VirtualBox 
OSE up and running on both FreeBSD and Windows. Both of you start with a 
system where it has to be compiled and installed. I guess your Windows 
friend will stop at compiler? what?.
You can't get the source code of vmware and compile it yourself, of 
course.. that's just another little detail.


Not the same? They get the thing pre-compiled? So could you.

Thing is, once you go trough the trouble to install VirtualBox on 
FreeBSD you get a lot more usable vrtualization platform, with things 
like ZFS that aren't going to be available on Windows.


It was mentioned a number of times already, that if you want to run 
binary only you would be better with PC-BSD -- and system based on 
FreeBSD (so it has most, but not all of the goodies), and someone else 
pre-compiles and pre-packages software for you. Just one click install.


So, if you used PC-BSD, you could have had VirtualBox running perhaps 
for the same time an Windows user would.


There is place for binary-only systems and systems where you are able to 
rebuild everything from source. FreeBSD tends to focus on the later 
while various folk (like PC-BSD) use the great FreeBSD platform to offer 
easier to use binary only systems. Of course, you could use the FreeBSD 
ports tree and build from source on PC-BSD too.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 04.06.12 18:04, xenophon\+freebsd wrote:

-Original Message-
From: owner-freebsd-sta...@freebsd.org [mailto:owner-freebsd-
sta...@freebsd.org] On Behalf Of Daniel Kalchev
Sent: Saturday, June 02, 2012 12:42 AM

I really see no reason why your 'mail or calendaring server'
should be able to wipe your devices.. This is the sort of bloat
that keeps me away. From Microsoft products.

I don't think that's fair to say.  Email/calendaring seems to be the
only connection point between a smartphone and an organization for at
least the current crop of devices (although I'm sure that at some point
soon, you'll be able to include organizational file servers as well).


Again, what does your e-mail or calendaring service have to do with 
wiping your device clean?? Wiping the device is task for your device 
management platform, which does not belong to the e-mail or calendaring 
platform. If you connect your desktop to Exchange, is it supposed to be 
wiped too? What if the  Exchange account is just one of the many e-mail 
accounts you use, as typically is the case?




Even if you're just a SOHO or SMB, you should want to be able to locate
or remotely wipe a device that's stolen, if only to ensure that someone
doesn't have access to potentially sensitive personal information.  Oh
and by the way, not only do the Windows phones feature this, but so do
the iPhones and the Android handsets - so this isn't just Microsoft.


I understand you don't like it, but apparently Apple got this right. 
They have device management tool that is in no way ties to your e-mail 
or calendaring server. Not only Apple, but any sane vendor too.


It is not excuse that because some (censored) at Microsoft has designed 
things this way, there are no other proper ways.



In this regard I rather prefer the way Apple handles things.
Shiny wrapper interface to pretty much generic technology. No
reinvention of the wheel and experiments to see if it can be made
square.

You can't damn Microsoft for being too proprietary in one paragraph and
then praise Apple for its openness in the next.  Does not compute.


I don't care how proprietary an proprietary thing is. If it is correctly 
implemented, it is ok, if it is not correctly implemented, it is not ok. 
Microsoft's wipe trough Exchange is weird, to put it mildly.
Apple too had a track record of doing many proprietary things, but in 
recent years their offerings are, as I mentioned earlier, pretty much 
generic standard and widespread protocols with a lot of sugar coating.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 02.06.12 09:23, David Magda wrote:

On Jun 2, 2012, at 00:51, Daniel Kalchev wrote:


On 02.06.2012, at 07:19, Freddie Cashfjwc...@gmail.com  wrote:


Glustre sits above the storage system, replicating data between systems.
So, disks -- ZFS (via Zvols) -- Glustre.


How is this different than ZFS using remote zvols via iSCSI? Can it tolerate 
down nodes better than ZFS?

Gluster ~ NFS++.


So Gluster is basically ZFS with NFS frontend? Something readily 
available in FreeBSD. The clients don't even have to learn new file 
sharing protocol.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 02.06.12 12:27, O. Hartmann wrote:


1a) On scietific production systems, FreeBSD has been banned due to
the lack of HPC compilers and appropriate mathematical libraries. The
lack of professional/academic support, like that from NAG in the late
1990s, has been droped for FreeBSD as well as the presence of C/C++/F95
compilers.


Could you please elaborate on this a bit. Scientific software, as such 
rarely depends on any hardware and should be practically OS agnostic. 
What are these libraries, that scientific production systems use that 
cannot be used on FreeBSD? Same about support. If someone supports an 
scientific library on an OS, chances are they can support it on 
FreeBSD just as well, except if they are religious fanatics, that is.


FreeBSD has always had more or less recent compilers available. Perhaps, 
not in the base system. As you say, this issue is strictly political (to 
not have cuitting edge [double the quote, for more pun] compilers). 
The policy with FreeBSD is stability over all else and that the base 
must be able to compile itself -- this is what any UNIX system is 
supposed to handle, but that's another long story. The recent 
developments with clang/llvm are very promising as well.


I can hardly imagine it being that difficult to maintain an advanced 
compiler around just to compile your highly specific code. Further, 
recent gcc is in ports anyway.



1b) The lack of GPGPU. This has become so important to HPC these days.
We use nVidia GPU based TESLA boards with OpenCL software (CUDA is
luckily not necessary). The lack of professional drivers for 64Bit on
FreeBSD was long time an issue, nVidia now provides drivers, but they
don't provide their CUDA/OpenCL libraries along with their nvcc compiler
natively for 64Bit FreeBSD/amd64. The Linuxulator isn't any option.


This one is regrettable. Outside of the scientific usage, it could let 
desktop users offload a lot of processing to their (in most cases more 
powerful than the CPU, video controller). But how is this FreeBSD fault?
I would attribute it more to inability of nVidia programmers, or their 
lack of resources (I doubt that many people do driver development there 
anyway) as the reason why we don't see it. If they have scarce resources 
available, it's understandable why they do not see the immediate need to 
port their code to FreeBSD. I am confident, given this hardware is not 
that cheap, that any bigger user asking for FreeBSD support could 
motivate them to just do it.
I also believe there is nothing hidden in FreeBSD and that in general 
FreeBSD has been more stable API-wise than other UNIX platforms around. 
And, I also believe should there be interest from nVidia, tey will see 
support and help from FreeBSD developers. Or they could just release 
their hardware spec, if they can't do it themselves for one reason or 
another. After all, much more complex tasks have been resolved with FreeBSD.




2) Disk and network I/O issues under load. We realized that FreeBSD has
some issues in multithreaded environments. Even on 6/12 or 12/24
core/thread systems, under heavy load (especially network and CPU load),
disk I/O was (is?) poor. This is a no-go in a HPC environment.


Could you please elaborate on this a bit?
On one hand, I am surprised that the HPC environment will have such 
requirements and on the other hand this is how typical higher-end 
storage systems are built with FreeBSD. I haven't seen anything like 
this and am willing to test on 24-32 core systems.


You said this is political for FreeBSD .. why? I don't get it? FreeBSD 
has no policy of failing under heavy load -- quite the contrary.




3) Outdated ports OR not available ports: some important software
maintained by the US government (USGS, NASA/JPL) is only provided for
Mac OS X and some Linux derivatives.


This may be for many, many reasons, including (most often and most 
unfortunately) licensing. But there is not much anyone working with 
FreeBSD can do, except create an port, if the license permits.
If the license does not permit this software run on FreeBSD -- then 
probably the only choice is to try and persuade it's author.
If it runs on OS X, chances are it will run on FreeBSD with very little 
effort. (except if it relies heavily on Cocoa)



4) The lack of clustering capabilities. The lack of a clustered
filesystem grows more and more important in the area of HPC, where
storage systems get spread over a department. I lost track in the
development on FreeBSD since around 2003. At the moment, for me
personally this issue isn't so important, but in combination with items
1) through 3) and the migration towards Linux (we use prefereably Ubuntu
server, some Suse and on some servers CentOS/RedHat, which suffers from
the Linux-narrowminded deseas as well, in my opinion, but you'll get
support by Dell and others - in times of strangling contracts, a more
and more restricted freedom of science in favor of business ...
another story ...)


Can you 

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 02.06.12 10:21, Marc Santhoff wrote:

Am Freitag, den 01.06.2012, 13:56 -0400 schrieb Michael R. Wayne:

On Fri, Jun 01, 2012 at 05:03:26AM -0700, Mehmet Erol Sanliturk wrote:

If you are NOT using FreeBSD for any area or some areas , would you please
list those areas with most important first to least important last ?

As mentioned by several others, once you have a single applicaiton
that demands Windows, you are mostly stuck running windows.

I was used to thinking the same way, but today there is VirtualBox and
it does very well. More simple applications work fine using wine, that
keeps you from having to install a complete Windows-OS.


I want to second this. For me, until recently the only Windows computer 
was an laptop. I was thinking along the same lines (being primary BSD 
UNIX for everything for over 20 years) - if you need Windows software, 
run it on Windows. But, I was more and more pissed of this stuff and 
eventually brought myself an Macbook. Never touched the Windows laptop 
ever since. Any windows only software that comes across, of any kind, 
runs in VirtualBox on either FreeBSD or OS X. No issues of any kind.



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You Using FreeBSD?

[Daniel Kalchev]

On 02.06.12 12:42, Erich Dollansky wrote:

On 02 June 2012 AM 9:14:28 Chris Rees wrote:

On Jun 2, 2012 4:04 AM, Erich Dollanskyer...@alogreentechnologies.com
wrote:

But I have to mention one disadvantage. The ports are in no way linked to

the releases. This leads to situations in which a small change in a basic
library will result in a complete update of the installed ports. I
expressed this already many time here. It would be of advantage if the
ports tree would also have tags like the base system itself.
Unfortunately this is a massive amount of extra work - we only just keep up
with updates as it is.

I do not think so. At least not for the first step as I see it. Just make 
snapshots of the ports tree when the release comes out. These snapshots are 
with the releases anyway.

What I did was very simple. I got the ports tree that comes with the release 
and installed the system back to the release status. Ok, it was some work for 
me - maybe not for others - to find this tree.

A simple link could help here.

I do not know if this is just an opinion which is too optimistic.




But this functionality is already here. As I mentioned earlier, FreeBSD 
is not an end-user product, but rather a software platform and a kit 
that you can use to assemble pretty much what you can imagine.


Here is one example, how to handle the 'port problem'. The example is 
with BSDRP: http://bsdrp.net/


This is an nanoBSD based system, that you can build yourself. For 
example, the 31 May 2012 svn code sets this environment variable

PORTS_DATE=date=2012.05.31.00.00.00
to pull the ports tree with that particular date (when it was tested to 
build sucessfuly)
It then proceeds to download it's own copy of /usr/src and /usr/ports 
and uses these to build the complete installation. More or less, 
controlled environment.


The /usr/src of -stable/-current and /usr/ports are in fact moving 
target. If you are uncomfortable with that, just sync to some date and 
you will have that date's snapshot and therefore known state. Most 
people who are bitten by the 'sudden change in ports' are just ignoring 
this option.


You don't have to use the (arguable old) 'release' ports tree. Ports get 
fixed/adapted for the new version usually months after release.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You Using FreeBSD?

[Daniel Kalchev]

On 02.06.12 15:32, Erich wrote:

I know that the ports tree is a moving target. But it stops moving during the 
release period. This could be used to give a fall back solution.

Or do I see this really too simple?


The ports tree is a moving target during release periods still, although 
there are efforts to make movements smaller. This is why, after a 
release it suddenly moves more :)


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You Using FreeBSD?

[Daniel Kalchev]

On 31.05.12 18:41, Damien Fleuriot wrote:

You missed the bit about 3 reboots, while these don't take 15 mins each,
they're still time consuming and disruptive.
1/ reboot after installing new kernel
2/ reboot after installing new world
3/ reboot after rebuilding ports


About the only time I ever do that is when moving from very distant 
versions, like from 6.4 to 9.0...


Upgrading from say, 8-stable from year ago, to today's 8-stable usually 
requires just one reboot: rebuild world, kernel, reinstall kernel, 
world, update configuration files, rebuild ports, reboot.
There are many cases where I do rebuild/reinstall kernel and world but 
do not reboot for one reason or other. Cases, where the kernel is 
incompatible with userspace are extremely rare and typically documented.


So yes, for example during port rebuild there might be glitches with 
services. You are better to shut down these services that will be 
affected, like web server. (Although usually say, apache would load all 
modules at startup time and replacing them under its feet will only be 
noticed after it is restarted). Most of the time however is spent just 
compiling... and unless your server is really underpowered or overloaded 
it does not impact anything. This again, is especially true for the OS. 
I wish ports could be rebuilt and reinstalled on a single step like FreeBSD.


In any case, if you have 'server farms', or like you said firewalls with 
CARP etc, you can usually shut down any of the members for as long as 
necessary and not impact any services. If you rebuild things on 
'central' server, the downtime will be indeed minimal.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You Using FreeBSD?

[Daniel Kalchev]

On 01.06.12 13:19, Katinka wrote:
There's a nice discussion going on, over at Phoronix. 
http://phoronix.com/forums/showthread.php?71263

For some reason, they don't seem to like us very much.



Do we really care?

The number of really bright people, or even people who are able to 
reasonably comprehend and understand things is very, very small and more 
or less constant over the years. The rest will always be more and 
there is really no point to convince them opf anything. Evangelizing 
those ignorant people to FreeBSD is just creating new (FreeBSD) 
religion, which is not what the bright minds are concerned. Attract the 
masses and you will definitely lose the leaders.


Instead, lead by example. Showcase. Demonstrate how superior FreeBSD is 
because the people who keep it going are not interested to be the Jack 
of All Trades (and master of none). Showcase implementations that are 
hard to do with any other OS.


Don't even complete on benchmarks. This is one thing I learned years ago 
working closely with Cisco: your competitor could always outspec you or 
win the benchmark, with system specifically designed for the task. Or 
tuned to the task. Just like with Linux.


This is not to say we should ignore opportunities to improve FreeBSD. 
Just don't slip for the popularity vote and stay within the framework 
and principles (even when you are seemingly outpaced) that made FreeBSD 
so successful.


Albert Einstein once said:

Any intelligent fool can make things bigger, more complex, and more 
violent. It takes a touch of genius -- and a lot of courage -- to move 
in the opposite direction.


Daniel

PS: This became longer than originally envisioned.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 01.06.12 15:15, Kurt Jaeger wrote:
- Telephony (ISDN to SIP gateways, Asterisk etc) -- I know, Hans 
Petter Selasky is doing wonderful work in that area, I had no time to 
dive into this. 


Asterisk (tested up to v 10) works wonderfully on FreeBSD. Mine even 
runs in jail. The few gateways to PSTN are external and spread over 
distance anyway. There are less and less PSTN physical interconnects 
anyway. About any sane telco will provide SIP trunks over TCP/IP so 
there is not much motivation for development in that area. Such 
connectivity was big thing say 10 years ago..



There is hardly anything FreeBSD cannot do. I tend to avoid proprietary 
technologies like the plague (this includes about anything from 
Microsoft). For example if one wants an e-mail server, that is better 
served in the long run by IMAP+MTA than any form of Exchange, because 
you are not tied to one single platform and that vendor's lunacy. 
Otherwise FreeBSD runs just fine as server for about any other OS 
client, provided those clients use standard Internet protocols.


Things I don't particularly do is use FreeBSD for mobile. I find OS X 
way better choice there. When you need to go mobile, you usually need 
the highest performing hardware (that is, lower power consumption, less 
heat etc) and those things usually are pretty much proprietary for quite 
long time. With OS X you get nice UNIX client.. for your FreeBSD 
servers, that is. I also find it increasingly tempting to use tablets 
for such remote client tasks :)


Another thing I don't use FreeBSD for is CAD. Unfortunately, there is no 
AutoCAD for anything but Windows or OS X.


It will sure be interesting to learn what people avoid to use FreeBSD for.

Best Regards,
Daniel Kalchev
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 02.06.2012, at 03:06, David Magda dma...@ee.ryerson.ca wrote:

 On Jun 1, 2012, at 08:33, Daniel Kalchev wrote:
 
 For example if one wants an e-mail server, that is better served in the long 
 run by IMAP+MTA than any form of Exchange, because you are not tied to one 
 single platform and that vendor's lunacy. Otherwise FreeBSD runs just fine 
 as server for about any other OS client, provided those clients use standard 
 Internet protocols.
 
 If all you want is e-mail, then there are certainly better options than 
 Exchange IMHO. However, once you get into calendars (private and shared, with 
 delegation to secretaries, etc.), meeting rooms, ActiveSync (to remotely wipe 
 lost devices), then it's a whole different game.

There are a lot of open source calendaring applications, of all kinds. Most run 
fine on FreeBSD.

I really see no reason why your 'mail or calendaring server' should be able to 
wipe your devices.. This is the sort of bloat that keeps me away. From 
Microsoft products.


 
 E-mail was solved a long time ago, but Exchange does many things on top of it 
 that many organizations find very handy, and where there doesn't seem to be a 
 decent open alternative.
 

Hope you are not of the opinion that first there was Exchange, then all other 
e-mail servers appeared, copying it. History was exactly the other way 
around. We were using it long before Microsoft discovered this Internet thing 
exists and first tried to kill it.
Again it is not about open source. It is about non-proprietary protocols. All 
proprietary platforms turn to be more expensive in every respect in a while.

In this regard I rather prefer the way Apple handles things. Shiny wrapper 
interface to pretty much generic technology. No reinvention of the wheel and 
experiments to see if it can be made square.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You NOT Using FreeBSD ?

[Daniel Kalchev]

On 02.06.2012, at 07:19, Freddie Cash fjwc...@gmail.com wrote:

 
 Glustre sits above the storage system, replicating data between systems.
 So, disks -- ZFS (via Zvols) -- Glustre.
 

How is this different than ZFS using remote zvols via iSCSI? Can it tolerate 
down nodes better than ZFS?

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Why Are You Using FreeBSD?

[Daniel Kalchev]

1) Been with BSD/OS since it's inception. Great OS and good example to 
follow. But BSD/OS was eventually killed and FreeBSD sort of inherited 
it's legacy. Both follow the simplicity and good architecture models, 
with FreeBSD improving on modularity.
2) The BSD license. Contrary to popular belief, it has brought a lot of 
high quality development to FreeBSD.
3) Universal toolkit. It scales easily from the thinnest embedded 
system, to various desktops to huge servers -- all with the same 
familiar tools and environment.


Sure, for consumption there are easier systems, such as PC-BSD 
(FreeBSD again), Ubuntu and, of course OS X. But there is no better 
platform, or kit to build whatever you need around.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: /usr/bin/unzip not being installed on 8.3-STABLE

[Daniel Kalchev]

According to the unzip(1) man page on 9-stable:

HISTORY
 The unzip utility appeared in FreeBSD 8.0.


So possibly the man page needs to be fixed as well.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Make filesystem type configurable for periodic(8)?

[Daniel Kalchev]

On May 4, 2012, at 7:05 PM, Freddie Cash wrote:

 A few of the periodic(8) scripts in FreeBSD have constructs similar to
 the following to get which filesystems to scan for various things:
MP=`mount -t ufs,zfs | awk '$0 !~ /no(suid|exec)/ { print $3 }'`
 
 For systems with large ZFS pools, and many ZFS filesystems, these
 periodic scripts can grind it to its knees, and then some.  For
 backups servers where we don't really care about the
 ownership/permissions of files from the FreeBSD perspective, we really
 don't want the ZFS filesytems to be scanned; 
[…]

The script already accommodates this scenario. Just mount your storage 
filesystems with 'nosuidexec' and they won't be scanned. 

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS: can't read MOS

[Daniel Kalchev]

It seems your RAID controlled goofed something.

I wonder, why did you use the hardware controller for RAID60, instead of 
ZFS (using each drive as single drive array, or JBOD).


About the only way I can think out of this situation, sans having a 
second box or huge tape backup is to convert it to proper ZFS.. in place :)


It helps that you have not used half of your capacity. You could do this:

1. remove as many drives from the RAID60 as you could, without breaking 
redundancy.
2. create single drive volumes out of these, or just JBOD the drives if 
this controller can.
3. create new zpool with these drives, again RAID60 if you like, that 
is vdevs of raidz2. You need enough drives to hold at least 24TB of 
data. You may 'trick' ZFS by creating raidz2 groups of say 8 drives (I 
guess this is what you mean by having 6x8 RAID60), then remove two 
drives from the vdev and use these for the next vdev. I guess two raidz2 
groups of 8 drives will be ok, as that gives you 24TB, but in fact 
slightly less, because the drives aren't really 2TB so you may need 3x8 
groups... You can create 3x8 groups without redundancy (actually 3x6) 
with 18 drives only.

4. copy over your old zpool to the new with zfs send and zfs receive.
5. rename both pools so that the new pool becomes zroot.
6. boot off from your new pool.
7. if everything is ok, destroy the remains of the RAID60, add two 
drives to each non-redundant vdev in the new zpool and add the other 
drives as 8 drive vdevs to the zpool.
8. Now you have true ZFS pool and ZFS will be able to detect and recover 
any data errors on your drives. With pool such huge and so much data it 
is very risky to depend on 'hardware raid', especially when ZFS is 
available.


Thinking about this again, you may not be able to extract enough 'spare' 
drives out of the RAID60 pool. You may get up to 12 drives and that is 
not enough to hold all your data. An option is to replace those with 
larger capacity drives, 3TB pr 4TB -- ZFS has not problem with different 
size drives in a zpool, but this has to be checked if this LSI 
controller can manage drives larger than 2TB. Or, you might be able to 
remove some data form the pool.


Another option is if you can attach an spare drive chassis to handle the 
transition..


In any case, my advice is to stay away from ZFS on top of 'hardware 
RAID'. ZFS will be able to detect data corruption, but not do anything 
except inform you, even if you have lots of redundant drives. Use ZFS as 
designed.


Daniel

On 09.04.12 21:50, Rumen Telbizov wrote:

Hello everyone,

I have a ZFS FreeBSD 8.2-STABLE (Aug 30, 2011) that I am having issues with
and might use some help.

In a nutshell, this machine has been running fine for about a year and a
half but after a recent power
outage (complete colo blackout) I can't boot of the ZFS pool any more.
Here's the error I get (attached screenshot as well):

ZFS: i/o error - all block copies unavailable
ZFS: can't read MOS
ZFS: unexpected object set type 0
ZFS: unexpected object set type 0

FreeBSD/x86 boot
Default: zroot:/boot/kernel/kernel
boot: ZFS: unexpected object set type 0

I've been searching the net high and low for an actual solution but all the
threads end up nowhere.
I hope I can get some clue here. Thanks in advance.

Here's the relevant hardware configuration of this box (serves as a backup
box).

- SuperMicro 4U + another 4U totalling 48 x 2TB disks
- Hardware raid LSI 9261-8i holding both shelves giving 1 mfid0 device
to the OS
- Hardware raid 60 -- 6 x 8 raid6 groups
- ZFS with gptzfsboot installed on the single mfid0 device. Partition
table is:

[root@mfsbsd /zroot/etc]# gpart show -l
=   34  140554616765  mfid0  GPT  (65T)
 34   128  1  (null)  (64k)
162  33554432  2  swap  (16G)
   33554594  140521062205  3  zroot  (65T)



- boot device is: vfs.root.mountfrom=zfs:zroot (as per loader.conf)
- zpool status is:

[root@mfsbsd /zroot/etc]# zpool status
   pool: zroot
  state: ONLINE
  scan: scrub canceled on Mon Apr  9 09:48:14 2012
config:

NAMESTATE READ WRITE CKSUM
zroot   ONLINE   0 0 0
  mfid0p3   ONLINE   0 0 0

errors: No known data errors



- zpool get all:

[root@mfsbsd /zroot/etc]# zpool get all zroot
NAME   PROPERTY   VALUE   SOURCE
zroot  size   65T -
zroot  capacity   36% -
zroot  altroot-   default
zroot  health ONLINE  -
zroot  guid   3339338746696340707  default
zroot  version28  default
*zroot  bootfs zroot   local*
zroot  delegation on  default
zroot  autoreplaceoff default
zroot  cachefile  -   default
zroot  failmode   waitdefault
zroot  listsnapshots  on  local
zroot  autoexpand off default
zroot  dedupditto 0   default
zroot  dedupratio 1.00x   -
zroot 

Re: grow zpool on a mirror setup

[Daniel Kalchev]

On 15.03.12 14:00, George Mamalakis wrote:

Hello everybody,

I have asked the same question in the freebsd forums, but had no luck. 
Apart of this, there might be a bug somewhere, so I re-ask the 
question to this list. Here how it goes (three posts):


post 1:

I am experimenting with one installation of FreeBSD-9-STABLE/amd64 on 
a VirtualBox that is using gptzfsboot on a raid-1 (mirrored) zfs pool. 
My problem is that I need to grow the filesystem size of zfs 
partitions. I followed this guide 
http://support.freenas.org/ticket/342(http://support.freenas.org/ticket/342), 
which is for FreeNAS, and encountered a few problems.




You are using FreeBSD 9, while you follow instructions for FreeNAS. The 
ZFS version support in FreeNAS is way behind that in FreeBSD 9.


In particular, ZFS v28, which is in FreeBSD 9 supports the autoexpand 
property, that does not require export/import of the pool.


You mentioned you did use autoexpand, but perhaps didn't really 'expand' 
the zpool vdevs.




Since nobody has an answer that far, let me ask another thing. 
Instead of deleting ada0p2 and ada1p2, and then recreating them from 
the same starting block but with a grater size, could I have just 
created two new filesystems (ada0p3 and ada1p3), and having them added 
in the pool as a new mirror? Because if that's the case, then I could 
try that out, since it seems to have the same result.




The proper way to expand an zpool is by replacing each underlying 
storage device with a larger one.
Replacing is the key word here. ZFS will not care if the same device 
suddenly became larger (if I am not mistaken).

So, to expand your zpool you have basically two options:

1. Detach one of the mirror members, make sure you clear ZFS metadata 
from the beginning and the end of the partition, recreate your 
partitions with larger sizes, then attach to the mirror. After 
resilvering and possibly verifying with scrub that you are not losing 
data, repeat with the other mirror member. If you have autoexpand=yes 
set, your zpool should grow.


2. If you can add more devices, then add a larger size device to the 
mirror. Or if you can, add two new larger devices to the mirror. After 
completing this, remove the old, smaller mirror members. As with the 
other option, your zpool should grow.


Do not add another mirror to the zpool, if you want to remove the old 
devices. Doing so will create second vdev in the zpool and will spread 
data over both (raid0 or stripe). Current versions of ZFS cannot remove 
vdevs from an zpool so in order to remove the smaller devices, you will 
have to backup, recreate the zpool and restore data.


Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Request for flowtable testers and actionable feedback RE: flowtable usable or not

[Daniel Kalchev]

On Mar 5, 2012, at 9:11 AM, H wrote:

 I have the right, even the obligation to point out what I think is wrong

So, you see yourself as speaking for others? You certainly do not speak for me! 
Never authorized you for this, never ever knew you actually exist. For various 
historical reasons, I don't particularly like the kind of people who self-elect 
themselves to defend other's rights. OK? :-)

So unlike you, Kip at least tries to achieve something. For the good of others. 
Even if he didn't do it in the most humble, democratic and whatever way. Even 
if he appears for many as being arrogant or whatever. People are different, 
some might actually prefer Kip's way, did you imagine that?

I happen to share the opinion and the experience of Mark Linimon in situations 
like this and yes, I do believe you have been rude here. For no reason 
whatsoever.

You either make the choice to help Kip in his experiment, or not. For me, 
personally, as long as you don't stay on my way, I don't really care what your 
position is.

Daniel

PS: In any case, this is an open forum, so you have your opinion heard. By a 
lot of people.___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Upgrade from 8.2-STABLE to 9.0-RELEASE wedges on SuperMicro H8DGiF-based system

[Daniel Kalchev]

On Jan 9, 2012, at 8:03 PM, Freddie Cash wrote:

 Small correction:  these are AMD Opteron 6218 CPUs, not 2218.
 
 Hardware (alphadrive):
  Chenbro 5U rackmount chassis with 24 hot-swap drive bays
  SuperMicro H8DGi-F motherboard
  AMD Opteron 6218 CPU (8-cores at 2.0 GHz)

You meant Opteron 6128 perhaps?

This looks weird coincidence indeed and considering the comments so far I too 
would question ACPI (BIOS revision, settings etc) and the possibility for some 
hardware going bad. 

Is it possible that you might have touched any hardware just before the 
upgrade? I had few cases an old system die on me when doing minor cleaning 
etc just before an update…

Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: SCHED_ULE should not be the default

[Daniel Kalchev]

On Dec 24, 2011, at 12:49 AM, Adrian Chadd wrote:

 Do you not have access to anything with 8 CPUs in it? It'd be nice to
 get clarification that this indeed was fixed.

I offered to do tests on 4x8 core Opteron system (32 cores total), but was 
discouraged that contention would be too much and results meaningless -- yet, 
such systems will be more and more popular.

 Does ULE care (much) if the nodes are hyperthreading or real cores?
 Would that play a part in what it tries to schedule/spread?

I could also run the tests on 2x4x2 cores Xeon, which uses hyper threading, 8 
real or 16 virtual cores in total.

I can torture both systems (actually two pairs) for a week or two. But I may 
not have enough time to prepare the core/setup so any advice is greatly 
appreciated. Be more descriptive :)

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Daniel Kalchev]

On Dec 15, 2011, at 3:25 PM, Stefan Esser wrote:

 Am 15.12.2011 11:10, schrieb Michael Larabel:
 No, the same hardware was used for each OS.
 
 In terms of the software, the stock software stack for each OS was used.
 
 Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with
 journaling enabled) should be an obvious choice since it is more similar
 in concept to ext4 and since that is what most FreeBSD users will use
 with FreeBSD?


Or perhaps, since it is server Linux distribution, use ZFS on Linux as well. 
With identical tuning on both Linux and FreeBSD. Having the same FS used by 
both OS will help make the comparison more sensible for FS I/O.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Daniel Kalchev]

On Dec 15, 2011, at 3:48 PM, Jeremy Chadwick wrote:

[…]
 That said: thrown out, data ignored, done.
 
 Now what?  Where are we?  We're right back where we were a day or two
 ago; meaning no closer to solving the dilemma reported by users and
 SCHED_ULE.  Heck, we're not even sure if there is an issue, other than
 some folks confirming that SCHED_4BSD performs better for them (that's
 what started this whole thread), and there are at least a couple which
 have stated this.

But, are any of these benchmarks really engaging the 4BSD/ULE scheduler 
differences? Most such benchmarks are run on a system with no other load 
whatsoever and in no way represent real world experience.

What is more, I believe in such benchmarks the system feels sluggish is not 
measured at all. Even if it is measured, if in such case the benchmark finishes 
better - that is, faster, or say, makes the system freeze for the user for 
the duration of the test -- it will be considered win, because the benchmark 
suite ran faster on that particular system -- whereas a system which ran the 
benchmark fast, provided good interactive response etc would be considered 
loser.

I think it is not good idea to hijack this thread, but instead focusing on the 
other SCHED_ULE bashing thread to define an reasonable benchmark or a set of 
benchmarks rather -- so that many would run it and provide feedback.


Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: SCHED_ULE should not be the default

[Daniel Kalchev]

On Dec 15, 2011, at 6:26 PM, Attilio Rao wrote:

 2011/12/13 Daniel Kalchev dan...@digsys.bg:
 
 
 On 13.12.11 09:36, Jeremy Chadwick wrote:
 
 I personally would find it interesting if someone with a higher-end system
 (e.g. 2 physical CPUs, with 6 or 8 cores per CPU) was to do the same test
 (changing -jX to -j{numofcores} of course).
 
 
 Is 4 way 8 core Opteron ok? That is 32 cores, 64GB RAM.
 
 Testing with buildworld in my opinion is not adequate, as it involves way
 too much I/O. Any advice on proper testing methodology?
 
 I'm sure that I/O and pmap subsystem contention (because of
 buildworld) and TLB shootdown overhead (because of 32 CPUs) will be so
 overwhelming that you are not really going to benchmark the scheduler
 activity at all.

Can't pmap / TLB be tuned for 32 CPUs and 64GB of RAM?

 
 However I still don't get what you want to verify exactly?

The obvious: is SCHED_ULE better or worse than SCHED_4BSD on such platform. 

Problem is how to test interactivity -- that is a blade server and doesn't 
really have a display and keyboard, nor does it have X etc.

I have spare pair of those, that might be put to crunch tests to see how things 
compare for different scenarios - but I need ideas what to test, really.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: SCHED_ULE should not be the default

[Daniel Kalchev]

On 15.12.11 01:39, O. Hartmann wrote:

On 12/14/11 18:54, Tom Evans wrote:

On Wed, Dec 14, 2011 at 11:06 AM, George Mitchell
george+free...@m5p.com  wrote:

Dear Secret Masters of FreeBSD: Can we have a decision on whether to
change back to SCHED_4BSD while SCHED_ULE gets properly fixed?


Please do not do this. This thread has shown that ULE performs poorly
in very specific scenarios where the server is loaded with NCPU+1 CPU
bound processes, and brought forward more complaints about
interactivity in X (I've never noticed this, and use a FreeBSD desktop
daily).

I would highly appreciate a decission against SCHED_ULE as the default
scheduler! SCHED_4BSD is considered a more mature entity and obviously
it seems that SCHED_ULE needs some refinements to achieve a better level
of quality.



My logic would be, if SCHED_ULE works better on multi-CPU systems, or if 
SCHED_4BSD works poor on multi-CPU systems, then by all means keep 
SCHED_ULE as default scheduler. We are at the end of 2011 and the number 
of single or dual core CPU systems is decreasing. Most people would just 
try the newest FreeBSD version on their newest hardware and on that base 
make an informed decision if it is worth it. If on newer hardware 
SCHED_ULE gives better performance, then again it should be the default.


Then, FreeBSD is used in an extremely wide set fo different 
environments. What scheduler might benefit an one CPU, simple 
architecture X workstation may be damaging for the performance of 
multiple CPU, NUMA based server with a large number of non-interactive 
processes running.


Perhaps an knob should be provided with sufficient documentation for 
those that will not go forward to recompile the kernel (the majority of 
users, I would guess).


I tried switching my RELENG8 desktop from SCHED_ULE to SCHED_4BSD 
yesterday and cannot see any measurable difference in responsiveness. My 
'stress test' is typically an FLASH game, that get's firefox in an 
almost unresponsive state, eats one of the CPU cores -- but no 
difference. Well, FLASH has it's own set of problems on FreeBSD, but 
these are typical desktop uses. Running 100% compute intensive 
processes in background is not.


Daniel

PS: As to why Linux is better in these usages: they do not care much 
to do things right, but rather to achieve performance. In my opinion, 
most of us are with FreeBSD for the do it right attitude.


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: SCHED_ULE should not be the default

[Daniel Kalchev]

On 13.12.11 09:36, Jeremy Chadwick wrote:
I personally would find it interesting if someone with a higher-end 
system (e.g. 2 physical CPUs, with 6 or 8 cores per CPU) was to do the 
same test (changing -jX to -j{numofcores} of course). 


Is 4 way 8 core Opteron ok? That is 32 cores, 64GB RAM.

Testing with buildworld in my opinion is not adequate, as it involves 
way too much I/O. Any advice on proper testing methodology?


These systems run ZFS, but could be booted diskless for the tests.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

igb hang when cable unplugged

[Daniel Kalchev]

I am observing an transmit hang of the igb driver when the cable is unplugged. 
It only recovers after unit reset, such as

ifconfig igb0 down up

This is with kernel

FreeBSD xxx 8.2-STABLE FreeBSD 8.2-STABLE #0: Fri Sep 30 16:17:47 EEST 2011 
root@xxx:/usr/obj/usr/src/sys/GENERIC  amd64

igb0: Intel(R) PRO/1000 Network Connection version - 2.2.3 port 0x3020-0x303f 
mem 0xb1d6-0xb1d7,0xb1d4-0xb1d5,0xb1e04000-0xb1e07fff irq 37 at 
device 0.0 on pci13
igb0: Using MSIX interrupts with 9 vectors
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: [ITHREAD]
igb0: Ethernet address: 00:25:90:36:ee:7c

The interface is quad port Supermicro branded PCI-E card with 

pciconf -vl

igb0@pci0:13:0:0:   class=0x02 card=0x10c915d9 chip=0x10c98086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
class  = network
subclass   = ethernet
igb1@pci0:13:0:1:   class=0x02 card=0x10c915d9 chip=0x10c98086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
class  = network
subclass   = ethernet
igb2@pci0:16:0:0:   class=0x02 card=0x10c915d9 chip=0x10c98086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
class  = network
subclass   = ethernet
igb3@pci0:16:0:1:   class=0x02 card=0x10c915d9 chip=0x10c98086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
class  = network
subclass   = ethernet


Has anyone experience something like this? Is there solution? It is very 
inconvenient to have to down/up the interfaces manually via the IPMI console 
when such thing happens.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: What about network virtualization for jails?

[Daniel Kalchev]

On Nov 25, 2011, at 10:36 PM, Shawn Webb wrote:

 I don't really know much about how using vnet with jails will affect
 NFS services. I would suggest setting up a test environment before
 attempting anything on production servers.

I was unsuccessful in setting up NFS from within a VIMAGE jail -- that had to 
run in the host environment. If there is a way to do it, that would be great 
news.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: CARP interfaces and mastership issue

[Daniel Kalchev]

On Sep 15, 2011, at 23:14 , Damien Fleuriot wrote:

 What would help here, is for a carp interface to wait a given delay
 (tunable through a sysctl ?) after creation or after being brought up
 from down.

I have the same observation. Perhaps it can just avoid going up initially --- 
it will become master anyway if it does not hear anything on the interface.

Daniel___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

CARP up at boot

[Daniel Kalchev]

I am trying to use a CARP/HAST setup for redundancy and reply on devd 
for the carp up/down events to trigger role switch for the nodes.


What is interesting is that upon reboot, the CARP interface always first 
comes up, like this:


carp0: link state changed to UP
carp0: MASTER - BACKUP (more frequent advertisement received)
carp0: link state changed to DOWN

This causes devd to execute the event scripts as 'master' first, then 
shortly after execute the script as 'backup'. This may cause undesired 
writing to the hast providers and possibly split-brain condition.


What is worse, on two hosts with the same advskew value if you reboot 
the BACKUP server, it would become MASTER. This results in all services 
teardown and starting them again on the new master.


I understand that for routers, which is the original design goal for 
CARP it does not matter much if roles switch from time to time, but for 
high-latency startup systems, this is not desirable. It is best when a 
node becomes MASTER it stays MASTER until failure and not change state 
when the other node is rebooted.


Perhaps CARP and devd are not the best tool, but it will help if at 
least the carp interface does not start as MASTER and if it waits 
longer, before requesting to become MASTER after reboot.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: bad sector in gmirror HDD

[Daniel Kalchev]

On Aug 20, 2011, at 06:24 , Jeremy Chadwick wrote:

 You might also be wondering that dd command writes 512 bytes of zero to
 that LBA; what about the old data that was there, in the case that the
 drive remaps the LBA?

If you write zeros at OS level to an LBA, you will end up with zeros at that 
LBA. What else did you expect???

The already remapped LBAs in ATA are not visible anymore to the user/OS. You 
get a perfectly readable sector. Of course not at the original location, but as 
you confirmed we are done with CHS addressing.

The pending bad sectors are almost always 'corrected', that is, remapped when 
you write to that LBA.

So your script will find only one readable sector and that will be the sector 
that is pending reallocation.

It may be that writing zeros to all free space, like

dd if=/dev/zero of=/filesystem/zero bs=1m; rm /filesystem/zero

is enough to remap the pending bad block and not have any unreadable sectors. 
But if the unreadable sector is in a file or directory -- bad luck -- these 
will need to be rewritten.

Once upon a time, BSD/OS had wonderful disk 'repair' utility. It could detect 
failing disks by reading every sector (had nice visual), or could re-write the 
drive by reading and writing back every sector. On bad blocks it would retry 
lots of times and eventually average what was read (with error).
Having said that, I doubt modern ATA drives will let anything be read by the 
pending bad block, but.. who knows.

Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: can not boot from RAIDZ with 8-STABLE

[Daniel Kalchev]

On 17.08.11 16:35, Miroslav Lachman wrote:
I tried mfsBSD installation on Dell T110 with PERC H200A and 4x 500GB 
SATA disks. If I create zpool with RAIDZ, the boot immediately hangs 
with following error:



May be it that the BIOS does not see all drives at boot?
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: 32GB limit per swap device?

[Daniel Kalchev]

On 09.08.11 18:16, David Wolfskill wrote:
While FreeBSD cannot address more than 32GB per swap space, it permits 
as many as 32 swap spaces to be active concurrently.


I am more concerned that with 32GB of swap in single device I could not 
dump kernel core, with 64GB of RAM.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: 32GB limit per swap device?

[Daniel Kalchev]

On 10.08.11 10:47, Jeremy Chadwick wrote:

On Wed, Aug 10, 2011 at 10:13:14AM +0300, Daniel Kalchev wrote:
I am more concerned that with 32GB of swap in single device I could 
not dump kernel core, with 64GB of RAM. 

My apologies if I've misunderstood something, but why does this of any
concern?  Machine has 64GB RAM.  You have a single swap slice that's
effectively 32GB.  How is a kernel panic worth of 64GB RAM going to fit
into a 32GB swap slice?

The swap partitions are 64GB, it is only that FreeBSD refuses to use 
more than 32GB of each for swap. But.. it might happily dump core to the 
whole partition, tests will show.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: 32GB limit per swap device?

[Daniel Kalchev]

On 10.08.11 11:47, Jeremy Chadwick wrote:
So we're back to where we started: swap slices/partitions can be 
greater than 32GBytes in size, but something is limiting the maximum 
amount of memory which can be dumped to a single swap swap to 32GBytes. 
It seems there is still some confusion. Partition size(s) is 64GB, but 
FreeBSD will use only 32GB of that for swap.


Trying:

sysctl debug.minidump=0
sysctl debug.kdb.panic=1

Produces 64GB dump, but.. I can't see the dump on the swap device 
(savecore doesn't find the magic number). My swap device is gmirror or 
two slices (/dev/mirror/swap).


Removing the gmirror and listing both slices for swap .. works.


I could use separate device for dumps, but the disk real estate on this 
particular blade is limited...


Well, I really asked two questions:
Q1: Is FreeBSD still limited to 32GB per swap slice?
A1: yes. There is limit set in /usr/src/sys/vm/swap_pager.c

Q2: If FreeBSD will only use 32GB of the slice for swap, will it dump 
larger (64GB in my case) core in there?

A2a: yes.
A2b: If the swap slice is gmirrored, you don't get any dump back.

I believe the gmirror bug might exist in smaller partitions as well, but 
haven't tested it yet (have few such systems that never duped core). It 
does not matter if I do full dump or minidump: on gmirrored 64GB 
partittion savecore does not find anything.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: 32GB limit per swap device?

[Daniel Kalchev]

On 10.08.11 14:19, Eugene Grosbein wrote:
You should read gmirror(8) manual page about Doing kernel dumps to 
gmirror providers.


Thanks, I totally forgot about the gmirror limitations.

When using the default minidump, the result is:

savecore: first and last dump headers disagree on /dev/mirror/swap

There seems to be no problem when a full dump is performed.

This is probably an entirely unrelated issue however.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

32GB limit per swap device?

[Daniel Kalchev]

I am trying to set up 64GB partitions for swap for a system that has 
64GB of RAM (with the idea to dump kernel core etc). But, on 8-stable as 
of today I get:


WARNING: reducing size to maximum of 67108864 blocks per swap unit

Is there workaround for this limitation?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS directory with a large number of files

[Daniel Kalchev]

On 06.08.11 09:24, Gary Palmer wrote:
Its been quite a while since I worked on the filesystem stuff in any 
detail but I believe, at least for UFS, it doesn't GC the directory, 
just truncate it if enough of the entries at the end are deleted to 
free up at least one fragment or block.


This was my point indeed. If you empty a directory or remove files form 
the end of the directory is it truncated, this is not really a GC, but 
rather a shortcut. I guess the reason why it does not  use GC is because 
of concurrency/locking reasons. Or maybe the code was just not written yet.


But with ZFS this should be much easier to implement. If it is the same 
in Solaris, then it is not done so far...  But then, the promise made by 
ZFS is to provide constant directory access timing.


I am just wondering.. does implementing such garbage collection merit a 
new ZFS filesystem version?


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS directory with a large number of files

[Daniel Kalchev]

On 02.08.11 12:46, Daniel O'Connor wrote:
I am pretty sure UFS does not have this problem. i.e. once you 
delete/move the files out of the directory its performance would be 
good again. 


UFS would be the classic example of poor performance if you do this.

If it is a limitation in ZFS it would be nice to know that, perhaps it 
truly, really is a bug that can be avoided (or it's inherent in the 
way ZFS handles such things)


It is possible  that there is not enough memory in ARC to cache that 
large directory.


Other than that, perhaps in ZFS it would be easier to prune the unused 
directory entries, than it is in UFS. It looks like this is not implemented.


Another reason might be some FreeBSD specific implementation issue for 
fstatfs.


In any case, the data available is not sufficient. More information 
would help, like how much RAM this system has, how much ARC uses, some 
ARC stats.


What made me wonder is .. how exactly the kernel and zpool disagree on 
zpool version? What is the pool version in fact?


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

On 10.06.11 20:07, Mikolaj Golub wrote:

On Fri, 10 Jun 2011 20:05:43 +0300 Mikolaj Golub wrote to Daniel Kalchev:

  MG  Could you please try this patch?

  MG  http://people.freebsd.org/~trociny/hastd.no_shutdown.patch

Sure you still have to have your kernel patched with uipc_socket.c.patch :-)



It is now running for about a day with both patches applied, without 
disconnects.


Also, now TCP/IP connections always stay in ESTABLISHED state. As I 
believe they should. Primary to secondary drain quickly on switching 
form init to primary etc. No troubles without checksums as well. Kernel 
is as of


FreeBSD b1a 8.2-STABLE FreeBSD 8.2-STABLE #1: Mon Jun 13 11:32:38 EEST 
2011 root@b1a:/usr/obj/usr/src/sys/GENERIC  amd64


Daniel


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

On 14.06.11 17:56, Mikolaj Golub wrote:

It has turned out that automatic receive buffer sizing works only for
connections in ESTABLISHED state. And with small receive buffer the connection
might stuck sending data only via TCP window probes -- one byte every few
seconds (see Scenario to make recv(MSG_WAITALL) stuck in net@ for details).

I have tried some TCP/IP tuning to help utilize the faster network, but 
for the moment it is likely local disks limit the throughput to about 
230 MB/sec peak. The peaks now are the same as before, but now the total 
performance is better.


However, it may turn out that single TCP/IP session across 10Gbit 
network would not be able to achieve very high throughput. It may be 
beneficial to support multiple parallel TCP/IP connections between 
primary/slave in order to utilize faster networks.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

Decided to apply the patch proposed in -current by Mikolaj Golub: 
http://people.freebsd.org/~trociny/uipc_socket.c.patch


This apparently fixed my issue as well. Running without checksums for a 
full bonnie++ run (~100GB write/rewrite) produced no disconnects, no 
stalls and generated up to 280MB/sec (4 drives in stripped zpool). 
Interestingly, the hast devices write latency as observed by gstat was 
under 30ms.


I believe this fix should be committed.

Here are the accumulated netstat -s from both hosts, for comparison with 
previous runs. Retransmits etc are much less.


http://news.digsys.bg/~admin/hast/test3jun-fix/b1a-netstat-s
http://news.digsys.bg/~admin/hast/test3jun-fix/b1b-netstat-s
http://news.digsys.bg/~admin/hast/test3jun-fix/b1b-systat-if-fix

Before applying the patch I verified there are no network problems. 
Created 1TB file from /dev/random on the first host. Copied over to the 
second host with ftp. Transfer speed was low, at 80MB/sec -- ftp would 
utilize one CPU core 100% at the receiving node. Then calculated md5 
checksums on both sides, matched.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

Well, apparently my HAST joy was short. On a second run, I got stuck with

Jun  3 19:08:16 b1a hastd[1900]: [data2] (primary) Unable to receive 
reply header: Operation timed out.


on the primary. No messages on the secondary.

On primary:

# netstat -an | grep 8457

tcp4   0  0 10.2.101.11.42659  10.2.101.12.8457   FIN_WAIT_2
tcp4   0  0 10.2.101.11.62058  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  0 10.2.101.11.34646  10.2.101.12.8457   FIN_WAIT_2
tcp4   0  0 10.2.101.11.11419  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  0 10.2.101.11.37773  10.2.101.12.8457   FIN_WAIT_2
tcp4   0  0 10.2.101.11.21911  10.2.101.12.8457   FIN_WAIT_2
tcp4   0  0 10.2.101.11.40169  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  97749 10.2.101.11.44360  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  0 10.2.101.11.8457   *.*LISTEN

on secondary

# netstat -an | grep 8457

tcp4   0  0 10.2.101.12.8457   10.2.101.11.42659  CLOSE_WAIT
tcp4   0  0 10.2.101.12.8457   10.2.101.11.62058  FIN_WAIT_2
tcp4   0  0 10.2.101.12.8457   10.2.101.11.34646  CLOSE_WAIT
tcp4   0  0 10.2.101.12.8457   10.2.101.11.11419  FIN_WAIT_2
tcp4   0  0 10.2.101.12.8457   10.2.101.11.37773  CLOSE_WAIT
tcp4   0  0 10.2.101.12.8457   10.2.101.11.21911  CLOSE_WAIT
tcp4   0  0 10.2.101.12.8457   10.2.101.11.40169  FIN_WAIT_2
tcp4   66415  0 10.2.101.12.8457   10.2.101.11.44360  FIN_WAIT_2
tcp4   0  0 10.2.101.12.8457   *.*LISTEN

on primary

# hastctl status
data0:
  role: primary
  provname: data0
  localpath: /dev/gpt/data0
  extentsize: 2097152 (2.0MB)
  keepdirty: 64
  remoteaddr: 10.2.101.12
  sourceaddr: 10.2.101.11
  replication: fullsync
  status: complete
  dirty: 0 (0B)
data1:
  role: primary
  provname: data1
  localpath: /dev/gpt/data1
  extentsize: 2097152 (2.0MB)
  keepdirty: 64
  remoteaddr: 10.2.101.12
  sourceaddr: 10.2.101.11
  replication: fullsync
  status: complete
  dirty: 0 (0B)
data2:
  role: primary
  provname: data2
  localpath: /dev/gpt/data2
  extentsize: 2097152 (2.0MB)
  keepdirty: 64
  remoteaddr: 10.2.101.12
  sourceaddr: 10.2.101.11
  replication: fullsync
  status: complete
  dirty: 6291456 (6.0MB)
data3:
  role: primary
  provname: data3
  localpath: /dev/gpt/data3
  extentsize: 2097152 (2.0MB)
  keepdirty: 64
  remoteaddr: 10.2.101.12
  sourceaddr: 10.2.101.11
  replication: fullsync
  status: complete
  dirty: 0 (0B)

Sits in this state for over 10 minutes.

Unfortunately, no KDB in kernel. Any ideas what other to look for?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

Here goes the second run, wihtout checksums.

systat -if

/0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
 Load Average

  Interface   Traffic   PeakTotal
lo0  in  0.000 KB/s 71.666 KB/s  361.825 KB
 out 0.000 KB/s 71.666 KB/s  361.825 KB

ix1  in  0.021 KB/s816.608 MB/s  625.751 GB
 out 0.016 KB/s  7.384 MB/s   23.032 GB

   igb0  in  0.025 KB/s  1.507 KB/s   11.547 MB
 out 0.069 KB/s  1.765 KB/s   17.140 MB

This time it managed to achieve 800MB/s wow! Anyway, no idea when this 
happened, as during my observation, it didn't manage to push much data, 
due to frequent disconnects. Typical good rate was lower than with 
checksums, like just over 100MB/s.


from primary
messages: http://news.digsys.bg/~admin/hast/test31may-2/b1a-messages
netstat -in: http://news.digsys.bg/~admin/hast/test31may-2/b1a-netstat-in
netstat-s: http://news.digsys.bg/~admin/hast/test31may-2/b1a-netstat-s

from secondary
messages: http://news.digsys.bg/~admin/hast/test31may-2/b1b-messages
netstat -in: http://news.digsys.bg/~admin/hast/test31may-2/b1b-netstat-in
netstat-s: http://news.digsys.bg/~admin/hast/test31may-2/b1b-netstat-s

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: PCIe SATA HBA for ZFS on -STABLE

[Daniel Kalchev]

On 01.06.11 09:34, TJ Varghese wrote:



SuperMicro AOC-USAS2-L8i works exceptionally well.  These are 8-port HBAs
using the LSI1068 chipset, supported by the mpt(4) driver.  Support 3 Gpbs
SATA/SAS, using multi-lane cables (2 connectors on the card, each connector
supports 4 SATA ports), hot-plug, hot-swap.

The USAS2 (6Gbps) is supported by the mps driver (on -CURRENT, not sure if
it's in 8-STABLE yet). Perhaps you're referring to the earlier USAS which
does 3Gbps and is supported by the mpt driver.



One should also bear in mind, that the 1068e based controllers 
(AOC-USAS-L8i) apparently have 2TB drive size limitation, therefore 
cannot be used with current 3TB (and who knows what capacities by the 
end of the year) drives. Otherwise, this is an well supported, high 
performance, stable and relatively cheap HBA to consider. The SAS2 (LSI 
2008 based) HBAs are also good, despite some firmware issues (mostly are 
related to use with SAS expanders) and do not have obvious limitations 
yet. These might be just a bit more expensive, but my Supermicro 
supplier advised delivery times for the older SAS (3Gbps) versions would 
be much longer.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

On 30.05.11 21:42, Mikolaj Golub wrote:

  DK  One strange thing is that there is never established TCP connection
  DK  between both nodes:

  DK  tcp4   0  0 10.2.101.11.48939  10.2.101.12.8457   
FIN_WAIT_2
  DK  tcp4   0   1288 10.2.101.11.57008  10.2.101.12.8457   
CLOSE_WAIT
  DK  tcp4   0  0 10.2.101.11.46346  10.2.101.12.8457   
FIN_WAIT_2
  DK  tcp4   0  90648 10.2.101.11.13916  10.2.101.12.8457   
CLOSE_WAIT
  DK  tcp4   0  0 10.2.101.11.8457   *.*LISTEN

It is normal. hastd uses the connections only in one direction so it calls
shutdown to close unused directions.
So the TCP connections are all too short-lived that I can never see a 
single one in ESTABLISHED state? 10Gbit Ethernet is indeed fast, so this 
might well be possible...

I suppose when checksum is enabled the bottleneck is cpu, the triffic rate is 
lower and the problem is not triggered.
I was thinking something like this. My later tests seems to suggest that 
when the network transfer rate is mugh higher than disk transfer rate 
this gets triggered.



Hash mismatch message suggests that actually you were using checksum then,
weren't you?
Yes, this occurs only when checksums are enabled. Happens with both 
crc32 and sha256.

I would like to look at full logs for some rather large period, with several
cases, from both primary and secondary (and be sure about synchronized time).
I have made sure clocks are synchronized and am currently running on a 
freshly rebooted nodes (with two additional SATA drives at each node) -- 
so far some interesting findings, like  I get hash errors and 
disconnects much more frequent now. Will post when an bonnie++ run on 
the ZFS filesystem on top of the HAST resources finishes.

Also, it might worth checking that there is no network packet corruption (some 
strange things in netstat -di, netstat -s, may be copying large files via net 
and comparing checksums).

I will post these as well, however so far no indication of any network 
problems was seen, no interface errors etc. Might be also the ix driver 
is not reporting such, of course.


One additional note: while playing with this setup, I tried to simulate 
local disk going away in the hope HAST will switch to using the remote 
disk. Instead of asking someone at the site to pull out the drive, I 
just issued on the primary


hastctl role init data0

which resulted in kernel panic. Unfortunately, there was no sufficient 
dump space for 48GB. I will re-run this again with more drives for the 
crash dump. Anything you want me to look for in particular? (kernels 
have no KDB compiled in yet)


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

On 31.05.11 17:08, Mikolaj Golub wrote:

As I wrote privately, it would be nice to see both netstat and hast logs (from 
both nodes) for the same rather long period, when several cases occured. It 
would be good to place them somewere on web so other guys could access them 
too, as I will be offline for 7-10 days and will not be able to help you until 
I am back.


The test finished running for almost three hours, and so here is the 
collected data:


(for the duration of test, on the secondary node)
systat -if
/0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
 Load Average

  Interface   Traffic   PeakTotal
lo0  in  0.000 KB/s  0.000 KB/s1.126 KB
 out 0.000 KB/s  0.000 KB/s1.126 KB

ix1  in  0.003 KB/s230.590 MB/s  614.688 GB
 out 0.054 KB/s  7.425 MB/s   19.910 GB

   igb0  in  0.025 KB/s  3.636 KB/s  566.897 KB
 out 0.072 KB/s  4.296 KB/s1.091 MB


The primary node is b1a, the secondary node is b1b.
kernel (built just after csup update):

FreeBSD b1a 8.2-STABLE FreeBSD 8.2-STABLE #1: Mon May 30 14:17:50 EEST 
2011 root@b1a:/usr/obj/usr/src/sys/GENERIC  amd64


from primary
messages: http://news.digsys.bg/~admin/hast/test31may/b1a-messages
netstat -in: http://news.digsys.bg/~admin/hast/test31may/b1a-netstat -in
netstat-s: http://news.digsys.bg/~admin/hast/test31may/b1a-netstat-s

from secondary
messages: http://news.digsys.bg/~admin/hast/test31may/b1b-messages
netstat -in: http://news.digsys.bg/~admin/hast/test31may/b1b-netstat -in
netstat-s: http://news.digsys.bg/~admin/hast/test31may/b1b-netstat-s


  DK  One additional note: while playing with this setup, I tried to
  DK  simulate local disk going away in the hope HAST will switch to using
  DK  the remote disk. Instead of asking someone at the site to pull out the
  DK  drive, I just issued on the primary

  DK  hastctl role init data0

  DK  which resulted in kernel panic. Unfortunately, there was no sufficient
  DK  dump space for 48GB. I will re-run this again with more drives for the
  DK  crash dump. Anything you want me to look for in particular? (kernels
  DK  have no KDB compiled in yet)

Well, removing physical disk (device /dev/gpt/data0 consumed by hastd
dissapears) and switching a resource to init role (devive /dev/hast/data0
consumed by FS dissapears) are two different things. Sure you should not
normally change the resource role (destroy hast device) before unmounting
(exporting) FS.
Then how do I proceed with a failed drive? Or  a flaky drive that is 
still visible to the OS, that I want to remove from HAST and replace 
with a different one? How do I ask HAST to switch I/O to the secondary? 
Is there other way to get a drive out of HAST? In any case, even if this 
is not allowed operation, it should not panic.


I am now going to reboot and run the same tests without checksums.

Daniel

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: HAST instability

[Daniel Kalchev]

Some further investigation:

The HAST nodes do not disconnect when checksum is enabled (either crc32 
or sha256).


One strange thing is that there is never established TCP connection 
between both nodes:


tcp4   0  0 10.2.101.11.48939  10.2.101.12.8457   FIN_WAIT_2
tcp4   0   1288 10.2.101.11.57008  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  0 10.2.101.11.46346  10.2.101.12.8457   FIN_WAIT_2
tcp4   0  90648 10.2.101.11.13916  10.2.101.12.8457   CLOSE_WAIT
tcp4   0  0 10.2.101.11.8457   *.*LISTEN

When using sha256 one CPU core is 100% utilized by each hastd process, 
while 70-80MB/sec per HAST resource is being transferred (total of up to 
140 MB/sec traffic for both);


When using crc32 each CPU core is at 22% utilization;

When using none as checksum, CPU usage is under 10%

Eventually after many hours, got corrupted communication:

May 30 17:32:35 b1b hastd[9827]: [data0] (secondary) Hash mismatch.
May 30 17:32:35 b1b hastd[9827]: [data0] (secondary) Unable to receive 
request data: No such file or directory.
May 30 17:32:38 b1b hastd[9397]: [data0] (secondary) Worker process 
exited ungracefully (pid=9827, exitcode=75).


and

May 30 17:32:27 b1a hastd[1837]: [data0] (primary) Unable to receive 
reply header: Operation timed out.
May 30 17:32:30 b1a hastd[1837]: [data0] (primary) Disconnected from 
10.2.101.12.
May 30 17:32:30 b1a hastd[1837]: [data0] (primary) Unable to send 
request (Broken pipe): WRITE(99128470016, 131072).


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

HAST instability

[Daniel Kalchev]

I am trying to get a basic HAST setup working on 8-stable (as of today). 
Hardware is two supermicro blades, each with 2x Xeon E5620 processors, 
48GB RAM, integrated LSI2008 controller, two 600GB SAS2 Toshiba drives, 
two Intel gigabit interfaces and two Intel 10Gbit interfaces.


On each of the drives there is an GPT partition intended to be used by 
HAST.  Each host thus has two HAST resources, data0 and data1 
respectively. HAST is run over the 10Gbit interfaces, connected via the 
blade chasis 10Gbit switch.


/etc/hast.conf is

resource data0 {
on b1a {
local /dev/gpt/data0
remote 10.2.101.12
}
on b1b {
local /dev/gpt/data0
remote 10.2.101.11
}
}

resource data1 {
on b1a {
local /dev/gpt/data1
remote 10.2.101.12
}
on b1b {
local /dev/gpt/data1
remote 10.2.101.11
}
}

On top of data0 and data1 I run ZFS mirror, although this doesn't seem 
to be relevant here.


What I am observing is very jumpy performance, both nodes often 
disconnect with primary:


May 29 13:06:33 b1b hastd[2372]: [data0] (primary) Unable to receive 
reply header: Socket is not connected.
May 29 13:06:33 b1b hastd[2372]: [data0] (primary) Unable to send 
request (Broken pipe): WRITE(60470853632, 131072).
May 29 13:06:33 b1b hastd[2372]: [data0] (primary) Disconnected from 
10.2.101.11.
May 29 13:06:33 b1b hastd[2372]: [data0] (primary) Unable to write 
synchronization data: Socket is not connected.


on secondary:

May 29 03:03:14 b1a hastd[28357]: [data1] (secondary) Unable to receive 
request header: RPC version wrong.
May 29 03:03:19 b1a hastd[11659]: [data1] (secondary) Worker process 
exited ungracefully (pid=28357, exitcode=75).
May 29 03:05:31 b1a hastd[35535]: [data0] (secondary) Unable to receive 
request header: RPC version wrong.
May 29 03:05:36 b1a hastd[11659]: [data0] (secondary) Worker process 
exited ungracefully (pid=35535, exitcode=75).


When it works, replication rate observed with 'systat -if' is over 
140MB/sec (perhaps limited by drives write troughput)


The only reference to this error messages I found in 
http://lists.freebsd.org/pipermail/freebsd-stable/2010-November/059817.html, 
and that thread indicated the fix was commited.


About the only tuning these machines have is to set 
kern.ipc.nmbclusters=51200, because with the default values 10Gbit 
interfaces would not work and anyway the system would run out of mbufs.


Has anyone observed something similar? Any ideas how to fix it?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: mps driver instability under stable/8

[Daniel Kalchev]

On 11.05.11 00:38, Dmitry Morozovsky wrote:

On Tue, 10 May 2011, Daniel Kalchev wrote:

DKWell, using
DKhttp://kb.lsi.com/KnowledgebaseArticle16414.aspx
DKI downgraded to version 8-fixed, and at least topology errors disappear.
DK  
DK  Would this work with the Supermicro integrated LSI2008, like in X8DTH-6F?
DK  Mine came with firmware version 7, is there instability to be expected?

I suppose you can upgrade to fixed firmware from the URL above; at least, I'd
been in mostly the same situation: SM server with onboard LSI and LSI expander,
and so far flashing 8-fixed firmware is good for me, at least machine did not
hang in find-related tasks as it were before...


This sort of worked, but:

Version 9 have me very poor performance with drives. While drives 
typically do 150MB/s sequential, I could never get more than half of 
that. Even when using only one drive. At some point, after reboot it 
started discovering four expanders.. It's BIOS complained at boot time 
loudly, etc. Performance did not change.


Version 8 (supposedly the fixed one), too had poor performance, had the 
same issues with multiple expanders, but also the disks devices were 
multiplied four times.


So I went back to flash the original SMC2008 firmware (version 7), which 
went fine, performance is ok, nothing is multiplied :) Weird, it claims 
to be IR firmware, but I ended up in IT mode, which is what I need anyway.


Looking at the batch file for the SMC firmware, it does erase with 7, 
instead of with 6. Probably it is necessary to erase the flash more 
than LSI suggests. Haven't tried yet.


Should I worry for not running the latest firmware?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: mps driver instability under stable/8

[Daniel Kalchev]

On 03.05.11 20:28, Dmitry Morozovsky wrote:

Well, using
http://kb.lsi.com/KnowledgebaseArticle16414.aspx
I downgraded to version 8-fixed, and at least topology errors disappear.

Would this work with the Supermicro integrated LSI2008, like in 
X8DTH-6F? Mine came with firmware version 7, is there instability to be 
expected?


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: correct way to setup gmirror on 7.4?

[Daniel Kalchev]

On 28.04.11 01:30, Freddie Cash wrote:

gmirror doesn't touch the start of the disk, but saves it's metadata
in the last sector of the disk, and creates a new GEOM provider that's
one sector shorter.

GPT stores it's partition table in the first sector of the disk, and
saves a backup copy of it in the last sector of the disk.

This looks like layering issue to me.

In theory, both gmirror and gpt should work on 'providers'. So if you 
give an gmirrored provider to gpt it should touch the last sector of the 
gmirror, but not the last sector of the disk - and not complain. It 
should not even be able to see the last sector of the real disk.


Is this hard to fix?

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS root on MB Intel S3420GP

[Daniel Kalchev]

On 17.04.11 21:54, Jeremy Chadwick wrote:

I don't recommend enabling ahci.ko after the OS has already been
installed on an adX disk, simply because I believe the combination of
GEOM+CAM+ahci may show different geometry details than GEOM+ata would.

With the disclaimer that I haven't studied all code to confirm this, but 
adX and adaX (with ahci.ko) have always played the same for me - 
geometry wise. I believe the translation is always 1:1. At least on 
modern SATA drives -- olderdrives from the transitional epoch of 
different CHS, LBA etc experiments might behave differently.


Migration from gmirror to zfs(root) is trivial. Having only two 
gmirror-ed disks, you could replace that with ZFS mirror. Having 4-way 
gmirror can let you do a 4-disk ZFS migration (raidz1 or raidz2).


I tend to do all of my new systems with ZFS on root. Probably because I 
no longer have to do systems that run with few MB of RAM :) - the I 
would use UFS alwats.


One nice feature of ZFS I have discovered is with USB flash media. You 
are not typically supposed to write much to that media, but using UFS on 
USB sticks is awful. On contrary, when used with ZFS, the USB sticks 
behave much differently, because ZFS will group writes and not do silly 
things like issue lots of 512 byte writes. So, you may have complete 
development system on an USB stick, or a pair of these. The only real 
trouble with USB stisks is that some motherboards behave unpredictable 
as to boot order, but this is improving.


My recent install procedure never used the FreeBSD release media. 
Instead, I have created myself USB stick distribution media (can work 
with CD/DVD as well, or over diskless boot), using a procedure like this:


- on an up to date FreeBSD system, do make buildworld; make buildkernel
- insert the USB stick, create filesystem. UFS or ZFS, doesn't matter
- make installworld, make installkernel, make distribution to the 
mounted USB stick

- fix fstab and loader.conf on the USB stick
(optional)
- copy over src and ports tree ro the USB stick
- do in place rebuild/reinstall of the world, kernel and any packages 
you may need

(end optional)
- put the USB stick in my pocket

Next time, I need to install a server on site, take the USB stick out of 
my pocket, plug it in one of the USB ports, boot the server, run small 
script (similar to that in the root-on-zfs guides), create ZFS on root 
and am done with it. I would use either pair of USB sticks for that, a 
separate set of (two) disk drives, or 'all' of the system's drives for 
this install, depending on the systems intended usage and hardware 
configuration. Typically on a multi-bay system I would do the root on a 
separate set of disks/USB flash in order to simplify documentation and 
operator training.


If the system needs to be installed remotely, I would typically use rKVM 
(most rackmount-intended motherboards have this functionality), 
attaching either the prepared USB stick or it's image are virtual media 
and booting over that the new system.


Many of these things can be done differently of course, it will depend 
on circumstances, but I hope the general idea is useful.


Jeremy, one of the reasons I switched many systems to pure ZFS was 
related to the memory allocation troubles between USF and ZFS we 
observed for quite long time. Having pure ZFS system eliminates these 
issues completely. I do have still few mixed systems - only laziness and 
lack of (down)time prevented me from switching these to pure-ZFS too. My 
rationale is that if something breaks, it is likely it will break with 
or without ZFS on root. In either case, I would have to load FreeBSD 
from other media. So it does not matter from where you boot the system.


One final note, on ZFS pool naming. I would traditionally name my 
root-on-zfs pool 'system'. However, this makes it difficult and error 
prone to create new zpools. Therefore, I have zpool of 'boot' for my 
install USB sticks and also have addopted the practice of naming the 
root pool after it's system's name. For example 'hostABCroot' or 
'hostABCsystem'. This has never been an issue with UFS, until filesystem 
labels appeared and still not that many people use these. With ZFS, you 
cannot escape.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS HAST config preference

[Daniel Kalchev]

On 06.04.11 15:55, Pete French wrote:

Or, I could use ZFS volumes and run HAST on top of these. This means,

that on each blade, I will have an local ZFS pool. Let's call this setup2.

...you would need to put a filesystem on to of the
HAST filesystem though, what would that be ?


Thanks for your comments Pete.

The idea here is to have local ZFS pool(s) on each of the nodes. Then, 
in each local ZFS pool create ZFS volumes and use these for HAST. On the 
HAST device, I would again run ZFS, just without any redundancy. Just to 
avoid having to run fsck after switching roles.


This should, I guess, permit safe use of cache and log devices for 
pools, thus increasing performance. However, no idea how reliability 
will be affected if sudden loss of one node happens, especially the 
active node.


It seems I am going with setup1 for now, going live in about a week.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

ZFS HAST config preference

[Daniel Kalchev]

This is more of an proof of concept question:

I am building an redundant cluster of blade servers, and toy with the 
idea to use HAST and ZFS for the storage.


Blades will work in pairs and each pair will provide various services, 
from SQL databases, to hosting virtual machines (jails and otherwise). 
Each pair will use CARP for redundancy.


My original idea was to set up blades so that they run HAST on pairs of 
disks, and run ZFS in number of mirror vdevs on top of HAST. The ZFS 
pool will exist only on the master HAST node. Let's call this setup1.


Or, I could use ZFS volumes and run HAST on top of these. This means, 
that on each blade, I will have an local ZFS pool. Let's call this setup2.


Third idea would be to have the blades completely diskless, boot from 
separate boot/storage server and mount filesystems or iSCSI volumes as 
needed from the storage server. HAST might not be necessary here. ZFS 
pool will exist on the storage server only. Let's call this setup3.


While setup1 is most straightforward, it has some drawbacks:
- disks handled by HAST need to be either identical or have matching 
partitions created;
- the 'spare' blade would do nothing, as it's disk subsystem will be 
gone as long as it is HAST slave. As the blades are quite powerful (4x8 
core AMD) that would be wasteful, at least in the beginning.


With setup2, I can get away with different size disks in each blade. All 
blades can also be used for whatever additional processing, shared data 
will be only presented by HAST to whichever node needs it, for 
important services. One drawback here:
- can't just pull off one of the blades, without stopping/transferring 
first all of it's services.


It seems that in larger scale, setup3 would be best. I am not yet here, 
although close (the storage server is missing).


HAST replication speed should not be an issue, there is 10Gbit network 
between the blade servers.


Has anyone already setup something similar? What was the experience? 
There were recently some bugs that sort of plagued setup1, but these 
seem to be resolved now.


Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: mps(4) driver (LSI 6Gb SAS) commited to stable/8

[Daniel Kalchev]

On 19.02.11 02:46, Bill Desjardins wrote:


Thank you Ken for getting this done. Any plan to support the LSI 9240
(skinny) cards?

I believe this is already supported by the mfi driver on the LSI site, 
it has support for the skinny and recently got a 64bit version. You may 
wish to try it.

Maybe these additions could be ported to the FreeBSD driver as well.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: 3TB disc and block alignment

[Daniel Kalchev]

da0:Hitachi HUA723030ALA640  Fixed Direct Access SCSI-5 device
da0: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)


Thanks -- is it also possible to have something like

da0: 2861588MB (732566646 4096 byte sectors: 255H 63S/T 364801C)


According to Hitachi, this is an 512b drive.

Daniel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: LSI SAS 2008 (mfi) on SuperMicro X8SI6-F

[Daniel Kalchev]

I have sucessfully used that motherboard with FreeBSD 9 and the mps 
driver. The mfi driver found on the LSI site does not support this 
controller.


Daniel

PS: My experiments were with the X8DTL-6F motherboard and Supermicro 
chassis with E16 expander. There is no reason the HBA chip in the single 
processor motherboard to be different.


On 16.02.11 17:48, Dmitry Morozovsky wrote:

Dear colleagues,

are there any success stories with using SuperMicro LSI SAS with stable/8 ?

I tried mfi drivers sources from LSI site (had to add one #include to make
kdump compilation happy) with no success

pciconf info is

none@pci0:0:2:0:class=0x010700 card=0x00721000 chip=0x00721000
rev=0x02 hdr=0x00
 vendor = 'LSI Logic (Was: Symbios Login, NCR)'
 class  = mass storage
 subclass   = SAS

Thanks in advance!


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: KERN - mfi driver for Dell raid h200 on r210 servers

[Daniel Kalchev]

On 30.1.2011 г. 13:30 ч., Damien Fleuriot wrote:


Ok I've loaded the newly patched mfi.ko and booted a MFS image.
Here's the relevant snip from dmesg.run :
at
mfi0:Dell PERC H200 Integrated  port 0xfc00-0xfcff mem
0xdf2b-0xdf2b,0xdf2c-0xdf2f irq 16 at device 0.0 on
pci1
mfi0: Megaraid SAS driver Ver 3.00
mfi0: firmware stuck in state 0
mfi0: Firmware not in READY state, error 6


I'll have to try mps then, unless someone has an idea about this
stuck in state 0 thing.
You should first try the mfi driver found in the LSI site 
(http://www.lsi.com). It does support few more variants than the FreeBSD 
driver.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks

[Daniel Kalchev]

For pure storage, that is a place you send/store files, you don't really 
need the ZIL. You also need the L2ARC only if you read over and over 
again the same dataset, which is larger than the available ARC (ZFS 
cache memory). Both will not be significant for 'backup server' 
application, because it's very unlikely to do lots of SYNC I/O (where 
separate ZIL helps), or serve the same files back (where the L2ARC might 
help).


You should also know that having large L2ARC requires that you also have 
larger ARC, because there are data pointers in the ARC that point to the 
L2ARC data. Someone will do good to the community to publish some 
reasonable estimates of the memory needs, so that people do not end up 
with large but unusable L2ARC setups.


It seems that the upcoming v28 ZFS will help greatly with the ZIL in the 
main pool..


You need to experiment with the L2ARC (this is safe with current v14 and 
v15 pools) to see if your usage will see benefit from it's use. 
Experimenting with ZIL currently requires that you recreate the pool. 
With the experimental v28 code things are much easier.


On 06.01.11 15:11, Damien Fleuriot wrote:

I see, so no dedicated ZIL device in the end ?

I could make a 15gb slice for the OS running UFS (I don't wanna risk
losing the OS when manipulating ZFS, such as during upgrades), and a
25gb+ for L2ARC, depending on the disk.

I can't afford a *dedicated* drive for the cache though, not enough room
in the machine.


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org