Re: Hardware.
Hi, On Sat, Apr 23, 2011 at 07:00:33AM +1000, David Crosswell wrote: I've been checking out your hardware page here: http://tinyurl.com/3qbp9ck and wanting to know which of these supermicro opteron server boards work best with Dragonfly off the shelf. They all do. I've never had trouble with Supermicro boards. The only parts not working out-of-the box I know of are the included SAS adapters on certain models. You'll be better of buying a real RAID card if you want to go that route. I'm looking at building a small server to familiarise myself with all the BSDs, for study purposes, and although I like to play, I don't have a lot of leeway as far as cash goes to make any mistakes. The cheapest board/cpu combination is the X7SPA-H: http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPA.cfm?typ=H The included Atom CPU is not very fast for compilation, so you may want to with this combo instead: X7SBL-LN2 + Core 2 Duo http://www.supermicro.com/products/motherboard/Xeon3000/3200/X7SBL-LN2.cfm I use both models in small servers, they're great. -- Francois Tigeot
Re: Filesystems
This post has me so perplexed, I just have to explore further. On 4/23/2011 2:15 AM, David Crosswell wrote: Yes, I understand that. I'm looking forward to doing something with Hammer, but I've spoken to a couple of guys at the local Users group who swear they'll never use anything else but ZFS You are apparently talking about random people at some local club. Why is their preference of filesystem impacting your intention to try out Hammer?What makes their opinion so special? - got it running on FreeBSD and I looked at Dragonfly with UFS and Hammer and thought with ZFS they'd have every scenario covered. Who is they? DragonFly community? Linux is working to incorporate ZFS compatibility into the kernel, No, Linux is not. As long as ZFS has the CDDL license, it won't be incorporated into the kernel. People are working on putting ZFS in a module that users can manually load in to work around license issues. It's not a technical incompatibility, it's a license incompatibility for which there is no solution other than Oracle changing the ZFS license. and even with various filesystem developers looking at substantial jail sentences for killing their wives, they've still got an over abundance of filesystems. Again, why is Linux filesystem situation relevant to DragonFly? How does Linux having too many filesystems (in your opinion) relate to DragonFly not having ZFS? I'm not following any logic train. It's going to have to wait for a while before I learn C then. Regards, What? There's going to be a delay in your learning C because of which following reason? A) Reiser was convicted of killing his wife B) Linux has too many filesystems C) Linux is putting ZFS in the kernel D) Random people in local user groups swear by ZFS E) Nobody is bothering to import ZFS to DragonFly. F) Other? -- John David Crosswell. On 23/04/2011, Justin Sherrilljus...@shiningsilence.com wrote: It's certainly possible. Nobody's working on it right now, to my knowledge. I'm more interesting in seeing Hammer grow, so I'm not that concerned about it. On Fri, Apr 22, 2011 at 5:03 PM, David Crosswell david.crosswe...@gmail.com wrote: I understand the availability of UFS and Hammer in the Dragonfly environment, but is ZFS possible, or are there any plans to facilitate it if it isn't? Regards, David Crosswell. -- In a world without walls and fences, what need have we for Windows or Gates? http://www.weavers-web.org
Re: Filesystems
On Sat, Apr 23, 2011 at 2:15 AM, David Crosswell david.crosswe...@gmail.com wrote: Yes, I understand that. I'm looking forward to doing something with Hammer, but I've spoken to a couple of guys at the local Users group who swear they'll never use anything else but ZFS - got it running on FreeBSD and I looked at Dragonfly with UFS and Hammer and thought with ZFS they'd have every scenario covered. Which version of Hammer was that? ;-) (in their test). Hammer has functions which are not in ZFS and are superb and Matt described quite well in one post why RAID is not catch all solution. With ZFS they depend on Oracle as it's released under CDDL and there are clauses which Oracle can use to close all ZFS ports if they wants (for example when it will start to be too much big concurrent for their own system). Anyway what are your options with ZFS - 1) Solaris with price from 1000$/socket/year without license it's unusable and just crazy people use systems without patches/updates in production connected to Internet 2) Illumos/OpenIndiana is good alternative and has some big companies behind to be able so stay somewhat resistent to Oracle 3) FreeBSD probably best port outside of Solaris, but main porter died (sad) and he was great regarding internals so it's quite harder now for them 4) Linux with some module or through FUSE. Can't be in kernel because of license and they don't care anymore as there is btrfs already 5) NetBSD still unusable, a lot of panics and long way ahead Linux is working to incorporate ZFS compatibility into the kernel, and even with various filesystem developers looking at substantial jail sentences for killing their wives, they've still got an over abundance of filesystems. see 4) above, ReiserFS is maintained quite well by community. What's the point to have all available filesystems included in some OS? Of course except of bigger mess in some systems ;-) MS-DOS for compatibility on USB flash disk or memory cards, NTFS for compatibility with Windows and iso9660/udf for CD/DVD media. Now about filesystems for disks in PCs/servers 1) ext2/3/4 for simplicity you can say that all are same 2) XFS 3) ReiserFS 4) ffs/ufs versions 1 and 2 and their brother HFS in Apple That's all because even those journaled filesystems are same/similar regard the design. Why it doesn't matter how much of those fs is supported in some OS? Because all of them are old by design and needs for modern storage. That's why ZFS/Hammer/btrfs born so you must care about those regarding feature and you can't care about those which are not in kernel because of speed and other issues via FUSE, Puffs, module or whatever which are fine for tests only. So you will end with what? Solaris, Ilumos/OI, FreeBSD for ZFS, DragonFlyBSD for Hammer and btrfs for Linux It's going to have to wait for a while before I learn C then. You don't need to know C to start learning ZFS/Hammer/btrfs all you need is a system (eg. in VM) which has mature implementation to play around with that and read man pages, papers, whatever. Regards, David Crosswell. On 23/04/2011, Justin Sherrill jus...@shiningsilence.com wrote: It's certainly possible. Nobody's working on it right now, to my knowledge. I'm more interesting in seeing Hammer grow, so I'm not that concerned about it. On Fri, Apr 22, 2011 at 5:03 PM, David Crosswell david.crosswe...@gmail.com wrote: I understand the availability of UFS and Hammer in the Dragonfly environment, but is ZFS possible, or are there any plans to facilitate it if it isn't? Regards, David Crosswell. -- In a world without walls and fences, what need have we for Windows or Gates? http://www.weavers-web.org -- In a world without walls and fences, what need have we for Windows or Gates? http://www.weavers-web.org
Re: Filesystems
On 23 April 2011 20:06, John Marino dragonfly...@marino.st wrote: This post has me so perplexed, I just have to explore further. Hello John, On 4/23/2011 2:15 AM, David Crosswell wrote: Yes, I understand that. I'm looking forward to doing something with Hammer, but I've spoken to a couple of guys at the local Users group who swear they'll never use anything else but ZFS You are apparently talking about random people at some local club. Yes, a 'NIX users group. Why is their preference of filesystem impacting your intention to try out Hammer? No, I didn't say anything like that. What makes their opinion so special? Because they are highly qualified. One is a professor of computer sciences and runs the University BSD servers. with the help of his more advanced students to give them a little cash in hand to keep them from starving to death and give them real world experience. - got it running on FreeBSD and I looked at Dragonfly with UFS and Hammer and thought with ZFS they'd have every scenario covered. Who is they? DragonFly community? I can't, for the life of me, see any other party referred to in that paragraph. Linux is working to incorporate ZFS compatibility into the kernel, No, Linux is not. As long as ZFS has the CDDL license, it won't be incorporated into the kernel. Yes, it will be. People are working on putting ZFS in a module that users can manually load in to work around license issues. Up until halfway through the 2,4 kernel, that's how it was done. If you wanted ext3, for example, it was as a module, because only ext2 was compiled in. It's not a technical incompatibility, it's a license incompatibility for which there is no solution other than Oracle changing the ZFS license. Ellison's not about to let anything go, because that's the way Ellison is. The CDDL will stay as it is. It is compatible with the BSD licence however, which is why it can be employed with Debian's KFreeBSD config., for example. There's nothing wrong with loading it as a module. Debian isn't exactly in the Microsoft patent heavy market scenario, but still supplies access to contrib and non-free apps. What's so different here? and even with various filesystem developers looking at substantial jail sentences for killing their wives, they've still got an over abundance of filesystems. Again, why is Linux filesystem situation relevant to DragonFly? I didn't say it was here. I seem to have stated that Linux have an over abundance of them. From what I can see a F.S suitable for end users and smaller server situations, another suited to the cluster environment and another that has some valuable features wouldn't do any harm. I can't see any use for more than three. Can you? How does Linux having too many filesystems (in your opinion) Here. You can count them if you like: *http://tinyurl.com/mym5k* relate to DragonFly not having ZFS? I'm not following any logic train. The train is there and quite clearly expressed, but you either can't see it, or, for some obscure, though no doubt interesting reason, don't want to. It's going to have to wait for a while before I learn C then. Regards, What? There's going to be a delay in your learning C because of which following reason? A) Reiser was convicted of killing his wife B) Linux has too many filesystems C) Linux is putting ZFS in the kernel D) Random people in local user groups swear by ZFS E) Nobody is bothering to import ZFS to DragonFly. F) Other? None of them, quite obviously, and I fail to see why you would need to know what details of my personal existence prevent me from doing so. Have you ever been accused of being a drama queen before this? Regards, David Crosswell. -- In a world without walls and fences, what need have we for Windows or Gates? http://www.weavers-web.org
Re: Hammer deduplication needs for RAM size
On Fri, Apr 22, 2011 at 10:12 PM, Matthew Dillon dil...@apollo.backplane.com wrote: :Hi all, : :can someone compare/describe need of RAM size by deduplication in :Hammer? There's something interesting about deduplication in ZFS :http://openindiana.org/pipermail/openindiana-discuss/2011-April/003574.html : :Thx The ram is basically needed to store matching CRCs. The on-line dedup uses a limited fixed-sized hash table to remember CRCs, designed to match recently read data with future written data (e.g. 'cp'). The off-line dedup (when you run 'hammer dedup ...' or 'hammer dedup-simulate ...' will keep track of ALL data CRCs when it scans the filesystem B-Tree. It will happily use lots of swap space if it comes down to it, which is probably a bug. But that's how it works now. Actual file data is not persistently cached in memory. It is read only when the dedup locates a potential match and sticks around in a limited cache before getting thrown away, and will be re-read as needed. Their discussion continues and they talk about rule 1 - 3GB of RAM per 1TB of data. Regarding this http://blogs.sun.com/roch/entry/dedup_performance_considerations1 it looks like those data are persistent as cache in memory. So is this a reason for higher RAM usage with ZFS and dedup when comparing with Hammer? -Matt Matthew Dillon dil...@backplane.com
Re: Filesystems
On Sat, Apr 23, 2011 at 7:46 AM, David Crosswell david.crosswe...@gmail.com wrote: Have you ever been accused of being a drama queen before this? Hey, it's the Internet. People get defensive easily. What David was asking originally - is ZFS going to be ported/in the process of porting to DragonFly - is a normal question, and one people have asked before. It wouldn't be easy to port, but it would be nice. Hammer is much less resource-intensive than ZFS, but ZFS has some really nice management commands. I would be happy to see both on DragonFly, or to at least take an example from ZFS's expressive commands for management for handling Hammer to meet those same needs. Hammer has some wonderful features, and we have a good open source filesystem that compares well to something produced by one of the world's larger Unix vendors. Rather than spending time complaining about someone's perfectly valid desire for ZFS, we should be looking at how we can make Hammer better. Antonio Huete has been working on a libhammer library to bring out some of the Hammer functions, which would make those richer management tools easier to put together. (nudge, nudge, tuxillo)
Re: Filesystems
On Sat, Apr 23, 2011 at 3:43 AM, Tomas Bodzar tomas.bod...@gmail.com wrote: 3) FreeBSD probably best port outside of Solaris, but main porter died (sad) and he was great regarding internals so it's quite harder now for them The main porter of ZFS to FreeBSD is Pawel Jakub Dawidek (probably spelt a bit wrong) aka p...@freebsd.org, who is most certainly still alive, and continuing work on ZFS, HAST, GEOM_GATE, and other interesting storage stuff. -- Freddie Cash fjwc...@gmail.com
Re: Filesystems
On Sat, Apr 23, 2011 at 8:38 PM, Freddie Cash fjwc...@gmail.com wrote: On Sat, Apr 23, 2011 at 3:43 AM, Tomas Bodzar tomas.bod...@gmail.com wrote: 3) FreeBSD probably best port outside of Solaris, but main porter died (sad) and he was great regarding internals so it's quite harder now for them The main porter of ZFS to FreeBSD is Pawel Jakub Dawidek (probably spelt a bit wrong) aka p...@freebsd.org, who is most certainly still alive, and continuing work on ZFS, HAST, GEOM_GATE, and other interesting storage stuff. Eh wrong technology in my head. I was talking about original porter of DTrace to FreeBSD. -- Freddie Cash fjwc...@gmail.com
Re: Filesystems
On 24 April 2011 03:39, Justin Sherrill jus...@shiningsilence.com wrote: On Sat, Apr 23, 2011 at 7:46 AM, David Crosswell david.crosswe...@gmail.com wrote: Have you ever been accused of being a drama queen before this? Hey, it's the Internet. People get defensive easily. What David was asking originally - is ZFS going to be ported/in the process of porting to DragonFly - is a normal question, and one people have asked before. It wouldn't be easy to port, but it would be nice. Hammer is much less resource-intensive than ZFS, but ZFS has some really nice management commands. I would be happy to see both on DragonFly, or to at least take an example from ZFS's expressive commands for management for handling Hammer to meet those same needs. Hammer has some wonderful features, and we have a good open source filesystem that compares well to something produced by one of the world's larger Unix vendors. Rather than spending time complaining about someone's perfectly valid desire for ZFS, we should be looking at how we can make Hammer better. Antonio Huete has been working on a libhammer library to bring out some of the Hammer functions, which would make those richer management tools easier to put together. (nudge, nudge, tuxillo) Thank you for your clarity and consideration, Justin. Regards, David. -- In a world without walls and fences, what need have we for Windows or Gates? http://www.weavers-web.org
Re: Fwd: pkgbox64 pkgsrc 2011Q1 DragonFly 2.10.0/x86_64 2011-04-12 03:54
On 4/12/11 1:13 PM, Justin Sherrill wrote: Packages breaking the most other packages Package Breaks Maintainer - pkgtools/rpm2pkg 128 t...@netbsd.org databases/postgresql84-client103 a...@netbsd.org multimedia/xine-lib 62 pkgsrc-us...@netbsd.org lang/ocaml37 a...@netbsd.org lang/mono 31 kef...@netbsd.org print/cups21 s...@netbsd.org sysutils/libgtop 18 pkgsrc-us...@netbsd.org devel/libthrift9 tonne...@netbsd.org www/w3m7 uebay...@netbsd.org textproc/cabocha 7 oba...@netbsd.org I looked into a couple of the big breakers to see if their was anything I could fix. pkgtools/rpm2pkg was fixed for DragonFly in master later in the day on April 12th. Just missed it :) lang/ocaml has an upstream fix they are releasing in 3.12.1 BUG: http://caml.inria.fr/mantis/view.php?id=5237 sysutils/libgtop had a fix April 13 th, not sure if it pertains to DragonFly or not. print/cups seems like a silly linking error, but I didn't look to much further into it. I'm looking into lang/mono now, I know this has been a sort of on going project to get it to build on DragonFly. It's pretty awesome how close that brings us to relatively few broken packages. I remember when were using pacman wa back, It's come a long way :) - Brian
