Re: [OmniOS-discuss] LX Zones and UID Mapping on LOFS Mounts
Oh shoot... I honestly don't know about lofs. Sorry, Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX Zones and UID Mapping on LOFS Mounts
Any examples on what needs to be done to make this work with LOFS? On Tue, Jan 10, 2017 at 7:19 PM Dan McDonaldwrote: > > > > On Jan 10, 2017, at 6:49 PM, Mini Trader > wrote: > > > > > > Currently the UID Mapping between the host (OmniOS) and my zone (Linux) > is based purely on UID. Obviously the UID's on my Linux zone are going to > be very different from my OmniOS setup. > > > > > > Is there any NFS4 IDMAPD concept available here? e.g. nobody/nogroup > for unrecognized users and groups or mapping from a numerical id to user? > > > > http://illumos.org/man/1m/idmap > > > > Dan > > > > ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX Zones and UID Mapping on LOFS Mounts
> On Jan 10, 2017, at 6:49 PM, Mini Traderwrote: > > Currently the UID Mapping between the host (OmniOS) and my zone (Linux) is > based purely on UID. Obviously the UID's on my Linux zone are going to be > very different from my OmniOS setup. > > Is there any NFS4 IDMAPD concept available here? e.g. nobody/nogroup for > unrecognized users and groups or mapping from a numerical id to user? http://illumos.org/man/1m/idmap Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX zones: configurations
> On Jan 10, 2017, at 6:50 PM, Mini Traderwrote: > > No Python 3.x ? Baby steps... Plus, the changes for 3.x would be even larger than for 2.6->2.7. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX zones: configurations
No Python 3.x ? On Tue, Jan 10, 2017 at 6:04 PM, Dale Ghentwrote: > > > On Jan 10, 2017, at 5:04 PM, Dominik Hassler wrote: > > > > @Dan: LX zones are considered BETA in r20 and r22 seems to be "late", is > there a chance to get LX bleeding edge in r20 w/o the risk of breaking > something else? > > Zones in 020 is still largely in sync with the LX code currently in bloody > (021). Since zones (beta) was released with 020, we’ve been primarily > focussed on other items required for 022, the most laborious of which is > the Python 2.6 -> 2.7 upgrade. This is needed for a number of reasons. > First, the benefits of getting off of 2.6 and on to 2.7 is > self-explanatory, but 2.7 is also needed for being able to stay in sync > with the pkg code, as well as the next big project for 022 - a > loader-enabled installer (text installer and kayak) to replace the current > one. > > In the mean-time, we’re always looking for ideas (and even contributions!) > from the community on how best to handle the management and administrivia > involved with LX zones. For now, we’d like this to stay within the realm of > zoneadm/zonecfg and family. > > /dale > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] LX Zones and UID Mapping on LOFS Mounts
Currently the UID Mapping between the host (OmniOS) and my zone (Linux) is based purely on UID. Obviously the UID's on my Linux zone are going to be very different from my OmniOS setup. Is there any NFS4 IDMAPD concept available here? e.g. nobody/nogroup for unrecognized users and groups or mapping from a numerical id to user? ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX zones: configurations
Dale, I am very satisfied to stay in the realm of zoneadm/zonecfg and family. This is what I wanted to read: On 01/11/2017 12:03 AM, Dan McDonald wrote: > If there are real, provable show-stoppers in LX, fixes may get backported. Cheers, Dom On 01/11/2017 12:04 AM, Dale Ghent wrote: On Jan 10, 2017, at 5:04 PM, Dominik Hasslerwrote: @Dan: LX zones are considered BETA in r20 and r22 seems to be "late", is there a chance to get LX bleeding edge in r20 w/o the risk of breaking something else? Zones in 020 is still largely in sync with the LX code currently in bloody (021). Since zones (beta) was released with 020, we’ve been primarily focussed on other items required for 022, the most laborious of which is the Python 2.6 -> 2.7 upgrade. This is needed for a number of reasons. First, the benefits of getting off of 2.6 and on to 2.7 is self-explanatory, but 2.7 is also needed for being able to stay in sync with the pkg code, as well as the next big project for 022 - a loader-enabled installer (text installer and kayak) to replace the current one. In the mean-time, we’re always looking for ideas (and even contributions!) from the community on how best to handle the management and administrivia involved with LX zones. For now, we’d like this to stay within the realm of zoneadm/zonecfg and family. /dale ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX zones: configurations
> On Jan 10, 2017, at 5:04 PM, Dominik Hasslerwrote: > > @Dan: LX zones are considered BETA in r20 and r22 seems to be "late", is > there a chance to get LX bleeding edge in r20 w/o the risk of breaking > something else? Zones in 020 is still largely in sync with the LX code currently in bloody (021). Since zones (beta) was released with 020, we’ve been primarily focussed on other items required for 022, the most laborious of which is the Python 2.6 -> 2.7 upgrade. This is needed for a number of reasons. First, the benefits of getting off of 2.6 and on to 2.7 is self-explanatory, but 2.7 is also needed for being able to stay in sync with the pkg code, as well as the next big project for 022 - a loader-enabled installer (text installer and kayak) to replace the current one. In the mean-time, we’re always looking for ideas (and even contributions!) from the community on how best to handle the management and administrivia involved with LX zones. For now, we’d like this to stay within the realm of zoneadm/zonecfg and family. /dale signature.asc Description: Message signed with OpenPGP ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX zones: configurations
> On Jan 10, 2017, at 5:04 PM, Dominik Hasslerwrote: > > @Dan: LX zones are considered BETA in r20 and r22 seems to be "late", is > there a chance to get LX bleeding edge in r20 w/o the risk of breaking > something else? Given how encompassing the 022 work is, don't hold your breath. If there are real, provable show-stoppers in LX, fixes may get backported. Also, the big other change for LX I want in 022 is ipadm(1M) being able to work in an LX zone, so you don't have to do zonecfg(1M) for your network configuration. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
[OmniOS-discuss] LX zones: configurations
Hi, I thought it might be good to have a public spreadsheet to post LX zone setups and how/if they work. It is not meant to be the book of wisdom, but should encourage people who are thinking of running the same service(s) in an LX zone, to actually try it if they see that someone else already did it before. Even if you plan to use a different distro, it might give you some confidence to step forward. This is just a basic start. Feel free to reformat it, add missing columns etc... If it is found to be useful, ownership could be transferred to OmniTI and linked to the wiki (w/ additional information). http://tinyurl.com/hajz4ap @Dan: LX zones are considered BETA in r20 and r22 seems to be "late", is there a chance to get LX bleeding edge in r20 w/o the risk of breaking something else? ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] A problem and puzzle with disappearing ZFS snapshots
For the interest of bystanders, here's the resolution to our puzzle. I wrote: > I wrote, about our mysteriously disappearing snapshots: > > [...] For example, is there some way where snapshots can be removed > > without that being logged in 'zpool history'? > > The answer to my question turns out to be 'yes'. If you do: > rmdir /.zfs/snapshot/ > > and ZFS accepts this, there is no entry made in 'zpool history'; the > snapshot is silently deleted. This is actually wrong, because it turns out that I hadn't read the zpool manpage about 'zpool history' carefully enough. By default 'zpool history' shows only ZFS commands; however, you can get it to show 'internally logged ZFS events' in addition with the -i switch, and this did show our snapshot deletions. ('zpool history -i' has some interesting output around scrubs and resilvers too.) > A successful rmdir of a snapshot can be done by root either locally > or over NFS, if the NFS client has been given root permissions. Like > other snapshot removals, this rmdir will be blocked if the snapshot > has a hold on it ('zfs hold'/'zfs release'). > > We have some client systems that have NFS root access to this > filesystem. Notably, one of them is a Samba server, and it is > possible that some Samba client is doing something that persuades the > Samba server to issue a rmdir against .zfs/snapshot/ as > root, possibly for all things in the directory that it sees at the > time. It turns out that I was unjustly suspicious of Samba. The real culprit was part of our home-grown NFS mount management system, which will unmount NFS filesystems that it thinks are not supposed to be there and as part of doing this will rmdir their mount point directories. When it was dealing with a snapshot NFS mount, this rmdir naturally deleted the snapshot (well, when done from our Samba server, with its root permissions). Snapshots were being accessed on our Samba server for the straightforward reason that some users were looking in them for deleted files. (Linux materializes synthetic NFS mounts for .zfs/snapshot/ filesystems when they're accessed on a NFS client. These are almost undistinguishable from regular NFS mounts, so our NFS mount management system saw them as extra, not-supposed-to-be-there NFS mounts that it should deal with.) I'd like to thank Dan McDonald and the others here for their help in leading me towards the ultimate solution. - cks ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
Hmm, as far as I know, Omni-Path is not an ethernet replacement, but an IB replacement. While Intel is investing heavily in Omni-Path, it should not impact their ethernet offerings as they really are two different markets. Ian On Tue, Jan 10, 2017 at 8:10 AM, Schweiss, Chipwrote: > > On Tue, Jan 10, 2017 at 9:58 AM, Dan McDonald wrote: > >> >> > On Jan 10, 2017, at 8:41 AM, Schweiss, Chip wrote: >> > >> > It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and >> SolarFlare. >> > >> > Can anyone comment on which of these is the most stable solution when >> running under OmniOS? What's the fastest NFS throughput you've been able >> to achieve? >> >> The Intel i40e driver is nascent, but it will receive more attention as >> time passes. Doug's point about SolarFlare is a good one. >> >> > I'm a bit concerned on the Intel because of posts like this: > https://news.ycombinator.com/item?id=11373848 and the fact that they > seem to have shifted their focus to Omni-Path which from my understanding > is incompatible with the existing 40G gear. > > SolarFlare seems promising, but I'd like to know of at least on success > story. > > -Chip > > > >> You may wish to ping the larger illumos community about this as well. >> > > >> Dan >> >> > > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss > > -- Ian Kaufman Research Systems Administrator UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
> On Jan 10, 2017, at 11:10 AM, Schweiss, Chipwrote: > > I'm a bit concerned on the Intel because of posts like this: > https://news.ycombinator.com/item?id=11373848 and the fact that they seem to > have shifted their focus to Omni-Path which from my understanding is > incompatible with the existing 40G gear. > Wow, I hadn't read that. That's depressing (and strikes familiar pain points, given I did work a bit on i40e early on). Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
On Tue, Jan 10, 2017 at 9:58 AM, Dan McDonaldwrote: > > > On Jan 10, 2017, at 8:41 AM, Schweiss, Chip wrote: > > > > It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and > SolarFlare. > > > > Can anyone comment on which of these is the most stable solution when > running under OmniOS? What's the fastest NFS throughput you've been able > to achieve? > > The Intel i40e driver is nascent, but it will receive more attention as > time passes. Doug's point about SolarFlare is a good one. > > I'm a bit concerned on the Intel because of posts like this: https://news.ycombinator.com/item?id=11373848 and the fact that they seem to have shifted their focus to Omni-Path which from my understanding is incompatible with the existing 40G gear. SolarFlare seems promising, but I'd like to know of at least on success story. -Chip > You may wish to ping the larger illumos community about this as well. > > Dan > > ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
> On Jan 10, 2017, at 8:41 AM, Schweiss, Chipwrote: > > It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and > SolarFlare. > > Can anyone comment on which of these is the most stable solution when running > under OmniOS? What's the fastest NFS throughput you've been able to achieve? The Intel i40e driver is nascent, but it will receive more attention as time passes. Doug's point about SolarFlare is a good one. You may wish to ping the larger illumos community about this as well. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX Program Issue
Hi MT! And which significant differences I see in zfs properties (sorry for HTML but without it -- the difference is not so evident): |Good | |Broken | | | | | |NAMEPROPERTY VALUE SOURCE| |NAMEPROPERTY VALUE SOURCE| |main/zones atime on default | |main/zones atime offlocal | |main/zones aclmode discarddefault | |main/zones aclmode passthroughlocal | |main/zones aclinheritrestricted default | |main/zones aclinheritpassthroughlocal | |main/zones utf8only off -| |main/zones utf8only on -| |main/zones normalization none -| |main/zones normalization formD -| |main/zones casesensitivity sensitive -| |main/zones casesensitivity insensitive -| |main/zones nbmandoffdefault | |main/zones nbmandon local | || 1. Atime. Maybe this backup software relies on atime some way? Anyway, atime for root FS is important. 2. Aclmode+aclinherit. These are for ACLs. Non-defaults are used for shares, for root FS the default is must. 3. Utf8only+normalization+casesensitivity. These are for file names. Maybe backup assumes that, ex., .lock and .LOCK are different files (usually on UNIX and Linux tis is true ;) ). BTW Linux (like UNIX or BSD or evem MacOS) likes case-sensitivity and "insensitive" is for network shares for windows-like clients. 4. Nbmand. This is some kind of locking, I don't know details. But I also think that non-default is not a good idea for linux root. So, I think that "right column" is not siutable as linux root filesystem - because of case insensitivity, atime=off and uncommon acls+nbmand. This is my opinion. And if You want - You definitely can fix these settings one by one and realize what is the unique cause of backup creash. It seems so. -- С уважением, Илья Кулагин smime.p7s Description: Firma criptográfica S/MIME ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] Network >10Gb/s
When I had it in production, I used SolarFlare for 10g (but not 40). I'm contributing this in case it helps..SolarFlare open sourced the code and contributed it and there have been a couple of posts to this list in the past with the updated stuff. (I contributed some fixes for add_drv). It worked very well. With the right workload, I could definitely nearly saturate the NIC (e.g. very large files streaming). say 85%+ utilization. I have not tried 40Gb. On 1/10/2017 8:41 AM, Schweiss, Chip wrote: It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and SolarFlare. Can anyone comment on which of these is the most stable solution when running under OmniOS? What's the fastest NFS throughput you've been able to achieve? Also is there any work being done by anyone to bring an Omni-Path compatible NIC to Illumos/OmniOS? Thanks! -Chip ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX Program Issue
Working + Broken Config below. On a positive note I'm very happy this is working. The performance improvement over a VM using NFS on a virtual 10GBE switch is significant. Working root@storage1:/main# zfs get all main/zones NAMEPROPERTY VALUE SOURCE main/zones type filesystem - main/zones creation Tue Jan 10 4:20 2017 - main/zones used 7.53G - main/zones available 3.71T - main/zones referenced226M - main/zones compressratio 1.06x - main/zones mounted yes- main/zones quota none default main/zones reservation none default main/zones recordsize128K default main/zones mountpoint/main/zonesdefault main/zones sharenfs offdefault main/zones checksum on default main/zones compression lz4inherited from main main/zones atime on default main/zones devices on default main/zones exec on default main/zones setuidon default main/zones readonly offdefault main/zones zoned offdefault main/zones snapdir hidden default main/zones aclmode discarddefault main/zones aclinheritrestricted default main/zones canmount on default main/zones xattr on default main/zones copies1 default main/zones version 5 - main/zones utf8only off- main/zones normalization none - main/zones casesensitivity sensitive - main/zones vscan offdefault main/zones nbmandoffdefault main/zones sharesmb offdefault main/zones refquota none default main/zones refreservationnone default main/zones primarycache alldefault main/zones secondarycachealldefault main/zones usedbysnapshots 0 - main/zones usedbydataset 226M - main/zones usedbychildren7.31G - main/zones usedbyrefreservation 0 - main/zones logbias latencydefault main/zones dedup offdefault main/zones mlslabel none default main/zones sync standard default main/zones refcompressratio 2.05x - main/zones written 226M - main/zones logicalused 7.96G - main/zones logicalreferenced 463M - main/zones filesystem_limit none default main/zones snapshot_limitnone default main/zones filesystem_count none default main/zones snapshot_countnone default main/zones redundant_metadataalldefault root@storage1:/main# /usr/bin/ls -lV zones/ total 2 drwxr-xr-x 2 root root 3 Jan 10 04:20 images owner@:rwxp-DaARWcCos:---:allow group@:r-x---a-R-c--s:---:allow everyone@:r-x---a-R-c--s:---:allow drwx-- 4 root root 5 Jan 10 13:20 lx0 owner@:rwxp-DaARWcCos:---:allow group@:--a-R-c--s:---:allow everyone@:--a-R-c--s:---:allow Broken root@storage1:/main# zfs get all main/zones NAMEPROPERTY VALUE SOURCE main/zones type filesystem - main/zones creation Tue Jan 10 13:57 2017 - main/zones used 226M - main/zones available 3.71T - main/zones referenced226M - main/zones compressratio 2.05x - main/zones mounted yes- main/zones quota none default main/zones reservation none default main/zones recordsize128K default main/zones mountpoint/main/zonesdefault main/zones
[OmniOS-discuss] Network >10Gb/s
It appears that my options for 40Gb/s Ethernet are Intel, Chelsio and SolarFlare. Can anyone comment on which of these is the most stable solution when running under OmniOS? What's the fastest NFS throughput you've been able to achieve? Also is there any work being done by anyone to bring an Omni-Path compatible NIC to Illumos/OmniOS? Thanks! -Chip ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] LX Program Issue
When you create a filesystem in napp-it it is writeable for everyone as this is what you mayexpect in an SMB filer use (be Windows alike) LX zone folder require a 700 permission. I have updated current napp-it 17.01 to restrict permissions automatically when you create a zone in napp-it. You can also set nics and mounts for ZFS filesystems now during the VM creation process. see http://www.napp-it.org/doc/downloads/zones.pdf Gea Am 10.01.2017 um 05:18 schrieb Mini Trader: So i have reproducible steps and also solve my problem. If I create my dataset via napp-it. It seems like it applies special permissions. LX will complain that other and group should not have write permissions on the zone directory. So I remove these via chmod o-w and chmod g-w. At this point I can create the zone and in doing so I can reproduce the error. If I create the data set with your standard zfs create tank/zones the permissions are fine by default and the bug is not reproducible. So somehow somewhere whatever is happening underneath the hood is sensitive to these permissions on the second write in this program. Strange indeed. On Mon, Jan 9, 2017 at 10:03 PM, Dan McDonald> wrote: > On Jan 9, 2017, at 10:01 PM, Mini Trader > wrote: > > Just to reiterate on a fresh install of 20 I got no error. Different hardware too. WEIRD. (Sorry I didn't catch that earlier. Juggling other balls concurrently!) Okay, thanks for the update. Not sure if there's anything I can do in the immediate term, but thanks for keeping me informed (and giving me the full zonecfg). Thanks! Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss