Re: [zones-discuss] zones-discuss Digest, Vol 87, Issue 1

[Habony, Zsolt]

>  If one zone is paged out completely, then the other zone won't need much 
> swap space, as there is plenty of free physical memory for it's pages to live 
> in.  
-Steve

I do not agree with that.  Another zone cannot use the physical memory of the 
"swapped out" zone, as there is strict hard cap on all the zones physical 
memory.
This hard cap prevents using of physical memory, and swap is filled up by the 
quiet zones pages.

I often face requirements when our customers would like have the same resouces 
available for the zones after consolidation,   what resources they had before 
consolidation.  

In my (theoretic) example, I have two physical boxes with 8G RAM and 16G swap 
area.  Thus, to reflect the similar limits I need to set physical memory limit 
to 8G, and max-swap to 8+16 = 24G for both zones. (Having 16G phys mem, and 32G 
swap area to serve these two zones.)

If zone A is swapped out completely, it might occupy 24G swap area, but zone B 
cannot use the physical memory instead, as there is 8G hard cap on it.



___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Mike Gerdts]

On Sat 11 Feb 2012 at 03:06PM, gerard henry wrote:
> i exactly have the same problem, but "detach; attach -u" didn't solve it
> 
> But it seems that the attach -u doesn't upgrade, according to the messages:
> # zoneadm -z www attach -u
> Log File: /var/tmp/www.attach_log.LGaqKg
>Attach Path: /zones/www/root
> Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4
> 
> Installing: Using pre-existing data in zonepath
>Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
>Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
>  Cache: Using /var/pkg/publisher.
>   Updating non-global zone: Output follows
> No updates necessary for this image.
> 
>   Updating non-global zone: Zone updated.
> Result: Attach Succeeded.
> 
> after the system has booted with:
> SunOS Release 5.11 Version 151.0.1.12 64-bit
> 
> 
> I don't understand what you said with "pkg -R ... image-update" ?
> 
> thanks in advance for help,

It looks to me like you didn't follow the steps to update software in
the global zone from Solaris 11 Express to Solaris first.  The procedure
for that is at:

http://docs.oracle.com/cd/E23824_01/html/E23811/glpgv.html#glpcn

After that, you can detach the zones, run dsconvert on them (see
/usr/lib/brand/shared/README.dsconvert after booting Solaris 11), then
attach them.  Once you are on Solaris 11, the "linked images" feature
of the packaging system will keep your global zone software and
non-global zone software in sync.  That is, you won't have to detach and
attach -u zones every time you update the global zone.

-- 
Mike Gerdts
Solaris Core OS / Zones http://blogs.oracle.com/zoneszone/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[gerard henry]

the problem is very hard to solve. excuse my bad english, i try to explain:
- remember that the initial OS is b151, first S11X release -> BE is
opensolaris-3
- first, i got the SRU13 iso and set a local repo with it
- i have to upgrade pkg, it creates a new BE -> opensolaris-4
- then,  i upgrade to SRU13, it creates opensolaris-5

the problem is that when i boot the BE opensolaris-4, the zones are broken,
so i tried the method "detach; attach -u" . But the zone remains broken

As you can see, from BE opensolaris-3 to BE opensolaris-5, the OS stays at
b151, not 175 because i cannot upgrade to S11X due to zones broken.

I'd hope to upgrade my machine during the day, but it seems impossible. I
open a SR on MOS.

thanks for help,

2012/2/11 gerard henry 

> your reply gave me an idea, but now the machine is rebooted with old BE:
> - when i booted the new BE, and did attach -u, i forgot one essential
> thing explained in 13352339: ORACLE SOLARIS 11 EXPRESS 2010.11 SRU 13 REPO
> ISO IMAGE
> - as i workoed with the lofiadm way, i forgot to re-mount it after reboot,
> and i guess this is why the "attach -u" does nothing.
>
> As soon as possible, i'll test it
>
> thanks for all replies,
>
>
>
> 2012/2/11 Enda O'Connor 
>
>> Hi
>> According to this your global zone is at
>> 151 ie express
>>
>>
>> Global zone version: entire@0.5.11,5.11-0.151.0.1:**20101105T054056Z
>>
>> in global zone run
>> pkg info entire
>> if that says 151 then can i see
>> pkg publisher
>>
>> If that points to s11
>> can I see
>> pkg update -nv '*@latest'
>>
>> Enda
>>
>> On 11/02/2012 14:06, gerard henry wrote:
>>
>>> i exactly have the same problem, but "detach; attach -u" didn't solve it
>>>
>>> But it seems that the attach -u doesn't upgrade, according to the
>>> messages:
>>> # zoneadm -z www attach -u
>>> Log File: /var/tmp/www.attach_log.LGaqKg
>>>Attach Path: /zones/www/root
>>> Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4
>>>
>>> Installing: Using pre-existing data in zonepath
>>>Global zone version: entire@0.5.11,5.11-0.151.0.1:**
>>> 20101105T054056Z
>>>Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:**
>>> 20101105T054056Z
>>>  Cache: Using /var/pkg/publisher.
>>>   Updating non-global zone: Output follows
>>> No updates necessary for this image.
>>>
>>>   Updating non-global zone: Zone updated.
>>> Result: Attach Succeeded.
>>>
>>> after the system has booted with:
>>> SunOS Release 5.11 Version 151.0.1.12 64-bit
>>>
>>>
>>> I don't understand what you said with "pkg -R ... image-update" ?
>>>
>>> thanks in advance for help,
>>>
>>>
>>>
>>> 2011/10/5 Ian Collins mailto:i...@ianshome.com>>
>>>
>>>
>>>  On 10/ 5/11 09:26 PM, casper@oracle.com
>>> wrote:
>>>
>>>  Before I go through the pain of logging a support call,
>>>has anyone
>>>seen or fixed the following problem:
>>>
>>>I ran an update on a fresh Solaris 11 Express system from
>>>the support
>>>repository and after restarting, all the systems zones are
>>>dead.  The
>>>zone consoles report:
>>>
>>>SunOS Release 5.11 Version 151.0.1.8 64-bit
>>>Copyright (c) 1983, 2010, Oracle and/or its affiliates. All
>>>rights reserved.
>>>Requesting System Maintenance Mode
>>>(See /lib/svc/share/README for more information.)
>>>svc:/system/early-manifest-__**import:default signalled: SYS
>>>
>>>
>>>
>>>The zones run an older version of the Solaris software and as a
>>>result its
>>>libc doesn't match the kernel and the binaries will fail.
>>>
>>>I think you will need to upgrade all your zones too
>>>
>>>It might be something simple as "zoneadm detach"; "zoneadm
>>>attach -u" but
>>>make sure that you keep sufficient save sufficient information to
>>>reinstall the zones; and make sure you try this on one zone
>>>first before
>>>you detach all of them.  It might be possible to update the
>>>zones also
>>>using pkg -R  image-update after you've mounted the root
>>>filesystem.
>>>
>>>Thanks Casper!
>>>
>>>I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside
>>>my eyelids!
>>>
>>>--
>>>Ian.
>>>
>>>
>>>__**___
>>>zones-discuss mailing list
>>>zones-discuss@opensolaris.org 
>>> >> >
>>>
>>>
>>>
>>>
>>>
>>> __**_
>>> zones-discuss mailing list
>>> zones-discuss@opensolaris.org
>>>
>>
>> __**_
>> zones-discuss mailing list
>> zones-discuss@opensolaris.org
>>
>
>
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Enda]

 Hi
just make sure via
pkg info entire
in global zone that it says 175 not 151, seems your global was still at 
151, ie express.


Enda
On 11/02/2012 15:03, gerard henry wrote
your reply gave me an idea, but now the machine is rebooted with old BE:
- when i booted the new BE, and did attach -u, i forgot one essential 
thing explained in 13352339: ORACLE SOLARIS 11 EXPRESS 2010.11 SRU 13 
REPO ISO IMAGE
- as i workoed with the lofiadm way, i forgot to re-mount it after 
reboot, and i guess this is why the "attach -u" does nothing.


As soon as possible, i'll test it

thanks for all replies,


2012/2/11 Enda O'Connor >


Hi
According to this your global zone is at
151 ie express


Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z

in global zone run
pkg info entire
if that says 151 then can i see
pkg publisher

If that points to s11
can I see
pkg update -nv '*@latest'

Enda

On 11/02/2012 14:06, gerard henry wrote:

i exactly have the same problem, but "detach; attach -u"
didn't solve it

But it seems that the attach -u doesn't upgrade, according to
the messages:
# zoneadm -z www attach -u
Log File: /var/tmp/www.attach_log.LGaqKg
   Attach Path: /zones/www/root
Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4

Installing: Using pre-existing data in zonepath
   Global zone version:
entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
   Non-Global zone version:
entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
 Cache: Using /var/pkg/publisher.
  Updating non-global zone: Output follows
No updates necessary for this image.

  Updating non-global zone: Zone updated.
Result: Attach Succeeded.

after the system has booted with:
SunOS Release 5.11 Version 151.0.1.12 64-bit


I don't understand what you said with "pkg -R ... image-update" ?

thanks in advance for help,



2011/10/5 Ian Collins mailto:i...@ianshome.com> >>


 On 10/ 5/11 09:26 PM, casper@oracle.com

>
wrote:

 Before I go through the pain of logging a support
call,
   has anyone
   seen or fixed the following problem:

   I ran an update on a fresh Solaris 11 Express
system from
   the support
   repository and after restarting, all the systems
zones are
   dead.  The
   zone consoles report:

   SunOS Release 5.11 Version 151.0.1.8 64-bit
   Copyright (c) 1983, 2010, Oracle and/or its
affiliates. All
   rights reserved.
   Requesting System Maintenance Mode
   (See /lib/svc/share/README for more information.)
   svc:/system/early-manifest-__import:default
signalled: SYS



   The zones run an older version of the Solaris software
and as a
   result its
   libc doesn't match the kernel and the binaries will fail.

   I think you will need to upgrade all your zones too

   It might be something simple as "zoneadm detach"; "zoneadm
   attach -u" but
   make sure that you keep sufficient save sufficient
information to
   reinstall the zones; and make sure you try this on one zone
   first before
   you detach all of them.  It might be possible to update the
   zones also
   using pkg -R  image-update after you've
mounted the root
   filesystem.

   Thanks Casper!

   I really should tattoo "zoneadm detach"; "zoneadm attach
-u" inside
   my eyelids!

   --
   Ian.


   _
   zones-discuss mailing list
zones-discuss@opensolaris.org

>





___
zones-discuss mailing list
zones-discuss@opensolaris.org



___
zones-discuss mailing list
zones-discuss@opensolaris.org 




___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[gerard henry]

your reply gave me an idea, but now the machine is rebooted with old BE:
- when i booted the new BE, and did attach -u, i forgot one essential thing
explained in 13352339: ORACLE SOLARIS 11 EXPRESS 2010.11 SRU 13 REPO ISOIMAGE
- as i workoed with the lofiadm way, i forgot to re-mount it after reboot,
and i guess this is why the "attach -u" does nothing.

As soon as possible, i'll test it

thanks for all replies,


2012/2/11 Enda O'Connor 

> Hi
> According to this your global zone is at
> 151 ie express
>
>
> Global zone version: entire@0.5.11,5.11-0.151.0.1:**20101105T054056Z
>
> in global zone run
> pkg info entire
> if that says 151 then can i see
> pkg publisher
>
> If that points to s11
> can I see
> pkg update -nv '*@latest'
>
> Enda
>
> On 11/02/2012 14:06, gerard henry wrote:
>
>> i exactly have the same problem, but "detach; attach -u" didn't solve it
>>
>> But it seems that the attach -u doesn't upgrade, according to the
>> messages:
>> # zoneadm -z www attach -u
>> Log File: /var/tmp/www.attach_log.LGaqKg
>>Attach Path: /zones/www/root
>> Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4
>>
>> Installing: Using pre-existing data in zonepath
>>Global zone version: entire@0.5.11,5.11-0.151.0.1:**
>> 20101105T054056Z
>>Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:**
>> 20101105T054056Z
>>  Cache: Using /var/pkg/publisher.
>>   Updating non-global zone: Output follows
>> No updates necessary for this image.
>>
>>   Updating non-global zone: Zone updated.
>> Result: Attach Succeeded.
>>
>> after the system has booted with:
>> SunOS Release 5.11 Version 151.0.1.12 64-bit
>>
>>
>> I don't understand what you said with "pkg -R ... image-update" ?
>>
>> thanks in advance for help,
>>
>>
>>
>> 2011/10/5 Ian Collins mailto:i...@ianshome.com>>
>>
>>
>>  On 10/ 5/11 09:26 PM, casper@oracle.com
>> wrote:
>>
>>  Before I go through the pain of logging a support call,
>>has anyone
>>seen or fixed the following problem:
>>
>>I ran an update on a fresh Solaris 11 Express system from
>>the support
>>repository and after restarting, all the systems zones are
>>dead.  The
>>zone consoles report:
>>
>>SunOS Release 5.11 Version 151.0.1.8 64-bit
>>Copyright (c) 1983, 2010, Oracle and/or its affiliates. All
>>rights reserved.
>>Requesting System Maintenance Mode
>>(See /lib/svc/share/README for more information.)
>>svc:/system/early-manifest-__**import:default signalled: SYS
>>
>>
>>
>>The zones run an older version of the Solaris software and as a
>>result its
>>libc doesn't match the kernel and the binaries will fail.
>>
>>I think you will need to upgrade all your zones too
>>
>>It might be something simple as "zoneadm detach"; "zoneadm
>>attach -u" but
>>make sure that you keep sufficient save sufficient information to
>>reinstall the zones; and make sure you try this on one zone
>>first before
>>you detach all of them.  It might be possible to update the
>>zones also
>>using pkg -R  image-update after you've mounted the root
>>filesystem.
>>
>>Thanks Casper!
>>
>>I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside
>>my eyelids!
>>
>>--
>>Ian.
>>
>>
>>__**___
>>zones-discuss mailing list
>>zones-discuss@opensolaris.org 
>> > >
>>
>>
>>
>>
>>
>> __**_
>> zones-discuss mailing list
>> zones-discuss@opensolaris.org
>>
>
> __**_
> zones-discuss mailing list
> zones-discuss@opensolaris.org
>
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Enda O'Connor]

On 11/02/2012 14:15, Enda O'Connor wrote:

Hi
According to this your global zone is at
151 ie express

Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z

in global zone run
pkg info entire
if that says 151 then can i see
pkg publisher


If pkg is set to s11  repo
and if entire is at 151
run
pkg update pkg
init 6
on reboot
pkg update

if any of these fail add -nv to see what gives.

Enda


If that points to s11
can I see
pkg update -nv '*@latest'

Enda
On 11/02/2012 14:06, gerard henry wrote:

i exactly have the same problem, but "detach; attach -u" didn't solve it

But it seems that the attach -u doesn't upgrade, according to the
messages:
# zoneadm -z www attach -u
Log File: /var/tmp/www.attach_log.LGaqKg
Attach Path: /zones/www/root
Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4

Installing: Using pre-existing data in zonepath
Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
Cache: Using /var/pkg/publisher.
Updating non-global zone: Output follows
No updates necessary for this image.

Updating non-global zone: Zone updated.
Result: Attach Succeeded.

after the system has booted with:
SunOS Release 5.11 Version 151.0.1.12 64-bit


I don't understand what you said with "pkg -R ... image-update" ?

thanks in advance for help,



2011/10/5 Ian Collins mailto:i...@ianshome.com>>

On 10/ 5/11 09:26 PM, casper@oracle.com
 wrote:

Before I go through the pain of logging a support call,
has anyone
seen or fixed the following problem:

I ran an update on a fresh Solaris 11 Express system from
the support
repository and after restarting, all the systems zones are
dead. The
zone consoles report:

SunOS Release 5.11 Version 151.0.1.8 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates. All
rights reserved.
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
svc:/system/early-manifest-__import:default signalled: SYS


The zones run an older version of the Solaris software and as a
result its
libc doesn't match the kernel and the binaries will fail.

I think you will need to upgrade all your zones too

It might be something simple as "zoneadm detach"; "zoneadm
attach -u" but
make sure that you keep sufficient save sufficient information to
reinstall the zones; and make sure you try this on one zone
first before
you detach all of them. It might be possible to update the
zones also
using pkg -R image-update after you've mounted the root
filesystem.

Thanks Casper!

I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside
my eyelids!

--
Ian.


_
zones-discuss mailing list
zones-discuss@opensolaris.org 




___
zones-discuss mailing list
zones-discuss@opensolaris.org


___
zones-discuss mailing list
zones-discuss@opensolaris.org


___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Enda O'Connor]

Hi
According to this your global zone is at
151 ie express

Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z

in global zone run
pkg info entire
if that says 151 then can i see
pkg publisher

If that points to s11
can I see
pkg update -nv '*@latest'

Enda
On 11/02/2012 14:06, gerard henry wrote:

i exactly have the same problem, but "detach; attach -u" didn't solve it

But it seems that the attach -u doesn't upgrade, according to the messages:
# zoneadm -z www attach -u
Log File: /var/tmp/www.attach_log.LGaqKg
Attach Path: /zones/www/root
 Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4

 Installing: Using pre-existing data in zonepath
Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
  Cache: Using /var/pkg/publisher.
   Updating non-global zone: Output follows
No updates necessary for this image.

   Updating non-global zone: Zone updated.
 Result: Attach Succeeded.

after the system has booted with:
SunOS Release 5.11 Version 151.0.1.12 64-bit


I don't understand what you said with "pkg -R ... image-update" ?

thanks in advance for help,



2011/10/5 Ian Collins mailto:i...@ianshome.com>>

  On 10/ 5/11 09:26 PM, casper@oracle.com
 wrote:

  Before I go through the pain of logging a support call,
has anyone
seen or fixed the following problem:

I ran an update on a fresh Solaris 11 Express system from
the support
repository and after restarting, all the systems zones are
dead.  The
zone consoles report:

SunOS Release 5.11 Version 151.0.1.8 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates. All
rights reserved.
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
svc:/system/early-manifest-__import:default signalled: SYS


The zones run an older version of the Solaris software and as a
result its
libc doesn't match the kernel and the binaries will fail.

I think you will need to upgrade all your zones too

It might be something simple as "zoneadm detach"; "zoneadm
attach -u" but
make sure that you keep sufficient save sufficient information to
reinstall the zones; and make sure you try this on one zone
first before
you detach all of them.  It might be possible to update the
zones also
using pkg -R  image-update after you've mounted the root
filesystem.

Thanks Casper!

I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside
my eyelids!

--
Ian.


_
zones-discuss mailing list
zones-discuss@opensolaris.org 




___
zones-discuss mailing list
zones-discuss@opensolaris.org


___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D.]

may be attach -U


On 2/11/2012 9:06 AM, gerard henry wrote:

i exactly have the same problem, but "detach; attach -u" didn't solve it

But it seems that the attach -u doesn't upgrade, according to the 
messages:

# zoneadm -z www attach -u
Log File: /var/tmp/www.attach_log.LGaqKg
   Attach Path: /zones/www/root
Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4

Installing: Using pre-existing data in zonepath
   Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
   Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
 Cache: Using /var/pkg/publisher.
  Updating non-global zone: Output follows
No updates necessary for this image.

  Updating non-global zone: Zone updated.
Result: Attach Succeeded.

after the system has booted with:
SunOS Release 5.11 Version 151.0.1.12 64-bit


I don't understand what you said with "pkg -R ... image-update" ?

thanks in advance for help,



2011/10/5 Ian Collins mailto:i...@ianshome.com>>

 On 10/ 5/11 09:26 PM, casper@oracle.com
 wrote:

 Before I go through the pain of logging a support call,
has anyone
seen or fixed the following problem:

I ran an update on a fresh Solaris 11 Express system from
the support
repository and after restarting, all the systems zones are
dead.  The
zone consoles report:

SunOS Release 5.11 Version 151.0.1.8 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates.
All rights reserved.
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
svc:/system/early-manifest-import:default signalled: SYS


The zones run an older version of the Solaris software and as
a result its
libc doesn't match the kernel and the binaries will fail.

I think you will need to upgrade all your zones too

It might be something simple as "zoneadm detach"; "zoneadm
attach -u" but
make sure that you keep sufficient save sufficient information to
reinstall the zones; and make sure you try this on one zone
first before
you detach all of them.  It might be possible to update the
zones also
using pkg -R  image-update after you've mounted the root
filesystem.

Thanks Casper!

I really should tattoo "zoneadm detach"; "zoneadm attach -u"
inside my eyelids!

-- 
Ian.



___
zones-discuss mailing list
zones-discuss@opensolaris.org 




___
zones-discuss mailing list
zones-discuss@opensolaris.org


--
Hung-Sheng Tsao Ph D.
Founder&  Principal
HopBit GridComputing LLC
cell: 9734950840

http://laotsao.blogspot.com/
http://laotsao.wordpress.com/
http://blogs.oracle.com/hstsao/

<>___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[gerard henry]

i exactly have the same problem, but "detach; attach -u" didn't solve it

But it seems that the attach -u doesn't upgrade, according to the messages:
# zoneadm -z www attach -u
Log File: /var/tmp/www.attach_log.LGaqKg
   Attach Path: /zones/www/root
Attach ZFS Dataset: rpool/zones/www/ROOT/zbe-4

Installing: Using pre-existing data in zonepath
   Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
   Non-Global zone version: entire@0.5.11,5.11-0.151.0.1:20101105T054056Z
 Cache: Using /var/pkg/publisher.
  Updating non-global zone: Output follows
No updates necessary for this image.

  Updating non-global zone: Zone updated.
Result: Attach Succeeded.

after the system has booted with:
SunOS Release 5.11 Version 151.0.1.12 64-bit


I don't understand what you said with "pkg -R ... image-update" ?

thanks in advance for help,



2011/10/5 Ian Collins 

>  On 10/ 5/11 09:26 PM, casper@oracle.com wrote:
>
>>  Before I go through the pain of logging a support call, has anyone
>>> seen or fixed the following problem:
>>>
>>> I ran an update on a fresh Solaris 11 Express system from the support
>>> repository and after restarting, all the systems zones are dead.  The
>>> zone consoles report:
>>>
>>> SunOS Release 5.11 Version 151.0.1.8 64-bit
>>> Copyright (c) 1983, 2010, Oracle and/or its affiliates. All rights
>>> reserved.
>>> Requesting System Maintenance Mode
>>> (See /lib/svc/share/README for more information.)
>>> svc:/system/early-manifest-**import:default signalled: SYS
>>>
>>
>> The zones run an older version of the Solaris software and as a result its
>> libc doesn't match the kernel and the binaries will fail.
>>
>> I think you will need to upgrade all your zones too
>>
>> It might be something simple as "zoneadm detach"; "zoneadm attach -u" but
>> make sure that you keep sufficient save sufficient information to
>> reinstall the zones; and make sure you try this on one zone first before
>> you detach all of them.  It might be possible to update the zones also
>> using pkg -R  image-update after you've mounted the root
>> filesystem.
>>
>>  Thanks Casper!
>
> I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside my
> eyelids!
>
> --
> Ian.
>
>
> __**_
> zones-discuss mailing list
> zones-discuss@opensolaris.org
>
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones talk: Oracle Solaris 11 Summit at LISA 2011

[Mike Gerdts]

Will you be at LISA 2011 or will you be hanging out in Boston looking
for something to do on Tuesday, December 6?  If so, I encourage you to
come to the Oracle Solaris 11 Summit at the Sheraton Boston Hotel.  LISA
registration is not required to attend the free Oracle Solaris Summit.

During the Summit, I'll be presenting Solaris 11 Zones.  Several of my
fellow engineers will be covering other areas such as installation,
packaging, ZFS, networking, security, integration with other Oracle
software, and Solaris Cluster.  Solaris 11 brings much better
integration between the various components of the operating system, and
as such you will learn important things about zones in the other talks
as well.

Find the agenda and registration link at:

http://www.oracle.com/us/dm/h2fy11/20741-wwmk11010781mpp004c003-oem-524681.html

I will also be one of the panelists at the Oracle Solaris 11 Engineering
Panel BoF sessions on Wednesday from 7:30 until 8:30 PM.

http://www.usenix.org/events/lisa11/bofs.html#Solaris11_table

And if that's not enough, you can also catch a variety of engineers at
the Oracle demo booth.  I'll be there from 2:00 - 4:00 on Wednesday.

I hope to see you there!

-- 
Mike Gerdts
Solaris Core OS / Zones http://blogs.oracle.com/zoneszone/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Ian Collins]

 On 10/ 5/11 09:26 PM, casper@oracle.com wrote:

  Before I go through the pain of logging a support call, has anyone
seen or fixed the following problem:

I ran an update on a fresh Solaris 11 Express system from the support
repository and after restarting, all the systems zones are dead.  The
zone consoles report:

SunOS Release 5.11 Version 151.0.1.8 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates. All rights reserved.
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
svc:/system/early-manifest-import:default signalled: SYS


The zones run an older version of the Solaris software and as a result its
libc doesn't match the kernel and the binaries will fail.

I think you will need to upgrade all your zones too

It might be something simple as "zoneadm detach"; "zoneadm attach -u" but
make sure that you keep sufficient save sufficient information to
reinstall the zones; and make sure you try this on one zone first before
you detach all of them.  It might be possible to update the zones also
using pkg -R  image-update after you've mounted the root
filesystem.


Thanks Casper!

I really should tattoo "zoneadm detach"; "zoneadm attach -u" inside my 
eyelids!


--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones toast after updating Solaris 11 Express system

[Casper . Dik]

>  Before I go through the pain of logging a support call, has anyone 
>seen or fixed the following problem:
>
>I ran an update on a fresh Solaris 11 Express system from the support 
>repository and after restarting, all the systems zones are dead.  The 
>zone consoles report:
>
>SunOS Release 5.11 Version 151.0.1.8 64-bit
>Copyright (c) 1983, 2010, Oracle and/or its affiliates. All rights reserved.
>Requesting System Maintenance Mode
>(See /lib/svc/share/README for more information.)
>svc:/system/early-manifest-import:default signalled: SYS
>
>The log file shows:
>
># more /var/svc/log/system-early-manifest-import:default.log
>svccfg: Loaded 100 smf(5) service descriptions
>/etc/svc system profiles not found: upgrade system profiles
>/lib/svc/method/manifest-import[447]: import_manifests: line 259: 1933: 
>Bad system call(coredump)
>/lib/svc/method/manifest-import[447]: import_manifests: line 259: 2424: 
>Bad system call(coredump)
>/lib/svc/method/manifest-import[447]: import_manifests: line 259: 3317: 
>Bad system call(coredump)

>/lib/svc/method/manifest-import is crashing:
>
>3482:   lstat64("/etc/svc/volatile/manifest_import.3465", 0x08047CE0) = 0
>3482:   unlinkat()  Err#89 ENOSYS
>3482:   Received signal #12, SIGSYS [default]
>3482: siginfo: SIG#0
>
>Any clues?


The zones run an older version of the Solaris software and as a result its 
libc doesn't match the kernel and the binaries will fail.

I think you will need to upgrade all your zones too

It might be something simple as "zoneadm detach"; "zoneadm attach -u" but
make sure that you keep sufficient save sufficient information to 
reinstall the zones; and make sure you try this on one zone first before 
you detach all of them.  It might be possible to update the zones also 
using pkg -R  image-update after you've mounted the root 
filesystem.

Casper



___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones toast after updating Solaris 11 Express system

[Ian Collins]

 Before I go through the pain of logging a support call, has anyone 
seen or fixed the following problem:


I ran an update on a fresh Solaris 11 Express system from the support 
repository and after restarting, all the systems zones are dead.  The 
zone consoles report:


SunOS Release 5.11 Version 151.0.1.8 64-bit
Copyright (c) 1983, 2010, Oracle and/or its affiliates. All rights reserved.
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
svc:/system/early-manifest-import:default signalled: SYS

The log file shows:

# more /var/svc/log/system-early-manifest-import:default.log
svccfg: Loaded 100 smf(5) service descriptions
/etc/svc system profiles not found: upgrade system profiles
/lib/svc/method/manifest-import[447]: import_manifests: line 259: 1933: 
Bad system call(coredump)
/lib/svc/method/manifest-import[447]: import_manifests: line 259: 2424: 
Bad system call(coredump)
/lib/svc/method/manifest-import[447]: import_manifests: line 259: 3317: 
Bad system call(coredump)


/lib/svc/method/manifest-import is crashing:

3482:   lstat64("/etc/svc/volatile/manifest_import.3465", 0x08047CE0) = 0
3482:   unlinkat()  Err#89 ENOSYS
3482:   Received signal #12, SIGSYS [default]
3482: siginfo: SIG#0

Any clues?

--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones disappeared fron zoneadm on OpenSolaris

[marria]

Are you still trouble with which site you can trust to buy 
[url=http://www.gameim.com/product/RuneScape_II_gold.html]RS Gold[/url] safely, 
I'll introduce one 

for you, I have bought 
[url=http://www.gameim.com/product/RuneScape_II_gold.html]Runescape Gold[/url] 
many times from here, if you want to buy 

[url=http://www.gameim.com/product/RuneScape_II_gold.html]RS Money[/url], trust 
me!!try!!
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones in Trusted Extensions and Display Variables

[marria]

Are you still trouble with which site you can trust to buy 
[url=http://www.gameim.com/product/RuneScape_II_gold.html]RS Gold[/url] safely, 
I'll introduce one 

for you, I have bought 
[url=http://www.gameim.com/product/RuneScape_II_gold.html]Runescape Gold[/url] 
many times from here, if you want to buy 

[url=http://www.gameim.com/product/RuneScape_II_gold.html]RS Money[/url], trust 
me!!try!!
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones in Trusted Extensions and Display Variables

[Glenn Faden]

  
  


On 4/28/11 12:25 PM, David Tarc wrote:

  What is the best practice for passing Display variables in labeled zones.  I have heard about all-zones interfaces and unix domain sockets for passing the display variable.  I however am finding it hard to find documentation on setting these configurations up.  Any help would be helpful.



It depends on what release of Solaris you are using. In Oracle
Solaris 11 Express, unix domain sockets are used by default. That
means that no networking configuration is required for labeled zones
to connect to the multilevel destktop services running in the global
zone. For Solaris 10 releases, you can share a single IP address
(and the associated hostname) with all zones, using txzonemgr. This
is known as an all-zones configuration. You can also just use
DISPLAY=localhost:0 since lo0 is an all-zones interface.

There are configuration instructions for various Solaris versions
available for Solaris 10 and Oracle Solaris 11 Express on the
OpenSolaris site:

http://hub.opensolaris.org/bin/view/Community+Group+security/tx-laptop-install

--Glenn


  
Glenn
Faden |
  Senior Principal Software Engineer
  Phone:
  +1 650 786 4003 | Mobile: +1 415 637 8181 
Oracle
  Solaris Security, Solaris Core OS Technology Engineering


  

  

___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones in Trusted Extensions and Display Variables

[David Tarc]

What is the best practice for passing Display variables in labeled zones.  I 
have heard about all-zones interfaces and unix domain sockets for passing the 
display variable.  I however am finding it hard to find documentation on 
setting these configurations up.  Any help would be helpful.

David
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones disappeared fron zoneadm on OpenSolaris

[Steve Lawrence]

 Never seen this failure before.

Is it possible that you ran out of disk space?

For reference, which opensolaris build ("cat /etc/release", and "uname 
-v").  Did you do any upgrading?  If so, from which prior build?


It seems that you've lost your /etc/zones/index file.  Perhaps there is 
a backup or temporary copy in /etc/zones.


- Steve L.




On 04/20/11 02:37 AM, Jonatan Walck wrote:

I'm new to this list, checked the archives and hope I found the right
place.

A few days ago I rebooted a opensolaris server, at the first boot
zones:default didn't go up properly, and after another reboot it did
but with one quirk: It finds no zones.

zoneadm list -cv shows only the global zone. zonecfg -z [zonename]
works for all zones though, all configs are there. zfs list shows all
the file systems, zpool status lists no errors.

Anyone have a idea for how to get zones back up? How is the zoneadm
list list built up?

Attaching part of the service log for zones:default, showing something
going wrong but I have been unable to track down the problem.

[ Apr 14 22:23:51 Enabled. ]
[ Apr 14 22:24:42 Executing start method ("/lib/svc/method/svc-zones start"). ]
Booting zones: badger cannonball lolcat mudkip cerberus rockdove tapir kangaroo 
molerat llama tuna narwhal tiger medusa dolphin tanukiERROR: error while 
acquiring slave h
andle of zone console for tapir: No such device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for kangaroo: No such 
file or directory
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for tanuki: No such 
device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for cerberus: No such 
device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for narwhal: No such 
file or directory
console setup: device initialization failed
zone 'tapir': could not start zoneadmd
zoneadm: zone 'tapir': call to zoneadmd failed
zone 'cerberus': zone 'zone 'could not start tanuki': zoneadmd
zone 'kangaroo': narwhal': could not start could not start could not start 
zoneadmd
zoneadmdzoneadmdzoneadm: zone 'cerberus': call to zoneadmd failed
zoneadm

: zoneadmzoneadm: zone 'tanuki': call to zoneadmd failed
zone ': narwhalzone '': kangaroo': call to zoneadmd failed
call to zoneadmd failed
.
[ Apr 14 22:24:56 Method "start" exited with status 0. ]
[ Apr 19 08:51:42 Enabled. ]
[ Apr 19 08:52:11 Executing start method ("/lib/svc/method/svc-zones start"). ]
Booting zones: badger cannonball lolcat mudkip cerberus rockdove tapir kangaroo 
molerat llama tuna narwhal tiger medusa dolphinERROR: error while acquiring 
slave handle of zone console for tuna: No such device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for cerberus: No such 
file or directory
console setup: device initialization failed
zone 'tuna': could not start zoneadmd
zone 'cerberus': zoneadm: could not start zoneadmd
zone 'tuna': zoneadm: zone 'cerberus': call to zoneadmd failed
call to zoneadmd failed
  tanuki[ Apr 19 09:09:43 Enabled. ]
[ Apr 19 09:10:16 Executing start method ("/lib/svc/method/svc-zones start"). ]
[ Apr 19 09:10:16 Method "start" exited with status 0. ]

Thanks for any advice,
Jonatan Walck


___
zones-discuss mailing list
zones-discuss@opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones disappeared fron zoneadm on OpenSolaris

[Jonatan Walck]

I'm new to this list, checked the archives and hope I found the right
place.

A few days ago I rebooted a opensolaris server, at the first boot
zones:default didn't go up properly, and after another reboot it did
but with one quirk: It finds no zones.

zoneadm list -cv shows only the global zone. zonecfg -z [zonename]
works for all zones though, all configs are there. zfs list shows all
the file systems, zpool status lists no errors.

Anyone have a idea for how to get zones back up? How is the zoneadm
list list built up?

Attaching part of the service log for zones:default, showing something
going wrong but I have been unable to track down the problem.

[ Apr 14 22:23:51 Enabled. ]
[ Apr 14 22:24:42 Executing start method ("/lib/svc/method/svc-zones start"). ]
Booting zones: badger cannonball lolcat mudkip cerberus rockdove tapir kangaroo 
molerat llama tuna narwhal tiger medusa dolphin tanukiERROR: error while 
acquiring slave h
andle of zone console for tapir: No such device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for kangaroo: No such 
file or directory
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for tanuki: No such 
device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for cerberus: No such 
device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for narwhal: No such 
file or directory
console setup: device initialization failed
zone 'tapir': could not start zoneadmd
zoneadm: zone 'tapir': call to zoneadmd failed
zone 'cerberus': zone 'zone 'could not start tanuki': zoneadmd
zone 'kangaroo': narwhal': could not start could not start could not start 
zoneadmd
zoneadmdzoneadmdzoneadm: zone 'cerberus': call to zoneadmd failed
zoneadm

: zoneadmzoneadm: zone 'tanuki': call to zoneadmd failed
zone ': narwhalzone '': kangaroo': call to zoneadmd failed
call to zoneadmd failed
.
[ Apr 14 22:24:56 Method "start" exited with status 0. ]
[ Apr 19 08:51:42 Enabled. ]
[ Apr 19 08:52:11 Executing start method ("/lib/svc/method/svc-zones start"). ]
Booting zones: badger cannonball lolcat mudkip cerberus rockdove tapir kangaroo 
molerat llama tuna narwhal tiger medusa dolphinERROR: error while acquiring 
slave handle of zone console for tuna: No such device or address
console setup: device initialization failed
ERROR: error while acquiring slave handle of zone console for cerberus: No such 
file or directory
console setup: device initialization failed
zone 'tuna': could not start zoneadmd
zone 'cerberus': zoneadm: could not start zoneadmd
zone 'tuna': zoneadm: zone 'cerberus': call to zoneadmd failed
call to zoneadmd failed
 tanuki[ Apr 19 09:09:43 Enabled. ]
[ Apr 19 09:10:16 Executing start method ("/lib/svc/method/svc-zones start"). ]
[ Apr 19 09:10:16 Method "start" exited with status 0. ]

Thanks for any advice,
Jonatan Walck

signature.asc
Description: Digital signature
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones zone.max-shm-memory setting.

[Jeff Victor]

Back to the original question (locked-shm-memory on servers):

If you are running multiple applications on a server, and at least one
of them uses shared memory, you should consider using max-shm-memory
or max-locked-memory for the zone that will use shared memory.

Any memory that a process locks down cannot be paged out. That helps
performance of the application, but reduces the amount of memory that
can be paged out. If the amount that can be paged out is too small,
Solaris cannot run as intended, and performance of all of the system's
workloads will suffer.

Some forms of shared memory (e.g. ISM) automatically lock those memory
pages. Other forms (e.g. DISM) allow the process to lock the shared
memory.

If you don't set a cap on shared memory or locked memory, the zone
might lock a significant portion of the system's RAM, leaving very
little that can be paged out if there isn't enough RAM for all of the
zones. In that situation, performance of all of the other zones will
suffer greatly, perhaps making them unusable. Performance of the zone
using shared memory may also be impacted.

However, if you set a shared memory cap on a zone that uses shared
memory, and you set it too low, performance of that zone will suffer,
or the application will fail. It is important to know how much memory
your applications will lock - if they lock any.

To determine how much memory a zone's processes lock, first find the
zone's current ID number:

GZ# zoneadm list -cv
ID NAME ...
 0 global ...
 1 myzone ...

Then use that number with the kstat command:

GZ# kstat 'caps:1:lockedmem_zone_1:usage'
module: caps  ...
name:   lockedmem_zone_1 ...
   usage:  4096

--JeffV

On Mon, Nov 29, 2010 at 2:23 PM, Enda O'Connor  wrote:
> Hi
> Locked memory is typically used by oracle database, ie ISM/DISM segments
> etc, not likely to be used on desktop, apps that use shared memory tend to
> try and pin it in memory to give max performance.
> I wouldn't think a desktop would need this typically.
>
> De
> On 29/11/2010 19:16, Jordan Vaughan wrote:
>>
>> "Locked memory" is the same as "pinned memory": In other words, pages
>> that won't be paged to disk. Applications can request that pages be
>> "locked" into memory. The pager won't page locked pages to disk.
>>
>> Regarding an "appropriate value for desktop usage": It depends on what
>> kinds of applications you're using. Most applications don't use
>> locked/pinned pages. I don't set this property on my desktop, but you
>> could set it to a small value. (0M?)
>>
>> Jordan
>>
>> On 11/27/10 01:15 PM, Orvar Korvar wrote:
>>>
>>> At the same time, I would like to ask exactly what is "locked" RAM?
>>> How much is an apropriate value for desktop usage? 2GB?
>>>
>>> add capped-memory
>>> set locked=2GB
>>> end
>>
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones zone.max-shm-memory setting.

[Enda O'Connor]

Hi
Locked memory is typically used by oracle database, ie ISM/DISM segments 
etc, not likely to be used on desktop, apps that use shared memory tend 
to try and pin it in memory to give max performance.

I wouldn't think a desktop would need this typically.

De
On 29/11/2010 19:16, Jordan Vaughan wrote:

"Locked memory" is the same as "pinned memory": In other words, pages
that won't be paged to disk. Applications can request that pages be
"locked" into memory. The pager won't page locked pages to disk.

Regarding an "appropriate value for desktop usage": It depends on what
kinds of applications you're using. Most applications don't use
locked/pinned pages. I don't set this property on my desktop, but you
could set it to a small value. (0M?)

Jordan

On 11/27/10 01:15 PM, Orvar Korvar wrote:

At the same time, I would like to ask exactly what is "locked" RAM?
How much is an apropriate value for desktop usage? 2GB?

add capped-memory
set locked=2GB
end

___
zones-discuss mailing list
zones-discuss@opensolaris.org


___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones zone.max-shm-memory setting.

[Jordan Vaughan]

"Locked memory" is the same as "pinned memory": In other words, pages 
that won't be paged to disk.  Applications can request that pages be 
"locked" into memory.  The pager won't page locked pages to disk.


Regarding an "appropriate value for desktop usage": It depends on what 
kinds of applications you're using.  Most applications don't use 
locked/pinned pages.  I don't set this property on my desktop, but you 
could set it to a small value.  (0M?)


Jordan

On 11/27/10 01:15 PM, Orvar Korvar wrote:

At the same time, I would like to ask exactly what is "locked" RAM? How much is 
an apropriate value for desktop usage? 2GB?

add capped-memory
   set locked=2GB
end

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones zone.max-shm-memory setting.

[Orvar Korvar]

At the same time, I would like to ask exactly what is "locked" RAM? How much is 
an apropriate value for desktop usage? 2GB?

add capped-memory
  set locked=2GB
end
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones zone.max-shm-memory setting.

[Ian Garbutt]

Hello experts, I'm after a bit help on the above.

We have a number of zones with the above being set.  When the systems were 
built we were told that its a setting to stop the oracle databases using up all 
the memory.

When I look into zone.max-shm-memory all I get is that its a limit on shared 
memory, but what is it actually limiting and is it required after all? and if 
it is required what should it be set as?  ie half the zones RAM etc.

Many thanks in advance.

Ian
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones and storage pools

[Ian Collins]

On 11/ 3/10 02:21 PM, Henrik Johansson wrote:

On Nov 3, 2010, at 1:22 AM, Ian Collins wrote:

On Nov 3, 2010, at 12:06 AM, Ian Collins wrote:

On 11/ 3/10 11:56 AM, Henrik Johansson wrote:

I would ideally like to do two things:
1. Have all filesystem configuration for the zone in the pool as 
we have with the global zone, only specify the pool(s) for the 
zone and all filesystems would be mounted inside the zone, this 
without giving away all control to the local zone.
Why don't you want the zones to be able to manage their own 
filesystems?  One of the main reasons for "zoned" filesystems is to 
allow filesystems to have mount points relative to the zone's root 
filesystem.
It would depend on what kind of users we have in the zone and how 
the zone is used, for some it would be fine to give away all control 
for other we would like to keep them from deleting 
snapshots/datasets or changing properties like quota. If the zones 
is compromised or if a privileged users does something nasty we 
would like to be able to rollback it from the global zone only.


I guess you can solve those two by setting the quota on the 
filesystem containing the zoned filesystem and replicating snapshots.


I guess replication would solve the problem but with lots of overhead.


What? no backups!

The quota problem should work if we dedicated a dataset below the pool 
itself to the zone as it's "root dataset". But for some zones we would 
really like to limit all zfs operations such as rename, destroy and 
create. The solution to this would be to make sure there are no users 
inside the zone with such privileges, as long as the zone is not 
compromised it would be fine.



The only option there would be to deny root access to the zone's users.

Changing properties for the zone could also affect the global zone 
and other zones on the same global zone, lets say you would enable 
gzip-9 compression and write lots of data, that would bypass all 
resource limits for the zone and drastically change the amount of 
cycles required for zones datasets . Even worse for zones in 
Solaris.Next you could enable deduplication which could eat the 
metadata part of the ARC. This would decrease the amount of 
 separation provided by zones with resource management.


I can see those being a problem if the cycles consumed by compression 
are not assigned to the zone (I'm not sure if they are or not), 
otherwise a zone cpu-cap would protect the rest of the system.  As 
for dedup, don't enable it on the zone dataset pool!


The zone is not accounted for the resources consumed by ZFS so that 
could be a problem. I don't and won't assign gzip compression or 
deduplication to any datasets, but a privileged user in the zone could 
do just that.


I thought as much, but wasn't sure.  I guess what you really want is 
some form of block on overriding inherited properties.


Maybe raise this issue on zfs-discuss?

--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones and storage pools

[Henrik Johansson]

On Nov 3, 2010, at 1:22 AM, Ian Collins wrote:
>> 
>> On Nov 3, 2010, at 12:06 AM, Ian Collins wrote:
>> 
>>> On 11/ 3/10 11:56 AM, Henrik Johansson wrote:
 
 I would ideally like to do two things:
 
 1. Have all filesystem configuration for the zone in the pool as we have 
 with the global zone, only specify the pool(s) for the zone and all 
 filesystems would be mounted inside the zone, this without giving away all 
 control to the local zone.
 
>>> Why don't you want the zones to be able to manage their own filesystems?  
>>> One of the main reasons for "zoned" filesystems is to allow filesystems to 
>>> have mount points relative to the zone's root filesystem.
>> 
>> It would depend on what kind of users we have in the zone and how the zone 
>> is used, for some it would be fine to give away all control for other we 
>> would like to keep them from deleting snapshots/datasets or changing 
>> properties like quota. If the zones is compromised or if a privileged users 
>> does something nasty we would like to be able to rollback it from the global 
>> zone only.
>> 
> I guess you can solve those two by setting the quota on the filesystem 
> containing the zoned filesystem and replicating snapshots.

I guess replication would solve the problem but with lots of overhead. The 
quota problem should work if we dedicated a dataset below the pool itself to 
the zone as it's "root dataset". But for some zones we would really like to 
limit all zfs operations such as rename, destroy and create. The solution to 
this would be to make sure there are no users inside the zone with such 
privileges, as long as the zone is not compromised it would be fine.

> 
>> We would like the administrative gains of having all "filesystem" 
>> configuration inside the pool but for some customers we handle their data 
>> needs even if they have privileged equal to root inside the zone so we would 
>> like to restrict what they can do.
>> 
> Would a quota be enough?

That could help but the root scenario would break the privileges as described 
above. I must look into what we can do with our roles for ZFS management inside 
the zone.

> 
>> Changing properties for the zone could also affect the global zone and other 
>> zones on the same global zone, lets say you would enable gzip-9 compression 
>> and write lots of data, that would bypass all resource limits for the zone 
>> and drastically change the amount of cycles required for zones datasets . 
>> Even worse for zones in Solaris.Next you could enable deduplication which 
>> could eat the metadata part of the ARC. This would decrease the amount of  
>> separation provided by zones with resource management.
>> 
> I can see those being a problem if the cycles consumed by compression are not 
> assigned to the zone (I'm not sure if they are or not), otherwise a zone 
> cpu-cap would protect the rest of the system.  As for dedup, don't enable it 
> on the zone dataset pool!

The zone is not accounted for the resources consumed by ZFS so that could be a 
problem. I don't and won't assign gzip compression or deduplication to any 
datasets, but a privileged user in the zone could do just that.


Henrik
http://sparcv9.blogspot.com

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones and storage pools

[Ian Collins]

On 11/ 3/10 12:56 PM, Henrik Johansson wrote:

Hello Ian,

Thanks four your answer, more below.


On Nov 3, 2010, at 12:06 AM, Ian Collins wrote:


On 11/ 3/10 11:56 AM, Henrik Johansson wrote:


I would ideally like to do two things:

1. Have all filesystem configuration for the zone in the pool as we 
have with the global zone, only specify the pool(s) for the zone and 
all filesystems would be mounted inside the zone, this without 
giving away all control to the local zone.


Why don't you want the zones to be able to manage their own 
filesystems?  One of the main reasons for "zoned" filesystems is to 
allow filesystems to have mount points relative to the zone's root 
filesystem.


It would depend on what kind of users we have in the zone and how the 
zone is used, for some it would be fine to give away all control for 
other we would like to keep them from deleting snapshots/datasets or 
changing properties like quota. If the zones is compromised or if a 
privileged users does something nasty we would like to be able to 
rollback it from the global zone only.


I guess you can solve those two by setting the quota on the filesystem 
containing the zoned filesystem and replicating snapshots.


We would like the administrative gains of having all "filesystem" 
configuration inside the pool but for some customers we handle their 
data needs even if they have privileged equal to root inside the zone 
so we would like to restrict what they can do.



Would a quota be enough?

Changing properties for the zone could also affect the global zone and 
other zones on the same global zone, lets say you would enable gzip-9 
compression and write lots of data, that would bypass all resource 
limits for the zone and drastically change the amount of cycles 
required for zones datasets . Even worse for zones in Solaris.Next you 
could enable deduplication which could eat the metadata part of the 
ARC. This would decrease the amount of  separation provided by zones 
with resource management.


I can see those being a problem if the cycles consumed by compression 
are not assigned to the zone (I'm not sure if they are or not), 
otherwise a zone cpu-cap would protect the rest of the system.  As for 
dedup, don't enable it on the zone dataset pool!


--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones and storage pools

[Henrik Johansson]

Hello Ian,

Thanks four your answer, more below.


On Nov 3, 2010, at 12:06 AM, Ian Collins wrote:

> On 11/ 3/10 11:56 AM, Henrik Johansson wrote:
>> 
>> 
>> I would ideally like to do two things:
>> 
>> 1. Have all filesystem configuration for the zone in the pool as we have 
>> with the global zone, only specify the pool(s) for the zone and all 
>> filesystems would be mounted inside the zone, this without giving away all 
>> control to the local zone.
>> 
> Why don't you want the zones to be able to manage their own filesystems?  One 
> of the main reasons for "zoned" filesystems is to allow filesystems to have 
> mount points relative to the zone's root filesystem.

It would depend on what kind of users we have in the zone and how the zone is 
used, for some it would be fine to give away all control for other we would 
like to keep them from deleting snapshots/datasets or changing properties like 
quota. If the zones is compromised or if a privileged users does something 
nasty we would like to be able to rollback it from the global zone only. 

We would like the administrative gains of having all "filesystem" configuration 
inside the pool but for some customers we handle their data needs even if they 
have privileged equal to root inside the zone so we would like to restrict what 
they can do.

Changing properties for the zone could also affect the global zone and other 
zones on the same global zone, lets say you would enable gzip-9 compression and 
write lots of data, that would bypass all resource limits for the zone and 
drastically change the amount of cycles required for zones datasets . Even 
worse for zones in Solaris.Next you could enable deduplication which could eat 
the metadata part of the ARC. This would decrease the amount of  separation 
provided by zones with resource management.

Henrik
http://sparcv9.blogspot.com

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones and storage pools

[Ian Collins]

On 11/ 3/10 11:56 AM, Henrik Johansson wrote:

Hi all,

I would like you take on this for a large zone installation.

I am going to create zones on zpools with a pool for the zoneroot and 
another pool for for application data, the second pool can differ in 
layout, disk system and properties and can easily be separated from 
the zone and moved to another zone, global or local.


Previously we have defined the filesystems for the application data 
specifically in the zone config for every filesystem, but to leverage 
some of the ZFS power to the users or have simpler zone configuration 
I would like to dedicate the pool to the zone.


I would ideally like to do two things:

1. Have all filesystem configuration for the zone in the pool as we 
have with the global zone, only specify the pool(s) for the zone and 
all filesystems would be mounted inside the zone, this without giving 
away all control to the local zone.


Why don't you want the zones to be able to manage their own 
filesystems?  One of the main reasons for "zoned" filesystems is to 
allow filesystems to have mount points relative to the zone's root 
filesystem.


2. Delegate ZFS operations to the zone so that privileged users only 
can perform a subset of ZFS operations from inside the zone (or 
deligate to local users), something like:

(zfs allow -z zone01snapshot,mount,rollback zone01_pool01).


Again, why?

3. Be able to do all administration of the pool from inside the global 
zone even if a dataset is exported to a pool. Today I am for example 
unable to create a dataset to a pool owned by a zone and set the 
mountpoint (which should be relative to the zone):



See my comment on mount points.

Today I can give away a pool to a zone but it will have control over 
without the ability to restrict it and I would the not be able to 
create new datasets for the pool with alternate mountpoints without 
going through zlogin. As an RFE I would also like to see an option to 
boot zones into single-user mode even if filesystems for pools besides 
zoneroot are unavalable.


Does anyone have similar setup?  How do you handle datasets for local 
zones?

All input is appreciated.

I tend to create all zones with a dedicated ZFS dataset, which is often 
on a different pool from the zone root.  This works well when there are 
different snapshot/replication cycles for user and system data.


--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones and storage pools

[Henrik Johansson]

Hi all,

I would like you take on this for a large zone installation.

I am going to create zones on zpools with a pool for the zoneroot and another 
pool for for application data, the second pool can differ in layout, disk 
system and properties and can easily be separated from the zone and moved to 
another zone, global or local.

Previously we have defined the filesystems for the application data 
specifically in the zone config for every filesystem, but to leverage some of 
the ZFS power to the users or have simpler zone configuration I would like to 
dedicate the pool to the zone.

I would ideally like to do two things:

1. Have all filesystem configuration for the zone in the pool as we have with 
the global zone, only specify the pool(s) for the zone and all filesystems 
would be mounted inside the zone, this without giving away all control to the 
local zone.

2. Delegate ZFS operations to the zone so that privileged users only can 
perform a subset of ZFS operations from inside the zone (or deligate to local 
users), something like:
(zfs allow -z zone01snapshot,mount,rollback zone01_pool01).

3. Be able to do all administration of the pool from inside the global zone 
even if a dataset is exported to a pool. Today I am for example unable to 
create a dataset to a pool owned by a zone and set the mountpoint (which should 
be relative to the zone):

Today I can give away a pool to a zone but it will have control over without 
the ability to restrict it and I would the not be able to create new datasets 
for the pool with alternate mountpoints without going through zlogin. As an RFE 
I would also like to see an option to boot zones into single-user mode even if 
filesystems for pools besides zoneroot are unavalable.

Does anyone have similar setup?  How do you handle datasets for local zones?  
All input is appreciated.

Thanks

Henrik
http://sparcv9.blogspot.com

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on NFS

[Ian Collins]

On 08/ 6/10 01:43 AM, Prasoon Bansal wrote:

Hello Benr,

I had read all the reply of your query posted on this blog.


Which blog?  Your post appears to be an orphan.


I have the same matching query with the others. As i had configured the 
non-global zone on nfs shared folder(nfsserver) and this shared folder is 
mapped onto another host(testzone). I am able to configured and see the status 
of non-global zone on both the hosts(nfsserver $ testzone).
I sucessfully detach my test zone from nfsserver but not able to attached onto 
testzone as showing the error as zonepath is configured on nfs share folder, 
local file system must be configured.
   


You should copy the zoneroot to the new host.

--
Ian.

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on NFS

[Prasoon Bansal]

Hello Benr,

I had read all the reply of your query posted on this blog. I have the same 
matching query with the others. As i had configured the non-global zone on nfs 
shared folder(nfsserver) and this shared folder is mapped onto another 
host(testzone). I am able to configured and see the status of non-global zone 
on both the hosts(nfsserver $ testzone). 
I sucessfully detach my test zone from nfsserver but not able to attached onto 
testzone as showing the error as zonepath is configured on nfs share folder, 
local file system must be configured.
Please let me know, if you have any of solution for this problem as of this i 
am not able to attache the non-global zone onto my testzone box. Once it get 
attach then can make it boot and can be used.
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on zfs on san - shared pool or separate ?

[Alexander J. Maidak]

On Wed, 2010-03-31 at 18:01 -0700, Brett wrote:
> Hi Folks,
> 
> I would like to source opinions of the forum on whether its better to share a 
> zfs storage pool for all zones on a machine or create one zpool per zone? The 
> premise is that this zpool (zonepool) would be sitting on san storage.
> 
> I posed that a consolidated pool (zonepool) with each zone zfs sitting under 
> that was good for ease of management and efficiency of resource. ie:
> zonepool/zone1
> zonepool/zone2
> zonepool/zone3
> 
> However a colleague suggests keeping separate zpools, one for each zone is 
> better for reasons of portability (ie being able to detach a zone and then 
> export the zonepool and move the san storage to another like machine if 
> resource becomes constrained). ie:
> zonepool1/zone1
> zonepool2/zone2
> zonepool3/zone3
> 
> Your thoughts are very welcome.
> Regards Rep

We deploy a zpool for each zone to allow for easier mobility.  We also
use the delegated dataset option, so our config is basically:

zone1/zonepath  <- this is were the zoneroot is stored
zone1/zp00  <- this dataset is delegated to the zone for zone data
zone2/zonepath
zone2/zp00

All application data is stored in child datasets of zp00.  This allows
live upgrade to create clones of the zonepath dataset and keeps our
application data isolated from the zoneroot.

Zone Migration consists of:

shutdown zone
detach zone
export zpool 
rezone LUNs to another machine
import zpool
attach zone

Putting the zones in one pool will use storage more efficiently, but
you're zone migration will require a zfs send -> zfs receive step, which
if you're dealing with large zone datasets could be time consuming.

If you're going to deploy lots of zones that don't use much disk I think
I'd go with option 1, because send/recv with a few gigs of data is
probably no big deal.  If you're zones are going to hold many gigs of
data you're life may be easier with SAN based migration.

-Alex

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on zfs on san - shared pool or separate ?

[Michael S. Keller]

Brett wrote:

Hi Folks,

I would like to source opinions of the forum on whether its better to share a 
zfs storage pool for all zones on a machine or create one zpool per zone? The 
premise is that this zpool (zonepool) would be sitting on san storage.

I posed that a consolidated pool (zonepool) with each zone zfs sitting under 
that was good for ease of management and efficiency of resource. ie:
zonepool/zone1
zonepool/zone2
zonepool/zone3

However a colleague suggests keeping separate zpools, one for each zone is 
better for reasons of portability (ie being able to detach a zone and then 
export the zonepool and move the san storage to another like machine if 
resource becomes constrained). ie:
zonepool1/zone1
zonepool2/zone2
zonepool3/zone3

Your thoughts are very welcome.
Regards Rep


Most of the zones I've been building have been separate VxVM disk groups
for reason of portability. They've also been under Veritas Cluster
Server control for start/stop and ease of switching among nodes, nothing
fancy, but it makes the zones more fault tolerant.

So if I had the luxury of multiple LUNs, I'd go with separate pools.

-Michael

___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones on zfs on san - shared pool or separate ?

[Brett]

Hi Folks,

I would like to source opinions of the forum on whether its better to share a 
zfs storage pool for all zones on a machine or create one zpool per zone? The 
premise is that this zpool (zonepool) would be sitting on san storage.

I posed that a consolidated pool (zonepool) with each zone zfs sitting under 
that was good for ease of management and efficiency of resource. ie:
zonepool/zone1
zonepool/zone2
zonepool/zone3

However a colleague suggests keeping separate zpools, one for each zone is 
better for reasons of portability (ie being able to detach a zone and then 
export the zonepool and move the san storage to another like machine if 
resource becomes constrained). ie:
zonepool1/zone1
zonepool2/zone2
zonepool3/zone3

Your thoughts are very welcome.
Regards Rep
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Frank Batschulat (Home)]

update on this one: 

a workaround if you so will, or the more appropriate way to do this is 
apparently
to use lofiadm(1M) to create a pseudo block device comprising the file hosted 
on NFS
and use the created lofi device (eg. /dev/lofi/1) as the device for zpool create
and all subsequent I/O (this was not producing the strange CKSUM errors), eg.:

osoldev.root./export/home/batschul.=> mount -F nfs opteron:/pool/zones /nfszone
osoldev.root./export/home/batschul.=> mount -v| grep nfs
opteron:/pool/zones on /nfszone type nfs 
remote/read/write/setuid/devices/xattr/dev=9080001 on Tue Feb  9 10:37:00 2010
osoldev.root./export/home/batschul.=> nfsstat -m
/nfszone from opteron:/pool/zones
 Flags: 
vers=4,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=1048576,wsize=1048576,retrans=5,timeo=600
 Attr cache:acregmin=3,acregmax=60,acdirmin=30,acdirmax=60

osoldev.root./export/home/batschul.=>  mkfile -n 7G /nfszone/remote.file
osoldev.root./export/home/batschul.=>  ls -la /nfszone
total 28243534
drwxrwxrwx   2 nobody   nobody 6 Feb  9 09:36 .
drwxr-xr-x  30 batschul other 32 Feb  8 22:24 ..
-rw---   1 nobody   nobody   7516192768 Feb  9 09:36 remote.file

osoldev.root./export/home/batschul.=> lofiadm -a /nfszone/remote.file
/dev/lofi/1

osoldev.root./export/home/batschul.=> lofiadm
Block Device File   Options
/dev/lofi/1  /nfszone/remote.file   -

osoldev.root./export/home/batschul.=> zpool create -m /tank/zones/nfszone 
nfszone /dev/lofi/1

Feb  9 10:50:35 osoldev zfs: [ID 249136 kern.info] created version 22 pool 
nfszone using 22

osoldev.root./export/home/batschul.=> zpool status -v nfszone
  pool: nfszone
 state: ONLINE
 scrub: none requested
config:

NAME   STATE READ WRITE CKSUM
nfszoneONLINE   0 0 0
  /dev/lofi/1  ONLINE   0 0 0

---
frankB
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones update on attach problem

[Brian Kolaci]

I've had success for zones update on attach in the past, but now ran into a 
problem.

I'm using Solaris x86 and just built a new system.  Since it has a CD drive (no 
DVD), I had to use the U7 disks to install, then I put the recommended patches, 
then used the U8 image to update the live upgrade packages and used live 
upgrade to bring it to U8.

I have a zone on another U8 box that I detached and want to move over.  When I 
try to attach, it gives me the following error:

# zoneadm -z scanner3 attach -u 
failed to initialize ZFS library

zoneadm: zone 'scanner3': ERROR: attempt to downgrade package SUNWhea, the 
source had patch 140566-02 which is not installed on this system

However the patch is installed:

# showrev -p | grep 140566
Patch: 140566-02 Obsoletes: 121395-03 Requires: 118844-24, 118855-36, 127128-11 
Incompatibles:  Packages: SUNWhea, SUNWckr

I tried removing the SUNWdetached.xml file (which sometimes helps), but still 
no glory.

any ideas as to what I can do to fix this?

Thanks,

Brian

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones patching issues using attach -u

[Gael]

On Fri, Jan 22, 2010 at 11:42 AM, Frank Batschulat (Home) <
frank.batschu...@sun.com> wrote:

> On Fri, 22 Jan 2010 18:30:39 +0100, Gael  wrote:
>
> >> This is bug:
> >>
> >> 6857294 zoneadm attach leads to partially installed packages
> >>
> >> I believe a T patch might be available for the S10 SVr4 packaging code
> >> if you need it, but I see that the fix has not yet been integrated
> >> into the nv SVr4 packaging code.  It is scheduled for b124.
> >
> > Was that fix ever released ?
>
> Yes, Solaris 10U9 will have it, ONNV/OSOL build 125 has it.
> and the following Solaris 10 patches have been released offically
> containing the fix for 6857294
>
> 119254-72 (sparc)
> 119255-72 (x86)
>
> (3 days ago)
>
> ---
> frankB
>

Exciting news, going to try that patch today :) Thanks!

-- 
Gael Martinez
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones patching issues using attach -u

[Frank Batschulat (Home)]

On Fri, 22 Jan 2010 18:30:39 +0100, Gael  wrote:

>> This is bug:
>>
>> 6857294 zoneadm attach leads to partially installed packages
>>
>> I believe a T patch might be available for the S10 SVr4 packaging code
>> if you need it, but I see that the fix has not yet been integrated
>> into the nv SVr4 packaging code.  It is scheduled for b124.
>
> Was that fix ever released ?

Yes, Solaris 10U9 will have it, ONNV/OSOL build 125 has it.
and the following Solaris 10 patches have been released offically
containing the fix for 6857294

119254-72 (sparc)
119255-72 (x86)

(3 days ago)

---
frankB
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones patching issues using attach -u

[Gael]

On Tue, Sep 15, 2009 at 1:02 PM, Jerry Jelinek wrote:

>  Gael wrote:
>
>> Hello
>>
>> I have been experimenting a few ways to speed up patching a bunch of
>> machines running whole zones (parallel patching, zoneadm attach -u).
>> I have encountered one issue with the attach -u way... Before initiating a
>> case with sun, I was wondering if it was a well known issue...
>>
>> The GZ is initially running Solaris 10 U6 with kernel patch 13-08 (and
>> other patches from the same period).
>>
>> I start by applying 119254-70, 119313-28 and 12-05 while the machine
>> is
>> in multiuser mode, then I shutdown and detach the zones.
>> I bring back the machine in single user mode and apply a collection of
>> about
>> 190 patches (smpatch analyze output from a few days ago) which brings the
>> machine at the kernel version 141414-10.
>>
>> The patching appears to go fine for the GZ
>>
>> apss8003:/var/sadm/patch #pkginfo -p
>> apss8003:/var/sadm/patch #
>>
>> But when zoneadm attaching -u the zones, pkginfo reports multiple
>> partially
>> failing packages adds ...
>>
>> apss8003:/var/sadm/patch #zlogin test pkginfo -p
>> system  SUNWcsr Core Solaris, (Root)
>> system  SUNWgsscGSSAPI CONFIG V2
>> system  SUNWkrbrKerberos version 5 support
>> (Root)
>> system  SUNWntprNTP, (Root)
>> system  SUNWppror   PatchPro core functionality
>> (Root)
>> system  SUNWsacom   Solstice Enterprise Agents
>> 1.0.3
>> files for root file system
>>
>> # cat /zones/test/root//var/sadm/system/logs/update_log | egrep
>> "partially|corrupt|pathname does not exist|"
>>
>> = SUNWcsr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWgssc 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWkrbr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWntpr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWppror 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed
>>
>> = SUNWsacom 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> If creating a new zone after the patching, there is no partial packages in
>> that newly build zone.
>>
>> The patch list being a little bit lengthy, I can send it privately when
>> asked...
>>
>
> This is bug:
>
> 6857294 zoneadm attach leads to partially installed packages
>
> I believe a T patch might be available for the S10 SVr4 packaging code
> if you need it, but I see that the fix has not yet been integrated
> into the nv SVr4 packaging code.  It is scheduled for b124.
>
> Jerry
>


Good morning

Was that fix ever released ?

Regards

-- 
Gael Martinez
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Tue, Jan 19, 2010 at 02:33:15PM -0700, Jerry Jelinek wrote:
> On 01/19/10 14:00, Edward Pilatowicz wrote:
> >>>   perhaps it would make sense to add some "tokens" to the comments in
> >>>   brand_asm.h like:
> >>>   32-BIT INTERPOSITION STACK
> >>>   32-BIT LCALL/INT STACK
> >>>   64-BIT INTERPOSITION STACK
> >>>   64-BIT LCALL/INT STACK
> >>>
> >>>   and then in the comments for each brand callback you could refer the
> >>>   reader to the appropriate tokens in brand_asm.h.
> >>
> >>I added this.
> >>
> >
> >could you add references to these tokens to lx_brand_asm.s as well?
>
> Ed,
>
> Will do, sorry for not adding this yesterday.  Do you want to re-review it
> again after that comment change?
>

not really.

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

On 01/19/10 14:00, Edward Pilatowicz wrote:

   perhaps it would make sense to add some "tokens" to the comments in
   brand_asm.h like:
32-BIT INTERPOSITION STACK
32-BIT LCALL/INT STACK
64-BIT INTERPOSITION STACK
64-BIT LCALL/INT STACK

   and then in the comments for each brand callback you could refer the
   reader to the appropriate tokens in brand_asm.h.


I added this.



could you add references to these tokens to lx_brand_asm.s as well?


Ed,

Will do, sorry for not adding this yesterday.  Do you want to re-review it
again after that comment change?

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Mon, Jan 18, 2010 at 12:51:27PM -0700, Jerry Jelinek wrote:
> Ed,
>
> Thanks for reviewing this, my responses are inline.
>
> On 01/15/10 19:26, Edward Pilatowicz wrote:
> >On Thu, Jan 14, 2010 at 09:18:20AM -0700, Jerry Jelinek wrote:
> >>I need a code review for my proposed fix for:
> >>
> >>6887823 brandz on x86 should ignore %gs and simplify brand hooks
> >>
> >>There is a webrev at:
> >>
> >>http://cr.opensolaris.org/~gjelinek/webrev.6887823/
> >>
> >>This simplifies some of the handling for the %gs register, cleans up
> >>the interfaces with the brand modules, and consolidates common code
> >>into a single file.  Although the webrev looks large, most of this is 
> >>because
> >>of moving the common code.
> >>
> >
> >hey jerry,
> >looks good.  a few comments below.
> >ed
> >
> >--
> >usr/src/uts/i86pc/ml/syscall_asm.s
> >
> >- so i think the following comment is true for lcall and int syscalls on
> >   32-bit kernels:
> >
> > * --
> > * | user's %ss |
> > *|| user's %esp|
> > *|| EFLAGS register|
> > *|| user's %cs |
> > *|| user's %eip (user return address)  |
> > *|| 'scratch space'|
> >
> >   but what about sysenter syscalls?  none of that stuff will be on the
> >   stack.  (that's why BRAND_CALLBACK used to explicitly push the user
> >   return address onto the stack.)  have you tested with sysenter
> >   syscalls on 32-bit kernels?
>
> I haven't really changed this.  If you look at the original code in 
> syscall_asm.s,
> all it was doing was re-pushing a value that was already on the stack.  This 
> is line
> 152 in the original code in syscall_asm.s.  The reason this works is that the
> sysenter callback doesn't use the data from the stack since the return address
> is in the %edx register (see the comment on the callback in
> s10_brand_asm.s or sn1_brand_asm.s).  The SYSCALL_EMUL macro
> expects the return address to be in a register, not on the stack.
>

ok.  i got it this time around.  man, this stuff is always confusing...

> >--
> >usr/src/uts/intel/brand/*/*_brand_asm.s
> >
> >- you've removed all the comments that explain how the stack frame looks
> >   and what the assumptions are when we enter the different handlers.  i
> >   realize this is all documented in brand_asm.h now, but now folks don't
> >   know how to map each of the handlers to their running conditions.  it'd
> >   be nice if there was a comment for each handler pointing people to that
> >   the correct comments that apply to each handler.
> >
> >   perhaps it would make sense to add some "tokens" to the comments in
> >   brand_asm.h like:
> > 32-BIT INTERPOSITION STACK
> > 32-BIT LCALL/INT STACK
> > 64-BIT INTERPOSITION STACK
> > 64-BIT LCALL/INT STACK
> >
> >   and then in the comments for each brand callback you could refer the
> >   reader to the appropriate tokens in brand_asm.h.
>
> I added this.
>

could you add references to these tokens to lx_brand_asm.s as well?

thanks for cleaning all this stuff up.
ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Ed,

Thanks for reviewing this, my responses are inline.

On 01/15/10 19:26, Edward Pilatowicz wrote:

On Thu, Jan 14, 2010 at 09:18:20AM -0700, Jerry Jelinek wrote:

I need a code review for my proposed fix for:

6887823 brandz on x86 should ignore %gs and simplify brand hooks

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6887823/

This simplifies some of the handling for the %gs register, cleans up
the interfaces with the brand modules, and consolidates common code
into a single file.  Although the webrev looks large, most of this is because
of moving the common code.



hey jerry,
looks good.  a few comments below.
ed

--
usr/src/uts/i86pc/ml/syscall_asm.s

- so i think the following comment is true for lcall and int syscalls on
   32-bit kernels:

* --
* | user's %ss |
*|| user's %esp|
*|| EFLAGS register|
*|| user's %cs |
*|| user's %eip (user return address)  |
*|| 'scratch space'|

   but what about sysenter syscalls?  none of that stuff will be on the
   stack.  (that's why BRAND_CALLBACK used to explicitly push the user
   return address onto the stack.)  have you tested with sysenter
   syscalls on 32-bit kernels?


I haven't really changed this.  If you look at the original code in 
syscall_asm.s,
all it was doing was re-pushing a value that was already on the stack.  This is 
line
152 in the original code in syscall_asm.s.  The reason this works is that the
sysenter callback doesn't use the data from the stack since the return address
is in the %edx register (see the comment on the callback in
s10_brand_asm.s or sn1_brand_asm.s).  The SYSCALL_EMUL macro
expects the return address to be in a register, not on the stack.


--
usr/src/uts/intel/brand/common/brand_asm.h

- could you create this file by doing an hg cp of sn1_brand_asm.h or
   s10_brand_asm.h?  (it preserves the history and also makes it easy
   to see changes to the defines.)


Done.


- this doesn't seem right:
#ifndef _SYS_BRAND_ASM_H
   the SYS_ prefix is normally used by header files in /sys/.
   for this file you should probably have:
#ifndef _BRAND_ASM_H
#ifndef _COMMON_BRAND_ASM_H


Changed.


- this file should probably #include the files which define macros
   that it uses.  for example, you should probably include:
#include
   i'm sure there are others as well.


Changed.


--
usr/src/uts/intel/brand/*/*_brand_asm.s

- you've removed all the comments that explain how the stack frame looks
   and what the assumptions are when we enter the different handlers.  i
   realize this is all documented in brand_asm.h now, but now folks don't
   know how to map each of the handlers to their running conditions.  it'd
   be nice if there was a comment for each handler pointing people to that
   the correct comments that apply to each handler.

   perhaps it would make sense to add some "tokens" to the comments in
   brand_asm.h like:
32-BIT INTERPOSITION STACK
32-BIT LCALL/INT STACK
64-BIT INTERPOSITION STACK
64-BIT LCALL/INT STACK

   and then in the comments for each brand callback you could refer the
   reader to the appropriate tokens in brand_asm.h.


I added this.



--
usr/src/uts/intel/brand/lx/lx_brand_asm.s

- in lx_brand_int80_callback() you have:
* See the comment on CALC_TABLE_ADDR in brand_asm.h for an
   but this function never uses CALC_TABLE_ADDR.  it still does the
   offset calculation manually:
shlq$4, %rax

   i think you could clean this up by changing CHECK_FOR_HANDLER and
   CALC_TABLE_ADDR in brand_asm.h to be de
CHECK_FOR_HANDLER(scr, handler)
...
cmp $0, handler(scr);   /* check handler */
   and:
CALC_TABLE_ADDR(scr, handler)
...
mov handler(scr), scr;  /* get p_brand_data->handler */ \

   then you also don't need this at the top of every brand:
#define USP_HANDLER ...

   and you can use CALC_TABLE_ADDR for jump table calculations in the lx
   brand.


This is a good idea, I rewrote things along these lines.

There is a new webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6887823/

Thanks,
Jerry

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Thu, Jan 14, 2010 at 09:18:20AM -0700, Jerry Jelinek wrote:
> I need a code review for my proposed fix for:
>
> 6887823 brandz on x86 should ignore %gs and simplify brand hooks
>
> There is a webrev at:
>
> http://cr.opensolaris.org/~gjelinek/webrev.6887823/
>
> This simplifies some of the handling for the %gs register, cleans up
> the interfaces with the brand modules, and consolidates common code
> into a single file.  Although the webrev looks large, most of this is because
> of moving the common code.
>

hey jerry,
looks good.  a few comments below.
ed

--
usr/src/uts/i86pc/ml/syscall_asm.s

- so i think the following comment is true for lcall and int syscalls on
  32-bit kernels:

* --
* | user's %ss |
*|| user's %esp|
*|| EFLAGS register|
*|| user's %cs |
*|| user's %eip (user return address)  |
*|| 'scratch space'|

  but what about sysenter syscalls?  none of that stuff will be on the
  stack.  (that's why BRAND_CALLBACK used to explicitly push the user
  return address onto the stack.)  have you tested with sysenter
  syscalls on 32-bit kernels?

--
usr/src/uts/intel/brand/common/brand_asm.h

- could you create this file by doing an hg cp of sn1_brand_asm.h or
  s10_brand_asm.h?  (it preserves the history and also makes it easy
  to see changes to the defines.)

- this doesn't seem right:
#ifndef _SYS_BRAND_ASM_H
  the SYS_ prefix is normally used by header files in /sys/.
  for this file you should probably have:
#ifndef _BRAND_ASM_H
#ifndef _COMMON_BRAND_ASM_H

- this file should probably #include the files which define macros
  that it uses.  for example, you should probably include:
#include 
  i'm sure there are others as well.


--
usr/src/uts/intel/brand/*/*_brand_asm.s

- you've removed all the comments that explain how the stack frame looks
  and what the assumptions are when we enter the different handlers.  i
  realize this is all documented in brand_asm.h now, but now folks don't
  know how to map each of the handlers to their running conditions.  it'd
  be nice if there was a comment for each handler pointing people to that
  the correct comments that apply to each handler.

  perhaps it would make sense to add some "tokens" to the comments in
  brand_asm.h like:
32-BIT INTERPOSITION STACK
32-BIT LCALL/INT STACK
64-BIT INTERPOSITION STACK
64-BIT LCALL/INT STACK

  and then in the comments for each brand callback you could refer the
  reader to the appropriate tokens in brand_asm.h.


--
usr/src/uts/intel/brand/lx/lx_brand_asm.s

- in lx_brand_int80_callback() you have:
* See the comment on CALC_TABLE_ADDR in brand_asm.h for an
  but this function never uses CALC_TABLE_ADDR.  it still does the
  offset calculation manually:
shlq$4, %rax

  i think you could clean this up by changing CHECK_FOR_HANDLER and
  CALC_TABLE_ADDR in brand_asm.h to be de
CHECK_FOR_HANDLER(scr, handler)
...
cmp $0, handler(scr);   /* check handler */
  and:
CALC_TABLE_ADDR(scr, handler)
...
mov handler(scr), scr;  /* get p_brand_data->handler */ \

  then you also don't need this at the top of every brand:
#define USP_HANDLER ...

  and you can use CALC_TABLE_ADDR for jump table calculations in the lx
  brand.
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jordan Vaughan]

On 01/14/10 08:18 AM, Jerry Jelinek wrote:

I need a code review for my proposed fix for:

6887823 brandz on x86 should ignore %gs and simplify brand hooks

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6887823/

This simplifies some of the handling for the %gs register, cleans up
the interfaces with the brand modules, and consolidates common code
into a single file.  Although the webrev looks large, most of this is 
because

of moving the common code.

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org


Hi Jerry,

This looks fine to me.

Jordan
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones code review

[Jerry Jelinek]

I need a code review for my proposed fix for:

6887823 brandz on x86 should ignore %gs and simplify brand hooks

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6887823/

This simplifies some of the handling for the %gs register, cleans up
the interfaces with the brand modules, and consolidates common code
into a single file.  Although the webrev looks large, most of this is because
of moving the common code.

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Frank Batschulat (Home)]

On Fri, 08 Jan 2010 18:33:06 +0100, Mike Gerdts  wrote:

> I've written a dtrace script to get the checksums on Solaris 10.
> Here's what I see with NFSv3 on Solaris 10.

jfyi, I've reproduces it as well using a Solaris 10 Update 8 SB2000 sparc client
and NFSv4.

much like you I also get READ errors along with the CKSUM errors which
is different from my observation on a ONNV client.

unfortunately your dtrace script did not worked for me, ie. it
did not spit out anything :(

cheers
frankB

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

On Fri, Jan 8, 2010 at 9:11 AM, Mike Gerdts  wrote:
> I've seen similar errors on Solaris 10 in the primary domain and on a
> M4000.  Unfortunately Solaris 10 doesn't show the checksums in the
> ereport.  There I noticed a mixture between read errors and checksum
> errors - and lots more of them.  This could be because the S10 zone
> was a full root SUNWCXall compared to the much smaller default ipkg
> branded zone.  On the primary domain running Solaris 10...

I've written a dtrace script to get the checksums on Solaris 10.
Here's what I see with NFSv3 on Solaris 10.

# zoneadm -z zone1 halt ; zpool export pool1 ; zpool import -d
/mnt/pool1 pool1 ; zoneadm -z zone1 boot ; sleep 30 ; pkill dtrace

# ./zfs_bad_cksum.d
Tracing...
dtrace: error on enabled probe ID 9 (ID 43443:
fbt:zfs:zio_checksum_error:return): invalid address (0x301b363a000) in
action #4 at DIF offset 20
dtrace: error on enabled probe ID 9 (ID 43443:
fbt:zfs:zio_checksum_error:return): invalid address (0x3037f746000) in
action #4 at DIF offset 20
cccdtrace:
error on enabled probe ID 9 (ID 43443:
fbt:zfs:zio_checksum_error:return): invalid address (0x3026e7b) in
action #4 at DIF offset 20
cc
Checksum errors:
   3 : 0x130e01011103 0x20108 0x0 0x400 (fletcher_4_native)
   3 : 0x220125cd8000 0x62425980c08 0x16630c08296c490c
0x82b320c082aef0c (fletcher_4_native)
   3 : 0x2f2a0a202a20436f 0x7079726967687420 0x2863292032303031
0x2062792053756e20 (fletcher_4_native)
   3 : 0x3c21444f43545950 0x452048544d4c2050 0x55424c494320222d
0x2f2f5733432f2f44 (fletcher_4_native)
   3 : 0x6005a8389144 0xc2080e6405c200b6 0x960093d40800
0x9eea007b9800019c (fletcher_4_native)
   3 : 0xac044a6903d00163 0xa138c8003446 0x3f2cd1e100b10009
0xa37af9b5ef166104 (fletcher_4_native)
   3 : 0xbaddcafebaddcafe 0xc 0x0 0x0 (fletcher_4_native)
   3 : 0xc4025608801500ff 0x1018500704528210 0x190103e50066
0xc34b90001238f900 (fletcher_4_native)
   3 : 0xfe00fc01fc42fc42 0xfc42fc42fc42fc42 0xfffc42fc42fc42fc
0x42fc42fc42fc42fc (fletcher_4_native)
   4 : 0x4b2a460a 0x0 0x4b2a460a 0x0 (fletcher_4_native)
   4 : 0xc00589b159a00 0x543008a05b673 0x124b60078d5be
0xe3002b2a0b605fb3 (fletcher_4_native)
   4 : 0x130e010111 0x32000b301080034 0x10166cb34125410
0xb30c19ca9e0c0860 (fletcher_4_native)
   4 : 0x130e010111 0x3a201080038 0x104381285501102
0x418016996320408 (fletcher_4_native)
   4 : 0x130e010111 0x3a201080038 0x1043812c5501102
0x81802325c080864 (fletcher_4_native)
   4 : 0x130e010111 0x3a0001c01080038 0x1383812c550111c
0x818975698080864 (fletcher_4_native)
   4 : 0x1f81442e9241000 0x2002560880154c00 0xff10185007528210
0x19010003e566 (fletcher_4_native)
   5 : 0xbab10c 0xf 0x53ae 0xdd549ae39aa1ba20 (fletcher_4_native)
   5 : 0x130e010111 0x3ab01080038 0x1163812c550110b
0x8180a7793080864 (fletcher_4_native)
   5 : 0x61626300 0x0 0x0 0x0 (fletcher_4_native)
   5 : 0x8003 0x3df0d6a1 0x0 0x0 (fletcher_4_native)
   6 : 0xbab10c 0xf 0x5384 0xdd549ae39aa1ba20 (fletcher_4_native)
   7 : 0xbab10c 0xf 0x0 0x9af5e5f61ca2e28e (fletcher_4_native)
   7 : 0x130e010111 0x3a201080038 0x104381265501102
0xc18c7210c086006 (fletcher_4_native)
   7 : 0x275c222074650a2e 0x5c222020436f7079 0x7269676874203139
0x38392041540a2e5c (fletcher_4_native)
   8 : 0x130e010111 0x3a0003101080038 0x1623812c5501131
0x8187f66a4080864 (fletcher_4_native)
   9 : 0x8a000801010c0682 0x2eed0809c1640513 0x70200ff00026424
0x18001d16101f0059 (fletcher_4_native)
  12 : 0xbab10c 0xf 0x0 0x45a9e1fc57ca2aa8 (fletcher_4_native)
  30 : 0xbaddcafebaddcafe 0xbaddcafebaddcafe 0xbaddcafebaddcafe
0xbaddcafebaddcafe (fletcher_4_native)
  47 : 0x0 0x0 0x0 0x0 (fletcher_4_native)
  92 : 0x130e01011103 0x10108 0x0 0x200 (fletcher_4_native)

Since I had to guess at what the Solaris 10 source looks like, some
extra eyeballs on the dtrace script is in order.

Mike

-- 
Mike Gerdts
http://mgerdts.blogspot.com/


zfs_bad_cksum.d
Description: Binary data
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

On Fri, Jan 8, 2010 at 5:28 AM, Frank Batschulat (Home)
 wrote:
[snip]
> Hey Mike, you're not the only victim of these strange CHKSUM errors, I hit
> the same during my slightely different testing, where I'm NFS mounting an
> entire, pre-existing remote file living in the zpool on the NFS server and use
> that to create a zpool and install zones into it.

What does your overall setup look like?

Mine is:

T5220 + Sun System Firmware 7.2.4.f 2009/11/05 18:21
   Primary LDom
  Solaris 10u8
  Logical Domains Manager 1.2,REV=2009.06.25.09.48 + 142840-03
  Guest Domain 4 vcpus + 15 GB memory
 OpenSolaris snv_130
(this is where the problem is observed)

I've seen similar errors on Solaris 10 in the primary domain and on a
M4000.  Unfortunately Solaris 10 doesn't show the checksums in the
ereport.  There I noticed a mixture between read errors and checksum
errors - and lots more of them.  This could be because the S10 zone
was a full root SUNWCXall compared to the much smaller default ipkg
branded zone.  On the primary domain running Solaris 10...

(this command was run some time ago)
primary-domain# zpool status myzone
  pool: myzone
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

NAMESTATE READ WRITE CKSUM
myzone  DEGRADED 0 0 0
  /foo/20g  DEGRADED 4.53K 0   671  too many errors

errors: No known data errors


(this was run today, many days after previous command)
primary-domain# fmdump -eV | egrep zio_err | uniq -c | head
   1zio_err = 5
   1zio_err = 50
   1zio_err = 5
   1zio_err = 50
   1zio_err = 5
   1zio_err = 50
   2zio_err = 5
   1zio_err = 50
   3zio_err = 5
   1zio_err = 50


Note that even though I had thousands of read errors the zone worked
just fine. I would have never known (suspected?) there was a problem
if I hadn't run "zpool status" or the various FMA commands.


> I've filed today:
>
> 6915265 zpools on files (over NFS) accumulate CKSUM errors with no apparent 
> reason

Thanks.  I'll open a support call to help get some funding on it...

> here's the relevant piece worth investigating out of it (leaving out the 
> actual setup etc..)
> as in your case, creating the zpool and installing the zone into it still 
> gives
> a healthy zpool, but immediately after booting the zone, the zpool served 
> over NFS
> accumulated CHKSUM errors.
>
> of particular interest are the 'cksum_actual' values as reported by Mike for 
> his
> test case here:
>
> http://www.mail-archive.com/zfs-disc...@opensolaris.org/msg33041.html
>
> if compared to the 'chksum_actual' values I got in the fmdump error output on 
> my test case/system:
>
> note, the NFS servers zpool that is serving and sharing the file we use is 
> healthy.
>
> zone halted now on my test system, and checking fmdump:
>
> osoldev.batschul./export/home/batschul.=> fmdump -eV | grep cksum_actual | 
> sort | uniq -c | sort -n | tail
>   2    cksum_actual = 0x4bea1a77300 0xf6decb1097980 0x217874c80a8d9100 
> 0x7cd81ca72df5ccc0
>   2    cksum_actual = 0x5c1c805253 0x26fa7270d8d2 0xda52e2079fd74 
> 0x3d2827dd7ee4f21
>   6    cksum_actual = 0x28e08467900 0x479d57f76fc80 0x53bca4db5209300 
> 0x983ddbb8c4590e40
> *A   6    cksum_actual = 0x348e6117700 0x765aa1a547b80 0xb1d6d98e59c3d00 
> 0x89715e34fbf9cdc0
> *B   7    cksum_actual = 0x0 0x0 0x0 0x0
> *C  11    cksum_actual = 0x1184cb07d00 0xd2c5aab5fe80 0x69ef5922233f00 
> 0x280934efa6d20f40
> *D  14    cksum_actual = 0x175bb95fc00 0x1767673c6fe00 0xfa9df17c835400 
> 0x7e0aef335f0c7f00
> *E  17    cksum_actual = 0x2eb772bf800 0x5d8641385fc00 0x7cf15b214fea800 
> 0xd4f1025a8e66fe00
> *F  20    cksum_actual = 0xbaddcafe00 0x5dcc54647f00 0x1f82a459c2aa00 
> 0x7f84b11b3fc7f80
> *G  25    cksum_actual = 0x5d6ee57f00 0x178a70d27f80 0x3fc19c3a19500 
> 0x82804bc6ebcfc0
>
> osoldev.root./export/home/batschul.=> zpool status -v
>  pool: nfszone
>  state: DEGRADED
> status: One or more devices has experienced an unrecoverable error.  An
>        attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>        using 'zpool clear' or replace the device with 'zpool replace'.
>   see: http://www.sun.com/msg/ZFS-8000-9P
>  scrub: none requested
> config:
>
>        NAME        STATE     READ WRITE CKSUM
>        nfszone     DEGRADED     0     0     0
>          /nfszone  DEGRADED     0     0   462  too many errors
>
> errors: No known data errors
>
> ==
>
> now compare this with Mike's error output as p

Re: [zones-discuss] Zones on shared storage - a warning

[James Carlson]

Mike Gerdts wrote:
> This unsupported feature is supported with the use of Sun Ops Center
> 2.5 when a zone is put on a "NAS Storage Library".

Ah, ok.  I didn't know that.

-- 
James Carlson 42.703N 71.076W 
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

On Fri, Jan 8, 2010 at 6:51 AM, James Carlson  wrote:
> Frank Batschulat (Home) wrote:
>> This just can't be an accident, there must be some coincidence and thus 
>> there's a good chance
>> that these CHKSUM errors must have a common source, either in ZFS or in NFS ?
>
> One possible cause would be a lack of substantial exercise.  The man
> page says:
>
>         A regular file. The use of files as a backing  store  is
>         strongly  discouraged.  It  is  designed  primarily  for
>         experimental purposes, as the fault tolerance of a  file
>         is  only  as  good  as  the file system of which it is a
>         part. A file must be specified by a full path.
>
> Could it be that "discouraged" and "experimental" mean "not tested as
> thoroughly as you might like, and certainly not a good idea in any sort
> of production environment?"
>
> It sounds like a bug, sure, but the fix might be to remove the option.

This unsupported feature is supported with the use of Sun Ops Center
2.5 when a zone is put on a "NAS Storage Library".

-- 
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[James Carlson]

Frank Batschulat (Home) wrote:
> This just can't be an accident, there must be some coincidence and thus 
> there's a good chance
> that these CHKSUM errors must have a common source, either in ZFS or in NFS ?

One possible cause would be a lack of substantial exercise.  The man
page says:

 A regular file. The use of files as a backing  store  is
 strongly  discouraged.  It  is  designed  primarily  for
 experimental purposes, as the fault tolerance of a  file
 is  only  as  good  as  the file system of which it is a
 part. A file must be specified by a full path.

Could it be that "discouraged" and "experimental" mean "not tested as
thoroughly as you might like, and certainly not a good idea in any sort
of production environment?"

It sounds like a bug, sure, but the fix might be to remove the option.

-- 
James Carlson 42.703N 71.076W 
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Frank Batschulat (Home)]

On Wed, 23 Dec 2009 03:02:47 +0100, Mike Gerdts  wrote:

> I've been playing around with zones on NFS a bit and have run into
> what looks to be a pretty bad snag - ZFS keeps seeing read and/or
> checksum errors.  This exists with S10u8 and OpenSolaris dev build
> snv_129.  This is likely a blocker for anything thinking of
> implementing parts of Ed's Zones on Shared Storage:
>
> http://hub.opensolaris.org/bin/view/Community+Group+zones/zoss
>
> The OpenSolaris example appears below.  The order of events is:
>
> 1) Create a file on NFS, turn it into a zpool
> 2) Configure a zone with the pool as zonepath
> 3) Install the zone, verify that the pool is healthy
> 4) Boot the zone, observe that the pool is sick
[...]
> r...@soltrain19# zoneadm -z osol boot
>
> r...@soltrain19# zpool status osol
>   pool: osol
>  state: DEGRADED
> status: One or more devices has experienced an unrecoverable error.  An
> attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
> using 'zpool clear' or replace the device with 'zpool replace'.
>see: http://www.sun.com/msg/ZFS-8000-9P
>  scrub: none requested
> config:
>
> NAME  STATE READ WRITE CKSUM
> osol  DEGRADED 0 0 0
>   /mnt/osolzone/root  DEGRADED 0 0   117  too many errors
>
> errors: No known data errors

Hey Mike, you're not the only victim of these strange CHKSUM errors, I hit
the same during my slightely different testing, where I'm NFS mounting an
entire, pre-existing remote file living in the zpool on the NFS server and use 
that to create a zpool and install zones into it.

I've filed today:

6915265 zpools on files (over NFS) accumulate CKSUM errors with no apparent 
reason

here's the relevant piece worth investigating out of it (leaving out the actual 
setup etc..)
as in your case, creating the zpool and installing the zone into it still gives
a healthy zpool, but immediately after booting the zone, the zpool served over 
NFS
accumulated CHKSUM errors.

of particular interest are the 'cksum_actual' values as reported by Mike for his
test case here:

http://www.mail-archive.com/zfs-disc...@opensolaris.org/msg33041.html

if compared to the 'chksum_actual' values I got in the fmdump error output on 
my test case/system:

note, the NFS servers zpool that is serving and sharing the file we use is 
healthy.

zone halted now on my test system, and checking fmdump:

osoldev.batschul./export/home/batschul.=> fmdump -eV | grep cksum_actual | sort 
| uniq -c | sort -n | tail
   2cksum_actual = 0x4bea1a77300 0xf6decb1097980 0x217874c80a8d9100 
0x7cd81ca72df5ccc0
   2cksum_actual = 0x5c1c805253 0x26fa7270d8d2 0xda52e2079fd74 
0x3d2827dd7ee4f21
   6cksum_actual = 0x28e08467900 0x479d57f76fc80 0x53bca4db5209300 
0x983ddbb8c4590e40
*A   6cksum_actual = 0x348e6117700 0x765aa1a547b80 0xb1d6d98e59c3d00 
0x89715e34fbf9cdc0
*B   7cksum_actual = 0x0 0x0 0x0 0x0
*C  11cksum_actual = 0x1184cb07d00 0xd2c5aab5fe80 0x69ef5922233f00 
0x280934efa6d20f40
*D  14cksum_actual = 0x175bb95fc00 0x1767673c6fe00 0xfa9df17c835400 
0x7e0aef335f0c7f00
*E  17cksum_actual = 0x2eb772bf800 0x5d8641385fc00 0x7cf15b214fea800 
0xd4f1025a8e66fe00
*F  20cksum_actual = 0xbaddcafe00 0x5dcc54647f00 0x1f82a459c2aa00 
0x7f84b11b3fc7f80
*G  25cksum_actual = 0x5d6ee57f00 0x178a70d27f80 0x3fc19c3a19500 
0x82804bc6ebcfc0

osoldev.root./export/home/batschul.=> zpool status -v
  pool: nfszone
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

NAMESTATE READ WRITE CKSUM
nfszone DEGRADED 0 0 0
  /nfszone  DEGRADED 0 0   462  too many errors

errors: No known data errors

==

now compare this with Mike's error output as posted here:

http://www.mail-archive.com/zfs-disc...@opensolaris.org/msg33041.html

# fmdump -eV | grep cksum_actual | sort | uniq -c | sort -n | tail

   2cksum_actual = 0x14c538b06b6 0x2bb571a06ddb0 0x3e05a7c4ac90c62 
0x290cbce13fc59dce
*D   3cksum_actual = 0x175bb95fc00 0x1767673c6fe00 0xfa9df17c835400 
0x7e0aef335f0c7f00
*E   3cksum_actual = 0x2eb772bf800 0x5d8641385fc00 0x7cf15b214fea800 
0xd4f1025a8e66fe00
*B   4cksum_actual = 0x0 0x0 0x0 0x0
   4cksum_actual = 0x1d32a7b7b00 0x248deaf977d80 0x1e8ea26c8a2e900 
0x330107da7c4bcec0
   5cksum_actual = 0x14b8f7afe6 0x915db8d7f87 0x205dc7979ad73 
0x4e0b3a8747b8a8
*C   6cksum_actual = 0x1184cb07d00 0xd2c5aab5fe80 0x69ef5922233f00 
0x280934efa6d20f4

Re: [zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

On Tue, Dec 22, 2009 at 9:34 PM, Mike Gerdts  wrote:
> On Tue, Dec 22, 2009 at 8:02 PM, Mike Gerdts  wrote:
>> I've been playing around with zones on NFS a bit and have run into
>> what looks to be a pretty bad snag - ZFS keeps seeing read and/or
>> checksum errors.  This exists with S10u8 and OpenSolaris dev build
>> snv_129.  This is likely a blocker for anything thinking of
>> implementing parts of Ed's Zones on Shared Storage:
>>
>> http://hub.opensolaris.org/bin/view/Community+Group+zones/zoss
>>
>> The OpenSolaris example appears below.  The order of events is:
>>
>> 1) Create a file on NFS, turn it into a zpool
>> 2) Configure a zone with the pool as zonepath
>> 3) Install the zone, verify that the pool is healthy
>> 4) Boot the zone, observe that the pool is sick
> [snip]
>
> An off list conversation and a bit of digging into other tests I have
> done shows that this is likely limited to NFSv3.  I cannot say that
> this problem has been seen with NFSv4.

I have been able to reproduce this with NFSv4 on build 130.  Taking
the discussion to the zfs list...

-- 
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

On Tue, Dec 22, 2009 at 8:02 PM, Mike Gerdts  wrote:
> I've been playing around with zones on NFS a bit and have run into
> what looks to be a pretty bad snag - ZFS keeps seeing read and/or
> checksum errors.  This exists with S10u8 and OpenSolaris dev build
> snv_129.  This is likely a blocker for anything thinking of
> implementing parts of Ed's Zones on Shared Storage:
>
> http://hub.opensolaris.org/bin/view/Community+Group+zones/zoss
>
> The OpenSolaris example appears below.  The order of events is:
>
> 1) Create a file on NFS, turn it into a zpool
> 2) Configure a zone with the pool as zonepath
> 3) Install the zone, verify that the pool is healthy
> 4) Boot the zone, observe that the pool is sick
[snip]

An off list conversation and a bit of digging into other tests I have
done shows that this is likely limited to NFSv3.  I cannot say that
this problem has been seen with NFSv4.

-- 
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones on shared storage - a warning

[Mike Gerdts]

I've been playing around with zones on NFS a bit and have run into
what looks to be a pretty bad snag - ZFS keeps seeing read and/or
checksum errors.  This exists with S10u8 and OpenSolaris dev build
snv_129.  This is likely a blocker for anything thinking of
implementing parts of Ed's Zones on Shared Storage:

http://hub.opensolaris.org/bin/view/Community+Group+zones/zoss

The OpenSolaris example appears below.  The order of events is:

1) Create a file on NFS, turn it into a zpool
2) Configure a zone with the pool as zonepath
3) Install the zone, verify that the pool is healthy
4) Boot the zone, observe that the pool is sick

r...@soltrain19# mount filer:/path /mnt
r...@soltrain19# cd /mnt
r...@soltrain19# mkdir osolzone
r...@soltrain19# mkfile -n 8g root
r...@soltrain19# zpool create -m /zones/osol osol /mnt/osolzone/root
r...@soltrain19# zonecfg -z osol
osol: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:osol> create
zonecfg:osol> info
zonename: osol
zonepath:
brand: ipkg
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
hostid:
zonecfg:osol> set zonepath=/zones/osol
zonecfg:osol> set autoboot=false
zonecfg:osol> verify
zonecfg:osol> commit
zonecfg:osol> exit

r...@soltrain19# chmod 700 /zones/osol

r...@soltrain19# zoneadm -z osol install
   Publisher: Using opensolaris.org (http://pkg.opensolaris.org/dev/
http://pkg-na-2.opensolaris.org/dev/).
   Publisher: Using contrib (http://pkg.opensolaris.org/contrib/).
   Image: Preparing at /zones/osol/root.
   Cache: Using /var/pkg/download.
Sanity Check: Looking for 'entire' incorporation.
  Installing: Core System (output follows)
DOWNLOAD  PKGS   FILESXFER (MB)
Completed46/46 12334/1233493.1/93.1

PHASEACTIONS
Install Phase18277/18277
No updates necessary for this image.
  Installing: Additional Packages (output follows)
DOWNLOAD  PKGS   FILESXFER (MB)
Completed36/36   3339/333921.3/21.3

PHASEACTIONS
Install Phase  4466/4466

Note: Man pages can be obtained by installing SUNWman
 Postinstall: Copying SMF seed repository ... done.
 Postinstall: Applying workarounds.
Done: Installation completed in 2139.186 seconds.

  Next Steps: Boot the zone, then log into the zone console (zlogin -C)
  to complete the configuration process.
6.3 Boot the OpenSolaris zone
r...@soltrain19# zpool status osol
  pool: osol
 state: ONLINE
 scrub: none requested
config:

NAME  STATE READ WRITE CKSUM
osol  ONLINE   0 0 0
  /mnt/osolzone/root  ONLINE   0 0 0

errors: No known data errors

r...@soltrain19# zoneadm -z osol boot

r...@soltrain19# zpool status osol
  pool: osol
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

NAME  STATE READ WRITE CKSUM
osol  DEGRADED 0 0 0
  /mnt/osolzone/root  DEGRADED 0 0   117  too many errors

errors: No known data errors

r...@soltrain19# zlogin osol uptime
  5:31pm  up 1 min(s),  0 users,  load average: 0.69, 0.38, 0.52


-- 
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Edward Pilatowicz wrote:

so we're actually changing our stack pointer after entry into the
kernel, so it's no longer necessarily matching the interrupt stack that
the processor switched in automatically and saved the parameters on.
notably we don't do this for 32-bit kernels.  this means that
de-referencing V_SSP is the right things todo.  sorry for taking so long
to understand this code...

so one last comment nit and then i promise i'm done.  could you update
the the descriptions of the stack setup by BRAND_CALLBACK on 64-bit
kernels to be more accurate for interrupt syscalls by changing:
"user stack pointer"
to:
"user (or interrupt) stack pointer"

thanks, and sorry again for the delays while i tried to understand the
code better.


Ed,

Thanks for all of your comments.  I'll make the update you suggested.

Thanks again,
Jerry

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Fri, Dec 18, 2009 at 08:19:02AM -0700, Jerry Jelinek wrote:
> Jerry Jelinek wrote:
> >Because its not right above, all of the other register values are
> >also pushed on the stack, so we need to go through the SSP to get
> >to the right spot.  I can add a comment explaining this but the
> >32bit and 64bit stacks are not identical.
>
> Ed,
>
> Actually, what I said above is not quite right.  I think that
> its not the other registers but the alignment that is making
> the stacks different.  I took another look at the AMD64 Architecture
> Programmers Manual, Volume 2: System Programming manual.  This is
> discussed in section 8.9 Long-Mode Interrupt Control Transfers.  You
> can see how the stack is different vs. the discussion in section 8.7.
>

all the comments in our source code would seem to indicate that the
stacks are the same.  also, looking at the stack diagram in the v2
manual they seem to be the same to me (aisde from the obvious 32 vs 64
bit word size differences):

Figure 8-9 Stack After Interrupt to Higher Privilege
p280 (p320)

Figure 8-14 Long-Mode Stack After Interrupt-Higher Privilege
p292 (p332)

also, right above figure 8-14 there is a discussion of the differences
between the long mode switch vs the classic legacy switch, with the only
difference being in how the SS is handled.

i guess this boils down to is, is there a difference between the value
of V_SSP and address of V_END, and if so why?  i set some breakpoints in
kmdb and compared these values and they are indeed different.  looking
at brand callback i can now see why, we do the following:
---8<---
#define BRAND_CALLBACK(callback_id) \
movq%rsp, %gs:CPU_RTMP_RSP  /* save the stack pointer   */ ;\
movq%r15, %gs:CPU_RTMP_R15  /* save %r15*/ ;\
movq%gs:CPU_THREAD, %r15/* load the thread pointer  */ ;\
movqT_STACK(%r15), %rsp /* switch to the kernel stack   */ ;\
---8<---

so we're actually changing our stack pointer after entry into the
kernel, so it's no longer necessarily matching the interrupt stack that
the processor switched in automatically and saved the parameters on.
notably we don't do this for 32-bit kernels.  this means that
de-referencing V_SSP is the right things todo.  sorry for taking so long
to understand this code...

so one last comment nit and then i promise i'm done.  could you update
the the descriptions of the stack setup by BRAND_CALLBACK on 64-bit
kernels to be more accurate for interrupt syscalls by changing:
"user stack pointer"
to:
"user (or interrupt) stack pointer"

thanks, and sorry again for the delays while i tried to understand the
code better.

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Jerry Jelinek wrote:

Because its not right above, all of the other register values are
also pushed on the stack, so we need to go through the SSP to get
to the right spot.  I can add a comment explaining this but the
32bit and 64bit stacks are not identical.


Ed,

Actually, what I said above is not quite right.  I think that
its not the other registers but the alignment that is making
the stacks different.  I took another look at the AMD64 Architecture
Programmers Manual, Volume 2: System Programming manual.  This is
discussed in section 8.9 Long-Mode Interrupt Control Transfers.  You
can see how the stack is different vs. the discussion in section 8.7.

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Edward Pilatowicz wrote:

so now you have:
---8<---
#define V_U_EIP (CLONGSIZE * 0)
...
GET_V(%rsp, 1, V_SSP, %rax) /* get saved stack pointer */
SET_V(%rax, 0, V_U_EIP, %r15)   /* save new return addr in %eip */
---8<---

but why can't this be identical to the 32-bit path?  afaik, it seems
like you could just do:

---8<---
#define V_U_EIP (V_END + (CLONGSIZE * 0))
...
SET_V(%rsp, 1, V_U_EIP, %r15)   /* save new return addr in %eip */
---8<---

why load V_SSP if we already know that the interrupt state is right on
the stack above the callback arguments?  (it seems we sholud just
access the state directly without first loading V_SSP.)


Ed,

Because its not right above, all of the other register values are
also pushed on the stack, so we need to go through the SSP to get
to the right spot.  I can add a comment explaining this but the
32bit and 64bit stacks are not identical.

Thanks,
Jerry


___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Thu, Dec 17, 2009 at 04:24:22PM -0700, Jerry Jelinek wrote:
> Edward Pilatowicz wrote:
> >to be:
> >---8<---
> >/*
> > * The saved stack pointer (V_SSP) points to the interrupt specific
> > * state, which is saved directly above the stack contents common to all
> > * callbacks.
> >...
> >*/
> >#define V_U_SS  (V_END + (CLONGSIZE * 4))
> >#define V_U_ESP (V_END + (CLONGSIZE * 3))
> >#define V_EFLAGS(V_END + (CLONGSIZE * 2))
> >#define V_U_CS  (V_END + (CLONGSIZE * 1))
> >#define V_U_EIP (V_END + (CLONGSIZE * 0))
> >
> >ENTRY(sn1_brand_int91_callback)
> >...
> > SET_V(%rsp, 1, V_U_EIP, %r15)   /* set user %eip to JMP table addr */
> > GET_V(%rsp, 1, V_URET_ADDR, %rax) /* save orig return addr in %eax */
> >---8<---
>
> Ed,
>
> Thanks for the correction on the comment.  I also updated the code as
> you suggested.  I'm not sure if what I have now is better than before
> but its the same number of instructions and its more similar to the
> the 32-bit code path (although it can't be identical).  I posted a new
> webrev at:
>
> http://cr.opensolaris.org/~gjelinek/webrev.6768950/
>
> Let me know if you have any other comments.


so now you have:
---8<---
#define V_U_EIP (CLONGSIZE * 0)
...
GET_V(%rsp, 1, V_SSP, %rax) /* get saved stack pointer */
SET_V(%rax, 0, V_U_EIP, %r15)   /* save new return addr in %eip */
---8<---

but why can't this be identical to the 32-bit path?  afaik, it seems
like you could just do:

---8<---
#define V_U_EIP (V_END + (CLONGSIZE * 0))
...
SET_V(%rsp, 1, V_U_EIP, %r15)   /* save new return addr in %eip */
---8<---

why load V_SSP if we already know that the interrupt state is right on
the stack above the callback arguments?  (it seems we sholud just
access the state directly without first loading V_SSP.)

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Edward Pilatowicz wrote:

On Thu, Dec 17, 2009 at 09:39:34AM -0700, Jerry Jelinek wrote:

Edward Pilatowicz wrote:

- CALC_TABLE_ADDR() a little clunky.  (it has seperate 32 and 64
 versions, it assumes the syscall number is in eax/rax, and it has a
 side effect of munging the syscall number.)  how about defining just one
 version of CALC_TABLE_ADDR() as:
---8<---
#define CALC_TABLE_ADDR(sysnum, rv) 
\
GET_P_BRAND_DATA(%rsp, 1, rv)   /* get p_brand_data ptr */; 
\
mov SPD_HANDLER(rv), rv /* get p_brand_data->spd_handler ptr 
*/;\
shl $4, sysnum  /* syscall_num * 16 */; 
\
add sysnum, result  /* calc JMP table address */;   
\
shr $4, sysnum  /* syscall_num / 16 */
---8<---

Ed,

I reworked the macros and I think its a lot cleaner now.
Let me know what you think.  The new webrev is at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/



this looks great.  things are much simpler and the callbacks are more
similar.  :)

i only have one nit.  i think the following comment is incorrect:

 * To 'return' to our user-space handler, we just need to place its
 * address into the user's %ss.

it should read:

 * To 'return' to our user-space handler we need to update the
 * user's %eip pointer in the saved interrupt state.  The
 * interrupt state was pushed onto our stack automatically when
 * the interrupt occured, see the comments above.

actually, now that i look at it some more...

i think you could make the int91 callback simpler by making it almost
the same as the like the 32-bit syscall callback.  after all, the stack
content basically is the same in both call paths...

specifically, you could change the following:
---8<---
/*
 * The saved stack pointer points at the state saved when we took
 * the interrupt:
...
*/
ENTRY(sn1_brand_int91_callback)
...
movq%r15, %rax  /* save new ret addr in %rax */
GET_V(%rsp, 1, V_SSP, %r15) /* Get saved %esp */
xchgq   (%r15), %rax/* swap %rax and return addr */
---8<---

to be:
---8<---
/*
 * The saved stack pointer (V_SSP) points to the interrupt specific
 * state, which is saved directly above the stack contents common to all
 * callbacks.
...
*/
#define V_U_SS  (V_END + (CLONGSIZE * 4))
#define V_U_ESP (V_END + (CLONGSIZE * 3))
#define V_EFLAGS(V_END + (CLONGSIZE * 2))
#define V_U_CS  (V_END + (CLONGSIZE * 1))
#define V_U_EIP (V_END + (CLONGSIZE * 0))

ENTRY(sn1_brand_int91_callback)
...
SET_V(%rsp, 1, V_U_EIP, %r15)   /* set user %eip to JMP table addr */
GET_V(%rsp, 1, V_URET_ADDR, %rax) /* save orig return addr in %eax */
---8<---


Ed,

Thanks for the correction on the comment.  I also updated the code as
you suggested.  I'm not sure if what I have now is better than before
but its the same number of instructions and its more similar to the
the 32-bit code path (although it can't be identical).  I posted a new
webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

Let me know if you have any other comments.

Thanks,
Jerry

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Thu, Dec 17, 2009 at 09:39:34AM -0700, Jerry Jelinek wrote:
> Edward Pilatowicz wrote:
> >- CALC_TABLE_ADDR() a little clunky.  (it has seperate 32 and 64
> >  versions, it assumes the syscall number is in eax/rax, and it has a
> >  side effect of munging the syscall number.)  how about defining just one
> >  version of CALC_TABLE_ADDR() as:
> >---8<---
> >#define  CALC_TABLE_ADDR(sysnum, rv) 
> >\
> > GET_P_BRAND_DATA(%rsp, 1, rv)   /* get p_brand_data ptr */; 
> > \
> > mov SPD_HANDLER(rv), rv /* get p_brand_data->spd_handler ptr 
> > */;\
> > shl $4, sysnum  /* syscall_num * 16 */; 
> > \
> > add sysnum, result  /* calc JMP table address */;   
> > \
> > shr $4, sysnum  /* syscall_num / 16 */
> >---8<---
>
> Ed,
>
> I reworked the macros and I think its a lot cleaner now.
> Let me know what you think.  The new webrev is at:
>
> http://cr.opensolaris.org/~gjelinek/webrev.6768950/
>

this looks great.  things are much simpler and the callbacks are more
similar.  :)

i only have one nit.  i think the following comment is incorrect:

 * To 'return' to our user-space handler, we just need to place its
 * address into the user's %ss.

it should read:

 * To 'return' to our user-space handler we need to update the
 * user's %eip pointer in the saved interrupt state.  The
 * interrupt state was pushed onto our stack automatically when
 * the interrupt occured, see the comments above.

actually, now that i look at it some more...

i think you could make the int91 callback simpler by making it almost
the same as the like the 32-bit syscall callback.  after all, the stack
content basically is the same in both call paths...

specifically, you could change the following:
---8<---
/*
 * The saved stack pointer points at the state saved when we took
 * the interrupt:
...
*/
ENTRY(sn1_brand_int91_callback)
...
movq%r15, %rax  /* save new ret addr in %rax */
GET_V(%rsp, 1, V_SSP, %r15) /* Get saved %esp */
xchgq   (%r15), %rax/* swap %rax and return addr */
---8<---

to be:
---8<---
/*
 * The saved stack pointer (V_SSP) points to the interrupt specific
 * state, which is saved directly above the stack contents common to all
 * callbacks.
...
*/
#define V_U_SS  (V_END + (CLONGSIZE * 4))
#define V_U_ESP (V_END + (CLONGSIZE * 3))
#define V_EFLAGS(V_END + (CLONGSIZE * 2))
#define V_U_CS  (V_END + (CLONGSIZE * 1))
#define V_U_EIP (V_END + (CLONGSIZE * 0))

ENTRY(sn1_brand_int91_callback)
...
SET_V(%rsp, 1, V_U_EIP, %r15)   /* set user %eip to JMP table addr */
GET_V(%rsp, 1, V_URET_ADDR, %rax) /* save orig return addr in %eax */
---8<---
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Edward Pilatowicz wrote:

- CALC_TABLE_ADDR() a little clunky.  (it has seperate 32 and 64
  versions, it assumes the syscall number is in eax/rax, and it has a
  side effect of munging the syscall number.)  how about defining just one
  version of CALC_TABLE_ADDR() as:
---8<---
#define CALC_TABLE_ADDR(sysnum, rv) 
\
GET_P_BRAND_DATA(%rsp, 1, rv)   /* get p_brand_data ptr */; 
\
mov SPD_HANDLER(rv), rv /* get p_brand_data->spd_handler ptr 
*/;\
shl $4, sysnum  /* syscall_num * 16 */; 
\
add sysnum, result  /* calc JMP table address */;   
\
shr $4, sysnum  /* syscall_num / 16 */
---8<---


Ed,

I reworked the macros and I think its a lot cleaner now.
Let me know what you think.  The new webrev is at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Wed, Dec 16, 2009 at 02:37:36PM -0700, Jerry Jelinek wrote:
> Ed,
>
> Edward Pilatowicz wrote:
> >On Wed, Dec 16, 2009 at 10:46:57AM -0700, Jerry Jelinek wrote:
> >>Ed,
> >>
> >>I've posted an updated webrev to address your comments.
> >>
> >>http://cr.opensolaris.org/~gjelinek/webrev.6768950/
> >>
> >
> >usr/src/uts/intel/brand/sn1/sn1_brand_asm.s
> >
> >- i'd think the "is 0 <= syscall <= MAX" check would have to be
> >  done after CHECK_FOR_NATIVE().
>
> It is.  I added it to the CHECK_FOR_INTERPOSITION macro which is
> called after the CHECK_FOR_NATIVE in the CALLBACK_PROLOGUE.
>

oops.  your right.  i was misread this.

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Ed,

Edward Pilatowicz wrote:

On Wed, Dec 16, 2009 at 10:46:57AM -0700, Jerry Jelinek wrote:

Ed,

I've posted an updated webrev to address your comments.

http://cr.opensolaris.org/~gjelinek/webrev.6768950/



usr/src/uts/intel/brand/sn1/sn1_brand_asm.s

- i'd think the "is 0 <= syscall <= MAX" check would have to be
  done after CHECK_FOR_NATIVE().


It is.  I added it to the CHECK_FOR_INTERPOSITION macro which is
called after the CHECK_FOR_NATIVE in the CALLBACK_PROLOGUE.


- CALC_TABLE_ADDR() a little clunky.  (it has seperate 32 and 64
  versions, it assumes the syscall number is in eax/rax, and it has a
  side effect of munging the syscall number.)  how about defining just one
  version of CALC_TABLE_ADDR() as:
---8<---
#define CALC_TABLE_ADDR(sysnum, rv) 
\
GET_P_BRAND_DATA(%rsp, 1, rv)   /* get p_brand_data ptr */; 
\
mov SPD_HANDLER(rv), rv /* get p_brand_data->spd_handler ptr 
*/;\
shl $4, sysnum  /* syscall_num * 16 */; 
\
add sysnum, result  /* calc JMP table address */;   
\
shr $4, sysnum  /* syscall_num / 16 */
---8<---


Since we don't care about preserving the syscall number that extra
instruction has no value.  However, I'll take another shot at trying
to streamline this a bit.

Thanks,
Jerry


___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Wed, Dec 16, 2009 at 10:46:57AM -0700, Jerry Jelinek wrote:
> Ed,
>
> I've posted an updated webrev to address your comments.
>
> http://cr.opensolaris.org/~gjelinek/webrev.6768950/
>

usr/src/uts/intel/brand/sn1/sn1_brand_asm.s

- i'd think the "is 0 <= syscall <= MAX" check would have to be
  done after CHECK_FOR_NATIVE().

- CALC_TABLE_ADDR() a little clunky.  (it has seperate 32 and 64
  versions, it assumes the syscall number is in eax/rax, and it has a
  side effect of munging the syscall number.)  how about defining just one
  version of CALC_TABLE_ADDR() as:
---8<---
#define CALC_TABLE_ADDR(sysnum, rv) 
\
GET_P_BRAND_DATA(%rsp, 1, rv)   /* get p_brand_data ptr */; 
\
mov SPD_HANDLER(rv), rv /* get p_brand_data->spd_handler ptr 
*/;\
shl $4, sysnum  /* syscall_num * 16 */; 
\
add sysnum, result  /* calc JMP table address */;   
\
shr $4, sysnum  /* syscall_num / 16 */
---8<---

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Ed,

I've posted an updated webrev to address your comments.

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

Let me know if you have any other comments or see anything
with the changes I made.

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Jordan Vaughan wrote:

--
usr/src/lib/brand/sn1/sn1_brand/amd64/sn1_handler.s

44: Shouldn't this function be named "sn1_handler_table"?


Jordan,

Good catch, I'll fix that.

Thanks,
Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jordan Vaughan]

On 12/15/09 07:39 AM, Jerry Jelinek wrote:

I have an initial code review for the fix for bug:

6768950 panic[cpu1]/thread=ff084ce0b3e0: syscall_asm_amd64.s:480
lwp ff0756a8cdc0, pcb_rupdate != 0

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

The code changes in the sn1 and solaris10 brands are basically
identical.  I know there is a lot of common code there but I
didn't want to clutter up this bug fix with the unrelated changes
necessary to make the code common.  I'll be addressing that with
a separate fix.

My initial testing of these changes looks good but I still need
to run more extensive tests.

Thanks,
Jerry

___
zones-discuss mailing list
zones-discuss@opensolaris.org


Hi Jerry,

I'll add one question to Ed's suggestions:


--
usr/src/lib/brand/sn1/sn1_brand/amd64/sn1_handler.s

44: Shouldn't this function be named "sn1_handler_table"?


Jordan
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Tue, Dec 15, 2009 at 01:28:01PM -0700, Jerry Jelinek wrote:
> Ed,
>
> Thanks for reviewing this.  My responses to your comments are
> in-line.
>
> Edward Pilatowicz wrote:
> >On Tue, Dec 15, 2009 at 08:39:12AM -0700, Jerry Jelinek wrote:
> >>I have an initial code review for the fix for bug:
> >>
> >>6768950 panic[cpu1]/thread=ff084ce0b3e0: syscall_asm_amd64.s:480
> >>lwp ff0756a8cdc0, pcb_rupdate != 0
> >>
> >>There is a webrev at:
> >>
> >>http://cr.opensolaris.org/~gjelinek/webrev.6768950/
> >>
> >>The code changes in the sn1 and solaris10 brands are basically
> >>identical.  I know there is a lot of common code there but I
> >>didn't want to clutter up this bug fix with the unrelated changes
> >>necessary to make the code common.  I'll be addressing that with
> >>a separate fix.
> >>
> >>My initial testing of these changes looks good but I still need
> >>to run more extensive tests.
> >>
...
> >- prior to these changes V_URET_ADDR wasn't always set, so the different
> >  brand syscall callbacks would get the userland return address from
> >  their syscall specific locations (registers, interrupt stack, etc).  but
> >  now since V_URET_ADDR is always set, perhaps the callback handlers could
> >  be made more consistent by always getting the value from the stack (ie,
> >  via V_URET_ADDR)?
> >
> >- so following up with the last comment (and getting more into potential
> >  comminization work) it seems to me like it might be benificial to move
> >  all the syscall mechanism specific handling code out of the actual brand
> >  callbacks and into BRAND_CALLBACK.  you've already started doing this by
> >  having BRAND_CALLBACK be aware of how to get the userland return
> >  address.  (prior to that it didn't have any dependancy upon the
> >  different syscall mechanisms, except when deciding which brand callback
> >  to invoke.)  continuing down that path we could move all the syscall
> >  specific handling code into BRAND_CALLBACK.  then each brand would only
> >  deliver a single callback which would take one parameter, the syscall
> >  number.  it would return one value, a userland return address.  then
> >  BRAND_CALLBACK could handle all the different syscall specific return
> >  paths.  this would also be benificial in the future since if a new
> >  syscall mechanism was introduced, we wouldn't have to update any actual
> >  brand callbacks, just BRAND_CALLBACK.  thoughts?
>
> For these last two I agree that there are some good opportunities here and
> I was torn between doing a bunch more clean up on this and deferring that
> work to the fix for:
>
> 6900207 code can be shared between solaris10 and ipkg brands
>
> Since bug 6768950 is serious and I'd like to get the fix done sooner
> rather than later, I'd like to defer some of these other changes to 6900207.
> I was about to start on that anyway so once 6768950 is done I'm going to
> immediately start work on a bunch of ideas I have for making the code shared
> and simpler.  I was also going to roll a fix for:
>
> 6887823 brandz on x86 should ignore %gs on 64-bit kernels
>
> into that same set of cleanup.  I definitely agree with your comments here
> but I'm worried about the fix for 6768950 taking too long.
>

sounds good.
ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones code review

[Jerry Jelinek]

Ed,

Thanks for reviewing this.  My responses to your comments are
in-line.

Edward Pilatowicz wrote:

On Tue, Dec 15, 2009 at 08:39:12AM -0700, Jerry Jelinek wrote:

I have an initial code review for the fix for bug:

6768950 panic[cpu1]/thread=ff084ce0b3e0: syscall_asm_amd64.s:480
lwp ff0756a8cdc0, pcb_rupdate != 0

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

The code changes in the sn1 and solaris10 brands are basically
identical.  I know there is a lot of common code there but I
didn't want to clutter up this bug fix with the unrelated changes
necessary to make the code common.  I'll be addressing that with
a separate fix.

My initial testing of these changes looks good but I still need
to run more extensive tests.



this looks great.  i have some initial comments.

--
usr/src/lib/brand/{sn1|solaris10}/*_brand/*/*_handler.s:

- could you update the following lines with comments:
xchgq   CPTRSIZE(%rbp), %rax/* swap JMP table offset and ret addr */
shrq$4, %rax/* JMP table offset / JMP size = syscall num */
movq%rax, EH_LOCALS_GREG(REG_RAX)(%rbp) /* save syscall num */


Will do.


--
usr/src/uts/i86pc/ml/syscall_asm.s:

- don't you need to update this file as well?  have you tested 32-bit
  kernels?


No, this doesn't need to be updated since this code doesn't touch the
user's stack.  I have done preliminary testing with 32 bit kernels and
the callbacks work correctly with the code as is.  Thats because the
32 bit code is more like the 64 bit code that handles an interrupt stack
where we already have the right data pushed.


--
usr/src/uts/i86pc/ml/syscall_asm_amd64.s

- perhaps you could do the following renames:
BRAND_GET_RET_REG -> BRAND_URET_FROM_REG
BRAND_GET_RET_STACK -> BRAND_URET_FROM_INTR_STACK


Will do.


- wrt this code:
cmpq$NSYSCALL, %rax /* is 0 <= syscall <= MAX?  */
jbe 0f  /* yes, syscall is OK */
xorq%rax, %rax  /* no, zero syscall number */

  it's duplicated in every brand callback right after
  CALLBACK_PROLOGUE().  why not make it part of CALLBACK_PROLOGUE?


Will do.


  also, if the syscall num is > NSYSCALL, why not just jump to 9: and
  let the normal syscall path detect and return the error?


OK.  I was modeling this on the way lx did it but your suggestion seems
better.


- it seems like there should be a macro for this rough block of code
  (which calculates the jmp table address):

GET_P_BRAND_DATA(%esp, 1, %edx);/* get p_brand_data ptr */
movlSPD_HANDLER(%edx), %edx /* get p_brand_data->spd_handler ptr */
shll$4, %eax
addl%eax, %edx  /* we'll return to our handler */


I'll put one together.


- prior to these changes V_URET_ADDR wasn't always set, so the different
  brand syscall callbacks would get the userland return address from
  their syscall specific locations (registers, interrupt stack, etc).  but
  now since V_URET_ADDR is always set, perhaps the callback handlers could
  be made more consistent by always getting the value from the stack (ie,
  via V_URET_ADDR)?

- so following up with the last comment (and getting more into potential
  comminization work) it seems to me like it might be benificial to move
  all the syscall mechanism specific handling code out of the actual brand
  callbacks and into BRAND_CALLBACK.  you've already started doing this by
  having BRAND_CALLBACK be aware of how to get the userland return
  address.  (prior to that it didn't have any dependancy upon the
  different syscall mechanisms, except when deciding which brand callback
  to invoke.)  continuing down that path we could move all the syscall
  specific handling code into BRAND_CALLBACK.  then each brand would only
  deliver a single callback which would take one parameter, the syscall
  number.  it would return one value, a userland return address.  then
  BRAND_CALLBACK could handle all the different syscall specific return
  paths.  this would also be benificial in the future since if a new
  syscall mechanism was introduced, we wouldn't have to update any actual
  brand callbacks, just BRAND_CALLBACK.  thoughts?


For these last two I agree that there are some good opportunities here and
I was torn between doing a bunch more clean up on this and deferring that
work to the fix for:

6900207 code can be shared between solaris10 and ipkg brands

Since bug 6768950 is serious and I'd like to get the fix done sooner
rather than later, I'd like to defer some of these other changes to 6900207.
I was about to start on that anyway so once 6768950 is done I'm going to
immediately start work on a bunch of ideas I have for making the code shared
and simpler.  I was also going to roll a fix for:

6887823 brandz on x86 should ignore %gs on 64-bit kernels

into that same set of cleanup.  I definitely agree with your comments

Re: [zones-discuss] zones code review

[Edward Pilatowicz]

On Tue, Dec 15, 2009 at 08:39:12AM -0700, Jerry Jelinek wrote:
> I have an initial code review for the fix for bug:
>
> 6768950 panic[cpu1]/thread=ff084ce0b3e0: syscall_asm_amd64.s:480
> lwp ff0756a8cdc0, pcb_rupdate != 0
>
> There is a webrev at:
>
> http://cr.opensolaris.org/~gjelinek/webrev.6768950/
>
> The code changes in the sn1 and solaris10 brands are basically
> identical.  I know there is a lot of common code there but I
> didn't want to clutter up this bug fix with the unrelated changes
> necessary to make the code common.  I'll be addressing that with
> a separate fix.
>
> My initial testing of these changes looks good but I still need
> to run more extensive tests.
>

this looks great.  i have some initial comments.

--
usr/src/lib/brand/{sn1|solaris10}/*_brand/*/*_handler.s:

- could you update the following lines with comments:
xchgq   CPTRSIZE(%rbp), %rax/* swap JMP table offset and ret addr */
shrq$4, %rax/* JMP table offset / JMP size = syscall num */
movq%rax, EH_LOCALS_GREG(REG_RAX)(%rbp) /* save syscall num */

--
usr/src/uts/i86pc/ml/syscall_asm.s:

- don't you need to update this file as well?  have you tested 32-bit
  kernels?

--
usr/src/uts/i86pc/ml/syscall_asm_amd64.s

- perhaps you could do the following renames:
BRAND_GET_RET_REG -> BRAND_URET_FROM_REG
BRAND_GET_RET_STACK -> BRAND_URET_FROM_INTR_STACK

- wrt this code:
cmpq$NSYSCALL, %rax /* is 0 <= syscall <= MAX?  */
jbe 0f  /* yes, syscall is OK */
xorq%rax, %rax  /* no, zero syscall number */

  it's duplicated in every brand callback right after
  CALLBACK_PROLOGUE().  why not make it part of CALLBACK_PROLOGUE?

  also, if the syscall num is > NSYSCALL, why not just jump to 9: and
  let the normal syscall path detect and return the error?

- it seems like there should be a macro for this rough block of code
  (which calculates the jmp table address):

GET_P_BRAND_DATA(%esp, 1, %edx);/* get p_brand_data ptr */
movlSPD_HANDLER(%edx), %edx /* get p_brand_data->spd_handler ptr */
shll$4, %eax
addl%eax, %edx  /* we'll return to our handler */

- prior to these changes V_URET_ADDR wasn't always set, so the different
  brand syscall callbacks would get the userland return address from
  their syscall specific locations (registers, interrupt stack, etc).  but
  now since V_URET_ADDR is always set, perhaps the callback handlers could
  be made more consistent by always getting the value from the stack (ie,
  via V_URET_ADDR)?

- so following up with the last comment (and getting more into potential
  comminization work) it seems to me like it might be benificial to move
  all the syscall mechanism specific handling code out of the actual brand
  callbacks and into BRAND_CALLBACK.  you've already started doing this by
  having BRAND_CALLBACK be aware of how to get the userland return
  address.  (prior to that it didn't have any dependancy upon the
  different syscall mechanisms, except when deciding which brand callback
  to invoke.)  continuing down that path we could move all the syscall
  specific handling code into BRAND_CALLBACK.  then each brand would only
  deliver a single callback which would take one parameter, the syscall
  number.  it would return one value, a userland return address.  then
  BRAND_CALLBACK could handle all the different syscall specific return
  paths.  this would also be benificial in the future since if a new
  syscall mechanism was introduced, we wouldn't have to update any actual
  brand callbacks, just BRAND_CALLBACK.  thoughts?
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones code review

[Jerry Jelinek]

I have an initial code review for the fix for bug:

6768950 panic[cpu1]/thread=ff084ce0b3e0: syscall_asm_amd64.s:480
lwp ff0756a8cdc0, pcb_rupdate != 0

There is a webrev at:

http://cr.opensolaris.org/~gjelinek/webrev.6768950/

The code changes in the sn1 and solaris10 brands are basically
identical.  I know there is a lot of common code there but I
didn't want to clutter up this bug fix with the unrelated changes
necessary to make the code common.  I'll be addressing that with
a separate fix.

My initial testing of these changes looks good but I still need
to run more extensive tests.

Thanks,
Jerry

___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

On Fri, Nov 27, 2009 at 02:47:37PM -0800, Frank Batschulat wrote:
> Hey Ed, addition to my previous posting as I just noticed something I've 
> totally
> forgotten about
>
> > afaik, determining the mount point should be pretty
> > strait forward. i was planning to get a list of all the shares
> > exported by the specified nfs server, and then do a strncmp() of all the
> > exported shares against the specified path.  the longest matching share name
> > is the mount path.
> >
> > for example.  if we have:
> > nfs://jurassic/a/b/c/d/file
> >
> > and jurassic is exporting:
> > jurassic:/a
> > jurassic:/a/b
> > jurassic:/a/b/c
> >
> > then our mount path with be:
> > /var/zones/nfsmount/jurassic/a/b/c
> >
> > and our encapsulated zvol will be accessible at:
> > /var/zones/nfsmount/jurassic/a/b/c/d/file
> >
> > afaik, this is acutally the only way that this could
> > be implemented.
>
> I just recognized (my bad) that the SO-URI of 
> 'nfs://[:port]/'.
> is actually compliant to the WebNFS URL syntax of  RFC 2224 / RFC  2054 / RFC 
> 2055
> ie.one could directly mount that.
>
> so it looks like we can avoid all the parsing handstands and directly mount
> such an URL aka. SO-URI if the server does support public file handles.
>
> I'll look into the URL mount schema and requirements in a bit more detail
> to discover potential problems laying around, generic support and 
> availability.
>
> ideally it should be something that should work with almost every NFS server 
> and
> presumably without much setup on the server side in order to serve our
> NFS implementation independend needs.
>

sounds good.

in the future i'll make sure to read all your follow up emails before
replying to your initial emails.  ;)

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

On Fri, Nov 27, 2009 at 09:12:33AM -0800, Frank Batschulat wrote:
> Hey Ed, I want to comment on the NFS aspects involed here,
>
> On Thu, May 21, 2009 at 3:55 AM, Edward Pilatowicz wrote:
> >
> >well, it all depends on what nfs shares are actually being exported.
>
> I definitively think we do want to abstain from that much programmatic
> attempts inside the Zones framework on making assumptions about what
> an NFS server does export, how the NFS servers exported namespace
> may look like and how the NFS client (who's running the Zone)
> handles those exports upon access as opposed to explicit mounting.
>
> It is merely okay for the NFS v2/v3 (and their helper) protocols world
> but it is not always adequate for the V4 protocol and all the work/features
> in V4 and V4.1 towards a unified, global namespace.
>
> I'll show why in the context of V4 on the examples you
> mentioned below.
>
> >if the nfs server has the following share(s) exported:
> >
> >nfsserver:/vol
> >
> >then you would have the following mount(s):
> >/var/zones/nfsmount/zone1/nfsserver/vol
> >/var/zones/nfsmount/zone2/nfsserver/vol
> >/var/zones/nfsmount/zone3/nfsserver/vol
> >
> >if the nfs server has the following share(s) exported:
> >
> >nfsserver:/vol/zones
> >
> >then you would have the following mount(s):
> >/var/zones/nfsmount/zone1/nfsserver/vol/zones
> >/var/zones/nfsmount/zone2/nfsserver/vol/zones
> >/var/zones/nfsmount/zone3/nfsserver/vol/zones
>
> in those 2 examples, we'd have to consider how the
> V4 server constructs it's pseudo namespace starting
> at the servers root, including what we call pseudo exports
> that build the bridge to the real exported share points
> at the server and how the V4 client may handle this.
>
> for instance, on the V4 server the export:
>
> /vol
>
> may (and probably will) have different ZFS datasets
> that host our zones underneath /vol eg.:
>
> /vol/zone1
> /vol/zone2
> /vol/zone3
>
> since they are separate ZFS datasets, we would
> cross file system boundaries while traversing from
> the exported servers root / over the share point /vol
> down to the (also presumably exported, otherwise it
> wouln't be usefull in our context anyways) share points
> zone1/zone2/zone3.
>
> We'll distingish between the different file systems
> based on the FSID attribute, if it changes, we'd cross
> server file system boundaries.
>
> With V2/V3 that'd stop us and the client can not travel into
> the new file system below the inital mount and a separate mount
> would have to be performed (unless we've explicitely mounted
> the entire path of course)
>
> However, with V4, the client has the (in our implementation)
> so called Mirror Mount feature. That allows the client
> to transparantly mount those new file systems on access below
> the starting share point /vol (provided they are shared as well)
> and make them immediately visible without requiring the user
> for perform any additional mounts.
>
> Those mirror mounts will be done automatically by our V4 client
> in the kernel as it detects it'd cross server side file system
> boundaries (based on the FSID) on any access other then
> VOP_LOOKUP() or VOP_GETATTR().
>
> Ie. if the global zone did already have mounted
>
> server:/vol
>
> an attempt by the zone utilities to access (as opposed to
> explicit mounting) of
>
> server:/vol/zone1
>
> will automatically mount server:/vol/zone1 into
> the clients namespace and you'd get on the client
> (nfsstat -m) 2 mounts:
>
> server:/vol (already existing regular mount)
> server:/vol/zone1 (the mirror mount done by the client)
>
> if we'd really perform a mount though, that'd just induce
> the mount of
>
> server:/vol/zone1
>
> into the clients namespace running the zone.
>
> With the advent of the upcomming NFS v4 Referrals support
> in the V4 server and V4 client, another 'automatism'
> in the client can possibly change our observation of
> the mounted server exports on the client running the zone.
>
> On the V4 server (that is hosting our zone image) the administrator
> might decide to relocate the export to a different server and
> then might establish a so called 'reparse point' (in essence
> a symlink containing special infos) that will redirect a client
> to a different server hosting this export.
>
> NB: other Vendors NFS servers might hand out referrals to
> NB2: the same feature will be supported by our CIFS client
>
> The V4 client can get a specific referral event (NFS4ERR_MOVED)
> on VOP_LOOKUP(), VOP_GETATTR() and during inital mount processing
> by observing the NFS4ERR_MOVED error and it'll fetch the new location
> information from the server via the 'fs_locations' attribute.
> Our client will then go off and automatically mount the file system
> from the different server it had been referred to from the inital
> server. Again like mirror mounts, this is done transparently for the
> user and inside the kernel.
>
> The minor but important quirk involved here as far as our
> observation from the Zone NFS cli

Re: [zones-discuss] zones on shared storage proposal

[Frank Batschulat]

Hey Ed, addition to my previous posting as I just noticed something I've totally
forgotten about

> afaik, determining the mount point should be pretty
> strait forward. i was planning to get a list of all the shares
> exported by the specified nfs server, and then do a strncmp() of all the
> exported shares against the specified path.  the longest matching share name
> is the mount path.
> 
> for example.  if we have:
>   nfs://jurassic/a/b/c/d/file
> 
> and jurassic is exporting:
>   jurassic:/a
>   jurassic:/a/b
>   jurassic:/a/b/c
> 
> then our mount path with be:
>   /var/zones/nfsmount/jurassic/a/b/c
> 
> and our encapsulated zvol will be accessible at:
>   /var/zones/nfsmount/jurassic/a/b/c/d/file
> 
> afaik, this is acutally the only way that this could
> be implemented.

I just recognized (my bad) that the SO-URI of 
'nfs://[:port]/'.
is actually compliant to the WebNFS URL syntax of  RFC 2224 / RFC  2054 / RFC 
2055
ie.one could directly mount that.

so it looks like we can avoid all the parsing handstands and directly mount 
such an URL aka. SO-URI if the server does support public file handles.

I'll look into the URL mount schema and requirements in a bit more detail
to discover potential problems laying around, generic support and availability.

ideally it should be something that should work with almost every NFS server and
presumably without much setup on the server side in order to serve our
NFS implementation independend needs.

cheers
frankB
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Frank Batschulat]

Hey Ed, I want to comment on the NFS aspects involed here,

On Thu, May 21, 2009 at 3:55 AM, Edward Pilatowicz wrote:  
>
>well, it all depends on what nfs shares are actually being exported.

I definitively think we do want to abstain from that much programmatic
attempts inside the Zones framework on making assumptions about what
an NFS server does export, how the NFS servers exported namespace
may look like and how the NFS client (who's running the Zone) 
handles those exports upon access as opposed to explicit mounting.

It is merely okay for the NFS v2/v3 (and their helper) protocols world
but it is not always adequate for the V4 protocol and all the work/features 
in V4 and V4.1 towards a unified, global namespace.

I'll show why in the context of V4 on the examples you
mentioned below.

>if the nfs server has the following share(s) exported:
>
>nfsserver:/vol
>
>then you would have the following mount(s):
>/var/zones/nfsmount/zone1/nfsserver/vol
>/var/zones/nfsmount/zone2/nfsserver/vol
>/var/zones/nfsmount/zone3/nfsserver/vol
>
>if the nfs server has the following share(s) exported:
>
>nfsserver:/vol/zones
>
>then you would have the following mount(s):
>/var/zones/nfsmount/zone1/nfsserver/vol/zones
>/var/zones/nfsmount/zone2/nfsserver/vol/zones
>/var/zones/nfsmount/zone3/nfsserver/vol/zones

in those 2 examples, we'd have to consider how the
V4 server constructs it's pseudo namespace starting
at the servers root, including what we call pseudo exports
that build the bridge to the real exported share points
at the server and how the V4 client may handle this.

for instance, on the V4 server the export:

/vol 

may (and probably will) have different ZFS datasets 
that host our zones underneath /vol eg.:

/vol/zone1
/vol/zone2
/vol/zone3

since they are separate ZFS datasets, we would
cross file system boundaries while traversing from
the exported servers root / over the share point /vol
down to the (also presumably exported, otherwise it
wouln't be usefull in our context anyways) share points
zone1/zone2/zone3.

We'll distingish between the different file systems
based on the FSID attribute, if it changes, we'd cross
server file system boundaries. 

With V2/V3 that'd stop us and the client can not travel into
the new file system below the inital mount and a separate mount
would have to be performed (unless we've explicitely mounted
the entire path of course)

However, with V4, the client has the (in our implementation)
so called Mirror Mount feature. That allows the client
to transparantly mount those new file systems on access below
the starting share point /vol (provided they are shared as well)
and make them immediately visible without requiring the user
for perform any additional mounts. 

Those mirror mounts will be done automatically by our V4 client
in the kernel as it detects it'd cross server side file system
boundaries (based on the FSID) on any access other then 
VOP_LOOKUP() or VOP_GETATTR().

Ie. if the global zone did already have mounted

server:/vol

an attempt by the zone utilities to access (as opposed to
explicit mounting) of

server:/vol/zone1

will automatically mount server:/vol/zone1 into
the clients namespace and you'd get on the client
(nfsstat -m) 2 mounts:

server:/vol (already existing regular mount)
server:/vol/zone1 (the mirror mount done by the client)

if we'd really perform a mount though, that'd just induce 
the mount of

server:/vol/zone1 

into the clients namespace running the zone.

With the advent of the upcomming NFS v4 Referrals support
in the V4 server and V4 client, another 'automatism' 
in the client can possibly change our observation of
the mounted server exports on the client running the zone.

On the V4 server (that is hosting our zone image) the administrator
might decide to relocate the export to a different server and
then might establish a so called 'reparse point' (in essence 
a symlink containing special infos) that will redirect a client
to a different server hosting this export. 

NB: other Vendors NFS servers might hand out referrals to
NB2: the same feature will be supported by our CIFS client

The V4 client can get a specific referral event (NFS4ERR_MOVED) 
on VOP_LOOKUP(), VOP_GETATTR() and during inital mount processing
by observing the NFS4ERR_MOVED error and it'll fetch the new location
information from the server via the 'fs_locations' attribute.
Our client will then go off and automatically mount the file system
from the different server it had been referred to from the inital
server. Again like mirror mounts, this is done transparently for the
user and inside the kernel.

The minor but important quirk involved here as far as our 
observation from the Zone NFS client is concered is that
we might get for our mount attempt (or access to)

server_A:/vol/zones1

a mount established instead for

server_B:/vol/zones1

It is planned to even provide our V2/V3 clients with
Referral support when taking to our NFS servers, although
the implementation

[zones-discuss] Zones/Projects bound to a pool

[Ketan]

How to i get to know that which project is bind to a pool like i have one pool 
called appPool to which i have 3 projects/zones bound how do i see that with 
command ?
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones

[Narayana Kadoor]

Alex George wrote:

Hi

I would like to know more abt zones, How zones work, what are types of zones 
,what is the best method to install zones.

Is zones and containers are both same.

Pls need help on above topic
  


The docs in this page should help.

http://opensolaris.org/os/community/zones/


-Narayana




Regards

Alexyy
  


<>___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones

[Alex George]

Hi

I would like to know more abt zones, How zones work, what are types of zones 
,what is the best method to install zones.

Is zones and containers are both same.

Pls need help on above topic


Regards

Alexyy
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[John Levon]

On Mon, Sep 21, 2009 at 10:23:21AM -0700, Edward Pilatowicz wrote:

> if we only support vdisks created via vdiskadm(1m), then we'll always
> have a directory and we can always use vdiskadm(1m) to sniff out if it's
> a valid vdisk and access it as such.
> 
> then for the implicit creation case we'll just support files containing
> a zpool.
> 
> sound good?

Yes.

> ok.  so how about we just generate an error if we need to create a file,
> and an explicity user id has not been specified, and root squashing is
> enabled.  (because under these conditions we'd generate a file owned by
> nobody.)

Sounds good to me.

regards
john
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

On Mon, Sep 21, 2009 at 04:25:30PM +0100, John Levon wrote:
> On Wed, Sep 16, 2009 at 09:33:11PM -0700, Edward Pilatowicz wrote:
>
> > there by implying that the vdisk path is a directory.  ok.  that's easy
>
> Right.
>
> > enough to detect.
>
> It's probably safer to directly use vdiskadm to sniff the directory, if
> you can.
>

sure.

> > > At import time, it's a combination of sniffing the type from the file,
> > > and some static checks on file name (namely .raw and .iso suffixes).
> >
> > well, as long as the suffixes above apply to directories and not to
> > files then i think we'd be ok.  if the extensions above will apply to
> > files then we have a problem.
>
> Once imported, the contents of the vdisk directory are private to vdisk.
> The name of the containing directory can be anything.
>
> That is, an import consists of taking the foo.raw file, and putting it
> into a directory along with an XML properties file.
>

so in this context, an import is one method for creating a vdisk.

> > so previously if the user specified:
> > file:///.../foo.raw
> > then we would create a zpool directly within the file, no label.
> >
> > and if the user specified:
> > vfile:///.../foo.raw
> >
> > then we would use lofi with the newly proposed -l option to access the
> > file, then we'd put a label on it (via the lofi device), and then create
> > a zpool in one of the partitions (and once again, zfs would access the
> > file through the lofi device).
> >
> > so in the two cases, how can we make the access mode determination
> > without having the seperate uri syntax?
>
> In the creation case, which I think we're talking about above, we create
> the vdisk directory (rather than direct file access, which vdiskadm
> can't do, even though vdisk itself can) and the container format is
> clear.
>
> If we want to configure access to a pre-existing raw file, I'm not sure
> we'd be doing the labelling ourselves. Perhaps I don't quite understand
> the use cases for what you're suggesting.
>

the two use cases above were creation use cases.

i think part of the confusion here is that in the raw case, i thought a
vdisk would just have a file, not a directory with an xml file and the
disk file.  (when i was using xvm that was the format of all the vdisks
i created.)

the other part of the confusion is that i was trying to support
automatic creation for raw vdisks.

if we only support vdisks created via vdiskadm(1m), then we'll always
have a directory and we can always use vdiskadm(1m) to sniff out if it's
a valid vdisk and access it as such.

then for the implicit creation case we'll just support files containing
a zpool.

sound good?

> > > How about defaulting to the owner of the containing directory? If it's
> > > root, you won't be able to write if you're root-squashed (or not root
> > > user) anyway.
> > >
> > > Failing that, I'd indeed prefer a different user, especially one that's
> > > configurable in terms of uid/gid.
> >
> > if a directory is owned by a non-root user and i want to create a file
> > there, i think it's a great idea to switch to the uid of the directory
> > owner todo my file operations.  i'll add that to the proposal.
> >
> > but, say i'm on a host that is not subject to root squashing and i need
> > to create a file on a share that is only writable by root.  in that
> > case, should i go ahead and create a file owned by root?  imho, no.
> > instead, i'd rather create the file as some other user.
>
> I don't agree that second-guessing the user's intentions when they've
> explicitly disabled root-squashing is a useful behaviour.
>
> > if the administrator then tries to migrate that zone to a host that is
> > subject to root squashing from the server, then i'd lose access to that
> > file.  eliminating all file accesses as root allows us to avoid
> > root-squashing and just help eliminate potential failure modes.
>
> Replacing it with a potentially more subtle issue: what's the zonenfs
> user ID, is it configured on the server, and is it unique and reserved
> across the organisation, and across all OSes?
>
> Having access fail with a clear message is an understandable failure
> mode, with a clear remedy: use a suitable uid /chosen by the
> administrator/. NFS users are surely comfortable and familiar with root
> squashing by now.
>
> Having a MySQL security hole allow access to all your virtual shared
> storage is a much more subtle problem (yes, I discovered despite my
> initial research that UID 60 is used by some Linux machines as mysqld).
>

ok.  so how about we just generate an error if we need to create a file,
and an explicity user id has not been specified, and root squashing is
enabled.  (because under these conditions we'd generate a file owned by
nobody.)

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[John Levon]

On Wed, Sep 16, 2009 at 09:33:11PM -0700, Edward Pilatowicz wrote:

> there by implying that the vdisk path is a directory.  ok.  that's easy

Right.

> enough to detect.

It's probably safer to directly use vdiskadm to sniff the directory, if
you can.

> > At import time, it's a combination of sniffing the type from the file,
> > and some static checks on file name (namely .raw and .iso suffixes).
> 
> well, as long as the suffixes above apply to directories and not to
> files then i think we'd be ok.  if the extensions above will apply to
> files then we have a problem.

Once imported, the contents of the vdisk directory are private to vdisk.
The name of the containing directory can be anything.

That is, an import consists of taking the foo.raw file, and putting it
into a directory along with an XML properties file.

> so previously if the user specified:
>   file:///.../foo.raw
> then we would create a zpool directly within the file, no label.
> 
> and if the user specified:
>   vfile:///.../foo.raw
> 
> then we would use lofi with the newly proposed -l option to access the
> file, then we'd put a label on it (via the lofi device), and then create
> a zpool in one of the partitions (and once again, zfs would access the
> file through the lofi device).
> 
> so in the two cases, how can we make the access mode determination
> without having the seperate uri syntax?

In the creation case, which I think we're talking about above, we create
the vdisk directory (rather than direct file access, which vdiskadm
can't do, even though vdisk itself can) and the container format is
clear.

If we want to configure access to a pre-existing raw file, I'm not sure
we'd be doing the labelling ourselves. Perhaps I don't quite understand
the use cases for what you're suggesting.

> > How about defaulting to the owner of the containing directory? If it's
> > root, you won't be able to write if you're root-squashed (or not root
> > user) anyway.
> >
> > Failing that, I'd indeed prefer a different user, especially one that's
> > configurable in terms of uid/gid.
> 
> if a directory is owned by a non-root user and i want to create a file
> there, i think it's a great idea to switch to the uid of the directory
> owner todo my file operations.  i'll add that to the proposal.
> 
> but, say i'm on a host that is not subject to root squashing and i need
> to create a file on a share that is only writable by root.  in that
> case, should i go ahead and create a file owned by root?  imho, no.
> instead, i'd rather create the file as some other user.

I don't agree that second-guessing the user's intentions when they've
explicitly disabled root-squashing is a useful behaviour.

> if the administrator then tries to migrate that zone to a host that is
> subject to root squashing from the server, then i'd lose access to that
> file.  eliminating all file accesses as root allows us to avoid
> root-squashing and just help eliminate potential failure modes.

Replacing it with a potentially more subtle issue: what's the zonenfs
user ID, is it configured on the server, and is it unique and reserved
across the organisation, and across all OSes?

Having access fail with a clear message is an understandable failure
mode, with a clear remedy: use a suitable uid /chosen by the
administrator/. NFS users are surely comfortable and familiar with root
squashing by now.

Having a MySQL security hole allow access to all your virtual shared
storage is a much more subtle problem (yes, I discovered despite my
initial research that UID 60 is used by some Linux machines as mysqld).

regards
john
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

On Thu, Sep 17, 2009 at 01:13:53AM +0100, John Levon wrote:
> On Wed, Sep 16, 2009 at 04:34:06PM -0700, Edward Pilatowicz wrote:
>
> > thanks for taking the time to look at this and sorry for the delay in
> > replying.
>
> Compared to /my/ delay...
>
> > > That is, I think vdisks should just use path:/// and nfs:// not have
> > > their own special schemes.
> >
> > this is easy enough to change.
> >
> > but would you mind explaning what is the detection techniques are for
> > the different vdisk formats?  are they files with well known extensions?
> > all directories with well known extensions?  directories with certain
> > contents?
>
> Well, the format comes from the XML property file present in the vdisk.

there by implying that the vdisk path is a directory.  ok.  that's easy
enough to detect.

> At import time, it's a combination of sniffing the type from the file,
> and some static checks on file name (namely .raw and .iso suffixes).
>

well, as long as the suffixes above apply to directories and not to
files then i think we'd be ok.  if the extensions above will apply to
files then we have a problem.

in the xvm world, you don't have any issues with accessing the files
above since you know that every object exported to a domain contains
a virtual disk, and there for contains a label.

but with zones this isn't the case.  in my proposal there are two access
modes for files.  raw file mode, where a zpool is created directly
inside a file.  and vdisk mode, where we first create a label within the
device and then create a zpool inside one of the partitions.

so previously if the user specified:
file:///.../foo.raw
then we would create a zpool directly within the file, no label.

and if the user specified:
vfile:///.../foo.raw

then we would use lofi with the newly proposed -l option to access the
file, then we'd put a label on it (via the lofi device), and then create
a zpool in one of the partitions (and once again, zfs would access the
file through the lofi device).

so in the two cases, how can we make the access mode determination
without having the seperate uri syntax?

> > > Hmmph. I really don't want the 'xvm' user to be exposed any more than it
> > > is. It was always intended as an internal detail of the Xen least
> > > privilege implementation. Encoding it as the official UID to access
> > > shared storage seems very problematic to me. Not least, it means xend,
> > > qemu-dm, etc. can suddenly write to all the shared storage even if it's
> > > nothing to do with Xen.
> > >
> > > Please make this be a 'user' option that the user can specify (with a
> > > default of root or whatever). I'm pretty sure we'd agreed on that last
> > > time we talked about this?
> >
> > i have no objections to adding a 'user' option.
> >
> > but i'd still like to avoid defaulting to root and being subject to
> > root-squashing.
>
> How about defaulting to the owner of the containing directory? If it's
> root, you won't be able to write if you're root-squashed (or not root
> user) anyway.
>
> Failing that, I'd indeed prefer a different user, especially one that's
> configurable in terms of uid/gid.
>

if a directory is owned by a non-root user and i want to create a file
there, i think it's a great idea to switch to the uid of the directory
owner todo my file operations.  i'll add that to the proposal.

but, say i'm on a host that is not subject to root squashing and i need
to create a file on a share that is only writable by root.  in that
case, should i go ahead and create a file owned by root?  imho, no.
instead, i'd rather create the file as some other user.  why?  because
if the administrator then tries to migrate that zone to a host that is
subject to root squashing from the server, then i'd lose access to that
file.  eliminating all file accesses as root allows us to avoid
root-squashing and just help eliminate potential failure modes.

this would be my argument for adding a new non-root user that could be
used as a fallback for remote file access in cases that would otherwise
default to the root user.

> > there wouldn't really be any problem which changing this from a lofi
> > service to be a vdisk service.  both services would do the same thing.
> > each would have a daemon that keeps track of the current vdisks on the
> > system and ensures that a vdisk utility remains running for each one.
> >
> > if you want smf to manage the vdisk utility processes directly, then
> > we'll have to create a new smf service each time a vdisk is accessed
> > and destroy that smf service each time the vdisk is taken down.
>
> Ah, right, I see now. Yes, out of the two options, I'd prefer each vdisk
> to have its own fault container (SMF service). You avoid the need for
> another hierarchy of fault management process (lofid), and get the
> benefit of enhanced visibility:
>
> # svcs
> ...
> online 15:33:19 svc:/system/lofi:dsk0
> online 15:33:19 svc:/system/lofi:dsk1
> maintenance15:33:19 svc:

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

hey illya,

thanks for reviewing this and sorry for the delay in replying.
my comments are inline below.

i've also attached a document that contains some of the design doc
sections i've revised based of your (and john's) feedback.  since the
document is large, i've only included sections that i've modified, and
i've included changebars in the right most column.

ed

On Mon, Sep 07, 2009 at 11:07:41AM +0300, Illya Kysil wrote:
> Hi Edward,
>
> See comments and questions below inline:
>
> 1. Section C.0
> >  ... That said, nothing in this proposal should not prevent us from adding 
> > support for...
> That "not" before "prevent" is superfluous.
>

done.

> 2. Section C.1.i
> How many instances of "rootzpool" and "zpool" resources is permitted?
> IMO "zero or one" for "rootzpool" and "zero or more" for "zpool" is enough.
>

correct.  i've updated the proposal to mention this.

> 3. Section C.1.iii
> > The newly created or imported root zpool will be named after the zone to
> > which it is associated, with the assigned name being "_rpool".
> What if zone name is changed later? Will the name of the zpool change as well?
> It is not clear how the association with the zpool will be maintained
> if its name will not change.
>

the pool name must be kept in sync with the zone name.

unfortunatly this presents a problem because currently zone renaming is
done from zonecfg and it can be done when a zone is installed, but since
zone zpools are imported at install time, we need to disallow re-naming
of installed zones.  hence, i've added the following new section:

C.1.x Zonecfg(1m) misc changes


> > This zpool will then be mounted on the zones zonepath and then the
> > install process will continue normally[07].
> >
> > XXX: use altroot at zpool creation or just manually mount zpool?
> What will happen on zoneadm move? IMO the zones framework have to
> remount the zpool in the new location.
>

that's correct.  i've added another new section:
C.1.ix Zoneadm(1m) move


> > If the user has specified a "zpool" resource, then the zones framework
> > will configure, initialize, and/or import it in a similar manner to a
> > zpool specified by the "rootzpool" resource.  The key differences are
> > that the name of the newly created or imported zpool will be
> > "_".
> What if zone name or zpool resource name is changed later? Will the
> name of the zpool change as well?
> It is not clear how the association with the zpool will be maintained
> if its name will not change.
>
> 4. Section C.1.viii

once again, the zpool name will need to be kept in sync with the
zonename.

> > Since zoneadm(1m) clone will be enhanced to support cloning between
> > encapsulated root zones and un-encapsulated root zones, zoneadm(1m)
> > clone will be documented as the recommended migration mechanism for
> > users who which to migrate existing zones from one format to another.
> "users who which" -> "users who wish"
>
> 5. Section C.5
> > For RAS purposes,...
> What is RAS?
>

a TLA.  :)

it stand for Reliability, Availability, and Serviceability.

> > Here's some examples of how this lofi functionality could be used
> > (outside of the zone framework).  If there are no lofi devices on
> > the system, and an admin runs the following command:
> > lofiadm -a -l /export/xvm/vm1.disk
> >
> > they would end up with the following device:
> > /dev/lofi/dsk0/p#   - for # == 0 - 4
> > /dev/lofi/dsk0/s#   - for # == 0 - 15
> > /dev/rlofi/dsk0/p#  - for # == 0 - 4
> > /dev/rlofi/dsk0/s#  - for # == 0 - 15
> >
> > If there are no lofi devices on the system, and an admin runs the
> > following command:
> > lofiadm -a -v /export/xvm/vm1.vmdk
> >
> > they would end up with the following device:
> > /dev/lofi/dsk0/p#   - for # == 0 - 4
> > /dev/lofi/dsk0/s#   - for # == 0 - 15
> > /dev/rlofi/dsk0/p#  - for # == 0 - 4
> > /dev/rlofi/dsk0/s#  - for # == 0 - 15
> The list of devices is the same in both examples. What's the difference?
>

the difference is in the invocation, not in the resulte.
i've re-worded this section to make things more clear.

> 6. Section D
> > D. INTERFACES
> >
> > Zonecfg(1m):
> > rootzpool   committed, resource
> > src committed, resource property
> > install-sizecommitted, resource property
> What is the meaning of "committed" here?
>

this is arc terminology.  basically, i'm presenting a new user interface
here and i'm saying that it isn't going to change incompatibaly in the
future.

> >Zones misc:
> > /var/zones/nfsmount///
> > project private, nfs mount point
> The mount point is different from what is described in section C.1.iii
> (see additional comment above):

oops.  fixed.

> > If an so-uri points to an explicit nfs server, the z

Re: [zones-discuss] zones on shared storage proposal

[John Levon]

On Wed, Sep 16, 2009 at 04:34:06PM -0700, Edward Pilatowicz wrote:

> thanks for taking the time to look at this and sorry for the delay in
> replying.

Compared to /my/ delay...

> > That is, I think vdisks should just use path:/// and nfs:// not have
> > their own special schemes.
> 
> this is easy enough to change.
> 
> but would you mind explaning what is the detection techniques are for
> the different vdisk formats?  are they files with well known extensions?
> all directories with well known extensions?  directories with certain
> contents?

Well, the format comes from the XML property file present in the vdisk.
At import time, it's a combination of sniffing the type from the file,
and some static checks on file name (namely .raw and .iso suffixes).

> > Hmmph. I really don't want the 'xvm' user to be exposed any more than it
> > is. It was always intended as an internal detail of the Xen least
> > privilege implementation. Encoding it as the official UID to access
> > shared storage seems very problematic to me. Not least, it means xend,
> > qemu-dm, etc. can suddenly write to all the shared storage even if it's
> > nothing to do with Xen.
> >
> > Please make this be a 'user' option that the user can specify (with a
> > default of root or whatever). I'm pretty sure we'd agreed on that last
> > time we talked about this?
> 
> i have no objections to adding a 'user' option.
> 
> but i'd still like to avoid defaulting to root and being subject to
> root-squashing.

How about defaulting to the owner of the containing directory? If it's
root, you won't be able to write if you're root-squashed (or not root
user) anyway.

Failing that, I'd indeed prefer a different user, especially one that's
configurable in terms of uid/gid.

> there wouldn't really be any problem which changing this from a lofi
> service to be a vdisk service.  both services would do the same thing.
> each would have a daemon that keeps track of the current vdisks on the
> system and ensures that a vdisk utility remains running for each one.
> 
> if you want smf to manage the vdisk utility processes directly, then
> we'll have to create a new smf service each time a vdisk is accessed
> and destroy that smf service each time the vdisk is taken down.

Ah, right, I see now. Yes, out of the two options, I'd prefer each vdisk
to have its own fault container (SMF service). You avoid the need for
another hierarchy of fault management process (lofid), and get the
benefit of enhanced visibility:

# svcs
...
online 15:33:19 svc:/system/lofi:dsk0
online 15:33:19 svc:/system/lofi:dsk1
maintenance15:33:19 svc:/system/lofi:dsk2

Heck, if we ever do represent zones or domains as SMF instances, we
could even build dependencies on the lofi instances. (Presuming we
somehow rewhack xVM to start a service instead of an isolated vdisk
process.)

regards
john
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

thanks for taking the time to look at this and sorry for the delay in
replying.  my comments are line below.
ed

On Sat, Sep 05, 2009 at 11:13:07PM +0100, John Levon wrote:
> On Thu, May 21, 2009 at 04:55:15PM +0800, Edward Pilatowicz wrote:
>
> > File storage objects:
> >
> > path:///
> > nfs://[:port]/
> >
> > Vdisk storage objects:
> >
> > vpath:///
> > vnfs://[:port]/
>
> This makes me uncomfortable. The fact it's a vdisk is derivable except
> in one case: creation. And when creating, we will already want some way
> to specify the underlying format of the vdisk, so we could easily hook
> the "make it a vdisk" option there.
>
> That is, I think vdisks should just use path:/// and nfs:// not have
> their own special schemes.
>

this is easy enough to change.

but would you mind explaning what is the detection techniques are for
the different vdisk formats?  are they files with well known extensions?
all directories with well known extensions?  directories with certain
contents?

> > In order to avoid root squashing, or requiring users to setup special
> > configurations on their NFS servers, whenever the zone framework
> > attempts to create a storage object file or vdisk, it will temporarily
> > change it's uid and gid to the "xvm" user and group, and then create the
> > file with 0600 access permissions.
>
> Hmmph. I really don't want the 'xvm' user to be exposed any more than it
> is. It was always intended as an internal detail of the Xen least
> privilege implementation. Encoding it as the official UID to access
> shared storage seems very problematic to me. Not least, it means xend,
> qemu-dm, etc. can suddenly write to all the shared storage even if it's
> nothing to do with Xen.
>
> Please make this be a 'user' option that the user can specify (with a
> default of root or whatever). I'm pretty sure we'd agreed on that last
> time we talked about this?
>

i have no objections to adding a 'user' option.

but i'd still like to avoid defaulting to root and being subject to
root-squashing.  the xvm user seems like a good way to do this.  but if
you don't like this then i could always introduce a new user just for
this purpose, say the zonesnfs user.

> > For RAS purposes, we will need to ensure that this vdisk utility is
> > always running.  Hence we will introduce a new lofi smf service
> > svc:/system/lofi:default, which will start a new /usr/lib/lofi/lofid
> > daemon, which will manage the starting, stopping, monitoring, and
> > possible re-start of the vdisk utility.  Re-starts of vdisk utility
>
> I'm confused by this bit: isn't startd what manages "starting, stopping,
> monitoring, and possible re-start" of daemons? Why isn't this
> svc:/system/vdisk:default ? What is lofid actually doing?
>

well, as specified in the proposal, the administrative interface for
accessing vdisks is via lofi:

---8<---
Here's some examples of how this lofi functionality could be used
(outside of the zone framework).  If there are no lofi devices on
the system, and an admin runs the following command:
lofiadm -a -l /export/xvm/vm1.disk

they would end up with the following device:
/dev/lofi/dsk0/p#   - for # == 0 - 4
/dev/lofi/dsk0/s#   - for # == 0 - 15
/dev/rlofi/dsk0/p#  - for # == 0 - 4
/dev/rlofi/dsk0/s#  - for # == 0 - 15
---8<---

so in this case, the lofi service would be started, and it would manage
starting and stopping a vdisk utility processes that services the
backend for this lofi device.

i originally made this a lofi service because i know that eventually it
would also be nice if we could persist lofi configuration across
reboots, and a lofi smf service would be a good way todo this.

there wouldn't really be any problem which changing this from a lofi
service to be a vdisk service.  both services would do the same thing.
each would have a daemon that keeps track of the current vdisks on the
system and ensures that a vdisk utility remains running for each one.

if you want smf to manage the vdisk utility processes directly, then
we'll have to create a new smf service each time a vdisk is accessed
and destroy that smf service each time the vdisk is taken down.

i don't really have a strong opinion on how this gets managed, so if you
have a preference then let me know and i can update the proposal.
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones patching issues using attach -u

[Mike Gerdts]

On Tue, Sep 15, 2009 at 2:02 PM, Jerry Jelinek  wrote:
> Gael wrote:
>>
>> Hello
>>
>> I have been experimenting a few ways to speed up patching a bunch of
>> machines running whole zones (parallel patching, zoneadm attach -u).
>> I have encountered one issue with the attach -u way... Before initiating a
>> case with sun, I was wondering if it was a well known issue...
>>
>> The GZ is initially running Solaris 10 U6 with kernel patch 13-08 (and
>> other patches from the same period).
>>
>> I start by applying 119254-70, 119313-28 and 12-05 while the machine
>> is
>> in multiuser mode, then I shutdown and detach the zones.
>> I bring back the machine in single user mode and apply a collection of
>> about
>> 190 patches (smpatch analyze output from a few days ago) which brings the
>> machine at the kernel version 141414-10.
>>
>> The patching appears to go fine for the GZ
>>
>> apss8003:/var/sadm/patch #pkginfo -p
>> apss8003:/var/sadm/patch #
>>
>> But when zoneadm attaching -u the zones, pkginfo reports multiple
>> partially
>> failing packages adds ...
>>
>> apss8003:/var/sadm/patch #zlogin test pkginfo -p
>> system      SUNWcsr                         Core Solaris, (Root)
>> system      SUNWgssc                        GSSAPI CONFIG V2
>> system      SUNWkrbr                        Kerberos version 5 support
>> (Root)
>> system      SUNWntpr                        NTP, (Root)
>> system      SUNWppror                       PatchPro core functionality
>> (Root)
>> system      SUNWsacom                       Solstice Enterprise Agents
>> 1.0.3
>> files for root file system
>>
>> # cat /zones/test/root//var/sadm/system/logs/update_log | egrep
>> "partially|corrupt|pathname does not exist|"
>>
>> = SUNWcsr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWgssc 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWkrbr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWntpr 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> = SUNWppror 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed
>>
>> = SUNWsacom 
>> pkgadd: ERROR: source path
>>
>> 
>> is corrupt
>>    pathname does not exist
>> Installation of  on zone  partially failed.
>>
>> If creating a new zone after the patching, there is no partial packages in
>> that newly build zone.
>>
>> The patch list being a little bit lengthy, I can send it privately when
>> asked...
>
> This is bug:
>
> 6857294 zoneadm attach leads to partially installed packages
>
> I believe a T patch might be available for the S10 SVr4 packaging code
> if you need it, but I see that the fix has not yet been integrated
> into the nv SVr4 packaging code.  It is scheduled for b124.
>
> Jerry

I stumbled across this a while back with SUNWservicetagr.  My workaround was:

d=/var/sadm/pkg/SUNWservicetagr/save/pspool/SUNWservicetagr/reloc/var/svc/manifest/network
mkdir -p $d
cp /var/svc/manifest/network/st*.xml $d

In the last week or so the CR (6833642) logged due to the case that I
opened related to this was changed to "cause known" and is now
"related to" 6857294.

-- 
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Zones patching issues using attach -u

[Jerry Jelinek]

Gael wrote:

Hello

I have been experimenting a few ways to speed up patching a bunch of
machines running whole zones (parallel patching, zoneadm attach -u).
I have encountered one issue with the attach -u way... Before initiating a
case with sun, I was wondering if it was a well known issue...

The GZ is initially running Solaris 10 U6 with kernel patch 13-08 (and
other patches from the same period).

I start by applying 119254-70, 119313-28 and 12-05 while the machine is
in multiuser mode, then I shutdown and detach the zones.
I bring back the machine in single user mode and apply a collection of about
190 patches (smpatch analyze output from a few days ago) which brings the
machine at the kernel version 141414-10.

The patching appears to go fine for the GZ

apss8003:/var/sadm/patch #pkginfo -p
apss8003:/var/sadm/patch #

But when zoneadm attaching -u the zones, pkginfo reports multiple partially
failing packages adds ...

apss8003:/var/sadm/patch #zlogin test pkginfo -p
system  SUNWcsr Core Solaris, (Root)
system  SUNWgsscGSSAPI CONFIG V2
system  SUNWkrbrKerberos version 5 support
(Root)
system  SUNWntprNTP, (Root)
system  SUNWppror   PatchPro core functionality
(Root)
system  SUNWsacom   Solstice Enterprise Agents 1.0.3
files for root file system

# cat /zones/test/root//var/sadm/system/logs/update_log | egrep
"partially|corrupt|pathname does not exist|"

= SUNWcsr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWgssc 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWkrbr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWntpr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWppror 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed

= SUNWsacom 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

If creating a new zone after the patching, there is no partial packages in
that newly build zone.

The patch list being a little bit lengthy, I can send it privately when
asked...


This is bug:

6857294 zoneadm attach leads to partially installed packages

I believe a T patch might be available for the S10 SVr4 packaging code
if you need it, but I see that the fix has not yet been integrated
into the nv SVr4 packaging code.  It is scheduled for b124.

Jerry
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Zones patching issues using attach -u

[Gael]

Hello

I have been experimenting a few ways to speed up patching a bunch of
machines running whole zones (parallel patching, zoneadm attach -u).
I have encountered one issue with the attach -u way... Before initiating a
case with sun, I was wondering if it was a well known issue...

The GZ is initially running Solaris 10 U6 with kernel patch 13-08 (and
other patches from the same period).

I start by applying 119254-70, 119313-28 and 12-05 while the machine is
in multiuser mode, then I shutdown and detach the zones.
I bring back the machine in single user mode and apply a collection of about
190 patches (smpatch analyze output from a few days ago) which brings the
machine at the kernel version 141414-10.

The patching appears to go fine for the GZ

apss8003:/var/sadm/patch #pkginfo -p
apss8003:/var/sadm/patch #

But when zoneadm attaching -u the zones, pkginfo reports multiple partially
failing packages adds ...

apss8003:/var/sadm/patch #zlogin test pkginfo -p
system  SUNWcsr Core Solaris, (Root)
system  SUNWgsscGSSAPI CONFIG V2
system  SUNWkrbrKerberos version 5 support
(Root)
system  SUNWntprNTP, (Root)
system  SUNWppror   PatchPro core functionality
(Root)
system  SUNWsacom   Solstice Enterprise Agents 1.0.3
files for root file system

# cat /zones/test/root//var/sadm/system/logs/update_log | egrep
"partially|corrupt|pathname does not exist|"

= SUNWcsr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWgssc 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWkrbr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWntpr 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

= SUNWppror 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed

= SUNWsacom 
pkgadd: ERROR: source path

is corrupt
pathname does not exist
Installation of  on zone  partially failed.

If creating a new zone after the patching, there is no partial packages in
that newly build zone.

The patch list being a little bit lengthy, I can send it privately when
asked...

Regards
-- 
Gael Martinez
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Illya Kysil]

Hi Edward,

See comments and questions below inline:

1. Section C.0
>  ... That said, nothing in this proposal should not prevent us from adding 
> support for...
That "not" before "prevent" is superfluous.

2. Section C.1.i
How many instances of "rootzpool" and "zpool" resources is permitted?
IMO "zero or one" for "rootzpool" and "zero or more" for "zpool" is enough.

3. Section C.1.iii
> The newly created or imported root zpool will be named after the zone to
> which it is associated, with the assigned name being "_rpool".
What if zone name is changed later? Will the name of the zpool change as well?
It is not clear how the association with the zpool will be maintained
if its name will not change.

> This zpool will then be mounted on the zones zonepath and then the
> install process will continue normally[07].
>
> XXX: use altroot at zpool creation or just manually mount zpool?
What will happen on zoneadm move? IMO the zones framework have to
remount the zpool in the new location.

> If the user has specified a "zpool" resource, then the zones framework
> will configure, initialize, and/or import it in a similar manner to a
> zpool specified by the "rootzpool" resource.  The key differences are
> that the name of the newly created or imported zpool will be
> "_".
What if zone name or zpool resource name is changed later? Will the
name of the zpool change as well?
It is not clear how the association with the zpool will be maintained
if its name will not change.

4. Section C.1.viii
> Since zoneadm(1m) clone will be enhanced to support cloning between
> encapsulated root zones and un-encapsulated root zones, zoneadm(1m)
> clone will be documented as the recommended migration mechanism for
> users who which to migrate existing zones from one format to another.
"users who which" -> "users who wish"

5. Section C.5
> For RAS purposes,...
What is RAS?

> Here's some examples of how this lofi functionality could be used
> (outside of the zone framework).  If there are no lofi devices on
> the system, and an admin runs the following command:
>   lofiadm -a -l /export/xvm/vm1.disk
>
> they would end up with the following device:
>   /dev/lofi/dsk0/p#   - for # == 0 - 4
>   /dev/lofi/dsk0/s#   - for # == 0 - 15
>   /dev/rlofi/dsk0/p#  - for # == 0 - 4
>   /dev/rlofi/dsk0/s#  - for # == 0 - 15
>
> If there are no lofi devices on the system, and an admin runs the
> following command:
>   lofiadm -a -v /export/xvm/vm1.vmdk
>
> they would end up with the following device:
>   /dev/lofi/dsk0/p#   - for # == 0 - 4
>   /dev/lofi/dsk0/s#   - for # == 0 - 15
>   /dev/rlofi/dsk0/p#  - for # == 0 - 4
>   /dev/rlofi/dsk0/s#  - for # == 0 - 15
The list of devices is the same in both examples. What's the difference?

6. Section D
> D. INTERFACES
>
> Zonecfg(1m):
>   rootzpool   committed, resource
>   src committed, resource property
>   install-sizecommitted, resource property
What is the meaning of "committed" here?

>Zones misc:
>   /var/zones/nfsmount///
>   project private, nfs mount point
The mount point is different from what is described in section C.1.iii
(see additional comment above):
> If an so-uri points to an explicit nfs server, the zones framework will
> need to mount the nfs filesystem containing storage object.  The nfs
> server share containing the specified object will be mounted at:
>   /var/zones/nfsmount//

7. What will happen to storage objects on "zonecfg delete"?

-- 
Illya Kysil
--
"EASY" is the word you use to describe other people's job.
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[John Levon]

On Thu, May 21, 2009 at 04:55:15PM +0800, Edward Pilatowicz wrote:

> File storage objects:
> 
> path:///
> nfs://[:port]/
> 
> Vdisk storage objects:
> 
> vpath:///
> vnfs://[:port]/

This makes me uncomfortable. The fact it's a vdisk is derivable except
in one case: creation. And when creating, we will already want some way
to specify the underlying format of the vdisk, so we could easily hook
the "make it a vdisk" option there.

That is, I think vdisks should just use path:/// and nfs:// not have
their own special schemes.

> In order to avoid root squashing, or requiring users to setup special
> configurations on their NFS servers, whenever the zone framework
> attempts to create a storage object file or vdisk, it will temporarily
> change it's uid and gid to the "xvm" user and group, and then create the
> file with 0600 access permissions.

Hmmph. I really don't want the 'xvm' user to be exposed any more than it
is. It was always intended as an internal detail of the Xen least
privilege implementation. Encoding it as the official UID to access
shared storage seems very problematic to me. Not least, it means xend,
qemu-dm, etc. can suddenly write to all the shared storage even if it's
nothing to do with Xen.

Please make this be a 'user' option that the user can specify (with a
default of root or whatever). I'm pretty sure we'd agreed on that last
time we talked about this?

> Additionally, whenever the zones framework attempts to access an storage
> object file or vdisk it will temporarily switch its uid and gid to match
> the owner and group of the file/vdisk, ensure that the file is readable
> and writeable by it's owner (updating the file/vdisk permissions if
> necessary), and finally setup the file/vdisk for access via a zpool
> import or lofiadm -a.  This should will allow the zones framework to
> access storage object files/vdisks that we created by any user,
> regardless of their ownership, simplifying file ownership and management
> issues for administrators.

+1 on this bit, for sure.

> For RAS purposes, we will need to ensure that this vdisk utility is
> always running.  Hence we will introduce a new lofi smf service
> svc:/system/lofi:default, which will start a new /usr/lib/lofi/lofid
> daemon, which will manage the starting, stopping, monitoring, and
> possible re-start of the vdisk utility.  Re-starts of vdisk utility

I'm confused by this bit: isn't startd what manages "starting, stopping,
monitoring, and possible re-start" of daemons? Why isn't this
svc:/system/vdisk:default ? What is lofid actually doing?

regards
john
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

On Sat, Sep 05, 2009 at 09:02:34AM +0300, Illya Kysil wrote:
> Hi Edward,
>
> On Sat, Sep 5, 2009 at 03:03, Edward
> Pilatowicz wrote:
> > i posted the latest version to our aliases in may after i incorporated
> > mike's feedback, but digging throught the archives i couldn't find any
> > decent readable copy.  hence i've gone ahead and posted the latest
> > version to zones community site here:
> >
> > http://www.opensolaris.org/os/community/zones/zones_design_docs/zoss/
>
> I've found readable proposal text attached here -
> http://www.opensolaris.org/jive/thread.jspa?threadID=103319&tstart=0

seems i'm forum searching foo isn't all that good.

> It is titled "Zones on shared storage (v1.1)" rather than "Zones on
> shared storage, v0.9"
> which you posted. It looks like it is the most recent version anybody has. :)
>

oops.  so first off, sorry about posting an older version.  that said,
since i wrote the doc i do really have the latest version.  ;)  i've
posted v1.2 (which incorporates mikes feedback) at the url above.

ed
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Illya Kysil]

Hi Edward,

On Sat, Sep 5, 2009 at 03:03, Edward
Pilatowicz wrote:
> i posted the latest version to our aliases in may after i incorporated
> mike's feedback, but digging throught the archives i couldn't find any
> decent readable copy.  hence i've gone ahead and posted the latest
> version to zones community site here:
>
> http://www.opensolaris.org/os/community/zones/zones_design_docs/zoss/

I've found readable proposal text attached here -
http://www.opensolaris.org/jive/thread.jspa?threadID=103319&tstart=0
It is titled "Zones on shared storage (v1.1)" rather than "Zones on
shared storage, v0.9"
which you posted. It looks like it is the most recent version anybody has. :)

-- 
Illya Kysil
--
"EASY" is the word you use to describe other people's job.
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

hey illya,

i posted the latest version to our aliases in may after i incorporated
mike's feedback, but digging throught the archives i couldn't find any
decent readable copy.  hence i've gone ahead and posted the latest
version to zones community site here:

http://www.opensolaris.org/os/community/zones/zones_design_docs/zoss/

fyi, the work described in the document is currently unfunded (ie, no
one is actually working on it), but if you have comments/feedback on the
design then please let me know and i can try to address it.

ed

On Sat, Sep 05, 2009 at 01:37:19AM +0300, Illya Kysil wrote:
> Hello Edward and Mike,
> 
> I've just discovered your thread from May 2009.
> Do you have any updates on the subject?
> I would like to read the latest version of the proposal.
> Where can I find it?
> 
> -- 
> Illya Kysil
> --
> "EASY" is the word you use to describe other people's job.
> ___
> zones-discuss mailing list
> zones-discuss@opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones on shared storage proposal

[Illya Kysil]

Hello Edward and Mike,

I've just discovered your thread from May 2009.
Do you have any updates on the subject?
I would like to read the latest version of the proposal.
Where can I find it?

-- 
Illya Kysil
--
"EASY" is the word you use to describe other people's job.
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] zones issues with SSL publisher (to get Sun support or not)

[Anil]

I've been playing around with how zones are integrated in a system running a 
/support (default publisher) version of OpenSolaris 2009.06. It seems when a 
new zone is installed, the SSL keys are also copied over to the zone (at least 
that's what the zone install messages seem to show - sorry didn't get a chance 
to actual verify what keys are copied over etc...). 

This is a bad thing, if we are providing the zone to a user/customer who does 
not have root access to the global zone. They would have access to the keys, 
free to distribute and use.

What is a solution to this?

If I set the default publisher of the global zone to be /release right before 
installing zone, then the zone and the global zone bits are different. But this 
does prevent the keys from being copied. Once installed, I could set it back to 
/support.

All of that sounds a bit of a hack and would rather not do that in the hopes of 
keeping the zones and the global zone in sync with the same bits.

But then how can I get Sun support (patches) and also prevent this problem? If 
there is no good solution at this point, I guess I will just have to stick with 
/release for now.
-- 
This message posted from opensolaris.org
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] zones on shared storage proposal

[Edward Pilatowicz]

comments inline below.
ed

On Fri, May 22, 2009 at 11:26:06AM -0500, Mike Gerdts wrote:
> On Fri, May 22, 2009 at 1:57 AM, Edward Pilatowicz
>  wrote:
> >
> > i've attached an updated version of the proposal (v1.1) which addresses
> > your feedback.  (i've also attached a second copy of the new proposal
> > that includes change bars, in case you want to review the updates.)
>
> As I was reading through it again, I fixed a few picky things (mostly
> spelling) that don't change the meaning.  I don't think that I "fixed"
> anything that was already right in British English.
>
> diff attached.
>

i've merged your fixes in.  thanks.

> > On Thu, May 21, 2009 at 11:59:22AM -0500, Mike Gerdts wrote:
> >> On Thu, May 21, 2009 at 3:55 AM, Edward Pilatowicz
> >>  wrote:
> >
> > nice catch.
> >
> > in early versions of my proposal, the nfs:// uri i was planning to
> > support allowed for the specification of mount options.  this required
> > allowing for per-zone nfs mounts with potentially different mount
> > options.  since then i've simplified things (realizing that most people
> > really don't need or want to specify mount options) and i've switched to
> > using the the nfs uri defined in rfc 2224.  this means we can do away
> > with the '' path component as you suggest.
>
> That was actually something I thought about after the fact.  When I've
> been involved in performance problems in the past, being able to tune
> mount options (e.g. protocol versions, block sizes, caching behavior,
> etc.) has been important.
>

yeah.  so the idea is to keep is simple for the initial functionality.
i figure that i'll evaluate different options and provide the best
defaults possible.  if customer requests come in for supporting
different options, well, first they can easily work around the issue by
using autofs + path:/// (and if the autofs config is in nis/ldap, then
migration will still work).  then we can just come up with a new uri
spec that allows the user to specify mount options.  the non-obvious and
unfortunate part of having a uri that allows for the specification of
mount options is that this we'll probably have to require that the user
percent-encode certain chars in the uri.  :(  leaving this off for now
gives me a simpler nfs uri format.  (that should be good enough for most
people.)

> >> Perhaps if this is what I would like I would be better off adding a
> >> global zone vfstab entry to mount nfsserver:/vol/zones somewhere and use
> >> the path:/// uri instead.
> >>
> >> Thoughts?
> >>
> >
> > i'm not sure i understand how you would like to see this functionality
> > behave.
> >
> > wrt vfstab, i'd rather you not use that since that moves configuration
> > outside of zonecfg.  so later, if you want to migrate the zone, you'll
> > need to remember about that vfstab configuration and move it as well.
> > if at all possible i'd really like to keep all the configuration within
> > zonecfg(1m).
> >
> > perhaps you could explanin your issues with the currently planned
> > approach in a different way to help me understand it better?
> >
>
> The key thing here is that all of my zones are served from one or two
> NFS servers.  Let's pretend that I have a T5440 with 200 zones on it.
> The way the proposal is written, I would have 200 mounts in the global
> zone of the form:
>
>$nfsserver:/vol/zones/zone$i
> on /var/zones/nfsmount/nfsserver/vol/zones/zone$i
>
> When in reality, all I need is a single mount (subject to
> implementation-specific details, as discussed above with ro vs. rw
> shares):
>
>$nfsserver:/vol/zones
> on /var/myzones/nfs/$nfsserver/vol/zones
>
> If my standard global zone deployment mechanism adds a vfstab entry
> for $nfsserver:/vol/zones and configure each zone via path:/// I avoid
> a storm of NFS mount requests at zone boot time as the global zone
> boots.  The NFS mount requests are UDP-based RPC calls, which
> sometimes get lost on the wire.  The timeout/retransmit may be such
> that we add a bit of time to the overall zone startup process.  Not a
> huge deal in most cases, but a confusing problem to understand.
>
> In this case, I wouldn't consider the NFS mounts as being something
> specific to a particular zone.  Rather, it is a common configuration
> setting across all members of a particular "zone farm".
>

so if your nfs server is exporting a bunch of filesystems like:
$nfsserver:/vol/zones/zone$i

then yes, you'll end up with mounts for each.  but if your nfs server
is exporting
$nfsserver:/vol/zones

then you'll only end up with one.

that said, if your nfs server is exporting
$nfsserver:/vol/zones
$nfsserver:/vol/zones/zone$i

i really don't see any way to avoid having mounts for each zone.  afaik,
if the nfs server has a nested export, the exported subdirectory is only
accessible via a mount.  so you couldn't mount $nfsserver:/vol/zones and
then access $nfsserver:/vol/zones/zone5 without first mounting
$nfsserver:/vol/zones/zone5.