Re: [developer] Re: zfs tests design/issues
On most unix-like platforms, sizeof(int) == 4, even for 64-bit binaries, and you still need ‘long int’ to get 64-bits. From: Igor Kozhukhov Reply: openzfs-developer Date: June 26, 2018 at 4:05:22 AM To: openzfs-developer Cc: Yuri Pankov Subject: Re: [developer] Re: zfs tests design/issues personally to me, KSH is wrong with result - because 64bit application should to use 64bit intager as others applications. BASH and ZSH are agreed with me and produce correct result. i didn’t check others shells. how is old/updated KSH? based on Debian changelog file - 93u+20120801-3.1 - it is code drop from 2012 year. but BASH is: https://packages.debian.org/stretch/bash 4.4 latest version http://git.savannah.gnu.org/cgit/bash.git?h=devel My primary goal was and is: to use more universal/the same tools as others platforms: Linux, FreeBSD, OSX. and i think - BASH will be much more better as default shell for zfs tests. because others platforms - like Linux, FreeBSD, OSX - have no KSH as default installed and should install it only for zfs tests. I’d like to see comments about this idea from OpenZFS community. also, about others ideas, described in first email with this subject. -Igor On Jun 26, 2018, at 2:59 AM, Joshua M. Clulow wrote: On Mon, 25 Jun 2018 at 16:52, Igor Kozhukhov wrote: the same test with BASH (64bit) Bash is a different shell, with different behaviour. For what it's worth, the "typeset -li" syntax also appears to work in bash. it is ksh93 bug - not Debian. i didn’t say about - it is illumos ksh bug - i just say about ksh93 issue with 2G value in bytes with 64bit application. As far as I can tell, it's working the way it's intended to work; i.e., that "typeset -i" gets you a 32-bit int and "typeset -li" gets you a longer (64-bit) integer. and - I have started thread about - move on from KSH to another shell for zfs tests - as part of another proposals.. This is not really a justification for making a change of that scope, merely for fixing the bug you found in the original script. It should be a simple fix now that we understand the problem. Cheers. -- Joshua M. Clulow Engineer @ Joyent http://blog.sysmgr.org -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M393ced4f2af3ece7621dd852 Delivery options: https://openzfs.topicbox.com/groups openzfs / openzfs-developer / see discussions + participants + delivery options Permalink signature.asc Description: Message signed with OpenPGP using AMPGpg smime.p7s Description: S/MIME cryptographic signature This is a multi-part message in MIME format... =_1530030683-45233-459--
Re: [developer] Re: zfs tests design/issues
> On Jun 26, 2018, at 4:03 AM, Igor Kozhukhov wrote: > > personally to me, KSH is wrong with result - because 64bit application should > to use 64bit intager as others applications. > BASH and ZSH are agreed with me and produce correct result. > i didn’t check others shells. > how is old/updated KSH? > based on Debian changelog file - 93u+20120801-3.1 - it is code drop from 2012 > year. > > but BASH is: > https://packages.debian.org/stretch/bash > 4.4 latest version > http://git.savannah.gnu.org/cgit/bash.git?h=devel > > My primary goal was and is: to use more universal/the same tools as others > platforms: Linux, FreeBSD, OSX. > and i think - BASH will be much more better as default shell for zfs tests. > because others platforms - like Linux, FreeBSD, OSX - have no KSH as default > installed and should install it only for zfs tests. > > I’d like to see comments about this idea from OpenZFS community. > > also, about others ideas, described in first email with this subject. > > -Igor > My $.02 is: By the time something is complicated enough you’re contemplating porting it to a different shell, just use python. I’ll be the first to admit I’m anti-shell. It’s a result of 25 years of experience with multi-platform shell scripts. Thanks, Josh Paetzel -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-Mc9819f9cbbcbbbc07a1f2f53 Delivery options: https://openzfs.topicbox.com/groups
Re: [developer] Re: zfs tests design/issues
On Mon, 25 Jun 2018 at 16:52, Igor Kozhukhov wrote: > the same test with BASH (64bit) Bash is a different shell, with different behaviour. For what it's worth, the "typeset -li" syntax also appears to work in bash. > it is ksh93 bug - not Debian. > i didn’t say about - it is illumos ksh bug - i just say about ksh93 issue > with 2G value in bytes with 64bit application. As far as I can tell, it's working the way it's intended to work; i.e., that "typeset -i" gets you a 32-bit int and "typeset -li" gets you a longer (64-bit) integer. > and - I have started thread about - move on from KSH to another shell for zfs > tests - as part of another proposals.. This is not really a justification for making a change of that scope, merely for fixing the bug you found in the original script. It should be a simple fix now that we understand the problem. Cheers. -- Joshua M. Clulow Engineer @ Joyent http://blog.sysmgr.org -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M393ced4f2af3ece7621dd852 Delivery options: https://openzfs.topicbox.com/groups
Re: [developer] Re: zfs tests design/issues
On Mon, 25 Jun 2018 at 16:37, Igor Kozhukhov wrote: > 64bit int max value > > where am i wrong? What makes you think it's a 64-bit integer? On a computer running Debian (the technology!) I get the same result with the ksh93 package they provide: bash$ uname -v #24-Ubuntu SMP Wed May 16 12:15:17 UTC 2018 bash$ ksh93 --version version sh (AT Research) 93u+ 2012-08-01 bash$ ksh93 $ typeset -i var=2147483648 $ echo $var -2147483648 Cheers. -- Joshua M. Clulow Engineer @ Joyent http://blog.sysmgr.org -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-Mc39b3683f49e6215af9381a9 Delivery options: https://openzfs.topicbox.com/groups
Re: [developer] Re: zfs tests design/issues
Igor Kozhukhov wrote: https://en.wikipedia.org/wiki/9,223,372,036,854,775,807 64bit int max value where am i wrong? 'typeset -i' is 'int', try using 'typeset -li' if you need something like 'long'. On Jun 26, 2018, at 2:33 AM, Yuri Pankov wrote: Igor Kozhukhov wrote: additional reason move on from illumos KSH93. I have found issue with zfs tests 'zvol_swap' whare we try to read 2G volsize in bytes. try to do simple test: #!/bin/ksh -p typeset -i var=2147483648 echo $var and i can see: # ksh93 v.ksh -2147483648 it is incorrect - it should not to be less 0 Why? You declare it as 'int' using 'typeset -i', and get the correct result. -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M0adc811139ddf60bbe656df2 Delivery options: https://openzfs.topicbox.com/groups
Re: [developer] Re: zfs tests design/issues
https://en.wikipedia.org/wiki/9,223,372,036,854,775,807 64bit int max value where am i wrong? -Igor > On Jun 26, 2018, at 2:33 AM, Yuri Pankov wrote: > > Igor Kozhukhov wrote: >> additional reason move on from illumos KSH93. >> I have found issue with zfs tests 'zvol_swap' whare we try to read 2G >> volsize in bytes. >> try to do simple test: >> #!/bin/ksh -p >> typeset -i var=2147483648 >> echo $var >> and i can see: >> # ksh93 v.ksh >> -2147483648 >> it is incorrect - it should not to be less 0 > > Why? You declare it as 'int' using 'typeset -i', and get the correct result. -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M631643178f715409a86344f6 Delivery options: https://openzfs.topicbox.com/groups
Re: [developer] Re: zfs tests design/issues
Igor Kozhukhov wrote: additional reason move on from illumos KSH93. I have found issue with zfs tests 'zvol_swap' whare we try to read 2G volsize in bytes. try to do simple test: #!/bin/ksh -p typeset -i var=2147483648 echo $var and i can see: # ksh93 v.ksh -2147483648 it is incorrect - it should not to be less 0 Why? You declare it as 'int' using 'typeset -i', and get the correct result. -- openzfs: openzfs-developer Permalink: https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M6291aec09bcfedb84c9c9bea Delivery options: https://openzfs.topicbox.com/groups
[developer] Re: zfs tests design/issues
additional reason move on from illumos KSH93.
I have found issue with zfs tests 'zvol_swap' whare we try to read 2G volsize
in bytes.
try to do simple test:
#!/bin/ksh -p
typeset -i var=2147483648
echo $var
and i can see:
# ksh93 v.ksh
-2147483648
it is incorrect - it should not to be less 0
best regards,
-Igor
> On Jun 24, 2018, at 10:20 PM, Igor Kozhukhov wrote:
>
> Hello All,
>
> I’d like to start thread about zfs tests.
>
> We have ideas for: to use zfs tests on DilOS and Debian based platform.
>
> Debian - it is not just Linux - it is thechnology.
>
> They can build varios of architectures - such as: Intel, MIPS, ARM, SPARC,
> etc - on one build env.
> also, they can use Lnux and FreeBSD kernels with the same userland code base
> with rebuilds on corresponded platform, etc.
>
> Debian is providing interest idea for splitting of packages to: with bins -
> platform depend and need rebuild on another platform/archivecture;
> text modules - for all platform/architectures - what can be reused as is
> without rebuilds.
>
> With these ideas on DilOS - we can reuse a big list of package with Perl,
> Python, Ruby, NodeJS, Java, JS, etc - just download and upload to DilOS repo.
> But Debian based platfroms are use it a long time.
>
> I’d like propose to split zfs tests to 2 parts:
> 1 - zfs hellper binaries - what can be built on platform like Linux, FreeBSD,
> Linux, OSX - and used for zfs tests
> 2 - scripting part - can be used as is on all platforms.
>
> about srcipts.
> current zfs tests based on KSH and it is outdated on illumos.
> I’d like propose to move on to more portable/usable scrupts based on BASH or
> ZSH.
>
> we can add ability to use based shell by additional environment variable and
> use screnps with extensions like:
> export SHELL_EXEC=ksh
> ${SHELL_EXEC} .${SHELL_EXEC}
>
> and for transition period to another shell - we can still to use KSH and
> working on another implementaions.
>
> additional problems with tests framework - some executions are failed if
> primary system shell is not as /bin/ksh
>
> on DilOS platform we are using DASH as primary sysytem shell - /bin/sh =
> /usr/bin/dash
>
> based on logs we can see FAILED tests where we do executions with SU and they
> are failed - it related to tests design and this issue can be fixed with idea
> to use ${SHELL_EXEC} too.
>
> Next problem: it is SUDO problem.
> current idea to use PATH to temporary directory with links to tools - failed,
> if /etc/sudoers contain full security set with definitions:
> Defaults env_reset
> Defaults
> secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
>
> yes - they ara changes are recommended with securoty needs and on DilOS we
> are using Debian proposed configs with security needs.
>
> for zfs tests - we need a hack local /etc/sudoers file and disable these
> optinos - they are not described in README file because probably others
> platforms are using another /etc/sudoers files.
> with these security options we can’t use PATH with sudo -E.
>
> for reduce this issue with hacks of local /etc/sudoers file i’d like propose
> try to change logic with tools: probably will be much more better to use
> 'alias zpool=/sbin/zpool' - and use 'zpool' with correct pach where we need
> it ?
>
> it will be applicable to others platforms(Linux, FreeBSD, OSX, etc) too -
> where we can remap some tools to another locations, becuase only illumos
> based platforms are using /usr/gnu/bin, etc.
>
> next proposal: to go to use universal tools for zfs tests - not illumos
> specific.
> for some tests - like ACL - we need specific tools based on platform - and
> every platform can remap and use own tools, but for some cases will be better
> to use DF, FILE, DD, etc - what are using on all platforms.
>
> Also - will be better to provide ability to define a platform where we run
> zfs tests - and use ${ztest_platform} variable where we need for
> enable/disable platform specific tests.
>
> Also, tests are failed with 4k drives with UFS (newfs) becasue:
> root@con6-vm-64:~# newfs /dev/dsk/c0t600144F07ADEAD185B2E7C150001d0s0
> newfs: construct a new file system
> /dev/rdsk/c0t600144F07ADEAD185B2E7C150001d0s0: (y/n)? y
> The device sector size 4096 is not supported by ufs!
>
> Have to reduce dependencies to newfs and probably change to use 'zfs' type.
> or - identify tests drives - are they 512b or 4k - and change some tests
> logic.
>
> What do you think about changes in zfs design to me used on others platforms ?
>
> best regards,
> -Igor
>
--
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/T42f8147492666f65-M1f7a11340219534c5310de22
Delivery options: https://openzfs.topicbox.com/groups
