Re: [Shr-User] UBI success story

[Martin Jansa]

On Wed, Jan 6, 2010 at 2:21 AM, Martin Jansa  wrote:

> 3) update Qi with dfu-util
> qi built here
> http://build.shr-project.org/shr-unstable/images/om-gta02/
> is still using jffs2 in kernel params
> but you can you binary from here
>
> http://jama.homelinux.org/org.openembedded.shr.images/om-gta02/qi-s3c2442-1.0.2+gitr0+c38b062a609f1442e6a9e13005cfbdfd59a5ac0d.udfu
> or build Qi yourself with bitbake from latest shr/merge branch + this
> patch:
>
> http://jama.homelinux.org/org.openembedded.shr.images/om-gta02/0014-qi-update-kernel-params-for-ubi-rootfs.patch
>

This is not true anymore, I've added qi-ubi as different recipe to OE and
rebuild here, so  this
qi-s3c2442-1.0.2+gitr0+c38b062a609f1442e6a9e13005cfbdfd59a5ac0d.udfu
has
jffs2 params again and you need
qi-ubi-s3c2442-1.0.2-r0+gitr1+c38b062a609f1442e6a9e13005cfbdfd59a5ac0d.udfu
for
ubifs params

but no need to download it from my host now.. I've built it on official shr
buildhost too, so download it here
http://build.shr-project.org/shr-unstable/images/om-gta02/

Sorry to those who downloaded it in between (I'll remove it from my host for
now..), I've updated wiki page
http://wiki.openmoko.org/index.php?title=UBIFS
and forgot to send this e-mail.

Cheers,
>
> --
> uin:136542059
> jid:martin.ja...@gmail.com
> Jansa Martin 
> sip:jama...@voip.wengo.fr
> JaMa
>
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Maksim 'max_posedon' Melnikau]

I just understand, that its wrong to test via boniee++ with default
params, which required write 300Mb data, neo have mtd just 256mb, and
both fs becomes **very** slow near ENOSPC.

So, I rerun all tests with -s 32 -m 16, also I tested ext3, and ext2
in sync/async mode.

=== ubifs ===   --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost   32M  1531  78  7594  73  4318  76  2195  98 42904  99  2805  87
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 16   887  94  2802  99   859  98   914  97 18084  99   659  98
localhost,32M,1531,78,7594,73,4318,76,2195,98,42904,99,2804.8,87,16,887,94,2802,99,859,98,914,97,18084,99,659,98


=== ext3, async === --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost   32M  1122  69  1911  40  2288  28  1832  82 43012  99  2923  98
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 16   111  96 18460  95  1315  85   111  95 20824  99   258  75
localhost,32M,1122,69,1911,40,2288,28,1832,82,43012,99,2923.0,98,16,111,96,18460,95,1315,85,111,95,20824,99,258,75


=== ext2, sync ===  --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost   32M70   5   122   3   970  11  2196  98 42969  99  56.1   1
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 1618  12 17609  8914   218  12 20165  98 6   1
localhost,32M,70,5,122,3,970,11,2196,98,42969,99,56.1,1,16,18,12,17609,89,14,2,18,12,20165,98,6,1


=== ext2, async === --Sequential Output-- --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost   32M  1319  70  2637  29  1997  16  1904  86 42504  99  3151  98
--Sequential Create-- Random Create
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
 16   166  95 19707  98  2588  97   170  96 20644  99   321  63
localhost,32M,1319,70,2637,29,1997,16,1904,86,42504,99,3151.4,98,16,166,95,19707,98,2588,97,170,96,20644,99,321,63

Can somebody run
bonnie\+\+ -u 0:0 -s 32 -r 16
on jffs2 with at least 64Mb free space?


Also I want repoint people about
http://shr-project.org/trac/ticket/445, we should ALWAYS mount
uSD/usb-etc async.

On 1/6/10, Vinzenz Hersche  wrote:
> could you make a wikientry with step by step-instructions for interested
> peobles (like me?)
>
> how much faster it is about?
>
> ___
> Openmoko community mailing list
> community@lists.openmoko.org
> http://lists.openmoko.org/mailman/listinfo/community
>

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Vinzenz Hersche]

could you make a wikientry with step by step-instructions for interested 
peobles (like me?)

how much faster it is about?

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Martin Jansa]

On Mon, Jan 04, 2010 at 02:37:14PM +0100, Christoph Mair wrote:
> > Ah great!, thanks
> Remember that ubi0:rootfs specifies the volume name. The current ubinize.cfg 
> sets this name to om-gta02-rootfs, so change this to ubi0:om-gta02-rootfs

I've removed some UBI debug after max_posedon request and those 2
success stories..

With disabled BGT it seems a bit better also here.

Today I successfully booted ubifs rootfs with Qi too (I'm using 2.6.32,
max_posedon seems happy with standard shr kernel - 2.6.29-rc3). 2.6.32
won't suspend yet (WSOD) but its imho a bit faster.

just after few steps:

1) booting 2.6.32 kernel on uSD
http://build.shr-project.org/tests/mrmoku/2.6.32/images/om-gta02/
download uImage to /boot 
download modules and untar it to /
update uImage link in /boot

2) flash image
flash_eraseall /dev/mtd6
nandwrite -p /dev/mtd6 shr-full-eglibc-ipk--20100105-om-gta02.rootfs.ubi
(almost the same ubinized image will finish build in few mins as
http://build.shr-project.org/shr-unstable/images/om-gta02/shr-full-eglibc-ipk--20100105-om-gta02.rootfs.ubi)

3) then test if it works 
ubiattach /dev/ubi_ctrl -O 2048 -m 6
mount -t ubifs ubi0:om-gta02-rootfs /media/om
ls /media/om

3) update Qi with dfu-util
qi built here
http://build.shr-project.org/shr-unstable/images/om-gta02/ 
is still using jffs2 in kernel params
but you can you binary from here
http://jama.homelinux.org/org.openembedded.shr.images/om-gta02/qi-s3c2442-1.0.2+gitr0+c38b062a609f1442e6a9e13005cfbdfd59a5ac0d.udfu
or build Qi yourself with bitbake from latest shr/merge branch + this patch:
http://jama.homelinux.org/org.openembedded.shr.images/om-gta02/0014-qi-update-kernel-params-for-ubi-rootfs.patch

4) flash 2.6.32 also to NAND

5) boot it, test it, play with it

max_posedon provided some bonnie++ results from UBI, please run bonnie++
on jffs2 if you have jffs2 partition (bonnie++ is now in shr-unstable feeds).

http://pastebin.ca/1739342";>
r...@localhost / $ bonnie\+\+ -u 0:0
Using uid:0, gid:0.
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03c   --Sequential Output-- --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
localhost  300M  1512  72  4878  60  2696  58  1587  98  5519  96
640.6  96
--Sequential Create-- Random
Create
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
 16   889  98  2619  99   802  95   883  97 15728  87
640  98
localhost,300M,1512,72,4878,60,2696,58,1587,98,5519,96,640.6,96,16,889,98,2619,99,802,95,883,97,15728,87,640,98



Cheers,

-- 
uin:136542059jid:martin.ja...@gmail.com
Jansa Martin sip:jama...@voip.wengo.fr 
JaMa 

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Martin Jansa]

On Mon, Jan 04, 2010 at 02:37:14PM +0100, Christoph Mair wrote:
> Quote  from http://www.linux-mtd.infradead.org/doc/ubi.html#L_subpage:
> "Indeed, let's consider a NAND flash with 128KiB eraseblocks and 2048-byte 
> pages. If it does not have sub-pages, UBI puts the the VID header at physical 
> offset 2048, so LEB size becomes 124KiB (128KiB minus one NAND page which 
> stores the EC header and minus another NAND page which stores the VID header."
> 
> Therefore the params should be:
> MKUBIFS_ARGS = "-m 2048 -e 126976 -c 2047"

Yeah, I notided the same yesterday in failing mount command (it said
that superblock is wrong because of having wrong leb size, so updated 
it locally too).

> With these parameters, I successfully wrote a ubifs image to a ubi volume:
> flash_eraseall /dev/mtd6
> ubiattach /dev/ubi_ctrl -m6 -O 2048
> ubimkvol /dev/ubi0 -m -Nrootfs
> ubiupdatevol /dev/ubi0_0 test.ubifs
> mount -t ubifs ubi0_0 /mnt/
> 
> The first step (flash_eraseall) is optional and only needed if ubiattach 
> fails.

I it "worked" for me too with just
flash_eraseall /dev/mtd6
nandwrite /dev/mtd6 shr-full-eglibc-ipk--20100103-om-gta02.rootfs.ubi

but first attempt it worked with only IO errors about few unreadable
PEBs. After reboot mount always failed after about a minute with
"Segmentation fault" and once with kernel panic - with 2.6.32.2.

> IMHO the ubinize params should be
> UBINIZE_ARGS = "-m 2048 -p 128KiB -s 2048"

I also had -O 2048 there..

> but until now I did not get this to work. I can write the image (sometimes it 
> needs a few tries) but mount does not work.

As said above for me it worked just once after nandwrite. :/

Cheers,

-- 
uin:136542059jid:martin.ja...@gmail.com
Jansa Martin sip:jama...@voip.wengo.fr 
JaMa 

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Christoph Mair]

[sent again to reach the mailing lists]
On Sunday 03 January 2010 13:11:52 Martin Jansa wrote:
> On Sat, Dec 26, 2009 at 09:25:57PM +0100, Christoph Mair wrote:
> > Hello,
> >
> > I tried to use SHR the ubi images on my freerunner, but they did not
> > work. Does someone know the parameters which are passed to mkfs.ubifs? I
> > think that the ubi fileystem is created for NAND flashes with subpage
> > support, but this feature does not work on the freerunner. Passing -s
> > 2048 should create a working image (but I did not verify this yet).
> 
> Params are in:
> http://git.openembedded.org/cgit.cgi/openembedded/tree/conf/machine/om-gta0
> 2.conf
> 
> MKUBIFS_ARGS = "-m 2048 -e 129024 -c 2047"
> for mkfs.ubifs
> 
> UBINIZE_ARGS = "-m 2048 -p 128KiB -s 512"
> for ubinize
> 
> So I don't see param for subpage setting for mkfs.ubifs, its only in
>  ubinize, but it didn't help for my NAND, even when I prepare ubi volume on
>  neo with -s 2048 -O 2048 and then try to updatevol with full image (small
>  image like initramfs-kexecboot works for me just fine). That's why I
>  didn't push patch for setting it in UBINIZE_ARGS.
Yes, my fault. mkfs.ubifs does not care about subpages. The critical point 
here is the logical eraseblock size:
Quote  from http://www.linux-mtd.infradead.org/doc/ubi.html#L_subpage:
"Indeed, let's consider a NAND flash with 128KiB eraseblocks and 2048-byte 
pages. If it does not have sub-pages, UBI puts the the VID header at physical 
offset 2048, so LEB size becomes 124KiB (128KiB minus one NAND page which 
stores the EC header and minus another NAND page which stores the VID header."

Therefore the params should be:
MKUBIFS_ARGS = "-m 2048 -e 126976 -c 2047"

With these parameters, I successfully wrote a ubifs image to a ubi volume:
flash_eraseall /dev/mtd6
ubiattach /dev/ubi_ctrl -m6 -O 2048
ubimkvol /dev/ubi0 -m -Nrootfs
ubiupdatevol /dev/ubi0_0 test.ubifs
mount -t ubifs ubi0_0 /mnt/

The first step (flash_eraseall) is optional and only needed if ubiattach fails.

IMHO the ubinize params should be
UBINIZE_ARGS = "-m 2048 -p 128KiB -s 2048"
but until now I did not get this to work. I can write the image (sometimes it 
needs a few tries) but mount does not work.

> > To fix this, we need to tell the kernel that the mtd partition 6 contains
> > a ubi volume which contains the ubifs rootfs: rootfstype=ubifs
> > ubi.mtd=6,2048 root=ubi0:rootfs.
> > For a "normal" boot qi passes rootfstype=jffs2 root=/dev/mtdblock6 to the
> > kernel. Just changing the boot params within the kernel configuration
> > does not work. Therefore I patched qi :)
> 
> Ah great!, thanks
Remember that ubi0:rootfs specifies the volume name. The current ubinize.cfg 
sets this name to om-gta02-rootfs, so change this to ubi0:om-gta02-rootfs

Christoph

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community

Re: [Shr-User] UBI success story

[Martin Jansa]

On Sat, Dec 26, 2009 at 09:25:57PM +0100, Christoph Mair wrote:
> Hello,
> 
> I tried to use SHR the ubi images on my freerunner, but they did not work. 
> Does someone know the parameters which are passed to mkfs.ubifs? I think that 
> the ubi fileystem is created for NAND flashes with subpage support, but this 
> feature does not work on the freerunner. Passing -s 2048 should create a 
> working image (but I did not verify this yet).

Params are in:
http://git.openembedded.org/cgit.cgi/openembedded/tree/conf/machine/om-gta02.conf

MKUBIFS_ARGS = "-m 2048 -e 129024 -c 2047"
for mkfs.ubifs

UBINIZE_ARGS = "-m 2048 -p 128KiB -s 512"
for ubinize

So I don't see param for subpage setting for mkfs.ubifs, its only in ubinize,
but it didn't help for my NAND, even when I prepare ubi volume on neo with 
-s 2048 -O 2048 and then try to updatevol with full image (small image
like initramfs-kexecboot works for me just fine). That's why I didn't
push patch for setting it in UBINIZE_ARGS.

> Using the ubi filesystem on the freerunner is not an easy task. A ubi image 
> can't be flashed using nandwrite. I'm not sure about dfu-util, but probably 
> it 
> uses the same technique as nandwrite and therefore won't work too. I did a 
> manual installation (untar SHR into a mounted ubifs) using archmobile 
> installed on SD. If the SHR ubi image is created with the right parameters 
> ubiformat or ubiupdatevol could be used to install the image. For a easy 
> installation with dfu-util, u-boot would need support for ubi images.
> 
> I used the newest kernel (om-gta02-2.6.32), but a older one should work too.

I had more issues with kernels older than 2.6.32.

> To fix this, we need to tell the kernel that the mtd partition 6 contains a 
> ubi 
> volume which contains the ubifs rootfs: rootfstype=ubifs ubi.mtd=6,2048 
> root=ubi0:rootfs.
> For a "normal" boot qi passes rootfstype=jffs2 root=/dev/mtdblock6 to the 
> kernel. Just changing the boot params within the kernel configuration does 
> not 
> work. Therefore I patched qi :)

Ah great!, thanks

-- 
uin:136542059jid:martin.ja...@gmail.com
Jansa Martin sip:jama...@voip.wengo.fr 
JaMa 

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community