Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
--- On Mon, 8/1/11, Daniel Drake d...@laptop.org wrote: From: Daniel Drake d...@laptop.org Subject: Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52] To: Yioryos Asprobounitis mavrot...@yahoo.com Cc: devel@lists.laptop.org Date: Monday, August 1, 2011, 3:23 PM On 1 August 2011 20:10, Yioryos Asprobounitis mavrot...@yahoo.com wrote: My original report was based on the `df' data. To test how accurate these might be I did the following: To save me the effort of trying to understood exactly what you did, could you just state how much data from /dev/urandom you were able to write to disk on freshly-installed 11.2.0, vs 11.3.0 build 1? What I said is that `df' is reliable in reporting free space as judged by urandom writes, within 5%. eg 253MB reported vs 266MB written in my current ubifs XO-1/os1 and 178MB reported vs 185 MB written in my current jffs2 XO-1/os874. So whatever intricacies ubifs and jffs2 have, `df' appears to handle them properly. Now, `df' reported ~270MB available on fresh os1 install vs ~340MB on a fresh os874 install. Given that dri has been removed from os1, this is an almost 100MB difference. I'm sure I can jump some more ropes to make the case on a fresh os1 vs a fresh os874, but I do not see any reason. If the opportunity appears I will. The need for an XO-1 reflashing is not that rare ;-). In the mean, time given the accuracy of `df', it may worth looking at why /dev/ubi0_0 is only 878MB. cheers Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
On 2 August 2011 15:35, Yioryos Asprobounitis mavrot...@yahoo.com wrote: I'm sure I can jump some more ropes to make the case on a fresh os1 vs a fresh os874, but I do not see any reason. The reason is that df is known to not produce good results on jffs2 nor on ubifs (but ubifs is supposedely much more truthful). Looking at those numbers is not a good basis for saying something is wrong. The only test that makes sense is looking at how much data can be stored in total - preferably even looking at the same data over each fs (i.e. not using /dev/urandom). Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
The reason is that df is known to not produce good results on jffs2 nor on ubifs (but ubifs is supposedely much more truthful). I do not know why you say that when the data show otherwise. (and although I can not find the post right now, I remember that James Cameron came on a similar conclusion - df being fairly accurate - in some tests of his own) I also not clear to me how random data can be different on average, over an extended length. Maybe you could investigate yourself in a way that you find more reliable. 100MB is considerable space for the XO-1. ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
On 2 August 2011 17:10, Yioryos Asprobounitis mavrot...@yahoo.com wrote: I do not know why you say that when the data show otherwise. You haven't provided data for the requested test. Maybe you could investigate yourself in a way that you find more reliable. 100MB is considerable space for the XO-1. Will do, after I get some higher priority tasks out of the way. Thanks for your help. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
On 28 July 2011 21:08, Yioryos Asprobounitis mavrot...@yahoo.com wrote: For what is worths, with the increasing size of the builds I had posted [1] a crude script I was using to remove dri, firmware, extra locales and 256x256 ions (61Mb worth on os874) which people used, and I did not hear any complains, yet ;). Going back to the issue of size, it would appear that in os1 tmpfs size (114MB) is subtracted from the available storage. Thus even if os1 is ~660MB the free space is only 270MB, since /dev/ubi0_0 is only 898MB instead on 1024M (for mtdblok0) in os874. Is this something specific to ubifs or a configuration issue? I'd be interested to know how you pulled up those numbers, but I doubt that they are valid. Calculating the amount of free space on NAND is a complicated issue, and even more complicated when UBI comes into play. And UBIFS provides much more conservative free space readings than jffs2, which often tells you that there is more space available than there actually is. That's not to say there isn't an issue here (there may well be), but you need to prove this with a test that looks at the quantity of data that can be stored. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
On Mon, Aug 1, 2011 at 10:25 AM, Daniel Drake d...@laptop.org wrote: That's not to say there isn't an issue here (there may well be), but you need to prove this with a test that looks at the quantity of data that can be stored. Yioryos, du -shx / before and after will give you uncompressed disk space usage delta ;-) m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
--- On Mon, 8/1/11, Daniel Drake d...@laptop.org wrote: From: Daniel Drake d...@laptop.org Subject: Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52] To: Yioryos Asprobounitis mavrot...@yahoo.com Cc: devel@lists.laptop.org Date: Monday, August 1, 2011, 10:25 AM On 28 July 2011 21:08, Yioryos Asprobounitis mavrot...@yahoo.com wrote: For what is worths, with the increasing size of the builds I had posted [1] a crude script I was using to remove dri, firmware, extra locales and 256x256 ions (61Mb worth on os874) which people used, and I did not hear any complains, yet ;). Going back to the issue of size, it would appear that in os1 tmpfs size (114MB) is subtracted from the available storage. Thus even if os1 is ~660MB the free space is only 270MB, since /dev/ubi0_0 is only 898MB instead on 1024M (for mtdblok0) in os874. Is this something specific to ubifs or a configuration issue? I'd be interested to know how you pulled up those numbers, but I doubt that they are valid. Calculating the amount of free space on NAND is a complicated issue, and even more complicated when UBI comes into play. And UBIFS provides much more conservative free space readings than jffs2, which often tells you that there is more space available than there actually is. That's not to say there isn't an issue here (there may well be), but you need to prove this with a test that looks at the quantity of data that can be stored. My original report was based on the `df' data. To test how accurate these might be I did the following: [olpc@xo-11-ea-51 ~]$ df -B M Filesystem 1M-blocks Used Available Use% Mounted on /dev/ubi0_0 878M 621M 253M 72% / tmpfs 114M1M 114M 1% /dev/shm /tmp 50M1M 50M 1% /tmp vartmp 50M1M 50M 1% /var/tmp varlog 20M1M 20M 1% /var/log /dev/mtdblock2 24M 12M 13M 47% /bootpart [olpc@xo-11-ea-51 ~]$ sudo dd if=/dev/urandom of=test_free.img bs=1M count=256 256+0 records in 256+0 records out 268435456 bytes (268 MB) copied, 1077.6 s, 249 kB/s [olpc@xo-11-ea-51 ~]$ df -B M Filesystem 1M-blocks Used Available Use% Mounted on /dev/ubi0_0 878M 871M3M 100% / tmpfs 114M1M 114M 1% /dev/shm /tmp 50M1M 50M 1% /tmp vartmp 50M1M 50M 1% /var/tmp varlog 20M1M 20M 1% /var/log /dev/mtdblock2 24M 12M 13M 47% /bootpart [olpc@xo-11-ea-51 ~]$ sudo dd if=/dev/urandom of=test_free2.img bs=1M count=6 6+0 records in 6+0 records out 6291456 bytes (6.3 MB) copied, 37.2785 s, 169 kB/s [olpc@xo-11-ea-51 ~]$ df -B M Filesystem 1M-blocks Used Available Use% Mounted on /dev/ubi0_0 878M 875M0M 100% / tmpfs 114M1M 114M 1% /dev/shm /tmp 50M1M 50M 1% /tmp vartmp 50M1M 50M 1% /var/tmp varlog 20M1M 20M 1% /var/log /dev/mtdblock2 24M 12M 13M 47% /bootpart [olpc@xo-11-ea-51 ~]$ sudo dd if=/dev/urandom of=test_free3.img bs=1M count=6 dd: writing `test_free3.img': No space left on device 5+0 records in 4+0 records out 4571136 bytes (4.6 MB) copied, 36.9996 s, 124 kB/s So `df' looks pretty good to me. Looking at the `df' output one can see that /dev/ubi0_0 + /dev/mtdblock2 + tmpfs(!?) = 1016 MB just 8MB short of the NAND capacity. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
On 1 August 2011 20:10, Yioryos Asprobounitis mavrot...@yahoo.com wrote: My original report was based on the `df' data. To test how accurate these might be I did the following: To save me the effort of trying to understood exactly what you did, could you just state how much data from /dev/urandom you were able to write to disk on freshly-installed 11.2.0, vs 11.3.0 build 1? cheers Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
On 07/29/2011 01:49 PM, Peter Robinson wrote: I don't have mine handy but I have a OLPC serial adapater What leads you to believe that the OLPC serial adapter needs firmware downloaded? -- Richard A. Smith rich...@laptop.org One Laptop per Child ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
On Thu, Jul 28, 2011 at 4:38 PM, Daniel Drake d...@laptop.org wrote: Hi, linux-firmware, the package that puts lots of stuff in /lib/firmware, is growing and growing and is mostly irrelevant for us, including firmware for (e.g.) SCSI controllers. I'm considering removing it - nothing in the XOs uses firmware from that package. (our wireless firmware comes from other packages) The question is, do any external devices used by our users pull on firmware from this directory? Browsing the contents, I doubt it, but I thought I'd ask anyway. I may also just remove it from the builds as an experiment, see if anything breaks. The devices that come to mind for me are serial adapters and wired ethernet adapters. I don't have mine handy but I have a OLPC serial adapater plus a pair of usb eth adapters that I use regularly on the XO that all need firmwares. Nothing else races to mind. Peter ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Removing linux-firmware from the build
Hi, linux-firmware, the package that puts lots of stuff in /lib/firmware, is growing and growing and is mostly irrelevant for us, including firmware for (e.g.) SCSI controllers. I'm considering removing it - nothing in the XOs uses firmware from that package. (our wireless firmware comes from other packages) The question is, do any external devices used by our users pull on firmware from this directory? Browsing the contents, I doubt it, but I thought I'd ask anyway. I may also just remove it from the builds as an experiment, see if anything breaks. Thanks, Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
On Thu, Jul 28, 2011 at 11:38 AM, Daniel Drake d...@laptop.org wrote: The question is, do any external devices used by our users pull on firmware from this directory? Browsing the contents, I doubt it, but I thought I'd ask anyway. I may also just remove it from the builds as an experiment, see if anything breaks. Just guessing, but I would think the most commonly used external devices would be projectors and external storage via USB. There are users that have posted about projector issues and they would be logical candidates to approach for testing of any dependency, which I don't expect either, but empirical testing is best. Other external devices would be sensors and robots which are obviously of growing importance. Just some thoughts about where to look for issues, not a prediction that they will occur. cjl ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
On 28 July 2011 16:44, Chris Leonard cjlhomeaddr...@gmail.com wrote: Just guessing, but I would think the most commonly used external devices would be projectors and external storage via USB. There are users that have posted about projector issues and they would be logical candidates to approach for testing of any dependency, which I don't expect either, but empirical testing is best. Other external devices would be sensors and robots which are obviously of growing importance. Thanks for your input. USB VGA and USB storage devices do not draw firmware from Linux, so they would be unaffected. I am fairly sure that the same is true for robotics. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
On Thu, Jul 28, 2011 at 12:04 PM, Daniel Drake d...@laptop.org wrote: Thanks for your input. USB VGA and USB storage devices do not draw firmware from Linux, so they would be unaffected. I am fairly sure that the same is true for robotics. +10 on removal :-) m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
I think this might be the same issue, if not, please forgive: On all of our deployed XO 1.0 machines, immediately after reflashing, we run one of Mavrothal's scripts that pretty much removes everything from that directory on the XO 1's. The script saves one file only: USB8388.bin If one deletes that file, I believe networking ceases to function. The code he uses is here: mv /versions/pristine/$OSVER/lib/firmware/usb8388.bin rm -rf /versions/pristine/$OSVER/lib/firmware/* mv usb8388.bin /versions/pristine/$OSVER/lib/firmware/ mv usb8388.bin /versions/run/$OSVER/lib/firmware/ rm -rf /versions/pristine/$OSVER/lib/firmware/* mv usb8388.bin /versions/run/$OSVER/lib/firmware/ The script also removes a bunch of dri/lib,icon 256, and language versions. Frees up about 60Mb on our machines. His full script, along with a swap script, and some bad/ugly installs script is in this thread http://www.olpcnews.com/forum/index.php?topic=4965.15 O, and most importantly, the USB microscopes and Vernier sensors continue to work after deleting all those files. I still haven't been able to get a USB2VGA projector to work on any 11-2 builds even beofre the firmware removal, so I cant comment on that part. Cheers KG On Thu, Jul 28, 2011 at 1:49 PM, Martin Langhoff martin.langh...@gmail.comwrote: On Thu, Jul 28, 2011 at 12:04 PM, Daniel Drake d...@laptop.org wrote: Thanks for your input. USB VGA and USB storage devices do not draw firmware from Linux, so they would be unaffected. I am fairly sure that the same is true for robotics. +10 on removal :-) m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build
Oops bad cut and paste of the code below ... better to just go look at his script. Line 5 should say run not pristine. :-) On Thu, Jul 28, 2011 at 4:05 PM, Kevin Gordon kgordon...@gmail.com wrote: I think this might be the same issue, if not, please forgive: On all of our deployed XO 1.0 machines, immediately after reflashing, we run one of Mavrothal's scripts that pretty much removes everything from that directory on the XO 1's. The script saves one file only: USB8388.bin If one deletes that file, I believe networking ceases to function. The code he uses is here: mv /versions/pristine/$OSVER/lib/firmware/usb8388.bin rm -rf /versions/pristine/$OSVER/lib/firmware/* mv usb8388.bin /versions/pristine/$OSVER/lib/firmware/ mv usb8388.bin /versions/run/$OSVER/lib/firmware/ rm -rf /versions/pristine/$OSVER/lib/firmware/* mv usb8388.bin /versions/run/$OSVER/lib/firmware/ The script also removes a bunch of dri/lib,icon 256, and language versions. Frees up about 60Mb on our machines. His full script, along with a swap script, and some bad/ugly installs script is in this thread http://www.olpcnews.com/forum/index.php?topic=4965.15 O, and most importantly, the USB microscopes and Vernier sensors continue to work after deleting all those files. I still haven't been able to get a USB2VGA projector to work on any 11-2 builds even beofre the firmware removal, so I cant comment on that part. Cheers KG On Thu, Jul 28, 2011 at 1:49 PM, Martin Langhoff martin.langh...@gmail.com wrote: On Thu, Jul 28, 2011 at 12:04 PM, Daniel Drake d...@laptop.org wrote: Thanks for your input. USB VGA and USB storage devices do not draw firmware from Linux, so they would be unaffected. I am fairly sure that the same is true for robotics. +10 on removal :-) m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 52]
From: Daniel Drake d...@laptop.org To: OLPC Devel devel@lists.laptop.org Subject: Removing linux-firmware from the build Hi, linux-firmware, the package that puts lots of stuff in /lib/firmware, is growing and growing and is mostly irrelevant for us, including firmware for (e.g.) SCSI controllers. I'm considering removing it - nothing in the XOs uses firmware from that package. (our wireless firmware comes from other packages) The question is, do any external devices used by our users pull on firmware from this directory? Browsing the contents, I doubt it, but I thought I'd ask anyway. I may also just remove it from the builds as an experiment, see if anything breaks. Thanks, Daniel For what is worths, with the increasing size of the builds I had posted [1] a crude script I was using to remove dri, firmware, extra locales and 256x256 ions (61Mb worth on os874) which people used, and I did not hear any complains, yet ;). Going back to the issue of size, it would appear that in os1 tmpfs size (114MB) is subtracted from the available storage. Thus even if os1 is ~660MB the free space is only 270MB, since /dev/ubi0_0 is only 898MB instead on 1024M (for mtdblok0) in os874. Is this something specific to ubifs or a configuration issue? [1] http://www.olpcnews.com/forum/index.php?topic=4965.msg33324#msg33324 ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 53]
Date: Thu, 28 Jul 2011 16:05:17 -0400 From: Kevin Gordon kgordon...@gmail.com Subject: Re: Removing linux-firmware from the build His full script, along with a swap script, and some bad/ugly installs script is in this thread I hope that by bad/ugly installs script you mean the scripts that install the gstreamer bad/ugly plugins (among other things) and not to the scripts themselves (lol) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Removing linux-firmware from the build [Devel Digest, Vol 65, Issue 53]
Yessir, the bad and ugly refer to the gstreamer addins, not the script. :-) BTW I added guvcview to the end of the yum line as well, as doing that seemed to handle some of the weird LIBAVCODEC50 and 52 compatibility issues that arose when I tried to install it separately. Mikus helped us implement the guvcview s/w elegantly for our USB external microscopes, instead of only having cheese, and now even the install all happens with a light-touch post-flash one-time install script. Many of the dependencies for the gstreamer and guvcview rpm's are the same, and this just lets the localinstall routine sort it all out for us :-) Cheers, KG On Thu, Jul 28, 2011 at 4:30 PM, Yioryos Asprobounitis mavrot...@yahoo.comwrote: Date: Thu, 28 Jul 2011 16:05:17 -0400 From: Kevin Gordon kgordon...@gmail.com Subject: Re: Removing linux-firmware from the build His full script, along with a swap script, and some bad/ugly installs script is in this thread I hope that by bad/ugly installs script you mean the scripts that install the gstreamer bad/ugly plugins (among other things) and not to the scripts themselves (lol) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
