Re: [LEDE-DEV] Call defines for minifying scripting languages

[Zefir Kurtisi]

On 10/03/2016 01:00 PM, Jan-Tarek Butt wrote:
> Hi,
> 
> My Idea ist to create calldefs to minify all scriptes there are 
> interpreted by run time.
> 
> As example see the mail "[PATCH] LuaSrcDiet call define for lua 
> code Minifying".
> 
> I plan to write also calldefs for shellscripts, python and perl.
> 
> [...]
> 
When it comes to shell-scripts, I'd start at a prior step: why does an embedded
system need to run and interpret so many shell-scripts at all?

OWRT has been shell-script heavy from the very beginning, but luckily over the
last few years started to move more and more of that scripted functionality into
binaries (e.g. procd, netifd). This transition only started, and imho the goal 
to
reach is to get rid of all shell-scripts on target (beside init-scripts, maybe).

Therefore, instead of optimizing them, I'd opt to treat them as technical debt
which needs to be resolved by replacing scripts with binaries - personally, I
won't spend too much time on something to-become-legacy mid-term.



Cheers,
Zefir


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Bastian Bittorf]

* Karl Palsson  [04.10.2016 08:38]:

[...]

> about the scripts _at all_ right up until they try and look in
> one to debug something behaving oddly. At _that_ point they will
> care _very much_ that it's all minified garbage, with some
> presumption that they could "just get the source somewhere else"

I have the same impression. But making it a non-default option
for advanced users/self-builders is ok. Lets see what he can
reach in the end...

bye, bastian

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Oswald Buddenhagen]

On Mon, Oct 03, 2016 at 01:00:19PM +0200, Jan-Tarek Butt wrote:
> My Idea ist to create calldefs to minify all scriptes there are 
> interpreted by run time.
> 
> 1. Reducing memory size on firmware images.
>
this is absolutely negligible. as others already pointed out, scripts
compress quite well. also, they are actually only a rather small
fraction of the image's total size.

> 2. Strip out all comments (which makes us able to do better code
> commenting)
>
you mean, we can make the scripts *yet bigger* because it won't
negatively affect the image's size. except that point 1, so this isn't
actually a limiting factor. writing said comments and keeping them
up-to-date is the problem.

> 3. and so on.
> 
i think mostly "whatever". ;)

i'll also pre-emptively point out that minification has a rather
insignificant impact on run time - the only thing that is slowed down is
the tokenization/parsing, and that is again rather insiginificant
compared to everything else executing a script involves.

but hey, make some actual measurements - maybe you'll surprise me.


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Bastian Bittorf]

* Jan-Tarek Butt  [03.10.2016 16:55]:
> currently I am working on a Shellscript minifyer
> there are not realy exsist one so I write a new one.

hopefully in haskell 8-)
you made my day 8-) ROFL!

I think this is a bad idea. In practise you
safe nothing but make debugging harder, the JFFS2 or
squashfs already compresses text/shellscripts very good.

bye, bastian

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Jan-Tarek Butt]

On 10/03/16 14:18, Piotr Dymacz wrote:
> Hello,
> 
> 2016-10-03 13:14 GMT+02:00 Karl Palsson :
>>
>> Jan-Tarek Butt  wrote:
> [snip]
>>> 1. Reducing memory size on firmware images.
>>
>> But will it? They're in the squashfs image, it's already been
>> demonstrated before that compressing things before can actually
>> have negative impacts.
> 
> That's true, my vote for 'optimization-only' approach and keep fs do
> the rest (compression).
> 
>>> 2. Strip out all comments (which makes us able to do better
>>> code commenting) 3. and so on.
> 
> IMHO, we could remove all comments from all scripts (not only Lua) in
> target rootfs and work on better documentation outside the code.
> 
> As an example of a huge, old comment, left in one of ar71xx base-files
> scripts, please see [1].
> It's inside every ar71xx image... does it really make sense to have it
> there, for devices without related hardware?

Exactly that's my point. I mean that is really not necessary.
And the comments will be stay inside the git. Only on the final images
there will be gone.

cheers
Tarek



signature.asc
Description: OpenPGP digital signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Jan-Tarek Butt]

On 10/03/16 13:14, Karl Palsson wrote:
> 
> Jan-Tarek Butt  wrote:
>> Hi,
>>
>> My Idea ist to create calldefs to minify all scriptes there are
>> interpreted by run time.
>>
>> As example see the mail "[PATCH] LuaSrcDiet call define for lua
>> code Minifying".
>>
>> I plan to write also calldefs for shellscripts, python and
>> perl.
>>
>> They brings vareous of features.
>>
>> 1. Reducing memory size on firmware images.
> 
> But will it? They're in the squashfs image, it's already been
> demonstrated before that compressing things before can actually
> have negative impacts.

The minifing proces is on compile time so I cant see negative
impacts for the squashfs.

>> 2. Strip out all comments (which makes us able to do better
>> code commenting) 3. and so on.
>>
>> One negative point will be there:
>>
>> the minified code an not realy human readable but if some one
>> want to read this script or work on it on routers, they can
>> easily copy the unminified script via scp.
> 
> This is a _massive_ downside IMO, and absolutely not something
> that should be enabled by default. There's already far too much
> undocumented/underdocumented internal behaviours of scripts and
> processes in LEDE/OpenWrt, and minifying scripts and stripping
> comments actively works to preserve that situation.

Sorry I think I have formulated it wrong.

Inside the git we can documented the code and while the compile
process the comments will drop from all shellscrips who run the
calldef in its makefile.

so only on the finish compiled images are the minified scripts.

and of cause this is optionaly and looks like this [0]. So you
can just call ist inside the package Makefile.

[0] https://lists.ffnw.de//pipermail/dev/2016-July/001444.html

cheers
Tarek



signature.asc
Description: OpenPGP digital signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Piotr Dymacz]

Hello,

2016-10-03 13:14 GMT+02:00 Karl Palsson :
>
> Jan-Tarek Butt  wrote:
[snip]
>> 1. Reducing memory size on firmware images.
>
> But will it? They're in the squashfs image, it's already been
> demonstrated before that compressing things before can actually
> have negative impacts.

That's true, my vote for 'optimization-only' approach and keep fs do
the rest (compression).

>> 2. Strip out all comments (which makes us able to do better
>> code commenting) 3. and so on.

IMHO, we could remove all comments from all scripts (not only Lua) in
target rootfs and work on better documentation outside the code.

As an example of a huge, old comment, left in one of ar71xx base-files
scripts, please see [1].
It's inside every ar71xx image... does it really make sense to have it
there, for devices without related hardware?

>> One negative point will be there:
>>
>> the minified code an not realy human readable but if some one
>> want to read this script or work on it on routers, they can
>> easily copy the unminified script via scp.
>
> This is a _massive_ downside IMO, and absolutely not something
> that should be enabled by default. There's already far too much
> undocumented/underdocumented internal behaviours of scripts and
> processes in LEDE/OpenWrt, and minifying scripts and stripping
> comments actively works to preserve that situation.
[snip]

+1 for keeping scripts human readable.
+10 for getting scripts documented, someday, somewhere... Wiki?

[1] https://goo.gl/yQFpDV

Cheers,
Piotr

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Call defines for minifying scripting languages

[Karl Palsson]

Jan-Tarek Butt  wrote:
> Hi,
> 
> My Idea ist to create calldefs to minify all scriptes there are
> interpreted by run time.
> 
> As example see the mail "[PATCH] LuaSrcDiet call define for lua
> code Minifying".
> 
> I plan to write also calldefs for shellscripts, python and
> perl.
> 
> They brings vareous of features.
> 
> 1. Reducing memory size on firmware images.

But will it? They're in the squashfs image, it's already been
demonstrated before that compressing things before can actually
have negative impacts.

> 2. Strip out all comments (which makes us able to do better
> code commenting) 3. and so on.
> 
> One negative point will be there:
> 
> the minified code an not realy human readable but if some one
> want to read this script or work on it on routers, they can
> easily copy the unminified script via scp.

This is a _massive_ downside IMO, and absolutely not something
that should be enabled by default. There's already far too much
undocumented/underdocumented internal behaviours of scripts and
processes in LEDE/OpenWrt, and minifying scripts and stripping
comments actively works to preserve that situation.

Sincerely,
Karl Palsson

signature.asc
Description: OpenPGP Digital Signature
___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev