Re: future?
Thanks for your comments and insights, Jakob. I just want to shortly assure you that I really liked your emails and your participation. I didn't answer/comment further due lack of time. Because of recent events (2017-03-01) I want to come back to this part: > Yes, but Google Code was no Github. Github is the singularity of > open source development. (As much as I dislike FB but hear me out...) Sidenote: Before GitHub, sourceforge.net was the singular main hoster for FOSS sourcecode. They lost their status when they bundled the downloads with adware (and malware). Always the same story: Nice little StartUp offers a comfortable, gratis service. Everyone starts using it. Then either the StartUp fails and the service gets shut down, or it gets sold to investors who want to see more profit with every year. Then the terms of service get worse, the prices go up, etc., e.g. see BitKeeper vs. Linux (the reason we have git now). GitHub: On 2017-03-01 GitHub introduced new Terms of Service (which you automatically accept by continueing to use their service) which demands from uploaders to grant GitHub a license. This is legal self-defense for GitHub, not unusual or bad in itself, but they further specifiy this license, which e.g. includes the uploader to waive any and all attribution rights. Basically, when uploading FOSS-licensed content to GitHub, the uploader automatically grants GitHub a special license, so actually a case of dual-licensing. The problem now is, that when the content is copyrighted by other authors beside the uploader, then the uploader cannot (in usual cases) actually grant this license legally to GitHub, as most FOSS-licenses forbid sublicensing and/or removal of the original attributions. Some projects already moved away from GitHub because of this. https://www.mirbsd.org/permalinks/wlog-10_e20170301-tg.htm http://joeyh.name/blog/entry/removing_everything_from_github/ I don't believe GitHub will take big damage because of this, especially as there is no serious contender on the rise. But it's surely a sign that GitHub is not a singularity from the usual development of such things. It's not an idealistic group of nerds anymore, but a big company (e.g. lookup the "github meritocracy" drama). For the other points mentioned by you and others about picolisp: Just do it. PicoLisp is MIT-licensed. Action speaks louder than words. As Lindsay wrote: > What would be great is to see more of an ecosystem built around the > picolisp core. Build something awesome with picolisp, document it and share > it with the world. This! I'm on it. /beneroth
Re: Arithmetic Coding
Hi Alex, Thank you. The (profile) tool is exactly what was needed here In the scaling loops, it made more of a difference than I thought to copy object properties into local variables. However, the idx tree change you pointed out is what helped most. : (prog (setq Msg (make (do (** 2 16) (link (rand (char "A") (char "Z")) (length Msg)) -> 65536 : (bench (/ (length (ACDC_Compress Msg)) 8)) 2.761 sec -> 38785 : (bench (= Msg (ACDC_Decompress (ACDC_Compress Msg 5.478 sec -> T Initially those numbers were larger by a factor of 5 I am sure there is lot more I will find to optimize as a I explore a bit more. /Lindsay # 1Mb list : (prog (setq Msg (make (do (** 2 20) (link (rand (char "A") (char "Z")) (length Msg)) -> 1048576 : (bench (= Msg (ACDC_Decompress (ACDC_Compress Msg 73.807 sec -> T : (mapc prof ...-> NIL : (bench (/ (length (ACDC_Compress Msg)) 8)) 36.574 sec -> 616497 : (profile) (2492 217 computeLower> . +ACDC_BasicModel) (602 48 emit> . +ACDC_BasicModel) (137 10 ACDC_Compress) (129 8 update> . +ACDC_BasicModel) (0 0 emitEof> . +ACDC_BasicModel) -> (0 0 emitEof> . +ACDC_BasicModel)
Re: Arithmetic Coding
Hi Lindsay, > As part of getting a handle on OO in picolisp I implemented a basic > arithmetic coder. > This is just the Coder and a simple adaptive model. > > https://github.com/thinknlive/picolisp-acdc Cool! Looks good (Though I didn't try to understand the details). > It is slower than I expected it to be, but I was focused more on getting > the implementation correct in this pass. > > Any optimization suggestions welcome. Maybe not use OO at all and leverage > namespaces. Though the method lookup in OO slows things down theoretically, I don't think it matters here. : (bench (ACDC_Compress (need (** 2 16) (char "A" 2.717 sec -> (0 1 0 0 0 0 0 0 1 ... I should explain that there is a way in PicoLisp to profile programs. This is not really documented, but the 'prof' function in @lib/prof.l is analog to 'debug' or 'trace'. You call it on the functions and methods you are interested in. Because 'ACDC_Compress' calls 'emit>', 'update> etc. I did: (load "@lib/prof.l") (mapc prof (quote ACDC_Compress (emit> . +ACDC_BasicModel) (update> . +ACDC_BasicModel) (emitEof> . +ACDC_BasicModel) (computeLower> . +ACDC_BasicModel) ) ) : (bench (ACDC_Compress (need (** 2 16) (char "A" T 2.703 sec -> T and then call 'profile': : (profile) (384 13 computeLower> . +ACDC_BasicModel) (114 1 emit> . +ACDC_BasicModel) (17 1 update> . +ACDC_BasicModel) (9 0 ACDC_Compress) (0 0 emitEof> . +ACDC_BasicModel) -> (0 0 emitEof> . +ACDC_BasicModel) The numbers in the CAR indicate ticks (see (doc 'ticks)) Now we know that 'computeLower>' takes most of the time. As this method does massive idx lookups, the efficiency of that tree comes to mind. I stopped at a breakpoint, and did ! (cons (depth (: counts)) @@) -> ((130 . 37) . 258) and notice that this tree is not optimally balanced. Looking at the 'T' method, I see that (let (Tmp NIL) # use idx tree for model probabilities (for X (: max) (idx 'Tmp (list (dec X) 1) T)) (balance 'Tmp Tmp) (=: counts Tmp) ) the usage of 'balance' is not correct. 'balance' does not take an idx tree, but a sorted list. So I changed these lines to (balance (:: counts) (make (for X (: max) (link (list (dec X) 1 ) and ran it again: 1.364 sec (73 5 computeLower> . +ACDC_BasicModel) (48 4 emit> . +ACDC_BasicModel) (3 0 update> . +ACDC_BasicModel) (3 0 ACDC_Compress) (0 0 emitEof> . +ACDC_BasicModel) Doubled the speed :) Perhaps there are other bottlenecks? ♪♫ Alex -- UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
