Re: [Bug-apl] Spell corrector - APL

[Juergen Sauermann]

Hi Ala'a In GNU APL every cell of a value takes 20 bytes. In your example in function hist, ∪⍵ is probably the entire alphabet with 26 characters and therefore the outer product has 26×6.2 Mio = 161,200,000 cells or 3.2 Gigabyte. Depending on your OS this will give you a WS full or the machine will start swapping. The nature of the underlying problem seems to be fairly sequential, so using the outer product (and reducing it right after having created it) may not be the best way of achieving the desired result. /// Jürgen On 09/09/2016 11:39 PM, Ala'a Mohammad wrote: Hi, I'm trying to create simple spell corrector (Norvig at http://norvig.com/spell-correct.html) in APL. I tried but stumbled upon the frequency/count stage and could not move further. The stopper was either WS Full, or apl process killed. I'm assuming the main issue is 'lack of experience with APL', and thus the inefficient coding. ftxt ← { ⎕FIO[26] ⍵ } a ← 'abcdefghijklmnopqrstuvwxyz' A ← 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' downcase ← { (a,⎕AV)[(A,⎕AV)⍳⍵] } nl ← ⎕UCS 13 cr ← ⎕UCS 10 tab ← ⎕UCS 9 nonalpha ← nl, cr, tab, ' 0123456789()[]!?%$,.:;/+*=<>-_#"`~@&' alphamask ← { ~ ⍵ ∊ nonalpha } hist ← { (⍪∪⍵),+/∨/¨(∪⍵)∘.⍷⍵ } fhist ← { hist (alphamask txt) ⊂ downcase txt ← ftxt ⍵ } ⍝ file ← '/misc/small.txt' ~ 28K ⍝ file ← '/misc/xaa' ~ 1.3M file ← '/misc/big.txt' ⍝ ~ 6.2M ⍝ following 2 lines for debugging ⎕ ← ⍴w ← (alphamask txt) ⊂ downcase txt ← ftxt file ⎕ ← ⍴u ← ∪w fhist file the errors happened inside 'hist' function, and I presume mostly due to the jot dot find (if understand correctly, operating on a matrix of length equal to : unique-length * words-length) Is there anyway to fix the issue? and then proceed to complete the solution. Also, Is this the way to create simple spell corrector in APL (that is a one which is capitalizing on APL strength as an array language)? I'm using LinuxMint 17.1 (kernel 3.13.0-37-generic #64-Ubuntu) Gnu APL 1.6 (794) Zsch 5.0.2 Emacs 25.1.50.1 Best, Ala'a P.S: I hoped that I could create the solution in APL and then get some wacks on the head from fellow experienced APL programmers before submitting it as 'another solution in X language'. but the hope stopped short before even getting the probability stage.