Thanks to Raul and Brian!
tomb=:(^/ i.@>:)~ , NB. It works miracuously! Thank you!
Raul wrote: "It sort of smells like you were working with matrices designed for
inversion and polynomial fitting"
Well, I want to find n numbers such that their mean powers are equal to the
mean powers of my input data set. I have n nonlinear equations in n unknowns.
The trick is to use the symmetry of the power sums. (A power sum does not
depend on the order of the variables). So the n unknowns are roots in a
polynomial of degree n. The coefficients of this polynomial are solutions to n
linear equations in n unknowns. See Newton's identities
|
|
| |
Newton's identities
Let x1, ..., xn be variables, denote for k ≥ 1 by pk(x1, ..., xn) the k-th
power sum:
|
|
|
So yes, I compute the n*n-matrix m from the n power sums s, and solve the
linear equation m +/ .* e = s (by e=.(%.m)s), and the the nonlinear equation e
p. x = 0 (by p. e). It is not polynomial fitting. Thank you for your help and
for your interest!
Bo
Den lørdag den 22. august 2020 15.44.33 CEST skrev Raul Miller
<[email protected]>:
I don't really follow what you are doing here -- for example, I look
at stuff like 11 summary@hist i.11 and do not understand how that
result corresponds to "size of the dataset" -- but even without
knowing your intentions, I can point out that you can simplify tomb:
tomb=:(^/ i.@>:)~ ,
It sort of smells like you were working with matrices designed for
inversion and polynomial fitting, but that doesn't tell me much (if
anything) about your destination.
Good luck,
--
Raul
On Sat, Aug 22, 2020 at 5:35 AM 'Bo Jacoby' via Programming
<[email protected]> wrote:
>
> Thanks to Michael and to ethiejiesa!
> The program is now:
>
> tomb=.[ (^/ i.@>:)~ ,@] NB. Powers of a tombola
>
> hist=.]* ((^/&i. >:)~ #) NB. Powers of a histogram
> s=.(*#)@(}.%{.)@(+/) NB. summation
> f1=.$~2 # # NB. n*n matrix
> f2=.>:@i.@#,}: NB. insert S0, remove Sn
> f3=.*_1^i.@# NB. change signs
> f4=.|:|.!.0"0 1~-@i.@# NB. shift in zeroes
> m=.|:@f4@(f3"1@f2@|:@f1) NB. Matrix
> e=.1,(%.m) NB. solve lin.eqns.
> pol=._1 x: -@|.@>@{:@p.@|. NB. solve alg.eqn.
> hp=.x: :: ] NB. high precision
> summary=.pol@e@s@hp f. NB. complete program
> (I need the comma in tomb, so now it is smelly again.)
> Michael wrote: "Any clue as to the input to summary, or what you're
> summarising? Sorry if it's obvious!"
> No sir, it is not obvious if it is not obvious to you.
> Left input to summary@tomb and summary@hist is the size of the output dataset.
> Right input to summary@tomb is the input dataset described as a tombola. The
> same number may occur on many tickets. The order of the tickets is immaterial.
> Right input to summary@hist is the input dataset described as a histogram.
> the number of zeroes, the number of ones, the number og twos &c.
> Examples:
>
> 11 summary@tomb i.11
>
>
> 0 1 2 3 4 5 6 7 8 9 10
>
> 11 summary@tomb 11 NB. this wouldn't work without the comma in tomb
> 11 11 11 11 11 11 11 11 11 11 11
>
> 1 summary@tomb i.11 NB. mean value
> 5
>
> 2 summary@tomb i.11 NB. mean plus/minus std.dev.
> 1.83772 8.16228
>
>
> 3 summary@tomb i.11 NB. generalized into 3 numbers
>
> 1.12702 5 8.87298
>
>
>
> Explanations:
>
>
>
> 5 tomb i.11
>
> 1 0 0 0 0 0
>
> 1 1 1 1 1 1
>
> 1 2 4 8 16 32
>
> 1 3 9 27 81 243
>
> 1 4 16 64 256 1024
>
> 1 5 25 125 625 3125
>
> 1 6 36 216 1296 7776
>
> 1 7 49 343 2401 16807
>
> 1 8 64 512 4096 32768
>
> 1 9 81 729 6561 59049
>
> 1 10 100 1000 10000 100000
>
> +/ hp 5 tomb i.11
> 11 55 385 3025 25333 220825
>
> (}.%{.) +/ hp 5 tomb i.11
> 5 35 275 2303 20075
>
> (*#) (}.%{.) +/ hp 5 tomb i.11
> 25 175 1375 11515 100375
>
> s hp 5 tomb i.11
> 25 175 1375 11515 100375
>
> f1 s hp 5 tomb i.11
> 25 175 1375 11515 100375
>
> 25 175 1375 11515 100375
>
> 25 175 1375 11515 100375
>
> 25 175 1375 11515 100375
>
> 25 175 1375 11515 100375
>
> |: f1 s hp 5 tomb i.11
> 25 25 25 25 25
>
> 175 175 175 175 175
>
> 1375 1375 1375 1375 1375
>
> 11515 11515 11515 11515 11515
>
> 100375 100375 100375 100375 100375
>
> f2 |: f1 s hp 5 tomb i.11
> 1 2 3 4 5
>
> 25 25 25 25 25
>
> 175 175 175 175 175
>
> 1375 1375 1375 1375 1375
>
> 11515 11515 11515 11515 11515
>
> f3"1 f2 |: f1 s hp 5 tomb i.11
> 1 _2 3 _4 5
>
> 25 _25 25 _25 25
>
> 175 _175 175 _175 175
>
> 1375 _1375 1375 _1375 1375
>
> 11515 _11515 11515 _11515 11515
>
> f4 f3"1 f2 |: f1 s hp 5 tomb i.11
> 1 25 175 1375 11515
>
> 0 _2 _25 _175 _1375
>
> 0 0 3 25 175
>
> 0 0 0 _4 _25
>
> 0 0 0 0 5
>
> |: f4 f3"1 f2 |: f1 s hp 5 tomb i.11
> 1 0 0 0 0
>
> 25 _2 0 0 0
>
> 175 _25 3 0 0
>
> 1375 _175 25 _4 0
>
> 11515 _1375 175 _25 5
>
> m s hp 5 tomb i.11
> 1 0 0 0 0
>
> 25 _2 0 0 0
>
> 175 _25 3 0 0
>
> 1375 _175 25 _4 0
>
> 11515 _1375 175 _25 5
>
> (%.m) s hp 5 tomb i.11
> 25 225 875 1340 450
>
> |.1,(%.m) s 5 tomb i.11
> 450 1340 875 225 25 1
>
> p.|.1,(%.m) s hp 5 tomb i.11
> ┌─┬─────────────────────────────────────┐
>
> │1│_9.54306 _7.0882 _5 _2.9118 _0.456938│
>
> └─┴─────────────────────────────────────┘
>
> _1 x: - |. > {: p.|.1,(%.m) s hp 5 tomb i.11
> 0.456938 2.9118 5 7.0882 9.54306
>
> 5 summary@tomb i.11
> 0.456938 2.9118 5 7.0882 9.54306
>
>
>
> Thanks!
> Bo. Den fredag den 21. august 2020 17.01.49 CEST skrev 'Michael Day' via
> Programming <[email protected]>:
>
> Any clue as to the input to summary, or what you're summarising?
>
> Sorry if it's obvious!
>
> Note these (display ?) errors:
>
> pol =. _1 [space] x: ...
>
> hp =. x [space] :: ] ...
>
> Cheers,
>
> Mike
>
>
> On 21/08/2020 15:41, 'Bo Jacoby' via Programming wrote:
> >
> > tomb=.(^/i.@>:)~ NB. tombola lottery powers
> > hist=.({."1~>:)~(*[:^/~i.@#@,)NB. histogram powers
> > s=.(*#)@ (}.%{.) @ (+/) NB. summation
> > f1=.$~2# # NB. n*n matrix
> > f2=.>:@i.@#, }: NB. insert S0, remove Sn
> > f3=.*_1^i.@# NB. change sign
> > f4=.|:|.!.0"01~-@i.@# NB. shift zeroes in
> > m=.|:@f4@(f3"1@f2@|:@f1) NB. matrix
> > e=.1,(%.m) NB. solve linear equations
> > pol=._1x: -@|.@>@{:@p.@|. NB.solve alg.eqn.
> > hp=.x::: ] NB. high precision
> > summary=.pol@e@s@hp f. NB. complete program
> > sorry for the still missing carriage returns.
> > Den fredag den 21. august 2020 16.37.19 CEST skrev Bo Jacoby
> ><[email protected]>:
> >
> > tomb=.(^/i.@>:)~ NB. tombola lottery
> >powershist=.({."1~>:)~(*[:^/~i.@#@,)NB. histogram powerss=.(*#)@ (}.%{.) @
> >(+/) NB. summationf1=.$~2# # NB. n*n matrixf2=.>:@i.@#, }: NB. insert S0,
> >remove Snf3=.*_1^i.@# NB. change signf4=.|:|.!.0"01~-@i.@# NB. shift zeroes
> >inm=.|:@f4@(f3"1@f2@|:@f1) NB. matrixe=.1,(%.m) NB. solve linear
> >equationspol=._1x: -@|.@>@{:@p.@|. NB.solve alg.eqn.hp=.x::: ] NB. high
> >precisionsummary=.pol@e@s@hp f. NB. complete program
> > This should be better. Sorry.
> > Bo. Den fredag den 21. august 2020 16.29.35 CEST skrev 'Bo Jacoby' via
> > Programming <[email protected]>:
> >
> > Thank you!
> > I was trying to accomplish this
> > tomb=.(^/i.@>:)~ NB. tombola lottery
> > powershist=.({."1~>:)~(*[:^/~i.@#@,)NB. histogram powerss=.(*#)@ (}.%{.) @
> > (+/) NB. summationf1=.$~2# # NB. n*n matrixf2=.>:@i.@#, }: NB. indsæt S0,
> > fjern Snf3=.*_1^i.@# NB. change signf4=.|:|.!.0"01~-@i.@# NB. shift zeroes
> > inm=.|:@f4@(f3"1@f2@|:@f1) NB. matrixe=.1,(%.m) NB. solve linear
> > equationspol=._1x: -@|.@>@{:@p.@|. NB.solve alg.eqn.hp=.x::: ] NB. high
> > precisionsummary=.pol@e@s@hpf. NB. complete program
> > 1&(summary@tomb) computes the mean value of a dataset.
> >
> > 1 summary@tomb i.11
> > 5
> >
> >
> > The mean value of 99 zeroes and 1 one is
> >
> > 1 summary@hist 99 1
> >
> > 0.01
> >
> >
> >
> > The mean value plus/minus the standard deviation is computed like this
> >
> > 2 summary@tomb i.11
> >
> > 1.83772 8.16228
> >
> > 2 summary@hist 99 1
> > _0.0894987 0.109499
> >
> >
> >
> > Of course these two numbers have the correct mean value
> >
> > 1 summary@tomb 2 summary@tomb i.11
> >
> > 5
> >
> > 1 summary@tomb 2 summary@hist 99 1
> > 0.01
> >
> >
> >
> > The computation is generalized to 3 numbers
> >
> > 3 summary@tomb i.11
> >
> > 1.12702 5 8.87298
> >
> > 3 summary@hist 99 1
> > _0.108204j0.16452 _0.108204j_0.16452 0.246409
> >
> > having the correct mean values and standarddeviations.
> >
> > 2 summary@tomb 3 summary@tomb i.11
> >
> > 1.83772 8.16228
> >
> > 2 summary@tomb 3 summary@hist 99 1
> > _0.0894987 0.109499
> >
> >
> >
> > Thus complicated probability distributions functions are summarized by a
> > few numbers. That is what I accomplished!
> > Thanks!
> > Bo.
> > Den fredag den 21. august 2020 15.33.50 CEST skrev ethiejiesa via
> >Programming <[email protected]>:
> >
> > Bo Jacoby <[email protected]> wrote:
> >> How to de-smell this:
> >> 3([* i.@#@,@[ ^/ i.@>:@])~ 0 0 1 1 1
> > 3 (]* ((^/&i. >:)~ #)) 0 0 1 1 1
> >
> > Maybe? That comes from just a little mechanical algebra:
> >
> >> 3([* i.@#@,@[ ^/ i.@>:@])~ 0 0 1 1 1
> > First notice the repeated (i.) on either argument of (^/). There is a
> > general
> > pattern, (f@u v f@w <--> u v&f w), which in this case specializes to
> >
> > 3([* #@,@[ ^/&i. >:@])~ 0 0 1 1 1
> >
> > Then, at least in this example the (@,) is superfluous, so we elide it:
> >
> > 3([* #@[ ^/&i. >:@])~ 0 0 1 1 1
> >
> > Finally, here is an idiom: ( (u@[ v w@]) <--> ((v w)~ u)~ ), and since
> > (x v~~ y <--> x v y), we have
> >
> > 3 (]* ((^/&i. >:)~ #)) 0 0 1 1 1
> >
> > The last idiom is really a matter of preference. I like the bare verbs, but
> > code golf-wise it doesn't win.
> >
> > More importantly, however, I really have no idea what you are trying to
> > accomplish. The above just performs some J syntax manipulations; however,
> > this
> > doesn't at all demonstrate the main point of avoiding code smell as I see
> > it,
> > which is more about finding places to improve the *algorithm* rather than
> > only
> > cleaning up the syntax.
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
>
> --
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
