Re: [Jprogramming] converting from 'floating' to 'rational'
Ian - could your display problem be solved by always formatting displays but retaining arbitrary internal precision? You probably already do this but thought I'd mention it because I just had to format a correlation matrix to show only two digits of precision, but was annoyed that my rounding fnc was showing me things like "0.01" and "0.2001" when I rediscovered "8!:0" and looked up how to format a number properly. On Wed, Mar 27, 2019 at 7:57 PM Don Kelly wrote: > Probalbly the main problem, as exemplified in the "sine(a) problem is > that a and sine a are the same within 10e-16 and the cosine is 1-7.5e-5 > but the display is only 6 figures and so the displayed result is rounded > to 1. as it is closer to 1 than 0.99. This is grade 8 > rounding.Increasing the numerical accuracy won't help but increasing the > display to say 12 digits might show something other (such as > 0.9993). > > Also, looking at the Tabula "church clock" somehow data which is of 2 > to 3 figure accuracy is represented as 6 figure print precision > accuracy. A height of 75 ft is of 2 figure accuracy or 75+/_ 0.5 lb > is being presented in display as 75.000 and calculated to a greater > number of digits internally. The is doesn't mean that 75 is 75.0 nor > 75.000 etc. Using 64 digit floating point simply reduces computer error > ((partly due to converting from base 2 to base 10) hich but doesn't give > more accuracy than is present in the input data (neither does hand > calculation). 1r3 will be, in decimal form 0.333 ad nauseum no > matter how you do it and somewhere at the limit of the computer and real > world, it is rounded at some point. > > Don Kelly > > On 2019-03-27 11:32 a.m., Ian Clark wrote: > > Raul wrote: > >> So I am curious about the examples driving your concern here. > > I'm in the throes of converting TABULA ( > > https://code.jsoftware.com/wiki/TABULA ) to work with rationals instead > of > > floats. Or more accurately, the engines CAL and UU which TABULA uses. > > > > No, I don't need to offer the speed of light to more than 10 significant > > digits. My motivation has little to do with performing a particular > > scientific calculation, and everything to do with offering a > > general-purpose tool and rubbing down the rough edges as they emerge. > It's > > a game of whack-the-rat. > > > > The imprecision of working with floating-point numbers shows up so often > > with TABULA that I haven't bothered to collect examples. In just about > > every session I hit an instance or ten. But it's worse when using the > tool > > to demonstrate technical principles to novices, rather than do mundane > > calculations, because in the latter case it would be used by an engineer > > well-versed in computers and the funny little ways of floating point, > > tolerant comparisons and binary-to-decimal conversion. But a novice is > apt > > to suffer a crisis of confidence when she sees (sin A)=1.23E-5 in the > same > > display as (cos A)=1. Even hardened physicists wince. > > > > Two areas stand out: > > • Infinitestimals, i.e. intermediate values which look like zero – and > > ought to be zero – but aren't. They act like grit in the works. > > • Backfitting, where the user overtypes a calculated value and CAL > backfits > > suitable input values, for which it mainly uses N-R algorithms – which > > don't like noisy values. > > > > It's too early to say yet – I have to finish the conversion and think up > > examples to stress-test it before I can be sure the effort is worthwhile. > > So far I've only upgraded UU (the units-conversion engine), but already > > some backfitting examples which were rather iffy are hitting the target > > spot-on: particularly where the slope of the "hill" being climbed is > nearly > > zero. Even I succumb to feelings of pleasure to see (sin A)=0 in the same > > display as (cos A)=1. > > > > But knowing the innards of CAL, I can't understand how it can possibly be > > showing benefits at this early stage. Perhaps UU's rational values are > > leaking further down the cascade of calculations than I expected? I'd > love > > to get to the bottom of it, but my systematic "rationalization" of the > CAL > > code will destroy the evidence, just as exploring Mars will destroy the > > evidence for indigenous life. Too bad: I'm not aiming at CAL working > > occasionally, but every time. > > > > Thanks for reminding me about digits separation. Yes, my numeral > converter > > (I find I'm mainly working with numerals than numeric atoms) can already > > handle standard scientific notation, like '6.62607015E-34' -- plus a few > > J-ish forms like '1p1'. I only had to type-in π to 50 places of decimals > to > > feel the need for some form of digit separation (…a good tool should > > support ALL forms!) e.g. '6.626,070,15E-34' but was unconsciously > assuming > > (y -. ',') would handle it. > > > > …It won't. SI specifies spaces as digit separators, and Germany uses > commas > > where
Re: [Jprogramming] converting from 'floating' to 'rational'
Probalbly the main problem, as exemplified in the "sine(a) problem is that a and sine a are the same within 10e-16 and the cosine is 1-7.5e-5 but the display is only 6 figures and so the displayed result is rounded to 1. as it is closer to 1 than 0.99. This is grade 8 rounding.Increasing the numerical accuracy won't help but increasing the display to say 12 digits might show something other (such as 0.9993). Also, looking at the Tabula "church clock" somehow data which is of 2 to 3 figure accuracy is represented as 6 figure print precision accuracy. A height of 75 ft is of 2 figure accuracy or 75+/_ 0.5 lb is being presented in display as 75.000 and calculated to a greater number of digits internally. The is doesn't mean that 75 is 75.0 nor 75.000 etc. Using 64 digit floating point simply reduces computer error ((partly due to converting from base 2 to base 10) hich but doesn't give more accuracy than is present in the input data (neither does hand calculation). 1r3 will be, in decimal form 0.333 ad nauseum no matter how you do it and somewhere at the limit of the computer and real world, it is rounded at some point. Don Kelly On 2019-03-27 11:32 a.m., Ian Clark wrote: Raul wrote: So I am curious about the examples driving your concern here. I'm in the throes of converting TABULA ( https://code.jsoftware.com/wiki/TABULA ) to work with rationals instead of floats. Or more accurately, the engines CAL and UU which TABULA uses. No, I don't need to offer the speed of light to more than 10 significant digits. My motivation has little to do with performing a particular scientific calculation, and everything to do with offering a general-purpose tool and rubbing down the rough edges as they emerge. It's a game of whack-the-rat. The imprecision of working with floating-point numbers shows up so often with TABULA that I haven't bothered to collect examples. In just about every session I hit an instance or ten. But it's worse when using the tool to demonstrate technical principles to novices, rather than do mundane calculations, because in the latter case it would be used by an engineer well-versed in computers and the funny little ways of floating point, tolerant comparisons and binary-to-decimal conversion. But a novice is apt to suffer a crisis of confidence when she sees (sin A)=1.23E-5 in the same display as (cos A)=1. Even hardened physicists wince. Two areas stand out: • Infinitestimals, i.e. intermediate values which look like zero – and ought to be zero – but aren't. They act like grit in the works. • Backfitting, where the user overtypes a calculated value and CAL backfits suitable input values, for which it mainly uses N-R algorithms – which don't like noisy values. It's too early to say yet – I have to finish the conversion and think up examples to stress-test it before I can be sure the effort is worthwhile. So far I've only upgraded UU (the units-conversion engine), but already some backfitting examples which were rather iffy are hitting the target spot-on: particularly where the slope of the "hill" being climbed is nearly zero. Even I succumb to feelings of pleasure to see (sin A)=0 in the same display as (cos A)=1. But knowing the innards of CAL, I can't understand how it can possibly be showing benefits at this early stage. Perhaps UU's rational values are leaking further down the cascade of calculations than I expected? I'd love to get to the bottom of it, but my systematic "rationalization" of the CAL code will destroy the evidence, just as exploring Mars will destroy the evidence for indigenous life. Too bad: I'm not aiming at CAL working occasionally, but every time. Thanks for reminding me about digits separation. Yes, my numeral converter (I find I'm mainly working with numerals than numeric atoms) can already handle standard scientific notation, like '6.62607015E-34' -- plus a few J-ish forms like '1p1'. I only had to type-in π to 50 places of decimals to feel the need for some form of digit separation (…a good tool should support ALL forms!) e.g. '6.626,070,15E-34' but was unconsciously assuming (y -. ',') would handle it. …It won't. SI specifies spaces as digit separators, and Germany uses commas where the UK and USA use dots, e.g. '6,626 070 15E-34'. Okay, fine… but in places I detect the first space in a (string) quantity to show where the numeral stops and the units begin. Ah well… another rat to whack. Ian On Wed, 27 Mar 2019 at 15:16, Raul Miller wrote: On Tue, Mar 26, 2019 at 7:38 PM Ian Clark wrote: I will still employ my "mickey-mouse" method, because it's easily checked once it's coded. I need built-into TABULA a number of physical constants which the SI defines exactly, e.g. • The thermodynamic temperature of the triple point of water, Ttpw , is 273.16 K *exactly*. • The speed of light in vacuo is 299792458 m/s *exactly*. The first I can generate and handle as: 27316r100 -whereas (x: 273.16) is 6829r25 . If you
Re: [Jprogramming] Working Directory and Changing Directory
I'm using the most recent version. I just called it ijx because that's what the intro book I was reading called it (it was either Learning J or the Primer). Thanks for the help. Kevin Wang On Wed, Mar 27, 2019 at 6:39 PM Devon McCormick wrote: > For quick and dirty execution of a J phrase from the command line, you can > use the "-js" option. So, if I want to time the inversion of a 1000 x 1000 > matrix in J, I could enter this on the command line: > e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 6!:2 '%.mat' > [ mat=. <:+:1000 1000?@$0" > The " 2!:55 '' " logs off the session after it outputs (smoutput) the > timing (6!:2) of the expression to invert the matrix "mat". > > Note that changing the directory in such a command-line string only affects > the J session, not the calling session, so, from my command line: > [Wed 03/27/2019 | 18:34:20.13 | E:\amisc\logs] > >e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 1!:43 '' [ > 1!:44 'C:/Temp'" > e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 1!:43 '' [ > 1!:44 'C:/Temp'" > C:\Temp > > [Wed 03/27/2019 | 18:34:45.08 | E:\amisc\logs] > > > Here I changed to "C:/Temp", showed that's my current directory, then > exited; you can see from my prompt that I remain in the original calling > directory after the J expression finished. > > > > On Wed, Mar 27, 2019 at 5:49 PM bill lam wrote: > > > strange that you mentioned ijx, which version of J are you using? > > > > On Thu, Mar 28, 2019, 5:25 AM Kevin Wang > wrote: > > > > > I am new to J and am trying to figure out how to find the directory my > > ijx > > > is running in and how to change it. I have looked in the Foreigns vocab > > > section and thought it would be better to ask now instead of scrambling > > all > > > over the wiki. > > > > > > Also, how can I run an ijs script file from the command line, either a > > > Windows command prompt or a Bash terminal? > > > > > > Thanks, > > > Kevin Wang > > > -- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > -- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > -- > > Devon McCormick, CFA > > Quantitative Consultant > -- > For information about J forums see http://www.jsoftware.com/forums.htm -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] Working Directory and Changing Directory
For quick and dirty execution of a J phrase from the command line, you can use the "-js" option. So, if I want to time the inversion of a 1000 x 1000 matrix in J, I could enter this on the command line: e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 6!:2 '%.mat' [ mat=. <:+:1000 1000?@$0" The " 2!:55 '' " logs off the session after it outputs (smoutput) the timing (6!:2) of the expression to invert the matrix "mat". Note that changing the directory in such a command-line string only affects the J session, not the calling session, so, from my command line: [Wed 03/27/2019 | 18:34:20.13 | E:\amisc\logs] >e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 1!:43 '' [ 1!:44 'C:/Temp'" e:/Users/DevonMcC/j64-807/bin/jconsole -js "2!:55'' [ smoutput 1!:43 '' [ 1!:44 'C:/Temp'" C:\Temp [Wed 03/27/2019 | 18:34:45.08 | E:\amisc\logs] > Here I changed to "C:/Temp", showed that's my current directory, then exited; you can see from my prompt that I remain in the original calling directory after the J expression finished. On Wed, Mar 27, 2019 at 5:49 PM bill lam wrote: > strange that you mentioned ijx, which version of J are you using? > > On Thu, Mar 28, 2019, 5:25 AM Kevin Wang wrote: > > > I am new to J and am trying to figure out how to find the directory my > ijx > > is running in and how to change it. I have looked in the Foreigns vocab > > section and thought it would be better to ask now instead of scrambling > all > > over the wiki. > > > > Also, how can I run an ijs script file from the command line, either a > > Windows command prompt or a Bash terminal? > > > > Thanks, > > Kevin Wang > > -- > > For information about J forums see http://www.jsoftware.com/forums.htm > -- > For information about J forums see http://www.jsoftware.com/forums.htm -- Devon McCormick, CFA Quantitative Consultant -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] Working Directory and Changing Directory
strange that you mentioned ijx, which version of J are you using? On Thu, Mar 28, 2019, 5:25 AM Kevin Wang wrote: > I am new to J and am trying to figure out how to find the directory my ijx > is running in and how to change it. I have looked in the Foreigns vocab > section and thought it would be better to ask now instead of scrambling all > over the wiki. > > Also, how can I run an ijs script file from the command line, either a > Windows command prompt or a Bash terminal? > > Thanks, > Kevin Wang > -- > For information about J forums see http://www.jsoftware.com/forums.htm -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] Working Directory and Changing Directory
I'd go with https://code.jsoftware.com/wiki/Guides/Standalone_Applications for running J from the command line. Also, 1!:43 queries current directory and 1!:44 changes it http://www.jsoftware.com/help/dictionary/dx001.htm -- those don't get much use, though, since usually it makes more sense to just use absolute references. Some shortcuts for absolute references include: jpath '~home' jpath '~user' jpath '~temp' Basically tilde expansions by jpath get you access to the paths listed in SystemFolders_j_ (and UserFolders_j_). I hope this helps, Thanks, -- Raul On Wed, Mar 27, 2019 at 5:25 PM Kevin Wang wrote: > > I am new to J and am trying to figure out how to find the directory my ijx > is running in and how to change it. I have looked in the Foreigns vocab > section and thought it would be better to ask now instead of scrambling all > over the wiki. > > Also, how can I run an ijs script file from the command line, either a > Windows command prompt or a Bash terminal? > > Thanks, > Kevin Wang > -- > For information about J forums see http://www.jsoftware.com/forums.htm -- For information about J forums see http://www.jsoftware.com/forums.htm
[Jprogramming] Working Directory and Changing Directory
I am new to J and am trying to figure out how to find the directory my ijx is running in and how to change it. I have looked in the Foreigns vocab section and thought it would be better to ask now instead of scrambling all over the wiki. Also, how can I run an ijs script file from the command line, either a Windows command prompt or a Bash terminal? Thanks, Kevin Wang -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] Server Maintenance
Done, and everything is back to normal. Chris On Wed, Mar 27, 2019 at 12:27 PM chris burke wrote: > I plan some maintenance work on the server starting in half an hour, 8pm > utc. During this time, pacman on Windows will not be available, but > everything else should continue working fine. > > The maintenance should only be for 20 minutes or so. I will advise when > things are back to normal. > > Chris > -- For information about J forums see http://www.jsoftware.com/forums.htm
[Jprogramming] Server Maintenance
I plan some maintenance work on the server starting in half an hour, 8pm utc. During this time, pacman on Windows will not be available, but everything else should continue working fine. The maintenance should only be for 20 minutes or so. I will advise when things are back to normal. Chris -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] converting from 'floating' to 'rational'
Raul wrote: > So I am curious about the examples driving your concern here. I'm in the throes of converting TABULA ( https://code.jsoftware.com/wiki/TABULA ) to work with rationals instead of floats. Or more accurately, the engines CAL and UU which TABULA uses. No, I don't need to offer the speed of light to more than 10 significant digits. My motivation has little to do with performing a particular scientific calculation, and everything to do with offering a general-purpose tool and rubbing down the rough edges as they emerge. It's a game of whack-the-rat. The imprecision of working with floating-point numbers shows up so often with TABULA that I haven't bothered to collect examples. In just about every session I hit an instance or ten. But it's worse when using the tool to demonstrate technical principles to novices, rather than do mundane calculations, because in the latter case it would be used by an engineer well-versed in computers and the funny little ways of floating point, tolerant comparisons and binary-to-decimal conversion. But a novice is apt to suffer a crisis of confidence when she sees (sin A)=1.23E-5 in the same display as (cos A)=1. Even hardened physicists wince. Two areas stand out: • Infinitestimals, i.e. intermediate values which look like zero – and ought to be zero – but aren't. They act like grit in the works. • Backfitting, where the user overtypes a calculated value and CAL backfits suitable input values, for which it mainly uses N-R algorithms – which don't like noisy values. It's too early to say yet – I have to finish the conversion and think up examples to stress-test it before I can be sure the effort is worthwhile. So far I've only upgraded UU (the units-conversion engine), but already some backfitting examples which were rather iffy are hitting the target spot-on: particularly where the slope of the "hill" being climbed is nearly zero. Even I succumb to feelings of pleasure to see (sin A)=0 in the same display as (cos A)=1. But knowing the innards of CAL, I can't understand how it can possibly be showing benefits at this early stage. Perhaps UU's rational values are leaking further down the cascade of calculations than I expected? I'd love to get to the bottom of it, but my systematic "rationalization" of the CAL code will destroy the evidence, just as exploring Mars will destroy the evidence for indigenous life. Too bad: I'm not aiming at CAL working occasionally, but every time. Thanks for reminding me about digits separation. Yes, my numeral converter (I find I'm mainly working with numerals than numeric atoms) can already handle standard scientific notation, like '6.62607015E-34' -- plus a few J-ish forms like '1p1'. I only had to type-in π to 50 places of decimals to feel the need for some form of digit separation (…a good tool should support ALL forms!) e.g. '6.626,070,15E-34' but was unconsciously assuming (y -. ',') would handle it. …It won't. SI specifies spaces as digit separators, and Germany uses commas where the UK and USA use dots, e.g. '6,626 070 15E-34'. Okay, fine… but in places I detect the first space in a (string) quantity to show where the numeral stops and the units begin. Ah well… another rat to whack. Ian On Wed, 27 Mar 2019 at 15:16, Raul Miller wrote: > On Tue, Mar 26, 2019 at 7:38 PM Ian Clark wrote: > > I will still employ my "mickey-mouse" method, because it's easily checked > > once it's coded. I need built-into TABULA a number of physical constants > > which the SI defines exactly, e.g. > > • The thermodynamic temperature of the triple point of water, Ttpw , is > > 273.16 K *exactly*. > > • The speed of light in vacuo is 299792458 m/s *exactly*. > > > > The first I can generate and handle as: 27316r100 -whereas (x: 273.16) > > is 6829r25 . If you multiply top and bottom by 4 you get my numeral. But > > (x:) will round decimal numerals with more than 15 sig figs and so lose > the > > exactness. > > I was looking at > https://en.m.wikipedia.org/wiki/2019_redefinition_of_SI_base_units but > I did not notice anything with more than 10 significant digits. So I > am curious about the examples driving your concern here. > > (That said, if you're going to go there, and you have not already done > so, I'd use your approach on strings, and I'd make it so that it would > work properly on values like '6.62607015e-34'. I might also be > tempted to make it accept group-of-three-digit representations to make > obvious typos stand out visually. Perhaps: '6.626 070 15e-34') > > Thanks, > > -- > Raul > -- > For information about J forums see http://www.jsoftware.com/forums.htm -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jprogramming] converting from 'floating' to 'rational'
On Tue, Mar 26, 2019 at 7:38 PM Ian Clark wrote: > I will still employ my "mickey-mouse" method, because it's easily checked > once it's coded. I need built-into TABULA a number of physical constants > which the SI defines exactly, e.g. > • The thermodynamic temperature of the triple point of water, Ttpw , is > 273.16 K *exactly*. > • The speed of light in vacuo is 299792458 m/s *exactly*. > > The first I can generate and handle as: 27316r100 -whereas (x: 273.16) > is 6829r25 . If you multiply top and bottom by 4 you get my numeral. But > (x:) will round decimal numerals with more than 15 sig figs and so lose the > exactness. I was looking at https://en.m.wikipedia.org/wiki/2019_redefinition_of_SI_base_units but I did not notice anything with more than 10 significant digits. So I am curious about the examples driving your concern here. (That said, if you're going to go there, and you have not already done so, I'd use your approach on strings, and I'd make it so that it would work properly on values like '6.62607015e-34'. I might also be tempted to make it accept group-of-three-digit representations to make obvious typos stand out visually. Perhaps: '6.626 070 15e-34') Thanks, -- Raul -- For information about J forums see http://www.jsoftware.com/forums.htm
