i just want to reiterate the point Alex Shroyer made about k dictionaries i found them extremely useful and upon occasion groused about not having them in J
i used them to represent an (English) dictionary very facile efficient and fast the fact is our world is (mostly) filled with ragged objects ... not the regular arrays that J folk so love eg HTML language dictionaries, encyclopedias... outlines... ~greg heil picsrp.github.io i.tgu.ca/real_cal -- from: Henry Rich <henryhr...@gmail.com> to: programm...@jsoftware.com date: Jan 31, 2022, 10:43 AM subject: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022 >Quite right. Jan-Pieter has at least a good start on the model. What I am >suggesting is that the 'more integrated implementation' might not need to be >much more integrated than the J script. Searching and sorting are already J >primitives. >The deficiency in J is that m&i. gives you a hashtable for searching m, but if >you modify m you have to recalculate the hash from scratch. This makes m&i. a >good read-mostly Dictionary but slow for frequent updates. If we can refine >the model to the point that such inefficiency is the worst problem we have, a >little integrated support could finish the package. Henry Rich -- from: Eric Iverson <eric.b.iver...@gmail.com> to: Programming forum <programm...@jsoftware.com> date: Jan 31, 2022, 10:37 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >A good and complete model is the first step. Then it is a matter of how much >it is used and what drawbacks it might have that would be addressed by a more >integrated implementation. -- from: Henry Rich <henryhr...@gmail.com> to: programm...@jsoftware.com date: Jan 31, 2022, 10:27 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >I have looked into this quite a bit. I am not convinced that Dictionary is a >fundamental datatype like number or letter, or that the current symbol support >is deficient. That makes the first questions What is a Dictionary? and Where >can a Dictionary be used in J? >The use case that would be important to me is a key-value store, aka >associative memory, where the Dictionary remember keys and values and later >returns a value when presented with the key. This feels to me like a package >written in J rather than a J primitive. A Dictionary would be a numbered >locale. The place where I could see added support inside J would be in adding >to and deleting from hashtables, specifically ones created/used by m&i. . >I invite proposals on what a Dictionary package needs to do. I have done this >before, and Jan-Pieter Jacobs responded with a package. Quoting from his >message of 20210327: *** QUOTE install'github:jpjacobs/types_dict@main' >(except on (my) android, where github installs don't seem to work, neither in >the Gui version JA nor on the commandline via termux) >It is pretty simple in use, you just use: d=: dict keys;vals for creating the >dictionary, which is a OOP object, having the following methods: get, set, >map, sort, destroy, whose documentation is contained in help_pdict_ >For performance, the create verb precomputes the lookup for the get verb, both >forward get (key->value) and backward get inv (value->first key) upon object >creation. >*** END QUOTE >I have not looked into this package, but it seems to me to have the right >entry points. Can we take that as a starting point to see what needs to be >added? The first step of course would be to put the addon under Package >Manager. Henry Rich -- from: Elijah Stone <elro...@elronnd.net> to: programm...@jsoftware.com date: Jan 31, 2022, 9:58 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >I agree with Eric regarding the challenges of adding dictionaries. >One issue: I think a necessary prerequisite is improved symbols. -- from: Eric Iverson <eric.b.iver...@gmail.com> to: Programming forum <programm...@jsoftware.com> date: Jan 31, 2022, 8:36 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >It seems to me that the big distinction between J and python where J falls far >short is not in the core language but in the addon libraries. I encourage J >fans to work away at adding important addons to J! -- from: Eric Iverson <eric.b.iver...@gmail.com> to: Programming forum <programm...@jsoftware.com> date: Jan 31, 2022, 8:28 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 -- >A J dictionary type is a good idea and it has been kicked around many times >before. The hard part is not implementing it, but doing a careful and thorough >spec that is backed by a complete model written in J. >Presented with a full spec that had community buy in, would probably be >followed by implementation. -- from: Alex Shroyer <ashro...@gmail.com> to: programm...@jsoftware.com date: Jan 31, 2022, 8:14 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 -- >agree with Raoul that competing with Python is not a good idea. But J can >learn from Python's decisions (good and bad) to grow and improve. In my >opinion, numpy is Python's "killer app" because it brings reasonable >performance without much conceptual overhead. The feature of Python that >enabled numpy is its extensibility, down to the C layer. There are other good features of Python that J could copy, in particular the 'dictionary' data type. The array language K has dictionaries, so J might take some inspiration from there for integrating them into the language. Cheers, Alex -- from: Raul Miller <rauldmil...@gmail.com> to: programm...@jsoftware.com date: Jan 31, 2022, 7:02 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 -- What does "competing with python" mean? >(I suspect that we must "compete" in some contexts, but I also imagine that >"competition" would be irrelevant in many, many contexts. But.. as a >programming exercise, I don't know how to conceptualize these distinctions.) Thanks, -- from: Raoul Schorer <raoul.scho...@gmail.com> to: programm...@jsoftware.com date: Jan 31, 2022, 2:30 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >Just my 2c, but I think that competing with python in general is somewhat >delusional. I think the key point for expanding J use to have a "killer J >app". For example, an improved clone of or excellent plugin for VisiData ( >https://www.visidata.org/) is my idea of a killer app. But someone here may >have a better idea? >Cheers, Raoul >-- from: Ric Sherlock <tikk...@gmail.com> to: Programming JForum <programm...@jsoftware.com> date: Jan 30, 2022, 6:50 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >Yes from a data structure point of view, inverted tables get you a lot of the >way (note they're also available in the 'general/misc' addon - load >'general/misc/inverted' ) and I've used them to good effect in my >'data/struct' addon (https://github.com/tikkanz/data_struct). I agree that J's >arrays are more general, flexible & powerful. But when you're dealing with a >tabular data set there is an overhead to keeping the fields in a table in sync >that dataframes can help with. Perhaps it's something abstracting the >structure so you don't have to deal so much with the mechanics of manipulating >it? At least for me :) -- from: Elijah Stone <elro...@elronnd.net> to: Programming JForum <programm...@jsoftware.com> date: Jan 30, 2022, 6:29 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 -- https://code.jsoftware.com/wiki/Essays/Inverted_Table, perhaps? >That said, I think a great strength of j is that data are _not_ explicitly >tabular. The associations are defined in an ad-hoc manner as needed by the >programmer. This is also an essential difference between the array paradigm >and the relational paradigm (cf SQL): in the former, pointers come with >implicit context; in the latter, that context must be explicit. -E >P.S. regarding analysis/optimization: I would love to see it, but for some >reason everybody is scared of building a compiler because of the parsing >problem. -- from: Ric Sherlock <tikk...@gmail.com> to: Programming JForum <programm...@jsoftware.com> date: Jan 30, 2022, 6:21 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >Yes, I've been thinking that a Dataframes equivalent in J would be useful. >Most things are already possible with J's arrays, but conceptually DataFrames >are well understood by many now, and they make it easy to work with datasets >as named fields. I've spent a reasonable amount of time working with Pandas, >but have recently been using Polars (Rust backend with Python bindings) which >really shines for larger datasets. Performance (especially read/write) is >awesome, and the LazyFrames which optimise your query/analysis plan make a big >difference too. I haven't taken enough time to explore it, but maybe Jd is the >starting point in this space for J? -- from: Michail L. Liarmakopoulos <m.l.liarm...@gmail.com> to: programm...@jsoftware.com date: Jan 30, 2022, 12:06 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 Hello Elijah, >Yes, I believe you're right. I don't have much experience with them yet >though, so I cannot compare. Best, --- Michail L. Liarmakopoulos, MSc -- from: Elijah Stone <elro...@elronnd.net> to: programm...@jsoftware.com date: Jan 30, 2022, 12:04 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 -- On Sun, 30 Jan 2022, Michail L. Liarmakopoulos wrote: >> There is also out there an amazing python library for symbolic calculations >> (like the ones you can do with Mathematica or WolframAlpha: symbolic >> evaluation of definite and indefinite integrals, symbolic solutions of diff. >> equations, symbolic solutions of algebraic and Diophantine equations >> etc...). It's called sympy. >> But I'm not sure if you'd like J to include symbolic computations too or if >> the aim of the language is to excel only in numerics, data analytics, stats, >> whatever can be quantified pretty much. Aren't there some symbolic calculus library functions? >(Though I doubt they're anywhere near the level of mathematica, let alone >sympy.) -- from: Michail L. Liarmakopoulos <m.l.liarm...@gmail.com> to: programm...@jsoftware.com date: Jan 30, 2022, 12:01 PM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 Hello all, >I find any parallels between python and J pretty interesting, being a person >with some python experience and an interest of the applications of both python >and J in mathematical modelling, analytics, computational math and perhaps >computational physics too. >If you'd like to bring some features from the python math/analytics >libraries/ecosystem in J, I'd suggest you to look at the features of three >libraries: >- numpy (I believe most features are already covered from the built in >features of an array language such as J) - pandas ( a nice library for manipulating csv files within python as dataframe objects -- see the dataframes from the R language) >- scipy (a collection of methods and functions ranging from solving >numerically: differential equations, evaluating definite integrals, >constrained and unconstrained optimization, and I believe statistics too) >There is also out there an amazing python library for symbolic calculations >(like the ones you can do with Mathematica or WolframAlpha: symbolic >evaluation of definite and indefinite integrals, symbolic solutions of diff. >equations, symbolic solutions of algebraic and Diophantine equations etc...). >It's called sympy. >But I'm not sure if you'd like J to include symbolic computations too or if >the aim of the language is to excel only in numerics, data analytics, stats, >whatever can be quantified pretty much. >My few cents as food for thought. Best regards, --- Michail L. Liarmakopoulos, MSc -- from: R.E. Boss <r.e.b...@outlook.com> to: "programm...@jsoftware.com" <programm...@jsoftware.com> date: Jan 30, 2022, 11:39 AM subject: Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022 >I copied the first chapter of the book A Journey to Core Python (in >https://drive.google.com/file/d/1p1uIANh-LFniNNRqjDeeWWd4_-ddEZmz/view?usp=sharing) > and have the question: do we want that J is competitive with Python? >If the answer is yes, the next question is: what is the to do list to be >competitive and how long will it take? >(And then the unavoidable question: WHY?) >Personally I think we must aim on the niches in the market, as there are the >mathematical oriented people, e.g. the broad scientific community. >Then all people experienced in Excel or other spreadsheets or calculation >tools. >Schools and universities. Financial and statistical oriented people. >What we should do, IMHO, is >- to emphasize the strengths of J, >- to improve (considerably) the error handling of J, >- to admit the steep learning curve, >- to facilitate the use of mnemonics instead of primitives (I tried this >afternoon the primitives.ijs for half an hour, but was not capable of use any >mnemonic, not even with >https://code.jsoftware.com/wiki/Primitives_to_Mnemonics) >- to decide which of the features, benefits or applications (of Python) we >want J to have. >Just my 2 cents. >R.E. Boss ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
