I hope you find me an idiot, but isn't that what a DBMS does? Maybe the
DBMS in J could be extended a little to do all this.
On Mon, Jan 31, 2022, 6:22 PM 'Pascal Jasmin' via Programming <
programm...@jsoftware.com> wrote:
> Beyond dictionaries is a "generic data structure" construct, and a large
> extended family of data structures that can respond to the following
> commands
>
> GET SET UPSERT APPEND DELETE INSERTafter(item) INSERTbefore(item) WHEREget
> CONVERTtoJnativearray UPSERTfromJnativearray
>
> in addition to dictionaries, ragged nested arrays or generally any binary
> encoding of datatype;shape/count;content
>
> for dictionaries, some applications (or at least language implementations)
> permit variant/inconsistent type keys and/or values.
>
> and generally, a table is a dictionary with
>
> multiple keys and multiple associated values to each unique combination of
> the multiple keys. With potential performance advantages to keeping it in
> a sorted order if retrieval of items by query, or having multiple fields
> indexed. Typed "tables"/dictionaries get inherent performance advantages
> due to not requiring item boxing.
>
> as a generic "pattern framework" for data structures without using classes:
>
> DEFAULT =: ''
>
> data dataconstructor DEFAULT NB. dataconstructor is conjunction that will
> return an adverb that is (conjunction data). example
>
> mydict = ('key1'; value1) dict DEFAULT NB. an adverb that is a
> conjunction bound to specific data, and where the conjunction code responds
> to commands in list above top.
>
> The list of commands near top of this message are actually numeric codes
> that get passed to the bound datastructure adverb.
>
> GET mydict becomes a verb that will retrieve items.
> SET mydict returns an conjunction with m as key, n as value that will
> return the same mydict adverb but with updated "data"
> (newkey; SET) mydict returns an adverb that is above conjunction with m
> bound as newkey
> (newkey: newdata; SET) mydict directly returns the updated mydict adverb.
>
> In the case of typed/consistent dictionaries, the internal data
> representation is just an inverted table of "fields". And while the
> underlying conjunction of mydict is bound to data to form an adverb, a
> generic (consistent typed) dictaccess conjunction can operate on any
> inverted table if prebound with GET/SET. And the GET/SET/OTHER "commands"
> can form a verb as a result of paired with a generic access adverb, all
> "command parameters" as x, and the inverted table/data structure as y.
>
> What makes this approach have more appeal to be over classes is besides
> issues like reference management and it is hard to send methods to
> class/objects in a "parameterized" way, is that arrays are J's
> first/primary datastructure, and in some datastructures can be
> transparently accessed by independent J code.
>
> Generally, putting the complexity in modifiers instead of classes is a
> functional approach I would prefer. I can volunteer to start it off at
> least if there is interest in this approach.
>
> It would be a datastructure access "language/dsl" on top of J.
>
>
>
>
> On Monday, January 31, 2022, 01:43:30 p.m. EST, Henry Rich <
> henryhr...@gmail.com> wrote:
>
>
>
>
>
> Quite right. Jan-Pieter has at least a good start on the model. What I
> am suggesting is that the 'more integrated implementation' might not
> need to be much more integrated than the J script. Searching and
> sorting are already J primitives.
>
> The deficiency in J is that m&i. gives you a hashtable for searching m,
> but if you modify m you have to recalculate the hash from scratch. This
> makes m&i. a good read-mostly Dictionary but slow for frequent updates.
> If we can refine the model to the point that such inefficiency is the
> worst problem we have, a little integrated support could finish the
> package.
>
> Henry Rich
>
> On 1/31/2022 1:37 PM, Eric Iverson wrote:
> > A good and complete model is the first step. Then it is a matter of how
> > much it is used and what drawbacks it might have that would be addressed
> by
> > a more integrated implementation.
> >
> > On Mon, Jan 31, 2022 at 1:27 PM Henry Rich <henryhr...@gmail.com> wrote:
> >
> >> I have looked into this quite a bit. I am not convinced that Dictionary
> >> is a fundamental datatype like number or letter, or that the current
> >> symbol support is deficient. That makes the first questions What is a
> >> Dictionary? and Where can a Dictionary be used in J?
> >>
> >> The use case that would be important to me is a key-value store, aka
> >> associative memory, where the Dictionary remember keys and values and
> >> later returns a value when presented with the key. This feels to me like
> >> a package written in J rather than a J primitive. A Dictionary would be
> >> a numbered locale. The place where I could see added support inside J
> >> would be in adding to and deleting from hashtables, specifically ones
> >> created/used by m&i. .
> >>
> >> I invite proposals on what a Dictionary package needs to do. I have
> >> done this before, and Jan-Pieter Jacobs responded with a package.
> >> Quoting from his message of 20210327:
> >>
> >> *** QUOTE
> >>
> >> install'github:jpjacobs/types_dict@main'
> >>
> >> (except on (my) android, where github installs don't seem to work,
> neither
> >> in the Gui version JA nor on the commandline via termux)
> >>
> >> It is pretty simple in use, you just use:
> >> d=: dict keys;vals
> >> for creating the dictionary, which is a OOP object, having the following
> >> methods:
> >> get, set, map, sort, destroy, whose documentation is contained in
> >> help_pdict_
> >>
> >> For performance, the create verb precomputes the lookup for the get
> verb,
> >> both forward get (key->value) and backward get inv (value->first key)
> upon
> >> object creation.
> >>
> >> *** END QUOTE
> >>
> >> I have not looked into this package, but it seems to me to have the
> >> right entry points. Can we take that as a starting point to see what
> >> needs to be added? The first step of course would be to put the addon
> >> under Package Manager.
> >>
> >> Henry Rich
> >>
> >>
> >>
> >> On 1/31/2022 12:58 PM, Elijah Stone wrote:
> >>> I agree with Eric regarding the challenges of adding dictionaries.
> >>>
> >>> One issue: I think a necessary prerequisite is improved symbols.
> >>>
> >>> On Mon, 31 Jan 2022, Alex Shroyer wrote:
> >>>
> >>>> I agree with Raoul that competing with Python is not a good idea.
> >>>> But J can learn from Python's decisions (good and bad) to grow and
> >>>> improve.
> >>>> In my opinion, numpy is Python's "killer app" because it brings
> >>>> reasonable
> >>>> performance without much conceptual overhead.
> >>>> The feature of Python that enabled numpy is its extensibility, down
> >>>> to the
> >>>> C layer.
> >>>>
> >>>> There are other good features of Python that J could copy, in
> particular
> >>>> the 'dictionary' data type.
> >>>> The array language K has dictionaries, so J might take some
> inspiration
> >>>> from there for integrating them into the language.
> >>>>
> >>>> Cheers,
> >>>> Alex
> >>>>
> >>>>
> >>>> On Mon, Jan 31, 2022 at 5:30 AM Raoul Schorer <
> raoul.scho...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Just my 2c, but I think that competing with python in general is
> >>>>> somewhat
> >>>>> delusional. I think the key point for expanding J use to have a
> >>>>> "killer J
> >>>>> app". For example, an improved clone of or excellent plugin for
> >>>>> VisiData (
> >>>>> https://www.visidata.org/) is my idea of a killer app. But someone
> >> here
> >>>>> may
> >>>>> have a better idea?
> >>>>>
> >>>>> Cheers,
> >>>>> Raoul
> >>>>>
> >>>>> Le lun. 31 janv. 2022 à 03:49, Ric Sherlock <tikk...@gmail.com> a
> >>>>> écrit :
> >>>>>
> >>>>>> Yes from a data structure point of view, inverted tables get you a
> >>>>> lot of
> >>>>>> the way (note they're also available in the 'general/misc' addon -
> >>>>> load
> >>>>>> 'general/misc/inverted' ) and I've used them to good effect in my
> >>>>>> 'data/struct' addon (https://github.com/tikkanz/data_struct).
> >>>>>> I agree that J's arrays are more general, flexible & powerful. But
> >>>>> when
> >>>>>> you're dealing with a tabular data set there is an overhead to
> >>>>> keeping
> >>>>> the
> >>>>>> fields in a table in sync that dataframes can help with. Perhaps
> it's
> >>>>>> something abstracting the structure so you don't have to deal so
> much
> >>>>> with
> >>>>>> the mechanics of manipulating it? At least for me :)
> >>>>>>
> >>>>>> On Mon, Jan 31, 2022 at 3:28 PM Elijah Stone <elro...@elronnd.net>
> >>>>> wrote:
> >>>>>>> https://code.jsoftware.com/wiki/Essays/Inverted_Table, perhaps?
> >>>>>>>
> >>>>>>> That said, I think a great strength of j is that data are _not_
> >>>>>> explicitly
> >>>>>>> tabular. The associations are defined in an ad-hoc manner as
> >>>>> needed by
> >>>>>>> the programmer. This is also an essential difference between
> >>>>> the array
> >>>>>>> paradigm and the relational paradigm (cf SQL): in the former,
> >>>>> pointers
> >>>>>>> come with implicit context; in the latter, that context must be
> >>>>> explicit.
> >>>>>>> -E
> >>>>>>>
> >>>>>>> P.S. regarding analysis/optimization: I would love to see it,
> >>>>> but for
> >>>>>> some
> >>>>>>> reason everybody is scared of building a compiler because of the
> >>>>> parsing
> >>>>>>> problem.
> >>>>>>>
> >>>>>>> On Mon, 31 Jan 2022, Ric Sherlock wrote:
> >>>>>>>
> >>>>>>>> Yes, I've been thinking that a Dataframes equivalent in J
> >>>>> would be
> >>>>>>> useful.
> >>>>>>>> Most things are already possible with J's arrays, but
> >>>>> conceptually
> >>>>>>>> DataFrames are well understood by many now, and they make it
> >>>>> easy to
> >>>>>> work
> >>>>>>>> with datasets as named fields.
> >>>>>>>> I've spent a reasonable amount of time working with Pandas,
> >>>>> but have
> >>>>>>>> recently been using Polars (Rust backend with Python bindings)
> >>>>> which
> >>>>>>> really
> >>>>>>>> shines for larger datasets. Performance (especially
> >>>>> read/write) is
> >>>>>>> awesome,
> >>>>>>>> and the LazyFrames which optimise your query/analysis plan
> >>>>> make a big
> >>>>>>>> difference too.
> >>>>>>>> I haven't taken enough time to explore it, but maybe Jd is the
> >>>>> starting
> >>>>>>>> point in this space for J?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Jan 31, 2022 at 9:01 AM Michail L. Liarmakopoulos <
> >>>>>>>> m.l.liarm...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hello all,
> >>>>>>>>>
> >>>>>>>>> I find any parallels between python and J pretty interesting,
> >>>>> being
> >>>>> a
> >>>>>>>>> person with some python experience and an interest of the
> >>>>> applications
> >>>>>>> of
> >>>>>>>>> both python and J in mathematical modelling, analytics,
> >>>>> computational
> >>>>>>> math
> >>>>>>>>> and perhaps computational physics too.
> >>>>>>>>>
> >>>>>>>>> If you'd like to bring some features from the python
> >>>>> math/analytics
> >>>>>>>>> libraries/ecosystem in J, I'd suggest you to look at the
> >>>>> features of
> >>>>>>> three
> >>>>>>>>> libraries:
> >>>>>>>>>
> >>>>>>>>> - numpy (I believe most features are already covered from the
> >>>>> built
> >>>>> in
> >>>>>>>>> features of an array language such as J)
> >>>>>>>>>
> >>>>>>>>> - pandas ( a nice library for manipulating csv files within
> >>>>> python
> >>>>> as
> >>>>>>>>> dataframe objects -- see the dataframes from the R language)
> >>>>>>>>>
> >>>>>>>>> - scipy (a collection of methods and functions ranging from
> >>>>> solving
> >>>>>>>>> numerically: differential equations, evaluating definite
> >>>>> integrals,
> >>>>>>>>> constrained and unconstrained optimization, and I believe
> >>>>> statistics
> >>>>>>> too)
> >>>>>>>>> There is also out there an amazing python library for symbolic
> >>>>>>> calculations
> >>>>>>>>> (like the ones you can do with Mathematica or WolframAlpha:
> >>>>> symbolic
> >>>>>>>>> evaluation of definite and indefinite integrals, symbolic
> >>>>> solutions
> >>>>> of
> >>>>>>>>> diff. equations, symbolic solutions of algebraic and Diophantine
> >>>>>>> equations
> >>>>>>>>> etc...). It's called sympy.
> >>>>>>>>>
> >>>>>>>>> But I'm not sure if you'd like J to include symbolic
> >>>>> computations
> >>>>> too
> >>>>>>> or if
> >>>>>>>>> the aim of the language is to excel only in numerics, data
> >>>>> analytics,
> >>>>>>>>> stats, whatever can be quantified pretty much.
> >>>>>>>>>
> >>>>>>>>> My few cents as food for thought.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Best regards,
> >>>>>>>>>
> >>>>>>>>> ---
> >>>>>>>>> Michail L. Liarmakopoulos, MSc
> >>>>>>>>>
> >>>>>>>>> On Sun, Jan 30, 2022, 20:39 R.E. Boss <r.e.b...@outlook.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I copied the first chapter of the book A Journey to Core
> >>>>> Python
> >>>>> (in
> >>
> https://drive.google.com/file/d/1p1uIANh-LFniNNRqjDeeWWd4_-ddEZmz/view?usp=sharing
> >>>>>>>>> )
> >>>>>>>>>> and have the question: do we want that J is competitive with
> >>>>> Python?
> >>>>>>>>>> If the answer is yes, the next question is: what is the to
> >>>>> do list
> >>>>>> to
> >>>>>>> be
> >>>>>>>>>> competitive and how long will it take?
> >>>>>>>>>>
> >>>>>>>>>> (And then the unavoidable question: WHY?)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Personally I think we must aim on the niches in the market, as
> >>>>> there
> >>>>>>> are
> >>>>>>>>>> the mathematical oriented people, e.g. the broad scientific
> >>>>>> community.
> >>>>>>>>>> Then all people experienced in Excel or other spreadsheets or
> >>>>>>> calculation
> >>>>>>>>>> tools.
> >>>>>>>>>>
> >>>>>>>>>> Schools and universities. Financial and statistical oriented
> >>>>> people.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> What we should do, IMHO, is
> >>>>>>>>>>
> >>>>>>>>>> - to emphasize the strengths of J,
> >>>>>>>>>>
> >>>>>>>>>> - to improve (considerably) the error handling of J,
> >>>>>>>>>>
> >>>>>>>>>> - to admit the steep learning curve,
> >>>>>>>>>>
> >>>>>>>>>> - to facilitate the use of mnemonics instead of primitives (I
> >>>>> tried
> >>>>>>> this
> >>>>>>>>>> afternoon the primitives.ijs for half an hour, but was not
> >>>>> capable
> >>>>>> of
> >>>>>>> use
> >>>>>>>>>> any mnemonic, not even with
> >>>>>>>>>> https://code.jsoftware.com/wiki/Primitives_to_Mnemonics)
> >>>>>>>>>>
> >>>>>>>>>> - to decide which of the features, benefits or applications
> >>>>> (of
> >>>>>>> Python)
> >>>>>>>>> we
> >>>>>>>>>> want J to have.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Just my 2 cents.
> >>>>>>>>>>
> >>>>>>>>>> R.E. Boss
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>
> ----------------------------------------------------------------------
> >>>>>>>>>> For information about J forums see
> >>>>>>> http://www.jsoftware.com/forums.htm
> >>>>>
> ----------------------------------------------------------------------
> >>>>>>>>> For information about J forums see
> >>>>>> http://www.jsoftware.com/forums.htm
> >>>>>
> ----------------------------------------------------------------------
> >>>>>>>> For information about J forums see
> >>>>> http://www.jsoftware.com/forums.htm
> >>>>>
> ----------------------------------------------------------------------
> >>>>>>> For information about J forums see
> >>>>> http://www.jsoftware.com/forums.htm
> >>>>>
> ----------------------------------------------------------------------
> >>>>>> For information about J forums see
> >>>>> http://www.jsoftware.com/forums.htm
> >>>>>
> ----------------------------------------------------------------------
> >>>>> For information about J forums see
> http://www.jsoftware.com/forums.htm
> >>>>>
> >>>> ----------------------------------------------------------------------
> >>>> For information about J forums see
> http://www.jsoftware.com/forums.htm
> >>> ----------------------------------------------------------------------
> >>> For information about J forums see http://www.jsoftware.com/forums.htm
> >>
> >> --
> >> This email has been checked for viruses by AVG.
> >> https://www.avg.com
>
> >>
> >> ----------------------------------------------------------------------
> >> For information about J forums see http://www.jsoftware.com/forums.htm
> >>
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
