Beyond dictionaries is a "generic data structure" construct, and a large
extended family of data structures that can respond to the following
commands
GET SET UPSERT APPEND DELETE INSERTafter(item) INSERTbefore(item) WHEREget
CONVERTtoJnativearray UPSERTfromJnativearray
in addition to dictionaries, ragged nested arrays or generally any binary
encoding of datatype;shape/count;content
for dictionaries, some applications (or at least language implementations)
permit variant/inconsistent type keys and/or values.
and generally, a table is a dictionary with
multiple keys and multiple associated values to each unique combination of
the multiple keys. With potential performance advantages to keeping it in
a sorted order if retrieval of items by query, or having multiple fields
indexed. Typed "tables"/dictionaries get inherent performance advantages
due to not requiring item boxing.
as a generic "pattern framework" for data structures without using classes:
DEFAULT =: ''
data dataconstructor DEFAULT NB. dataconstructor is conjunction that will
return an adverb that is (conjunction data). example
mydict = ('key1'; value1) dict DEFAULT NB. an adverb that is a
conjunction bound to specific data, and where the conjunction code responds
to commands in list above top.
The list of commands near top of this message are actually numeric codes
that get passed to the bound datastructure adverb.
GET mydict becomes a verb that will retrieve items.
SET mydict returns an conjunction with m as key, n as value that will
return the same mydict adverb but with updated "data"
(newkey; SET) mydict returns an adverb that is above conjunction with m
bound as newkey
(newkey: newdata; SET) mydict directly returns the updated mydict adverb.
In the case of typed/consistent dictionaries, the internal data
representation is just an inverted table of "fields". And while the
underlying conjunction of mydict is bound to data to form an adverb, a
generic (consistent typed) dictaccess conjunction can operate on any
inverted table if prebound with GET/SET. And the GET/SET/OTHER "commands"
can form a verb as a result of paired with a generic access adverb, all
"command parameters" as x, and the inverted table/data structure as y.
What makes this approach have more appeal to be over classes is besides
issues like reference management and it is hard to send methods to
class/objects in a "parameterized" way, is that arrays are J's
first/primary datastructure, and in some datastructures can be
transparently accessed by independent J code.
Generally, putting the complexity in modifiers instead of classes is a
functional approach I would prefer. I can volunteer to start it off at
least if there is interest in this approach.
It would be a datastructure access "language/dsl" on top of J.
On Monday, January 31, 2022, 01:43:30 p.m. EST, Henry Rich <
henryhr...@gmail.com> wrote:
Quite right. Jan-Pieter has at least a good start on the model. What I
am suggesting is that the 'more integrated implementation' might not
need to be much more integrated than the J script. Searching and
sorting are already J primitives.
The deficiency in J is that m&i. gives you a hashtable for searching m,
but if you modify m you have to recalculate the hash from scratch. This
makes m&i. a good read-mostly Dictionary but slow for frequent updates.
If we can refine the model to the point that such inefficiency is the
worst problem we have, a little integrated support could finish the
package.
Henry Rich
On 1/31/2022 1:37 PM, Eric Iverson wrote:
A good and complete model is the first step. Then it is a matter of how
much it is used and what drawbacks it might have that would be addressed
by
a more integrated implementation.
On Mon, Jan 31, 2022 at 1:27 PM Henry Rich <henryhr...@gmail.com> wrote:
I have looked into this quite a bit. I am not convinced that Dictionary
is a fundamental datatype like number or letter, or that the current
symbol support is deficient. That makes the first questions What is a
Dictionary? and Where can a Dictionary be used in J?
The use case that would be important to me is a key-value store, aka
associative memory, where the Dictionary remember keys and values and
later returns a value when presented with the key. This feels to me like
a package written in J rather than a J primitive. A Dictionary would be
a numbered locale. The place where I could see added support inside J
would be in adding to and deleting from hashtables, specifically ones
created/used by m&i. .
I invite proposals on what a Dictionary package needs to do. I have
done this before, and Jan-Pieter Jacobs responded with a package.
Quoting from his message of 20210327:
*** QUOTE
install'github:jpjacobs/types_dict@main'
(except on (my) android, where github installs don't seem to work,
neither
in the Gui version JA nor on the commandline via termux)
It is pretty simple in use, you just use:
d=: dict keys;vals
for creating the dictionary, which is a OOP object, having the following
methods:
get, set, map, sort, destroy, whose documentation is contained in
help_pdict_
For performance, the create verb precomputes the lookup for the get
verb,
both forward get (key->value) and backward get inv (value->first key)
upon
object creation.
*** END QUOTE
I have not looked into this package, but it seems to me to have the
right entry points. Can we take that as a starting point to see what
needs to be added? The first step of course would be to put the addon
under Package Manager.
Henry Rich
On 1/31/2022 12:58 PM, Elijah Stone wrote:
I agree with Eric regarding the challenges of adding dictionaries.
One issue: I think a necessary prerequisite is improved symbols.
On Mon, 31 Jan 2022, Alex Shroyer wrote:
I agree with Raoul that competing with Python is not a good idea.
But J can learn from Python's decisions (good and bad) to grow and
improve.
In my opinion, numpy is Python's "killer app" because it brings
reasonable
performance without much conceptual overhead.
The feature of Python that enabled numpy is its extensibility, down
to the
C layer.
There are other good features of Python that J could copy, in
particular
the 'dictionary' data type.
The array language K has dictionaries, so J might take some
inspiration
from there for integrating them into the language.
Cheers,
Alex
On Mon, Jan 31, 2022 at 5:30 AM Raoul Schorer <
raoul.scho...@gmail.com>
wrote:
Just my 2c, but I think that competing with python in general is
somewhat
delusional. I think the key point for expanding J use to have a
"killer J
app". For example, an improved clone of or excellent plugin for
VisiData (
https://www.visidata.org/) is my idea of a killer app. But someone
here
may
have a better idea?
Cheers,
Raoul
Le lun. 31 janv. 2022 à 03:49, Ric Sherlock <tikk...@gmail.com> a
écrit :
Yes from a data structure point of view, inverted tables get you a
lot of
the way (note they're also available in the 'general/misc' addon -
load
'general/misc/inverted' ) and I've used them to good effect in my
'data/struct' addon (https://github.com/tikkanz/data_struct).
I agree that J's arrays are more general, flexible & powerful. But
when
you're dealing with a tabular data set there is an overhead to
keeping
the
fields in a table in sync that dataframes can help with. Perhaps
it's
something abstracting the structure so you don't have to deal so
much
with
the mechanics of manipulating it? At least for me :)
On Mon, Jan 31, 2022 at 3:28 PM Elijah Stone <elro...@elronnd.net>
wrote:
https://code.jsoftware.com/wiki/Essays/Inverted_Table, perhaps?
That said, I think a great strength of j is that data are _not_
explicitly
tabular. The associations are defined in an ad-hoc manner as
needed by
the programmer. This is also an essential difference between
the array
paradigm and the relational paradigm (cf SQL): in the former,
pointers
come with implicit context; in the latter, that context must be
explicit.
-E
P.S. regarding analysis/optimization: I would love to see it,
but for
some
reason everybody is scared of building a compiler because of the
parsing
problem.
On Mon, 31 Jan 2022, Ric Sherlock wrote:
Yes, I've been thinking that a Dataframes equivalent in J
would be
useful.
Most things are already possible with J's arrays, but
conceptually
DataFrames are well understood by many now, and they make it
easy to
work
with datasets as named fields.
I've spent a reasonable amount of time working with Pandas,
but have
recently been using Polars (Rust backend with Python bindings)
which
really
shines for larger datasets. Performance (especially
read/write) is
awesome,
and the LazyFrames which optimise your query/analysis plan
make a big
difference too.
I haven't taken enough time to explore it, but maybe Jd is the
starting
point in this space for J?
On Mon, Jan 31, 2022 at 9:01 AM Michail L. Liarmakopoulos <
m.l.liarm...@gmail.com> wrote:
Hello all,
I find any parallels between python and J pretty interesting,
being
a
person with some python experience and an interest of the
applications
of
both python and J in mathematical modelling, analytics,
computational
math
and perhaps computational physics too.
If you'd like to bring some features from the python
math/analytics
libraries/ecosystem in J, I'd suggest you to look at the
features of
three
libraries:
- numpy (I believe most features are already covered from the
built
in
features of an array language such as J)
- pandas ( a nice library for manipulating csv files within
python
as
dataframe objects -- see the dataframes from the R language)
- scipy (a collection of methods and functions ranging from
solving
numerically: differential equations, evaluating definite
integrals,
constrained and unconstrained optimization, and I believe
statistics
too)
There is also out there an amazing python library for symbolic
calculations
(like the ones you can do with Mathematica or WolframAlpha:
symbolic
evaluation of definite and indefinite integrals, symbolic
solutions
of
diff. equations, symbolic solutions of algebraic and Diophantine
equations
etc...). It's called sympy.
But I'm not sure if you'd like J to include symbolic
computations
too
or if
the aim of the language is to excel only in numerics, data
analytics,
stats, whatever can be quantified pretty much.
My few cents as food for thought.
Best regards,
---
Michail L. Liarmakopoulos, MSc
On Sun, Jan 30, 2022, 20:39 R.E. Boss <r.e.b...@outlook.com>
wrote:
I copied the first chapter of the book A Journey to Core
Python
(in
https://drive.google.com/file/d/1p1uIANh-LFniNNRqjDeeWWd4_-ddEZmz/view?usp=sharing
)
and have the question: do we want that J is competitive with
Python?
If the answer is yes, the next question is: what is the to
do list
to
be
competitive and how long will it take?
(And then the unavoidable question: WHY?)
Personally I think we must aim on the niches in the market, as
there
are
the mathematical oriented people, e.g. the broad scientific
community.
Then all people experienced in Excel or other spreadsheets or
calculation
tools.
Schools and universities. Financial and statistical oriented
people.
What we should do, IMHO, is
- to emphasize the strengths of J,
- to improve (considerably) the error handling of J,
- to admit the steep learning curve,
- to facilitate the use of mnemonics instead of primitives (I
tried
this
afternoon the primitives.ijs for half an hour, but was not
capable
of
use
any mnemonic, not even with
https://code.jsoftware.com/wiki/Primitives_to_Mnemonics)
- to decide which of the features, benefits or applications
(of
Python)
we
want J to have.
Just my 2 cents.
R.E. Boss
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
