Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

I dug a little for an answer to Henry's first question: what's a dictionary? I found 2 things: 1. We're not the only ones who seem confused about this definition. 2. I found this: https://cs.stackexchange.com/a/30025/59926 that gives precise mathematical definitions, although still subject to deb

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Same here. Let J be free to arrange at will. Conceptually having an order on the domain means talking about yet another kind of ds. (in the integral case, that’s good old arrays) Am 01.02.22 um 08:55 schrieb Elijah Stone: I still do not think of maps as having order. -- --

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Interesting discussion. My little addon (https://github.com/jpjacobs/types_dict ) works pretty well in my test, with any rectangular data (including boxes) as either keys or values. My only gripe with it was that due to subtle details of inverse in combin

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Following this thread I recall long-ago disputes in IBM R&D and university IT departments about supporting language features such as non-ASCII writing systems, database access (especially relational), *enum*, *class*, *struct* and *const*, right back to the egregiously misnamed "real" numbers. Fo

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

I would second looking at how k approaches dictionaries regarding operations available and their domain. Uniformity between lists, dictionaries and function calls is really elegant there too (although likely out of reach - but maybe still having some parts of syntactic sugar is possible - like havi

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

Dictionaries in q/k are actually a core part of the language, not just a tool or package. Dictionaries are the tool behind namespaces. Dictionaries are also the ‘items’ in the table data structures. If you have a list of dictionaries of keys and equal length values, then in the special case:

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

I did attach as a txt file in my sent email, but it seems to have been stripped … (I thought .txt files were ok to the mailing list now ??). I’ll try once more here, if not people can ask me and I’ll email them or put on a google drive and share with whoever requests. Thanks Rob, plain txt fil

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

OK that went through, anyone who wants a copy of the full PDF, please just email me directly and I will share it. Rob (email me directly at http://me.com/>> (no spaces) > On 2 Feb 2022, at 12:22 am, 'Rob Hodgkinson' via Programming > wrote: > > I did attach as a txt file in my sent email, b

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Henry Rich wrote: > I think I agree with all your statements, but you are not responding to > my questions, which will help focus the discussion: > > 1. What is a Dictionary, EXACTLY? FWIW, I find myself confused by this question. Maybe I am just failing to pick up on an implicit convention he

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

A large share of real data is organized in tree-like structures where lists (integer indexes) are mixed with dictionaries (arbitrary type indexes). An obvious example would be http centric APIs where json use is pervasive -- accessing an element at arbitrary depth is a trivial task in other languag

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

In the past, I have implemented 2-column "pseudo-dictionaries" in J. It works, but feels very awkward compared to working with arrays: ]d=: (;:'a b c'),.1 2 3;'foo bar';(<;:'x y z') ┌─┬───┐ │a│1 2 3 │ ├─┼───┤ │b│foo bar│ ├─┼───┤ │c│┌─┬─┬─┐│ │ ││x│y│z││ │ │└─┴─┴─┘│ └─┴───┘ get

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

You might search for "Beginner's Regatta" on the J wiki to see if the beginner-oriented sections of our NYCJUG meetings are helpful. On Mon, Jan 31, 2022 at 7:44 PM Raul Miller wrote: > I would not worry too much about "the J way of thinking" -- J is a > tools, but it's not the only tool. > > Se

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

This discussion is important.  I'd like to continue from this post. To respond to Ian, who asked Who really needs this anyway?  I say that I got great use from C++ classes for ordered_map and unordered_map.  The operations can be done in J, but the speed advantage of hashing for lookups is so

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

For what it's worth below an implementation I've started toying with a while ago. A dictionary here is a boxed pair of lists representing ordered and uniq keys and values. regards, Danil == Transcript == [d1 =: dnew NB. empty dictionary is a pair of empty lists ┌┬┐ │││ └┴┘

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

One of the problems you're going to run into right away with making dictionaries a core part of J is the concept of rank, which doesn't really exist in K (everything is a rank 1 list, hooked together by dictionaries). At the end of the day, what I think most people want from maps/dictionaries is a

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

The distinction between an x column table and a x column dictionary (as inverted table) is unique keys: unique values that make up 1+ column.  While a strict definition of a dictionary is a datastructure that supports just value lookup through keys, the ability to make any query at all on the da

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

unique keys is a pretty important aspect of dictionaries. It permits SET (or Alex's merge) functionality to determine whether it is an update or an append.  Where users appreciate this "auto-functionality".  Without unique keys, the user must determine if they want update vs append, and if it i

[Jprogramming] Sierpinski triangle

I have found in a forum the following J code that generates a Sierpinski triangle, and am trying to understand how it works. sierpinski =: {{ (, ((72#:~8#2){~3#.\0,],0:)@{:)^:y ,: 1 y} (>:+:y) $ 0 }} ' #'{~ sierpinski 15 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #

Re: [Jprogramming] Sierpinski triangle

You already understood that some kinds of brackets don’t come in balanced pairs. But parentheses () nearly always do, the exceptions being inside of strings, obviously, and closing ) on a line of its own (which doesn’t occur here, so I won’t explain that). (oh, and {{…}} introduced another possibi

Re: [Jprogramming] Sierpinski triangle

Hello, Besides dissect that I really like, Art's flowtree ( https://code.jsoftware.com/wiki/User:Art_Anger/FlowTree) looks pretty interesting too, but I haven't used it much yet. Best regards, On Tue, Feb 1, 2022 at 7:59 PM Hauke Rehr wrote: > You already understood that some kinds of brackets

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

To respond to Henry (…I guess) I wrote: > Swift boasts a me-too *dictionary* à-la Python, but I've never found a use for it. I need to eat my words. I may indeed have an important use for *Dictionary* in Swift. I'm just about to implement syntax coloring in j901 for iOS (and – no I can't use Qt

Re: [Jprogramming] Sierpinski triangle

Look at https://code.jsoftware.com/wiki/Essays/Parentheses_Matching and run the first 5 definitions. Then run paren '(, ((72#:~8#2){~3#.\0,],0:)@{:)^:y ,: 1 y} (>:+:y) $ 0 ' (, ((72#:~8#2){~3#.\0,],0:)@{:)^:y ,: 1 y} (>:+:y) $ 0 ++ +--+ +--

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Personally, I’m fine with the little syntax highlighting I get in vim. Much could be improved but I don’t feel I need it. But there doesn’t seem to be a pygments lexer for J. I find an APL lexer, though [1] Has anyone tried to provide one already? @Michal: could you talk about the way you did you

Re: [Jprogramming] Sierpinski triangle

another approach that could be helpful with respect to that question: try removing that , and rearranging )@{:)^:y to become )^:y)@{: in the definition what does it compute now? Sierpinski =: {{ (((72#:~8#2){~3#.\0,],0:)^:y)@{: ,: 1 y} (>:+:y) $ 0 }} ' #'{~ Sierpinski"0 i.>:15 This gives a

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

The present threat seems to be proving useful to Henry. I didn't mean to divert it off-topic into a discussion of syntax coloring per-se. I offered it as a case-study to inform the design of a serviceable dictionary feature for J. On Tue, 1 Feb 2022 at 19:49, Hauke Rehr wrote: > Personally, I’m

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

…Freudian slip there. For 'threat' read 'thread'. On Tue, 1 Feb 2022 at 20:35, Ian Clark wrote: > The present threat seems to be proving useful to Henry. I didn't mean to > divert it off-topic into a discussion of syntax coloring per-se. I offered > it as a case-study to inform the design of a s

Re: [Jprogramming] Syntax highlighting WAS: Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

Apologies, I should have known better. If anyone wants to answer, start from this one instead. Am 01.02.22 um 21:35 schrieb Ian Clark: The present threat seems to be proving useful to Henry. I didn't mean to divert it off-topic into a discussion of syntax coloring per-se. I offered it as a case-

Re: [Jprogramming] Sierpinski triangle

Andrew, your observations are correct. Now consider this part which is an amend to create and amend a long list of zeroes. 1 (15)} (>:+:15) $ 0NB. beware, of long result 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 $,:1 (15)} (>:+:15) $ 0 NB. ,: forces a new dimension

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

I'm sure that previous discussions of dictionaries in J resulted in a consensus that J doesn't need dictionaries. >  How about storing the different-colored J words in dictionaries, one for >each color? A J way to do this without dictionaries is to have your words grouped by boxing, and a colo

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Mon, Jan 31, 2022 at 9:16 PM Elijah Stone wrote: > I think not; keys should not be ordered. I think I need to disagree here. (1) K (and Q) have shown that you get great efficiencies with ordered keys. (2) Also, arrays require ordered data for meaningful operations. (3) Lack of order tends t

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

1. What is the efficiency gain from ordered keys? (In general, the more restrictive your semantics are, the more freedom you have about implementing them, and hence the more efficient you can be. So this doesn't make sense to me. Ordering keys may happen to be faster as an _implementation_ st

Re: [Jprogramming] Sierpinski triangle

It's probably worth noting here that the {: here is used to grab the last row of the partial result which is being built. Initially, this is a list of y zeros, followed by a 1, followed by y zeros. ,: 1 (15)} (>:+:15) $ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Note that

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, Feb 1, 2022 at 2:31 PM Elijah Stone wrote: > 1. What is the efficiency gain from ordered keys? (a) Ordered access is "close to the hardware" (cache friendly), (b) Many of J's verbs are defined on ordered sequences. So demanding unordered access is demanding that those verbs do not functi

[Jprogramming] class method returning the instance

Dear all, I'm following the J for C tutorial, and I'd like to know whether J allows methods to return the instance, i.e. 'this' in Java or 'self' in SmallTalk? Thanks! Raoul -- For information about J forums see http://www.jsoftw

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

I stumbled across better performance (kdb+ vs. Jd) in my own stuff: http://blog.vmchale.com/article/q-perf Basically joins take longer (which I think makes sense?). It’s easy to imagine that selections and joins (on the ordered keys) are going to be faste

Re: [Jprogramming] class method returning the instance

I think you are looking for the result of coname'' It's a function, rather than a special variable, but yes -- if you are in an instance, it will identify that instance for you, and you can use that value to refer to that instance from elsewhere. I hope this helps, -- Raul On Tue, Feb 1, 2022

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, 1 Feb 2022, Raul Miller wrote: 1. What is the efficiency gain from ordered keys? (a) Ordered access is "close to the hardware" (cache friendly), 1. Dictionaries are specifically about random access. That's the point. If you want a list of key-value pairs, you can do that already

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, Feb 1, 2022 at 2:57 PM Elijah Stone wrote: > On Tue, 1 Feb 2022, Raul Miller wrote: > >> 1. What is the efficiency gain from ordered keys? > > > > (a) Ordered access is "close to the hardware" (cache friendly), > > 1. Dictionaries are specifically about random access. That's the point. >

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, 1 Feb 2022, Raul Miller wrote: I think you are thinking about scalar access -- operations which focus on a single key and the associated value. And, while that does happen, remember that J also supports operation on the data structure as a whole. I am thinking about operations on ma

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, 1 Feb 2022, Vanessa McHale wrote: easy to imagine that selections and joins (on the ordered keys) are going to be faster since one can implement them differently. I thought k maintains insertion order? I can imagine fast joins for sorted keys, but not for keys which happen to be in t

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, Feb 1, 2022 at 3:21 PM Elijah Stone wrote: > I don't care what you call it; forcing an order on keys removes > implementation freedom which may have a negative impact on performance. At no point did I suggest "forcing an order on keys" -- I instead said that there needed to be an order.

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

I imagine so! In my example, sorting/insertion was fast enough that q was still faster than Jd. > On Feb 1, 2022, at 5:28 PM, Elijah Stone wrote: > > On Tue, 1 Feb 2022, Vanessa McHale wrote: > >> easy to imagine that selections and joins (on the ordered keys) are going to >> be faster since

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

Ric For the dataframes equivalent, are you thinking of the more recent tibble structure in tidyverse? This is easier to understand than the original dataframes. If so, this is a worthwhile addon for J. I worked on something similar recently. If you or anyone else is interested, we could collabora

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

So there was some kind of misunderstanding. Perceived point of view: There should be a total ordering on the (in theory) infinite set of J values. Fine. And maybe more than one (Elijah already pointed at <. vs /:) in different contexts. This again drives us to the question: Do we actually want an

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Mon, 31 Jan 2022, Henry Rich wrote: The deficiency in J is that m&i. gives you a hashtable for searching m, but if you modify m you have to recalculate the hash from scratch.  This makes m&i. a good read-mostly Dictionary but slow for frequent updates.  Why not cache the hash table for x w

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, Feb 1, 2022 at 5:05 PM Hauke Rehr wrote: > So there was some kind of misunderstanding. > Perceived point of view: > There should be a total ordering on the (in theory) > infinite set of J values. Fine. Oh... I do not think I would go there (unless we consider dictionary dimensions to be i

Re: [Jprogramming] Sierpinski triangle

Hello everyone, First of all, thank you so much for all the responses. After reading each one multiple times and spending several hours looking at this program, I think that I have made progress. Hauke: I hadn't noticed that parentheses were, as a rule of thumb, matched. About dissect: it is not

Re: [Jprogramming] Sierpinski triangle

On Wed, 2 Feb 2022, Andrew P wrote: A question that I have is: can V0 V1 V2 V3 V4 can become V0 V1 V2 (V3 V4) instead of V0 V1 (V2 V3 V4)? Nope. A way to think about it is that the parser eats verbs from the right, as many as it can handle. So if you start with V0 V1 V2 V3 V4 V5, it first

Re: [Jprogramming] Sierpinski triangle

I know this question is about understanding some J code, but perhaps it is worth noting that the Jqt Studio|Showcase|isigraph demo has other examples, including Cliff Reiter's elegant: load 'viewmat' viewmat (,,.~)^:8 ,1 -- Fo

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

Hi Chris, I haven't been using R much at all over the last few years, so haven't had much to do with tibbles. Just looked them up now and they do seem more sane. Most of my recent work has been with Polars Dataframes ( https://github.com/pola-rs/polars) which are based on Apache Arrow Columnar form

Re: [Jprogramming] Dictionaries WAS: Report on the J wiki meeting of January 27, 2022

On Tue, 1 Feb 2022, 14:32:43 +10 Henry Rich wrote: > Many ideas have been aired. But please, every now and again address the > questions I need answers for: > > 1. What is a Dictionary, EXACTLY? 1.1. Dictionary (I prefer shorter term: "map") is a data structure establishing a discrete surjectiv

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

Ric, You might want to check out DuckDB (https://duckdb.org/), I recently used it for reading and writing Parquet files. It's similar to SQLite but intended to be used for analytics. Stefan. On Wed, Feb 2, 2022 at 5:29 AM Ric Sherlock wrote: > I spend a fair bit of time wrangling data formatted

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

There's also these J bindings that can be used to read/write parquet files directly from J: https://github.com/interregna/JArrow On Wed, Feb 2, 2022 at 5:27 PM Stefan Baumann wrote: > > Ric, You might want to check out DuckDB (https://duckdb.org/), I recently > used it for reading and writing Par