[Jprogramming] Polish my wavelet

So, I'm trying to cook up a wavelet package for you guys. I want to use it to illustrate "notation as a tool of thought," and also so everyone has wavelets with which to wavelet things. Wavelets have multiple levels, and calculating them is a recursive process on filtered values of the origina

Re: [Jprogramming] Polish my wavelet

My apologies guys: I should have included the other bits, but I thought something might pop out immediately, and I didn't want to spam up the list with lots of code. Adverbs and conjunctions, I am still a bit intimidated by; wouldn't have thought of that. FWIIW, I have to think about higher dim

Re: [Jprogramming] Polish my wavelet

L yvals -SL On 10/27/2014 08:45 PM, Scott Locklin wrote: My apologies guys: I should have included the other bits, but I thought something might pop out immediately, and I didn't want to spam up the list with lots of code. Adverbs and conjunctions, I am still a bit intimidated by; woul

Re: [Jprogramming] Polish my wavelet

My apologies from being away from this for a few days. Raul wrote: dwtL=: 1 : 0 : 'wn yn'=. m dwt y wn; (<:x) m dwtL^:(x>1) yn ) But that might be getting too obscure? That's really awesome, actually, and conveys well what I was trying to accomplish with code clarity! Thank you very muc

[Jprogramming] Dumb ffi question

So, I have this pile of C code that uses rank-2 arrays as double**, then subsets them with loops like "array[i][j]." I'm so used to the J FFI working flawlessly on C code (most of which sensibly uses double*), I forgot that a J-array doesn't work like that, and was kind of shocked when the code

Re: [Jprogramming] How is it that people are oblivious to APL/J?

The learning curve is pretty steep, and with all respect due this group, there is not yet a Paul Graham who has both the chops to get rich using the tool, and the literary skill to enthrall people on the subject. Personally I am a novelty seeker. I liked Lisp, but was unhappy with it as a nume

[Jprogramming] hierarchical clustering for J

I've got something working here if anybody needs hierarchical clustering: https://github.com/locklin/j-cluster I'll be updating and modifying this in the next few months as needed for a personal project, adding things like D3 output for the dendrograms, but it's useful now. Thanks to Bill Lam

[Jprogramming] Quandl bits

I don't follow the J list as much as I should, but noticed Joe's examination of his portfolio holdings. I've been fooling around with futures data, and had need for some interest rates and have the beginnings of a useful verb for querying quandl. The only thing you need to set is the apitoken.

[Jprogramming] jtags

I'm certain that we had a jtags script once upon a time in addons/general. I generally don't need such things, but I've been using ctags for another project; is there a pointer to the old code? -Scott -- For information abou

Re: [Jprogramming] J strengths?

On 11/28/2017 3:59 PM, Andrew Dabrowski wrote: > Do you loveJ most because of (pick only one) Background: applied math/numerics guy. I've worked around big data and tsdb problems for a long time, so the legends around Kx systems is what originally brought me to APL land. I even worked on a "more

Re: [Jprogramming] jtags

Dec 01, 2017; 8:46pmbill-lam-2 wrote: >Qt ide dropped support for jtag. Use find in file should be more intuitive  >in ide. >If you use jconsole, then jtag may be useful. Yeah, I'm working with emacs most of th

Re: [Jprogramming] Jdb vs Spark

On Nov 29, 2017; 12:17pm, Miles Wells wrote: > Did Scott ever write that blog post he mentioned in the linked post? I am  > quite interested in reading more about his testing.  No, I never did. I quit the company a few months later and attempted to found a TSDB company called Kerf, so I was a li

Re: [Jprogramming] SVMs in J

On Dec 02, 2017; 1:58pmDevon McCormick wrote: > Does anyone have J code to implement support vector machines that they would like to share?  I started a wrapper for sequential SVM using Leon Bottou's "lasvm."

Re: [Jprogramming] SVMs in J

On Dec 04, 2017; 7:37pm Devon McCormick wrote: > I searched the J wiki for "quadratic" and found a mention from a 2005  > NYCJUG meeting where we were lamenting that other environments simply > have  > quadratic solvers available but J does not.  Looking into it more, I was  > unsure "quadratic" m

Re: [Jprogramming] SVMs in J

On Tue, Dec 5, 2017 at 7:36 PM Devon McCormick wrote: Thanks for the image-processing suggestions.  I have lots of pictures - "million" is not an exaggeration - and I'd like to do some kind of automated recognition and aesthetic ranking of them. Big data problem for sure. Dig around in loc

[Jprogramming] HMAC

This verb might be useful to someone at some point. Pretty straightforward stuff, but fiddley and no point in reinventing the wheel if someone else needs it. https://en.wikipedia.org/wiki/Hash-based_message_authentication_code NB. hmac signature NB. x key, y needs signature hmac=: 4 : 0  key=. x

Re: [Jprogramming] HMAC

Dec 19, 2017; 8:17pmBill Lam wrote: >Wouldn't it type less if  > key=. 128{.x,128#{.a.  > or  > key=. 128{.!.({.a.)x I was just happy to make it work at all! Had totally forgotten about the bit manipulations po

Re: [Jprogramming] Haar wavelet

Jan 07, 2018; 9:41am — by Raul Miller-4  Raul Miller-4 FWIIW I wrote a decent and general wavelet implementation in native

Re: [Jprogramming] JGMM, Mixture Model in J

Nice work! FWIIW, Mixture, Naive Bayes and semisupervised versions of each with the EM-algo can often be cooked up from the same raw materials. And multivariate/Bernoulli, I think can be derived from the mean term in the Gaussian example. Multivariate and Bernoulli should probably be done with spar

Re: [Jprogramming] Getting LBFGS back online

I think I fixed these. I  haven't tested these repos since 2014 or so though. https://github.com/locklin/j-lbfgs https://github.com/locklin/j-lbfgsb -- For information about J forums see http://www.jsoftware.com/forums.htm

Re: [Jprogramming] algotrade and backtest platform

>>"Y-01"> > 1. J practise; > 2. interesting task; > 3. the effect of social reinforcement (international team); > 4. experiments with strategies; > 5. possibility to earn some candy wrappers (like btc or usd) > 6. pride oneself and pride for the lost time. >Also I propose to start with Mtgox.com

[Jprogramming] SVD performance comments and CUR decomp

I noticed that the c-lapack svd function is very slow and memory inefficient on my (linux) laptop. The native J version in math/misc/svd is actually many times faster and more memory efficient, so I used imath/misc/svd instead. Now, this flags a bug in C-lapack for certain, as it isn't so bad w

Re: [Jprogramming] SVD performance comments and CUR decomp

Bill: The SVD operation returns the values: 'u s v' =. svd trn The J version returns the diagonal, which is what you want. The lapack version for some reason returns the whole array with all the 0's. So, to match them up: 'ul sl vl' =. gesvd_jlapack_ trn +/ s=(<0 1)&|: sl In this case, yo

[Jprogramming] emacs & jconsole plots

I know some of you are emacs types like me. I recall being able to call up multiple plots in emacs using jconsole/J7.x/emacs, but I can't for the life of me remember the configuration magic for achieving this. The symptom, just to be clear, is you need to kill the plot window to return to the

Re: [Jprogramming] emacs & jconsole plots

that code is never reached when I use plotting from jconsole/emacs. The relevant line there is fork_jtask_ (dquote PDFReader_j_),' ', (dquote file) -SL Bill Lam wrote: >I know almost nothing about emacs. for J7 Jconsole , there is a >cairo output which generates png image. at ~temp/

[Jprogramming] A humbling linear algebra reminder

So, I've been screwing around with neural nets, and built a learner with simple linear regression output nodes. The nature of this beast is you have to invert a big matrix which is close to singular. Because I am a big numerics nerd, I remembered to add a small number to the diagonal in order t

Re: [Jprogramming] A humbling linear algebra reminder

Erm, apologies, I meant: inv=: [: gesv_jlapack_ ] ; [: =@i. $ On 09/01/2014 05:10 PM, Scott Locklin wrote: So, I've been screwing around with neural nets, and built a learner with simple linear regression output nodes. The nature of this beast is you have to invert a big matrix whi

Re: [Jprogramming] J Kernel

Nice work, Devon. One of the things I would like to do eventually is a sort of "J for Matlab/R programmers." I figure J is vastly more useful as a general purpose language than these, but it is also first rate for all kinds of numerics and mathematical applications. There is a large population

Re: [Jprogramming] deoptim and lbfgs minimization

For what it is worth, lbfgs works on linux. I stuck it on github, along with fortran source and Makefile in case it helps others. https://github.com/locklin/j-lbfgs I also took the liberty of using the same code to do the box constrained version of the algorithm. The example is no good, and I t

[Jprogramming] 64 bit lapack

I'm in the process of modifying the lapack addon for 64 bit linux. I had a very hard time building working shared libraries with the old CLAPACK source (last major update ... 1994 or so), so this is oriented towards the modern LAPACKE C interface. LAPACKE is part of the main Lapack distribution,

Re: [Jprogramming] 64 bit lapack

re is additional package liblapacke which depends on liblapack3 > but not installed in my computer. Can you confirm which of > these packages corresponds to your shared library? Is it > possible to just use linux distro provided package instead > building it from source? Вт, 26 фев 2013, Sc

Re: [Jprogramming] 64 bit lapack

Putting on my thinking cap a little more, I was able to build CLAPACK-3.2.1 with all the dependencies cooked in and make it work with the old code as a standalone shared library. That's probably more convenient for others, and I can put off working on j-lapacke later, when I need more speed. It'

[Jprogramming] JDB client/server missing from latest addons

I noticed in the latest j701 updates, all the demo files, as well as the client/server are zeroed out: -rw-r--r-- 1 scott scott 0 Feb 1 20:03 jdbclient.ijs -rw-r--r-- 1 scott scott 39446 Dec 4 16:50 jdb.ijs -rw-r--r-- 1 scott scott 0 Feb 1 20:03 jdbserver.ijs -rw-r--r-- 1 scott scott

Re: [Jprogramming] JDB client/server missing from latest addons

Thanks for the quick fix! I looked for other empty files in the addons directory, but I guess files only disappear when you need them. Probably, they were hanging out with my keys and Schroedinger's cat. -Scott Bill Lam wrote: > Not sure why this had happened, files in svn are ok.  Any ways,

Re: [Jprogramming] Learning J language - initial thoughts

I'll add my voice as a  beginner: I agree with Greg that JforC is a tough slog for a beginner. However, it contains most of the actual information needed to get things done. I find myself returning to it all the time when I don't know how to do something. The others are great as introductions.

[Jprogramming] Information theory in J

Dan Bron wrote in the "Learning J language" thread: "If you post your explicit (that's what we call the 3 : 'x stuff y other stuff' style of code), I can take a whack at showing you how to translate it to tacit code (that's what we call the cartoon-characters-cursing style of code).  "  Than

Re: [Jprogramming] Information theory in J

This line looks suspicious to me because #y is always an integer already, and <. seems to do nothing. nbins=. 1 Round (<.#y)^0.5 The following result also looks suspicious because the sum of the result is only 19, not 20.    discretize i. 20 1 4 5 5 4 Good eyes; thank you. The <. should have

[Jprogramming] ZeroMQ

I'm wondering if anyone has yet had a go at integrating ZeroMQ with J. I'd like to code up the pub/sub pattern for a ticker plant. I was just going to use the 15!:0 FFI and do calls on ZeroMQ pointers. The only thing which gives me pause is ZeroMQ has an underlying threading model which might d

Re: [Jprogramming] ZeroMQ

Ah, nevermind, zeromq seems to Just Work with J fairly trivially. I'll be coding up the parts I need first (pub-sub), but look for J hooks in this github in the next week or two: https://github.com/locklin/j-zeromq Should be pretty helpful for talking to other processes, as well as IPC. This sho

Re: [Jprogramming] lapack

Raul Miller wrote: >currently, I can't use lapack even on 32bit j6.  Works on 64 bit linux/j7. I had to noodle around with clapack to get the library to build. The noodlings are here if that helps you: https://github.com/locklin/clapack-j I think these noodlings are what is in the present add on

Re: [Jprogramming] Socket client

Oh man, someone noticed. I wouldn't characterize that as "fully tested" yet. Also, unlike Henry Rich's examples, 0MQ is pretty much only good for talking to 0MQ. Without serialization (I may add XDR later), it's really only useful for talking to other 0MQ processes running in J. The main upside

Re: [Jprogramming] Socket client

(10,000 pardons for my munging the subject line in the quantum computing thread; it's been a while since I used an elist outside of nabble). >>  only useful for talking to other 0MQ processes running in J >looks useful.  Is there anything you think is missing or need help with? >instead of xdr

Re: [Jprogramming] Socket client

Pascal wrote: >for larger data, 3!:1 (or 3!:3) is very fast and you don't have to worry about >running ". on what is only assumed to be a noun (data). I'm going to give this a shot this evening and see if it generalizes well. Looks like just the doodad I needed. FWIIW, I sat down and looked

[Jprogramming] Documenting addons via joxygen

Hello: Is joxygen going to be the preferred way of documenting addons moving forward? I have a few interesting addons brewing, and am missing the literate programming toolchain which exists in R. If joxygen integration is on its way, I'd like to be ready for it (and would be happy to pitch in to

Re: [Jprogramming] Documenting addons via joxygen

Ric wrote: >Hi Scott, >The answer is yes. At least in my mind! >Unfortunately I haven't made as much progress on this as I'd have liked. >The types/datetime addon is documented according to current joxygen Ah, OK. I'll follow the conventions, which seem pretty straightforward to me (Lush had a s

Re: [Jprogramming] code golf

Marshall wrote: > Here's 18: >   '_*'{~4=+/~4<.|i:5 Dang: 18 bytes. FWIIW, the Shannon information in the generated string is ... about 17 bytes. Kolomogorov is arguably smaller (and of course, the 12 byte train produces the same information), but that's really damn cool. -Scott ---

[Jprogramming] big csvs

So, I'm having a hard time loading some very simple, but relatively large csv files into J for processing. The problem arises because J boxes all the atoms individually. Since in general (headers excluded), columns of the csv are all of the same type, this seems wasteful. In this case, everythin

Re: [Jprogramming] big csvs

Pascal wrote: >Can you be more specific about the code? >I assume that you looked into cut ;. ? >an alternative to boxing might be to strip out the commas, and then run 0&". >on the string.  Not sure that is faster though. I'm sorry for being unclear. I did something like this: loadd 'tables/cs

Re: [Jprogramming] big csvs

"bill lam-2" wrote: >   ".;._2 a > 1 2 3 > 4 5 6 Oh chrikey, that is a whole lot better (and a lot easier than writing a C gizmo to do this). Thank you for your patience in showing me that one. -SL -- For information about J

Re: [Jprogramming] wiki: task scheduler and zeromq intro

Dang, thanks for working on this, Pascal. This is one of the use cases I had in mind for J-ZMQ, though it was taking me some time to figure out how to do it in my clumsy way. I had been looking at joebo's fork thing for a little inspiration, but I got busy with a half dozen other things. I will

Re: [Jprogramming] wiki: task scheduler and zeromq intro

Joe Bogner wrote: >Can anyone share specific examples where it was needed to scale out to >multiple cores and machines? I >am interested in learning about the types of problems this would be applied >to. I have read some >examples while researching but haven't ran into anyone who has. >For ex

Re: [Jprogramming] GNU Scientific bindings?

Kind of curious: what is compelling about GSL? I ported a few things from it to a Lisp that had partially completed bindings, mostly because it was already basically there, but it's generally a trashfire of very little utility to a language like J which already has most of it built in. For example

Re: [Jprogramming] Question about least squares with %.

Been following this conversation with interest. I actually went through this exercise a long time ago (I think 7.x, could have been 6.x). I was building a framework for Kernel regression. There were problems then, though I think the present situation is actually considerably better. The solution I

Re: [Jprogramming] [Jgeneral] interpolation (equidistant re-sampling)

FWIIW there's a splines mini package here: https://www.astro.umd.edu/~jph/J_page.html -SL On Fri, May 7, 2021 at 11:58 AM 'Rudolf Sykora' via Programming < programm...@jsoftware.com> wrote: > Dear Raul, > > > thanks for your writing, and also for directing this from the general to > the program

Re: [Jprogramming] Report on the J wiki meeting of January 27, 2022

One of the problems you're going to run into right away with making dictionaries a core part of J is the concept of rank, which doesn't really exist in K (everything is a rank 1 list, hooked together by dictionaries). At the end of the day, what I think most people want from maps/dictionaries is a

Re: [Jprogramming] HMAC and PBKDF2

I wrote this a couple of years ago. It's a bit more bit fiddly, and I know it works as it was used in production to talk to an exchange. NB. hmac signature NB. x key, y needs signature NB. key may need to be made raw first hmac=: 4 : 0 key=. x,(128 - #x) # 0{a. ixorkey =. ((22 b.)&(16b36) a. i.

Re: [Jprogramming] math/flann

FWIIW I wrote the original version of this. Never tried to build FLANN on windows (I've never run J on windows), so no idea what it does there. I haven't touched it in a long time, but I confirm there is something wrong with it on my laptop/flann1.9/ubuntu 18.04 LTS and J901. Either something in t

Re: [Jprogramming] math/flann

s valid on older versions of J. (The J package release process is versioned against the J engine.) Thanks, -- Raul On Tue, Mar 10, 2020 at 7:27 PM Scott Locklin <locklin.scott@ > wrote: > FWIIW I wrote the original version of this. Never tried to build FLANN on > windows (I

[Jprogramming] DLL documentation update request

I haven't called a custom DLL in some time now; since I have more spare time, I was noodling around with old code. Took me a while to notice what was going on; "pass by reference" where you send a J array to a DLL to be written to wasn't working the way it had in the past. It actually is documente

Re: [Jprogramming] DLL documentation update request

>The safest way to convert pre-j807 code that will modify content of *data >is continue to use * but >remove the leading > (if there) and read result of cd . eg >pre-j807 >'...' cd cmd;nr;XX >now >r=. '...' cd cmd;nr;XX >XX=. _1{::r >or >XX=. _1{:: '...' cd cmd;nr;XX Ah, yikes, I see what's goi

Re: [Jprogramming] stock data programming how-to #4

There is definitely a way to do this by mapping the files, but if you don't know what mmap is you're going to have a hard time doing it. I think you might also have a hard time writing a better parser than is in the csv addon (which is pretty slow last I checked). Jd is really the right way to go

Re: [Jprogramming] Brave GitHub Rewards and dbrun ''

fwiiw sorry for OT, but if you guys have any troubles with Brave rewards, please let me know. On Mon, Nov 30, 2020 at 4:16 PM Henry Rich wrote: > The PFkey problem results from my changes in the most recent beta. > We'll look into it. > > Henry Rich > > On 11/30/2020 10:57 AM, John Baker wrote:

Re: [Jprogramming] J and Spreadsheets

Two alternatives: there's R methods for dealing with OpenOffice, and a J package for calling R methods. Probably not the experience you're looking for. There's also a couple of QT demos which have a spreadsheet like interface if you just want the experience of seeing the data laid out in graphical

[Jprogramming] Spatial trees

Mini intro: new to J; only been looking at it for two days now. Very impressed so far. I do finance/machine learning/numerics nerdery and need to stash my larger TS in a proper column DB. As such, JDB is what landed me here (I'm hoping it solves my TSDB problems), but I found a lot more than I e

Re: [Jprogramming] Matrix inverse query

>You may want to take a look at NuVoc, the jwiki section that I think that Ian >Clark organized just for the reason that you are describing. :) >http://www.jsoftware.com/jwiki/NuVoc If I could make a humble suggestion: NuVoc is not the language reference, but for most people, it should be the

[Jprogramming] Sparse array behavior

So, I have never used sparse arrays in J before, and am puzzled by some of the behavior. If I initialize an integer sparse vector BID, and set element 10 to be 90: BID=: 1 $. 100;0;3-3 BID=: (90) 10}BID   BID 10 │ 90 Cool;  but then, if I want to zero out the 10 element BID=: (90-90) 10}BID   

Re: [Jprogramming] Sparse array behavior

when not needed. >The dictionary says: "8&$. removes any rows where the value equals the zero >element" , so >BID =: 8$.BID >>2014-05-21 11:18 GMT+02:00 'Scott Locklin' via Programming < programming@>: >> So, I have never used sparse arr

Re: [Jprogramming] k-NN in Rust / F# / ocaml

I've finally got a kd-tree library I'm comfortable with wired up to J. It links to v1.8.4 of libflann, which can be found here: http://www.cs.ubc.ca/research/flann/ I can provide a linux dll if you need one, but you should probably build your own. My stubs can be found here: locklin/j-nearest-n

Re: [Jprogramming] k-NN in Rust / F# / ocaml

Erm, please excuse yahoo eating my link, but the link should have been: https://github.com/locklin/j-nearest-neighbor More to come, hopefully fairly soon. *previous message*** I've finally got a kd-tree library I'm comfortable with wired up to J. It links to v1.8.4 of libflann, whic

Re: [Jprogramming] k-NN in Rust / F# / ocaml

The Debian and Ubuntu builds for 1.7.x both dump core for me, so I'm assuming the 1.7 API is very different (though at a glance, it doesn't look real different; perhaps it is the flannparam struct). I would imagine they'll get around to using 1.8.4, as even that version is over a year and a half