Thanks for reply Dan,
I don't really know what I'm doing, but using your definitions
pwdcheck2 =: ('password' -: ]) NB. unlocked
PWDCHECK2 =: (<'pwdcheck2') <@]^:(1:`(<'@.')) 0
a =. memr (15!:14 <'PWDCHECK'), 0, 1, JINT
b =. memr (15!:14 <'PWDCHECK2'), 0, 1, JINT
memr a,0,16,JINT
56 0 184 262144 2 1 0 6443034000 6443033120 626935840960 0 0 0
9223372036854775807 9223372036854775807 9223372036854775807
memr b,0,16,JINT
56 0 184 262144 1 1 0 6443034000 6443033120 625002615568 0 0 0
9223372036854775807 9223372036854775807 9223372036854775807
They look very similar. I think you are suggesting that the first could be
unlocked by flipping one bit in here?
Though I don't see a single bit difference in an int there.
Does 9!:25 disable the 15!:n commands?
I guess even if it did, external processes can read memory.
Btw, the only secret I want to keep safe is the noun 'password' by embedding it
in a locked verb that is set at run time, and while the verb is much more
complicated, its template source is public, as is knowledge that the locked
verb name is 'pwdcheck', and the byte size of the noun is guessable (though
that can actually be obfuscated significantly). I'm much more worried about
external processes (virus/malware) than a process that could attach itself to
the J session and query it (though I may be unaware of things I'm unaware of).
Total process memory usage is not at all guessable, the timing for when the
verb will be set is not predictable, and the overall program is large for J.
The planned structure of verb is more like this template that can be easily
conjunctioned.
pwdcheck =: localvar&('password' [:`(dostuff@[)@.-: ])
----- Original Message -----
From: Dan Bron <[email protected]>
To: J Programming <[email protected]>
Cc:
Sent: Monday, August 10, 2015 5:08 PM
Subject: Re: [Jprogramming] unlock.ijs
REB wrote:
> No, Dan made a typo, he wrote /wsvn/ instead of /svn/
Blasphemy. Dan does not make mistakes ;)
In this case, I had used the recommended best practice of MoinMoin interwiki
links: I never typed wsvn or svn at all; instead, what I had written was:
[[JSvn:DanBron/trunk/general/unlock.ijs|unlock.ijs]]
The “JSvn:” interwiki shortcut is a layer of indirection which is supposed to
protect Wiki editors from volatile URLs (while at the same time offering typing
shortcuts which make linking more convenient).
However, in this case, the Interwiki repository has not been kept up to date.
In http://www.jsoftware.com/jwiki/InterWiki
<http://www.jsoftware.com/jwiki/InterWiki>, we see that JSvn is still mapped to
http://www.jsoftware.com/wsvn/ <http://www.jsoftware.com/wsvn/>, though svn has
been rebased to /svn/ (as you said) since.
I’m not a MoinMoin administrator, so I can’t change the official Interwiki
page, so I hard-coded the direct /svn/ link in the Unlock page. I don’t have
the time or will to go through other pages I control to make more systematic
changes. Better for Chris or some other MoinMoin admin to update all the
/wsvn/ links on the central Interwiki page*.
Pascal (originally) asked:
> does the unlock script need access to the locked file, or
> does it do everything in memory?
It does everything in memory, but it relies on an exploit that (I believe) has
since been patched. In short, this used to produce the definition of the quoted
name, even when locked:
'pwdcheck' f.
pwdcheck
As you can see, as of J803 (at most), it no longer does.
The only other trick the unlock script exploited was 4!:5, used precisely as
intended: to discover which names a script defines. That’s necessary because
the “interesting” parts of a script may not be exposed in the documented
interface.
> Dan alluded that he could attack it in memory with undocumented 15!:6
The unlock script depended only on the (now patched) exploit ‘name’ f. . It
didn’t make any use of 15!:6 .
But you’re right that I made the claim:
> I want to stress that the exploit above isn't the problem, only an
> demonstration of it. Even if Roger "fixed" f. and 4!:5, other
> approaches are possible. For example, I could use 15!:6 and 15!:7.
So, now that the very situation I described has occurred (i.e. Roger fixed f.),
can I carry out my threat?
Well … not with a simple application of 15!:6 **. However unintentional, that
foreign can no longer be applied to pro-verbs, pro-adverbs, or pro-conjunctions
[1]. It can only be used to get the address of pro-nouns these days. Which has
frustrated me on more than one occasion (see a list of personal utilities this
change broke in the postscript to [2]).
However, while the easy paths have been closed, the point still remains: 3!:6
doesn’t encrypt files, it merely encodes them. As I said in the original
message (and Wiki page), and others have pointed out in this very thread, it is
not possible keep secrets from a user when the user himself holds the key.
If you’re not entering a key, personally, every time a name is unlocked,
someone else is. In this case, that someone is your user (via J).
With all that said, Raul and the others are right: security is a matter of
degree, not binary, and J itself is a kind of encoding layer which will
dissuade all but the most persistent of snoopers. Add a bit of 3!:6 encoding on
top, and I’d call that “secure enough” for non-critical secrets.
-Dan
[1] J602 introduces breaking change to undocumented foreign 15!:6 :
http://www.jsoftware.com/pipermail/general/2008-November/032554.html
<http://www.jsoftware.com/pipermail/general/2008-November/032554.html>
[2] List of personal utilities the change to 15!:6 broke for me
(see postscript of this message):
http://www.jsoftware.com/pipermail/programming/2013-October/033760.html
<http://www.jsoftware.com/pipermail/programming/2013-October/033760.html>
* Upon investigation, it appears that /wsvn/ is also a valid URL, but only for
web-browsing, i.e. prettified, for a subset of repositories hosted at /svn/.
** Argh, ok. I don’t like to make assertions I can’t back up; I don’t want to
develop a reputation for making empty threats. So…
Earlier, Roger had said he didn’t know what had changed in J602 that made it
stop working for proverbs etc [3], so I had a little look in the source.
Here’s what I found:
x.c (foreign definitions), line 201:
https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/x.c#L201
<https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/x.c#L201>
case XC(15,6): R SDERIV(CIBEAM,jtdllsymget,0,RMAX,0,0);
s.c (symbol table), line 222:
https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/s.c#L222
<https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/s.c#L222>
F1(jtdllsymget){R dllsymaddr(w,0);}
and line 207:
https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/s.c#L207
<https://github.com/openj/core/blob/18fd23bbdc2f50770eb3047e978cd5e4e3b47039/s.c#L207>
static A jtdllsymaddr(J jt,A w,C flag){A*wv,x,y,z;I i,n,wd,*zv;L*v;
RZ(w);
n=AN(w); wv=AAV(w); wd=(I)w*ARELATIVE(w);
ASSERT(!n||BOX&AT(w),EVDOMAIN);
GA(z,INT,n,AR(w),AS(w)); zv=AV(z);
for(i=0;i<n;++i){
x=WVR(i); v=syrd(nfs(AN(x),CAV(x)),0L);
ASSERT(v,EVVALUE);
y=v->val;
ASSERT(NOUN&AT(y),EVDOMAIN);
zv[i]=flag?(I)AV(y):(I)v;
}
R z;
} /* 15!:6 (0=flag) or 15!:14 (1=flag) */
So the culprit here is line 216:
ASSERT(NOUN&AT(y),EVDOMAIN);
That is, J has the answer I want in-hand already — it just refuses to give it
to me. (To interpret this line, one must understand that there’s only one
single data structure in the J engine, which is used to represent nouns, verbs,
adverbs, conjunctions, and even some other stuff. The NOUN&(AT) part says
“throw an error unless this data structure is representing a noun".)
Now, since 15!:6 was originally introduced for the purposes of passing J data
to other processes, it makes sense, and is a reasonable safety check, to
restrict its use to nouns. But I have other uses for this function, which are
hampered by this restriction.
So what can we do about it? J will refuse to return the address of any name
which is not a noun. But we want the address of a verb. Somehow we need to
pass 15!:6 a verb, but have J believe it’s a noun.
Our mind first wanders to things like PWDCHECK=:’pwdcheck’, but then we realize
nothing along those lines will work (the string ‘pwdcheck’ is just
coincidentally spelled the same as the name of a function, but 15!:6 won’t be
able to make that connection; it will just return the address of the literal
noun ‘pwdcheck’ - wherever those 8 characters are stored in memory).
Then we remember the “disaster which smells like a breakthrough”, as Tracy so
eloquently put it on Twitter [4]. Let’s use Marshall’s discovery [5] that that
“breakthrough” permits us to box verbs:
PWDCHECK =: (<'pwdcheck') <@]^:(1:`(<'@.')) 0
PWDCHECK
+--------+
|pwdcheck|
+--------+
type'PWDCHECK'
+----+
|noun|
+----+
PWDCHECKOPENED=:>pwdcheck
type'PWDCHECKOPENED'
+----+
|verb|
+----+
(>PWDCHECK) 'password'
1
(>PWDCHECK) 'not the password'
0
Ok, here we have it. A verb cloaked as a noun, so that it can pass the test
ASSERT(NOUN&AT(y),EVDOMAIN) . Let’s try it:
15!:6 <'PWDCHECK'
140618867255944
No domain error. Great. But now we have to remember that 140618867255944 is the
address of the box which contains the verb pwdcheck, not the verb pwdcheck
itself. What we need is the value of the first element of the boxed array,
which, going off the comment on the final line of the function jtdllsymaddr(),
looks like we can get using the (also undocumented) foreign 15!:14 (the macro
AV(x) returns the address of the “ravel”, or value, of the struct x):
15!:14 <'PWDCHECK'
140618884704072
But this being a box, that address itself probably contains a pointer to the
header of the function pwdcheck:
memr (15!:14 <'PWDCHECK'), 0, 1, JINT
140618882427152
Which looks like a reasonable address, not far from the block which contains
PWDCHECK itself (which makes sense).
Indeed, if we use 15!:7 to set a new symbol pointing to this address:
‘someothername' =:15!:7 {. memr (15!:14 <'PWDCHECK'), 0, 1, JINT
someothername
pwdcheck
So we’re on the right track, for sure. But of course we’re not home yet -
we’re pointing ‘someothername’ right back to the _name_ ‘pwdcheck’, which
doesn’t get us anywhere.
The next step would be to chase pointers to the value underlying ‘pwdcheck’ and
manipulate the “flags” field to turn off the VLOCK flag (bit #16), as in:
(memw~ (26 b. VLOCK=:65536) 17 b. memr) CORRECT_POINTER, 8,1,JINT
Unfortunately, I’ve run out of time for pointer-chasing today. Nevertheless, in
my mind the rest of this is mere stamp-collecting: we have the beginning of the
thread, now we only need to pull it carefully to unravel the whole sweater.
[3] Roger on the change to 15!:6:
http://www.jsoftware.com/pipermail/programming/2010-January/017946.html
<http://www.jsoftware.com/pipermail/programming/2010-January/017946.html>
[4] Tracy on twitter:
https://twitter.com/kaleidic/status/296101561459953664
<https://twitter.com/kaleidic/status/296101561459953664>
[5] Marshall discovers that the stupid ^:(<‘@.’) trick allows us to box verbs:
http://www.jsoftware.com/pipermail/programming/2013-January/031260.html
<http://www.jsoftware.com/pipermail/programming/2013-January/031260.html>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
