I was hacking together a little script a few days ago whose purpose was to take
a downloaded data.csv file and change fields around a bit and then emit a tab
delimited result suitable for pasting into a spreadsheet.
When doing things like this, I always work from a command line jconsole
interface in either OSX or Linux. I got my logic sorted out and producing the
result I wanted and then added the bits to my script to accept stdin which is
also my custom for creating command line scripts. What surprised me was that
when I gave the file name and did a "readcsv" for it, things were good, but
when I passed the data through a pipe to the jscript I got a segmentation
fault...
I was working in j901 Beta-i in Mac OSX when I first encountered this problem.
I verified that the same problem occurred in Linux. I don't have access to
windows (or any work experience with command line interface there...)
I first thought it might be a Beta problem, but then noted that switching
Jengines to j807 did not clear the problem. Switching to j806 did cause the
problem to go away and gave me the expected results. In j901 both AVX and
nonAVX engines have the problem
I've spent several hours chasing this and trying to reduce the places to look
and this evening feel like I finally made some progress. Here is a simplified
version of the script in question -
iMi7:jseg jkt$ cat bfun # a jscript
#! /usr/local/bin/ja
3 : 0 ARGV
try. NB. catch errors.
echo JVERSION
require 'csv'
'call parms stin' =: getargs 'no fancy parms...' NB. parms e.g. -a10 <->
(,'a');'10'
if. ((0 = #stin) *. 3 > # call) do.
stderr 'Usage: ',(;1{call), ' argument list',NL
exit 2
tmp=: 1 2 3
else.
if. ( *#stin ) do.
dat=: (',';'"') fixdsv CR-.~ > stin
(3!:1 dat) fwrite 'bfstin'
else.
(3!:1 dat=. readcsv >2{y) fwrite 'bffin'
end.
tmp=: |. dat,. a:
echo $tmp
end.
catch.
stderr 13!:12 '' NB. at the very least...
end.
exit 0
)
I could eliminate the error catching, but this is a template that I use over
and over again so I left that in. Here is a result of running that script -
iMi7:jseg jkt$ ./bfun t2.csv
Engine: j807/j64nonavx/darwin
Release-d: commercial/2019-03-18T16:08:19
Library: 8.07.26
Platform: Darwin 64
Installer: J807 install
InstallPath: /applications/j64-807
Contact: www.jsoftware.com
16 6
# that is the expected result, a 16 6 boxed table
iMi7:jseg jkt$ cat t2.csv | ./bfun
Engine: j807/j64nonavx/darwin
Release-d: commercial/2019-03-18T16:08:19
Library: 8.07.26
Platform: Darwin 64
Installer: J807 install
InstallPath: /applications/j64-807
Contact: www.jsoftware.com
Segmentation fault: 11
The file "t2.csv" is a relatively small (972 bytes) text file containing
nothing unusual.
Here is where it gets even stranger, if I make the t2.csv file 16 bytes shorter
by removing characters from a text field, (or, as I originally discovered,
removing the CR chapters from the 16 line DOS file) then the output from my
test script (bfun) is what I would expect and the binary files written by (3!:1
dat) fwrite are identical.
If, instead of 16, I remove 10 characters from t2.csv then I get the following
result which may be a clue (but I'm depending on someone with much more
experience in the underlying code to comment, Henry, Roger, Bill??)
iMi7:jseg jkt$ cat t2.csv | ./bfun
Engine: j807/j64nonavx/darwin
Release-d: commercial/2019-03-18T16:08:19
Library: 8.07.26
Platform: Darwin 64
Installer: J807 install
InstallPath: /applications/j64-807
Contact: www.jsoftware.com
ja(56541,0x7fffbb8843c0) malloc: *** error for object 0x101841e0a: pointer
being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
iMi7:jseg jkt$
and, as I said this happens in j901 and j807 on Linux and Mac OSX (both AVX and
nonAVX) so it is clearly a problem that should get some attention.
The problem disappears if the require 'csv' scripts are not involved, and
that may be an additional hint for what is going sideways.
Strangely (or so it seems to me) the problem also disappears if the line
tmp=: |. dat,. a:
is removed from the script. So that seems to be the culprit line...
If needed, I can supply the 972 byte file to help someone else reproduce the
error. Let me know and I can send a zip file to a private address (since they
are, appropriately, not allowed in the forum).
- joey
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
