I've already done one trial build and if the shell script issue gets
sorted out, and I have time, (it's already taken quiet a lot of time)
then I hope to do the vote-able process as soon as possible, like
tomorrow. No promises but.
Andy
On 24/07/15 15:54, Claude Warren wrote:
I have one more piece to add to JENA-992. I have it at home working and
will add it this evening. Following that a clean build will close the
issue.
On Fri, Jul 24, 2015 at 11:41 AM, Rob Vesse <[email protected]> wrote:
Comments inline
On 24/07/2015 10:12, "Andy Seaborne" <[email protected]> wrote:
Rob,
Still some problems :-(
1/ FREE_MEM
For production use, are you expecting warnings on every run when the
file size is larger than free memory? I have typically used tdblaoder2
on large files way bigger than memory so a WARN each time the program
runs is a bit excessive.
Agreed, a couple of warnings have been reduced to debug
It isn't being calculated correctly - on non-darwin it calls
FREE_MEM=$(free -b)
and does no further processing but that is a long messy string.
Which figure from multi-line output from "free -b" is it looking for?
(what's {6} on OSX top output?)
echo $OSTYPE
==>
linux-gnu
This appears to be a 2.x vs 3.x kernel issue
When I tested this on some of our internal Linux servers (which are 2.x
kernels) free -b just returns an integer, as you note on newer (3.x I
assume) kernels it instead prints a more complex output
I have changed the logic to now check for complex output and extract the
desired value and to do an extra check for numerics with errors suppressed
and to return the not available value if not numeric
I have also restricted the functionality to just OSTYPE linux*
2/ The use of pv is causing terminal problems (sic).
In a gnome terminal, character echo is turned off when tdbloader2 exits.
In an emacs shell buffer, the load hangs.
Haven't found out why pv is doing this. I wonder if two in the same
line are causing character mix up on output leading to broken terminal
control sequences, especially with the small file I was using.
It's a really nice feature to have to see the progress. Would it be
safer for this release to switch off pv to give time for testing in
different environments (other *nixes, cygwin) with a simple HAS_PV=0 ?
Agreed, actually it is HAS_PV=1 because I just use the return of `which
pv` to detect it so 0 is enabled and non-zero is disabled
Have removed the auto-detection and defaulted it to off but users can set
HAS_PV=0 in their environment if they known pv works reliably in their
environment
Rob
Andy
On 23/07/15 15:40, Rob Vesse wrote:
Comments inline:
On 23/07/2015 14:41, "Andy Seaborne" <[email protected]> wrote:
Trying to do a release, I came across some issues.
JENA-992: (Refactor graph/permissions interface layer)
Not sure of the status of this but I'm assuming that the code already
in
'master' is releasable.
JENA-997: (tdbloader2 script refactoring)
The new scripts misbehave on Linux - there isn't one (obvious) issue.
To unblock the release, if there is a small fix, then great. Another
possibility is to revert to the older scripts for 3.0.0, so as to fix
afterwards. This gives more time and space for testing.
Looks to be relatively simple, think I have the bugs you identified
resolved
For Case 1 I needed to look up the drive info based on the directory
where
the work files will be created and not the work file itself because that
doesn't exist yet. As part of fixing this I also made the script
resistant to errors where the drive information was unavailable
For Case 2 I was checking the directory before I had ensured it existed
and was a directory so that just required changing the order of checks
Rob
It looks to me like "bash" on OSX is bash 3.2 derived (3.2 was
originally 2006) whereas on Ubuntu currently it is 4.3. There might be
other issues that arise if the current ones are resolved with bash or
other commands.
Andy
