Comments inline
On 24/07/2015 10:12, "Andy Seaborne" <[email protected]> wrote:
>Rob,
>
>Still some problems :-(
>
>
>1/ FREE_MEM
>
>For production use, are you expecting warnings on every run when the
>file size is larger than free memory? I have typically used tdblaoder2
>on large files way bigger than memory so a WARN each time the program
>runs is a bit excessive.
Agreed, a couple of warnings have been reduced to debug
>
>It isn't being calculated correctly - on non-darwin it calls
>
>FREE_MEM=$(free -b)
>
>and does no further processing but that is a long messy string.
>
>Which figure from multi-line output from "free -b" is it looking for?
>(what's {6} on OSX top output?)
>
>
>echo $OSTYPE
>==>
>linux-gnu
This appears to be a 2.x vs 3.x kernel issue
When I tested this on some of our internal Linux servers (which are 2.x
kernels) free -b just returns an integer, as you note on newer (3.x I
assume) kernels it instead prints a more complex output
I have changed the logic to now check for complex output and extract the
desired value and to do an extra check for numerics with errors suppressed
and to return the not available value if not numeric
I have also restricted the functionality to just OSTYPE linux*
>
>
>
>2/ The use of pv is causing terminal problems (sic).
>
>In a gnome terminal, character echo is turned off when tdbloader2 exits.
>In an emacs shell buffer, the load hangs.
>
>Haven't found out why pv is doing this. I wonder if two in the same
>line are causing character mix up on output leading to broken terminal
>control sequences, especially with the small file I was using.
>
>It's a really nice feature to have to see the progress. Would it be
>safer for this release to switch off pv to give time for testing in
>different environments (other *nixes, cygwin) with a simple HAS_PV=0 ?
Agreed, actually it is HAS_PV=1 because I just use the return of `which
pv` to detect it so 0 is enabled and non-zero is disabled
Have removed the auto-detection and defaulted it to off but users can set
HAS_PV=0 in their environment if they known pv works reliably in their
environment
Rob
>
> Andy
>
>On 23/07/15 15:40, Rob Vesse wrote:
>> Comments inline:
>>
>> On 23/07/2015 14:41, "Andy Seaborne" <[email protected]> wrote:
>>
>>> Trying to do a release, I came across some issues.
>>>
>>> JENA-992: (Refactor graph/permissions interface layer)
>>>
>>> Not sure of the status of this but I'm assuming that the code already
>>>in
>>> 'master' is releasable.
>>>
>>> JENA-997: (tdbloader2 script refactoring)
>>>
>>> The new scripts misbehave on Linux - there isn't one (obvious) issue.
>>>
>>> To unblock the release, if there is a small fix, then great. Another
>>> possibility is to revert to the older scripts for 3.0.0, so as to fix
>>> afterwards. This gives more time and space for testing.
>>
>> Looks to be relatively simple, think I have the bugs you identified
>> resolved
>>
>> For Case 1 I needed to look up the drive info based on the directory
>>where
>> the work files will be created and not the work file itself because that
>> doesn't exist yet. As part of fixing this I also made the script
>> resistant to errors where the drive information was unavailable
>>
>> For Case 2 I was checking the directory before I had ensured it existed
>> and was a directory so that just required changing the order of checks
>>
>> Rob
>>
>>>
>>> It looks to me like "bash" on OSX is bash 3.2 derived (3.2 was
>>> originally 2006) whereas on Ubuntu currently it is 4.3. There might be
>>> other issues that arise if the current ones are resolved with bash or
>>> other commands.
>>>
>>> Andy
>>
>>
>>
>>
>
