Further information, using the initial e-mail to keep all relevant information
in one e-mail.
Today I uninstalled the 32-bit ooRexx 5.0beta and installed the 64-bit ooRexx
5.0beta (both from
SourceForge, downloaded February 4th 2017).
This time the program ran successfully to the end:
* there were more than 247,000 XML elements processed
o as already mentioned this may translate into million(s) call backs from
Java to Rexx as all
SAX events will cause callbacks to Rexx
+ some of these callbacks carry Java objects as arguments which will
get wrapped as a peer
ooRexx object (ultimately an instance of the class .BSF that has an
uninit method
defined), namely the Java types: "char[]", "Attributes", "Local"
from the callbacks
<https://docs.oracle.com/javase/7/docs/api/org/xml/sax/ContentHandler.html>
+ there are no return values (all callbacks are defined to be void)
* it took a total of 2:37 hours to SAX-parse a 10MB XML file,
o the SAX-parsing was over after 1:45 hours
+ the program constantly slowed down and in between paused for a
while (which got longer
and longer over time)
o after the SAX-parseing was over the Rexx program needed another 0:52
hours to complete
+ speculating/assuming that in this time all the uninits got executed
(which is fine)
* it consumed 8.7GB of memory
o ProcessExplorer reports a
+ Physical Memory "PeekWorking Set" of 8.795.796 K
+ Virtual Memory "Virtual Size" of 13.404.424 K with "Peak Private
Bytes" of 9.138.664 K
* it used 24% of the CPU over all 2:37 hours
o ProcessExplorer reports a
+ "Kernel Time" of 0:01:10.184
+ "User Time" of 2.32.57.960
+ "Total Time" of 2:34:08.144
+ "Cycles": 23.898.290.997.693
Will check the BSF4ooRexx call and code paths.
---rony
On 26.04.2017 20:28, Rony G. Flatscher wrote:
>
> A student is using BSF4ooRexx to parse a huge XML file using Java's SAX
> parser.
>
> The SAX (simple API for XML) works with a callback concept. In order to be
> eligible for these SAX
> callbacks the ContentHandler interface
> <https://docs.oracle.com/javase/7/docs/api/org/xml/sax/ContentHandler.html>
> must be implemented,
> which the Rexx program does with the help of BSF4ooRexx.
>
> As a result each single SAX event will cause an appropriate Rexx message to
> be generated and
> dispatched. The Rexx message is named after the ContentHandler method name
> and will receive all
> arguments documented there. All arguments are processed in native code, a
> Rexx array object will
> get created from the Java arguments.
>
> This works and was presented on one of the International Rexx symposiums and
> has become part of my
> lecture using Java camouflaged as ooRexx with the help of BSF4ooRexx. Here is
> a link to the
> symposium which contains links to the slides:
> <http://rexxla.org/events/2013/schedule.html>.
>
> Now, a student is in the need to parse a rather huge XML file (10 MB) and
> uses the SAX parser. She
> always hits the Rexx "Error 5: System resources exhausted"! I asked her to
> come up with a very
> small program to demonstrate the error and I am able to reproduce it reliably!
>
> Here is the rather short Rexx program:
> --- #!/usr/bin/rexx parse arg xmlFile if xmlFile="" then
> XMLFile="..\LB-HB.onlb" if
> sysFileExists(xmlFile)=.false then do say "File" pp(xmlFile) "does not exist,
> aborting." exit -1
> end rexxObject=.saxHandler~new();
> javaProxy=BSFCreateRexxProxy(rexxObject,,"org.xml.sax.ContentHandler")
> parser=bsf.loadClass("org.xml.sax.helpers.XMLReaderFactory")~createXMLReader
> parser~setContentHandler(javaProxy) eh=.errorHandler~new
> javaEH=BsfCreateRexxProxy(eh,,"org.xml.sax.ErrorHandler")
> parser~setErrorHandler(javaEH) signal on
> syntax start=.dateTime~new say pp(start)": start of parsing"
> parser~parse(XMLFile)
> end=.dateTime~new say pp(end)": end of parsing" say "done, needed:"
> pp(end-start) exit syntax: say
> "error at" rexxObject~nrOfElements "element" co=condition('o') say
> ppCondition2(co) ::requires
> "BSF.CLS" /* get the Java support */ ::requires "rgf_util2.rex" ::CLASS
> "SaxHandler" -- a Rexx
> content handler ("org.xml.sax.ContentHandler") ::attribute nrOfElements
> ::METHOD init --ooRexx
> constructor expose nrOfElements nrOfElements=0 ::METHOD startElement expose
> nrOfElements USE ARG
> uri, localName, qName, attributes nrOfElements+=1 if nrOfElements//1000=0
> then say nrOfElements
> "elements so far ..." ::METHOD characters -- the callback method for
> characters (text) USE ARG
> charArray, start, length ::METHOD endElement USE ARG uri, localName, qName,
> attributes ::method
> unknown ::class ErrorHandler ::method unknown use arg methName, argArray
> exception=argArray[1]s
> .error~say(methName": " "line="exception~getLineNumber
> ",col="exception~getColumnNumber": "
> pp(exception~getMessage)) ---
> As you might see, there is nothing "special" going on here, other than being
> able to supply a Rexx
> object as the callback object for the Java implemented SAX parser!
>
> It seems that running all these callback from Java to Rexx will cause
> eventually that ooRexx gets
> exhausted of system resources. While running the program it can be observed
> that from time to time
> there are little lags as if some garbage collecting takes place.
>
> Eventually the error 5 is thrown, while executing in native code:
>
> result_obj=rtc->SendMessage(ro, c_msg, ra); // with arguments
>
> Please note that the following error message only gives the numbers of XML
> elements that the SAX
> parser reported. In reality, *all* callbacks are carried out from Java to
> Rexx, the unprocessed
> ones will be intercepted by the Rexx UNKNOWN method. So there is a flurry of
> context switches and
> creation of Rexx array objects for the arguments and the like.
>
> Here is the error message on my system.
> --- 83000 elements so far ... 84000 elements so far ... error at 84926
> element [ADDITIONAL] =[an
> Array (2 items) id#_-1457160369] ["BSF4ooRexx/routine/BSF(), error 3: Java
> exception occurred:
> [org.rexxla.bsf.engines.rexx.RexxException:
> BSF4ooRexx/routine/jniRexxSendMessageToRexxObject(),
> error 4:" || "0A"x || " *-* Compiled method ""OBJECTNAME="" with scope
> ""Object""." || "0A"x ||
> "Error 5 running C:\Program Files (x86)\BSF4ooRexx\BSF.CLS line 2644: System
> resources exhausted."
> || "0A"x || "Error 5.0: message n/a]"]
> [org.rexxla.bsf.engines.rexx.RexxException@182d820
> id#_-1478994985] [CODE] =[40.900] [CONDITION] =[SYNTAX] [DESCRIPTION]=[]
> [ERRORTEXT] =[Incorrect
> call to routine.] [INSTRUCTION]=[SIGNAL] [MESSAGE]
> =["BSF4ooRexx/routine/BSF(), error 3: Java
> exception occurred: [org.rexxla.bsf.engines.rexx.RexxException:
> BSF4ooRexx/routine/jniRexxSendMessageToRexxObject(), error 4:" || "0A"x || "
> *-* Compiled method
> ""OBJECTNAME="" with scope ""Object""." || "0A"x || "Error 5 running
> C:\Program Files
> (x86)\BSF4ooRexx\BSF.CLS line 2644: System resources exhausted." || "0A"x ||
> "Error 5.0: message
> n/a]."] [PACKAGE] =[a Package id#_-2109388833] [POSITION] =[25] [PROGRAM]
> =[F:\tmp\orx\kirisits\20170425\SAX_noAlerts.rxj] [PROPAGATED] =[1] [RC] =[40]
> [STACKFRAMES]=[a
> List (4 items) id#_-1467321073] [ *-* Compiled routine "BSF".
> id#_-1939043649] [ 1811 *-* res=BSF(
> "invoke", "com.sun.org.apache.xerces.internal.parsers.SAXParser@117ae12",
> "PARSE" , a.1 )
> id#_-1938526417] [ 1813 *-* interpret code ")" -- execute this dynamically
> created Rexx string
> id#_-1937397361] [ 25 *-* parser~parse(XMLFile) id#_-1936143753] [TRACEBACK]
> =[a List (4 items)
> id#_-1458379065] [ *-* Compiled routine "BSF".] [ 1811 *-* res=BSF( "invoke",
> "com.sun.org.apache.xerces.internal.parsers.SAXParser@117ae12", "PARSE" , a.1
> )] [ 1813 *-*
> interpret code ")" -- execute this dynamically created Rexx string] [ 25 *-*
> parser~parse(XMLFile)] ---
> The traceback points to the invocation of the SAX parser from Rexx which only
> returns upon
> completion of the Java parse method, or in this case when an error interrupts
> the program.
>
> ---
>
> So now my question/request: how to debug this problem/error? How can I get
> more information from
> the ooRexx interpreter? How can one determine what kind of "system resources"
> got exhausted and
> what the cause would be? Can one e.g. determine how many Rexx objects there
> are and how many are
> dead, in the need of getting garbage collected?
>
> In this case whenever (on each SAX callback) the Java side invokes the native
> code which then
> sends the message to the Rexx object there will be an AttachThread() followed
> by a DetachThread().
>
> Please advise!
>
> ---rony
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
