Re: Xalan 2.7.1 - File Handle Limitation?

Tue, 01 Nov 2016 14:27:59 -0700

As I remember it -- I'd have to re-read the code -- the Document Table Model used a 32-bit node id, split into two bitfields identifying documents and node within the document. This was parameterized, so in theory it would be possible to support more documents at the cost of reducing the maximum size of each -- but I don't remember whether that was a compile-time decision or if it could be adjusted when Xalan was started.

If you only need to scan those documents sequentially rather than having them all in memory at once, Xalan has (had?) an extension that discarded each one when the next was retrieved from the document() call. I'm far from certain, but that *might* also reuse document IDs, and thus solve your problem.

IBM's XTQHP optimizing compiler for XSLT used a different internal API to access documents, and hence avoided this field-width limitation -- but as far as I know that processor is still available only as the XSLT Feature for WebSphere Traditional and hasn't been made available any other way. Last I checked, IBM Java still shipped with a version of Xalan (though at the time, it was a significantly more recent version of Xalan than the one Sun was packaging with their JRE).

Hope that was at least somewhat useful...

______________________________________
"Your data is important to us. Please stay on the port, and your connection will be accepted by the next available thread."

-----Scott Boag/Cambridge/IBM wrote: -----
To: [email protected]
From: Scott Boag/Cambridge/IBM
Date: 11/01/2016 04:48PM
Cc: [email protected], Joseph Kesselman/Watson/IBM@lotus
Subject: Re: Xalan 2.7.1 - File Handle Limitation?

From my deep memory, I think there is indeed a limitation imposed by a 16-bit space embedded doc id in the node id, or something like that.  Joe may remember more details.

Sent from Lotus Traveler

basZero --- Xalan 2.7.1 - File Handle Limitation? ---

From:"basZero" <[email protected]>
To:[email protected]
Date:Tue, Nov 1, 2016 12:54
Subject:Xalan 2.7.1 - File Handle Limitation?

Hi,

I'm observing the following: 
my Java 1.8 application (web app on Tomcat 8) is running on Ubuntu 16 under a user that has a ulimit of 65000.

A certain application logic within my Java app is looping through 80000 different files. When this loop is performed in Java code, there is no issue, all data can be processed as expected. 

When I run the loop via XSLT by using the document() function (Xalan 2.7) to open the files, it stops opening files after approx. 32000, depending on the current load on our system. By stopping I mean that XSL can not access the content of the requested file anymore.

I'm wondering if there is a limitation besides the ulimit of Ubuntu within my technology stack here?

Any help regarding Xalan would be highly appreciated.
Thanks
Cheers, Bas


--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Reply via email to