new-javacc commented on issue #1324:
URL: https://github.com/apache/jena/issues/1324#issuecomment-1179574359
So yeah the MORE pattern still keeps the whole thing in memory :( so I
nornally just use SKIP and collect image myself into a buffer like the attached
grammar which works well with the default and independent of the Charstream
logic/buffering. Some of these idioms were developed in 1996 for Java 1.0 with
32MB RAM machines lol so yeah time for updating them.
Anyway, the chunking charstream is not well tested for correctness or
performance so until that happens maybe you can simply get that and use it as a
USER_CHARSTREAM (unless you user my SKIP version).
```
TOKEN_MGR_DECLS:
{
static StringBuilder sb = new StringBuilder();
}
SKIP:
{
< STRING_LITERAL_BEGIN: "'''"> : LIT_BODY
}
<LIT_BODY> TOKEN:
{
<STRING_LITERAL_LONG1: "'''"> { matchedToken.image = (sb.toString());
}: DEFAULT
}
<LIT_BODY> SKIP: {
< ~[]> { sb.append(image); }
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
