hansva opened a new issue, #9465:
URL: https://github.com/apache/hop/issues/9465

   ### Apache Hop version?
   
   2.20.0
   
   ### Java version?
   
   21
   
   ### Operating system
   
   Other
   
   ### What happened?
   
   The EDI to XML transform sends every data element of a message through 
`StringEscapeUtils.escapeXml10` 
(`FastSimpleGenericEdifactDirectXMLParser.sanitizeText`). Almost no EDIFACT 
value contains a character that needs escaping, yet the call costs about 180 ns 
per value: 38% of the transform's CPU in a profile.
   
   The method is also untested. The only unit test of the plugin checks that 
the settings are saved and loaded. The integration test `main-0052-edi-to-xml` 
parses one sample message that has no character to escape and no release 
character.
   
   ### Proposed changes
   
   **1. Tests first, on the current code**
   
   Add unit tests for the parser's output, so that the change below is checked 
against today's behavior:
   
   - a message with values that need escaping: `&`, `<`, `>`, `"` and `'`;
   - the release character: `?+`, `?:`, `?'` and `??` inside a value;
   - accented characters, characters outside the basic plane, a lone surrogate, 
and control characters;
   - an empty value and an empty composite;
   - the sample message of the integration test, as a whole.
   
   Add a second sample to the integration test, or extend the existing one, 
with a value that needs escaping and one with a release character.
   
   **2. Escape only when needed**
   
   In `sanitizeText`, look at the characters of the value first. When none of 
them is one that `escapeXml10` changes, return the value as it is; otherwise 
call `escapeXml10` as today. The characters that pass are: tab, line feed, 
carriage return, the printable ASCII characters except `& < > " '`, and the 
characters from U+00A0 up, except the surrogates, U+FFFE and U+FFFF.
   
   The output does not change. Checked for every one of the 65,536 characters: 
`escapeXml10` changes none of the 63,419 that pass.
   
   `FastSimpleGenericEdifactDirectXMLParser` was generated from an ANTLR 
grammar that is not in the repository, so the method is changed in the Java 
file.
   
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: Transforms


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to