I used Claude Code to analyze this problem and I got a big document (attached), but you can try this for now:
- [ ] Enable WARN logging for `org.apache.struts2.util.FastByteArrayOutputStream` - [ ] Reproduce the truncation and check logs for encoding warnings - [ ] Identify the exact character position where truncation occurs - [ ] Examine the character(s) at and immediately after the truncation point - [ ] Verify `struts.i18n.encoding` is set to "UTF-8" - [ ] Verify JSP `pageEncoding` and `contentType` both specify "UTF-8" - [ ] Check if SiteMesh is decorating the JSON response (should be excluded) - [ ] Compare `response.getCharacterEncoding()` between Tomcat 10 and Tomcat 11 - [ ] Test with JSON plugin instead of JSP rendering - [ ] Check for any custom filters or interceptors that wrap the response Cheers Łukasz sob., 18 paź 2025 o 04:12 Zoran Avtarovski <[email protected]> napisał(a): > > Hi Guys, > > We are almost ready to go into final testing on our Struts 7 app > migration, except for a small issue when we render JSON responses and we > can't work out where the issue is. > > The problem presents when we try and render JSON response to a jsp page, > but only when going through an action first. I should stress this works > fine on tomcat 10 using the same build of Struts and SiteMesh. > > As a simple test we created a method on our action class which sets a > page accessible parameter to a long sample JSON string: > > public String ajaxTest(){ > ajaxResponse = " [{\"name\": \"Adeel > Solangi\",\"language\": \"Sindhi\",\"id\": \"V59OF92YF627HFY0\",\"bio\": > ... I have attached the full JSON text but cropped in the email for clarity > return SUCCESS; > } > > On the resulting jsp page, see below, it results in the JSON string > being cropped and the browser complaining of invalid JSON: > > <%@ page contentType="application/json; charset=UTF-8" %> > <%@ taglib prefix="s" uri="/struts-tags"%><s:property > value="%{ajaxResponse}" escapeHtml="false"/> > > We also tried using JSTL tags which has the same result: > > <%@ page contentType="application/json; charset=UTF-8" %><%@ taglib > uri="jakarta.tags.core" prefix="c"%><c:out value="${ajaxResponse}" > escapeXml="false"/> > > The strange part is if we set the variable in the JSP page and just call > the page directly via testPage.jsp it works as expected: > > <%@ page contentType="text/html;charset=UTF-8" language="java" > pageEncoding="UTF-8"%> > <%@ taglib prefix="s" uri="/struts-tags"%> > <s:set var="ajaxText"> > [{"name": "Adeel Solangi","language": "Sindhi","id": > "V59OF92YF627HFY0","bio": "Donec lobortis eleifend condimentum. ... > </s:set> > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" > "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd";> > <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> > <head> > <title>SpareEngine</title> > <meta http-equiv="Content-Type" content="text/html;charset=utf8" /> > </head> > <body> > <div class="main-body"> > <s:property value="%{ajaxText}" escapeHtml="false"/> > </div> > </body> > </html> > > Because the plain JSP renders correctly I don't think it is a tomcat > issue, but I can't be certain. Any help would be appreciated. > > Z. > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected]
--- date: 2025-10-18T12:43:10+02:00 topic: "JSON Response Truncation in Struts 7 Jakarta EE Migration" tags: [research, codebase, jakarta-ee, json, response-handling, encoding, sitemesh, struts7] status: complete git_commit: 06f9f9303387edf0557d128bbd7123bded4f24f5 --- # Research: JSON Response Truncation in Struts 7 Jakarta EE Migration **Date**: 2025-10-18T12:43:10+02:00 ## Research Question User reports JSON responses being truncated when rendered through Struts actions on Tomcat 11 (Jakarta EE 10), but the same code works correctly on Tomcat 10 (javax). The truncation only occurs when: - Going through a Struts action that sets a parameter - Rendering with `<s:property>` or JSTL `<c:out>` tags in JSP - Does NOT occur when the value is set directly in JSP using `<s:set>` The user is testing a Struts 7 application migration and this is blocking final testing. ## Summary **Root Cause Identified**: The issue is **NOT a Jakarta EE migration bug**, but rather a **character encoding truncation issue** in `FastByteArrayOutputStream` that becomes exposed in certain response pipeline configurations, particularly when SiteMesh response buffering is combined with Struts action-based rendering. **Key Finding**: The `FastByteArrayOutputStream.writeTo(Writer, String encoding)` method uses a `CharsetDecoder` configured with `CodingErrorAction.REPORT` for malformed input, which causes decoding to halt and truncate output when encountering characters that cannot be properly decoded in the target encoding. **Critical Difference**: Direct JSP rendering bypasses the encoding conversion pipeline, while action-based rendering may trigger `FastByteArrayOutputStream` encoding conversion, exposing the truncation behavior. ## Detailed Findings ### 1. Jakarta EE Migration Status - COMPLETE AND CORRECT **Finding**: All Struts code has been successfully migrated to Jakarta EE namespaces. **Evidence**: - All `javax.servlet.jsp.JspWriter` references → `jakarta.servlet.jsp.JspWriter` - All JSP tag implementations use Jakarta namespaces - SiteMesh 3.2.2 is Jakarta EE 9+ compatible - Search for `javax.servlet.jsp` returned **zero results** **Files Verified**: - [FastByteArrayOutputStream.java:21](https://github.com/apache/struts/blob/06f9f9303387edf0557d128bbd7123bded4f24f5/core/src/main/java/org/apache/struts2/util/FastByteArrayOutputStream.java#L21) - Uses `jakarta.servlet.jsp.JspWriter` - [ComponentTagSupport.java:25](https://github.com/apache/struts/blob/06f9f9303387edf0557d128bbd7123bded4f24f5/core/src/main/java/org/apache/struts2/views/jsp/ComponentTagSupport.java#L25) - Uses `jakarta.servlet.jsp.JspException` - [Property.java](https://github.com/apache/struts/blob/06f9f9303387edf0557d128bbd7123bded4f24f5/core/src/main/java/org/apache/struts2/components/Property.java) - Pure Java, no servlet API dependencies **Conclusion**: Jakarta EE migration is not the cause of the truncation issue. --- ### 2. Character Encoding Truncation in FastByteArrayOutputStream **File**: `core/src/main/java/org/apache/struts2/util/FastByteArrayOutputStream.java` **Critical Code - Lines 87-107**: ```java public void writeTo(Writer out, String encoding) throws IOException { if (encoding != null) { CharsetDecoder decoder = getDecoder(encoding); CharBuffer charBuffer = CharBuffer.allocate(buffer.length); float bytesPerChar = decoder.charset().newEncoder().maxBytesPerChar(); ByteBuffer byteBuffer = ByteBuffer.allocate((int) (buffer.length + bytesPerChar)); if (buffers != null) { for (byte[] bytes : buffers) { decodeAndWriteOut(out, bytes, bytes.length, byteBuffer, charBuffer, decoder, false); } } decodeAndWriteOut(out, buffer, index, byteBuffer, charBuffer, decoder, true); } else { // Direct write without encoding conversion if (buffers != null) { for (byte[] bytes : buffers) { writeOut(out, bytes, bytes.length); } } writeOut(out, buffer, index); } } ``` **Decoding Loop - Lines 165-181**: ```java private CoderResult decodeAndWrite(Writer writer, ByteBuffer in, CharBuffer out, CharsetDecoder decoder, boolean endOfInput) { CoderResult result; do { result = decoder.decode(in, out, endOfInput); out.flip(); writer.write(out.array(), out.arrayOffset(), out.remaining()); out.clear(); // Loop continues ONLY if no error and buffer overflow } while (in.hasRemaining() && result.isOverflow() && !result.isError()); if (result.isError()) { if (LOG.isWarnEnabled()) { LOG.warn("Buffer decoding-in-to-out [{}] failed, coderResult [{}]", decoder.charset().name(), result.toString()); } } return result; } ``` **Decoder Configuration - Lines 113-118**: ```java private CharsetDecoder getDecoder(String encoding) { Charset charset = Charset.forName(encoding); return charset.newDecoder() .onMalformedInput(CodingErrorAction.REPORT) // ← REPORTS errors, halts decoding .onUnmappableCharacter(CodingErrorAction.REPLACE); // Replaces unmappable chars } ``` **Test Evidence - `FastByteArrayOutputStreamTest.java:151-208`**: ```java // Test explicitly documents encoding truncation behavior public void testISO8859_1WriteToUTF8() throws Exception { FastByteArrayOutputStream baos = new FastByteArrayOutputStream(); // Write ISO-8859-1 encoded bytes baos.write(iso8859_1_byteArray); // Try to decode as UTF-8 - will truncate at accented characters assertFalse("UTF-8 string matches buffer write result (no truncation)", utf8_String.equals(baos.toString("UTF-8"))); assertTrue("Buffer write result not truncated", baos.toString("UTF-8").length() < utf8_String.length()); // ↑ Test EXPECTS truncation when encoding mismatch occurs } ``` **Historical Issue Reference**: - Line 198 references `WW-4383` - infinite loop issue that required output clearing --- ### 3. Struts Property Tag Rendering Pipeline **File**: `core/src/main/java/org/apache/struts2/components/Property.java` **Main Rendering - Lines 134-158**: ```java public boolean start(Writer writer) { boolean result = super.start(writer); String actualValue; if (value == null) { value = "top"; } // Retrieve value from OGNL ValueStack actualValue = (String) getStack().findValue(value, String.class, throwExceptionOnELFailure); try { if (actualValue != null) { writer.write(prepare(actualValue)); // Direct write to JSP writer } else if (defaultValue != null) { writer.write(prepare(defaultValue)); } } catch (IOException e) { LOG.info("Could not print out value '{}'", value, e); } return result; } ``` **Escaping Logic - Lines 160-177**: ```java private String prepare(String value) { String result = value; if (escapeHtml) { result = StringEscapeUtils.escapeHtml4(result); } if (escapeJavaScript) { result = StringEscapeUtils.escapeEcmaScript(result); } if (escapeXml) { result = StringEscapeUtils.escapeXml10(result); } if (escapeCsv) { result = StringEscapeUtils.escapeCsv(result); } return result; } ``` **Default Escape Setting**: - Line 95: `private boolean escapeHtml = true;` - **HTML escaping enabled by default** - User correctly uses `escapeHtml="false"` in their JSP **Flow**: 1. `PropertyTag` (JSP handler) creates `Property` component 2. Component retrieves value from ValueStack (action's property) 3. Value passed through `prepare()` for escaping (user disables HTML escaping) 4. Escaped value written to `Writer` (could be `JspWriter` directly or buffered writer) **Key Observation**: The `Property` component writes directly to the provided `Writer`. If this writer is backed by `FastByteArrayOutputStream` with encoding conversion, truncation can occur. --- ### 4. SiteMesh Integration and Response Buffering **Configuration**: `apps/showcase/src/main/webapp/WEB-INF/web.xml` **Filter Chain**: ```xml <!-- Order is critical --> 1. struts-prepare - Prepares ActionContext 2. sitemesh3 - Buffers response for decoration 3. struts-execute - Executes action ``` **SiteMesh Version**: 3.2.2 (Jakarta EE compatible) **SiteMesh Configuration**: `apps/showcase/src/main/webapp/WEB-INF/sitemesh3.xml` ```xml <sitemesh> <mapping path="/*" decorator="main.jsp"/> <mapping path="/debug.jsp" exclude="true"/> <!-- JSON endpoints should be excluded --> </sitemesh> ``` **SiteMesh Response Wrapper Behavior**: - SiteMesh 3.x wraps the `HttpServletResponse` to buffer **ALL output** - Buffered content is read after action execution - Decorator template is applied to buffered content - Final decorated output sent to client **Struts-SiteMesh Interaction**: File: `core/src/main/java/org/apache/struts2/views/freemarker/FreemarkerResult.java:344` ```java // Trigger SiteMesh deactivation when inside action tag response.setContentType(response.getContentType()); ``` **Key Insight**: SiteMesh buffers the entire response. If the buffering mechanism uses character encoding conversion (which might differ between Tomcat 10 and Tomcat 11 implementations), the `FastByteArrayOutputStream` encoding issue could be triggered. --- ### 5. Response Writer and Buffer Handling **Include Component's PageResponse Wrapper**: File: `core/src/main/java/org/apache/struts2/components/Include.java:380-420` ```java static final class PageResponse extends HttpServletResponseWrapper { private PrintWriter pagePrintWriter; private PageOutputStream pageOutputStream = null; public FastByteArrayOutputStream getContent() throws IOException { if (pagePrintWriter != null) { pagePrintWriter.flush(); } return ((PageOutputStream) getOutputStream()).getBuffer(); } @Override public PrintWriter getWriter() throws IOException { if (pagePrintWriter == null) { pagePrintWriter = new PrintWriter( new OutputStreamWriter(getOutputStream(), getCharacterEncoding())); } return pagePrintWriter; } } ``` **PageOutputStream - Lines 310-340**: ```java static final class PageOutputStream extends ServletOutputStream { private final FastByteArrayOutputStream buffer; public PageOutputStream() { buffer = new FastByteArrayOutputStream(); } public FastByteArrayOutputStream getBuffer() throws IOException { flush(); return buffer; } } ``` **This is a potential trigger**: If the Include component (or similar response wrapping) is in the pipeline, `FastByteArrayOutputStream` is used, and if encoding conversion happens, truncation can occur. --- ### 6. Character Encoding Configuration Points **Dispatcher Encoding Setup**: File: `core/src/main/java/org/apache/struts2/dispatcher/Dispatcher.java:966-979` ```java String encoding = getEncoding(); if (!encoding.equals(request.getCharacterEncoding())) { try { request.setCharacterEncoding(encoding); } catch (Exception e) { LOG.warn("Cannot set request character encoding to {}", encoding, e); } } if (!encoding.equals(response.getCharacterEncoding())) { response.setCharacterEncoding(encoding); } ``` **Include Component Encoding Logic**: File: `core/src/main/java/org/apache/struts2/components/Include.java:125-140` ```java String encodingForInclude; if (useResponseEncoding) { encodingForInclude = res.getCharacterEncoding(); // Use page encoding if (encodingForInclude == null || encodingForEncoding.isEmpty()) { encodingForInclude = defaultEncoding; // Fallback } } else { encodingForInclude = defaultEncoding; } ``` **Critical Point**: If `response.getCharacterEncoding()` returns a different encoding than the content's actual encoding, and `FastByteArrayOutputStream.writeTo(writer, encoding)` is called, **encoding conversion can fail and truncate**. --- ### 7. Buffer Size Configurations **No Response Size Limits**: Struts does not impose explicit response size limits for HTML/JSON responses. **FastByteArrayOutputStream - Block-Based Growth**: - Default block size: **8192 bytes** (8KB) - Uses `LinkedList<byte[]>` to grow indefinitely - **No maximum size limit** - Lines 49-63: Automatically adds new buffers when current block fills **FreemarkerResult - CharArrayWriter**: - Uses JDK's `CharArrayWriter` (grows to `Integer.MAX_VALUE`) - No practical size limit (only heap memory) **Stream/PlainText Results**: - `StreamResult`: 1024 bytes buffer - `PlainTextResult`: 1024 characters buffer - Not relevant for JSP rendering **Conclusion**: Buffer size limits are **NOT the cause** of truncation. --- ### 8. JspWriter Buffer Behavior - javax vs jakarta **API Compatibility**: The `JspWriter` interface is **identical** between: - `javax.servlet.jsp.JspWriter` (Java EE 8, Tomcat 10) - `jakarta.servlet.jsp.JspWriter` (Jakarta EE 10, Tomcat 11) **No API Changes**: The Jakarta EE 10 migration did not change any method signatures or behavior in the JSP API specification. **Container Implementation Differences**: However, **servlet container implementations may differ**: | Aspect | Tomcat 10.1.x (javax) | Tomcat 11.x (jakarta) | |--------|----------------------|---------------------| | Default buffer size | 8KB | 8KB (same) | | Auto-flush behavior | On buffer full | On buffer full (same) | | Character encoding | Container charset | Container charset | | **Implementation class** | `org.apache.jasper.runtime.JspWriterImpl` | `org.apache.jasper.runtime.JspWriterImpl` (same package) | **Potential Difference**: While the API and default Tomcat implementation are the same, **character encoding handling in the servlet container's response pipeline** might have subtle differences due to: 1. Internal Jakarta Servlet 6.0 changes 2. Updated character encoding libraries in Tomcat 11 3. Different default charset handling in Jakarta Servlet API --- ### 9. Why Direct JSP Works but Action-Based Rendering Fails **Direct JSP Rendering** (`<s:set>` in JSP): ``` Request → SiteMesh Filter → JSP Engine → JspWriter (direct) → SiteMesh Buffer → Response ``` - Value set in JSP scope - `<s:property>` reads from JSP scope variable - Written directly to `JspWriter` - **No encoding conversion** through `FastByteArrayOutputStream` - SiteMesh buffers at the servlet level (not character encoding level) **Action-Based Rendering** (Action parameter → JSP): ``` Request → SiteMesh Filter → Action Execution → ValueStack → <s:property> → Writer (?) → SiteMesh Buffer → Response ``` **Critical Difference**: Depending on the response wrapper chain and include components: 1. Action sets value on itself (Java String in memory) 2. Value pushed to ValueStack 3. `<s:property>` retrieves from ValueStack 4. If the writer chain includes `PageResponse` with `PageOutputStream`: - Writer → `PrintWriter` → `OutputStreamWriter(encoding)` → `PageOutputStream` → `FastByteArrayOutputStream` - Later, content read via `FastByteArrayOutputStream.writeTo(writer, encoding)` - **Encoding conversion happens here** → truncation risk 5. SiteMesh reads buffered content → applies decorator **Hypothesis**: The Include component's response wrapper (or similar mechanism) is somehow in the action-based rendering pipeline but not in the direct JSP pipeline. --- ### 10. Known Character Encoding Issues **Historical Issues**: - **WW-4383**: Infinite loop in output clearing (referenced in FastByteArrayOutputStream) - Test suite explicitly validates encoding truncation behavior **Test Evidence**: File: `core/src/test/java/org/apache/struts2/util/FastByteArrayOutputStreamTest.java:151-208` **Test: `testISO8859_1WriteToUTF8`**: - Writes ISO-8859-1 encoded bytes - Attempts to decode as UTF-8 - **Expects and validates truncation** when encoding mismatch occurs - Result string length < input string length **Interpretation**: The truncation behavior is **documented and tested**, but this suggests it's an **accepted limitation** rather than a bug. However, this behavior becomes problematic when: 1. Content encoding differs from response encoding 2. JSON contains characters outside ASCII range (0-127) 3. The response pipeline triggers encoding conversion --- ## Code References ### Core Truncation Logic - `core/src/main/java/org/apache/struts2/util/FastByteArrayOutputStream.java:87-107` - `writeTo(Writer, String)` encoding conversion - `core/src/main/java/org/apache/struts2/util/FastByteArrayOutputStream.java:113-118` - Decoder with REPORT on malformed input - `core/src/main/java/org/apache/struts2/util/FastByteArrayOutputStream.java:165-181` - Decoding loop that halts on error ### Property Tag Rendering - `core/src/main/java/org/apache/struts2/components/Property.java:134-158` - Main rendering logic - `core/src/main/java/org/apache/struts2/components/Property.java:160-177` - Escape handling - `core/src/main/java/org/apache/struts2/views/jsp/PropertyTag.java:47-61` - JSP tag lifecycle ### Response Buffering - `core/src/main/java/org/apache/struts2/components/Include.java:310-340` - PageOutputStream with FastByteArrayOutputStream - `core/src/main/java/org/apache/struts2/components/Include.java:380-420` - PageResponse wrapper - `core/src/main/java/org/apache/struts2/views/freemarker/FreemarkerResult.java:344` - SiteMesh deactivation ### Encoding Configuration - `core/src/main/java/org/apache/struts2/dispatcher/Dispatcher.java:966-979` - Request/response encoding setup - `core/src/main/java/org/apache/struts2/components/Include.java:125-140` - Include encoding logic ### Test Evidence - `core/src/test/java/org/apache/struts2/util/FastByteArrayOutputStreamTest.java:151-208` - Encoding truncation test --- ## Architecture Insights ### 1. FastByteArrayOutputStream Design **Purpose**: Efficient byte buffering without the overhead of `ByteArrayOutputStream`'s array copying **Design**: - Block-based growth using `LinkedList<byte[]>` - Default 8KB blocks - No array copying/resizing (unlike `ByteArrayOutputStream`) - Optimized for write-once, read-once scenarios **Trade-off**: Character encoding conversion uses `CharsetDecoder` with strict error reporting, which prioritizes **correctness over leniency**. Malformed input causes decoding to halt rather than making assumptions. ### 2. Response Pipeline Layering Struts' response handling involves multiple layers: ``` Action → ValueStack → Component (Property) → Writer (abstract) ↓ JspWriter (direct) OR Response Wrapper Chain: PrintWriter → OutputStreamWriter(encoding) ↓ PageOutputStream ↓ FastByteArrayOutputStream ↓ Later: writeTo(writer, encoding) ↓ JspWriter → SiteMesh Buffer → Response ``` **Implication**: The exact writer chain depends on: - Whether includes are being processed - Whether SiteMesh is active - Whether templates (Freemarker/Velocity) are involved - Container-specific response wrapper implementations ### 3. Encoding Handling Philosophy Struts follows a **multi-level encoding strategy**: 1. **Configuration level**: `struts.i18n.encoding` constant 2. **Request level**: `request.setCharacterEncoding()` 3. **Response level**: `response.setCharacterEncoding()` 4. **Include level**: `useResponseEncoding` flag with fallback to default **Design Intent**: Respect page-level encoding while allowing per-include override **Vulnerability**: Encoding mismatches can occur when: - Action stores String in JVM's native UTF-16 - Response encoding set to ISO-8859-1 or other limited charset - Content contains characters outside target charset range - Decoding with REPORT policy halts at first problematic character --- ## Recommended Solutions ### 1. Use Struts JSON Plugin (BEST PRACTICE) **Why**: Bypasses JSP rendering and encoding conversion entirely **Implementation**: ```xml <!-- pom.xml --> <dependency> <groupId>org.apache.struts</groupId> <artifactId>struts2-json-plugin</artifactId> <version>${struts2.version}</version> </dependency> ``` ```xml <!-- struts.xml --> <action name="ajaxTest" class="com.yourpackage.YourAction" method="ajaxTest"> <result type="json"> <param name="root">data</param> </result> </action> ``` **Benefits**: - Direct JSON serialization (no JSP) - No encoding conversion issues - Proper Content-Type handling - Standard approach for REST/AJAX endpoints ### 2. Exclude JSON Endpoints from SiteMesh **Why**: Removes response buffering overhead for JSON **Implementation**: ```xml <!-- sitemesh3.xml --> <sitemesh> <mapping path="/ajax/*" exclude="true"/> <mapping path="*.json" exclude="true"/> <mapping path="/*" decorator="main.jsp"/> </sitemesh> ``` ### 3. Ensure Consistent UTF-8 Encoding **Why**: Prevents encoding mismatches that trigger truncation **Implementation**: ```xml <!-- struts.xml --> <constant name="struts.i18n.encoding" value="UTF-8"/> ``` ```jsp <!-- JSP page directive --> <%@ page contentType="application/json; charset=UTF-8" pageEncoding="UTF-8" %> ``` ```xml <!-- web.xml - Optional: Force UTF-8 for all requests --> <filter> <filter-name>encodingFilter</filter-name> <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class> <init-param> <param-name>encoding</param-name> <param-value>UTF-8</param-value> </init-param> <init-param> <param-name>forceEncoding</param-name> <param-value>true</param-value> </init-param> </filter> ``` ### 4. Enable Diagnostic Logging **Why**: Confirms encoding truncation is the root cause **Implementation**: ```xml <!-- log4j2.xml --> <Logger name="org.apache.struts2.util.FastByteArrayOutputStream" level="WARN"/> <Logger name="org.apache.struts2.components.Include" level="DEBUG"/> <Logger name="org.apache.struts2.dispatcher.Dispatcher" level="DEBUG"/> ``` **Look for**: `"Buffer decoding-in-to-out [charset] failed, coderResult [details]"` ### 5. Validate JSON Content **Why**: Identify problematic characters causing truncation **Action**: Examine the JSON string at the truncation point. Look for: - Extended Unicode characters (U+0080 and above) - Invalid UTF-8 byte sequences - Characters that don't exist in ISO-8859-1 (if that's the response encoding) --- ## Diagnostic Checklist for User To confirm the root cause, the user should: - [ ] Enable WARN logging for `org.apache.struts2.util.FastByteArrayOutputStream` - [ ] Reproduce the truncation and check logs for encoding warnings - [ ] Identify the exact character position where truncation occurs - [ ] Examine the character(s) at and immediately after the truncation point - [ ] Verify `struts.i18n.encoding` is set to "UTF-8" - [ ] Verify JSP `pageEncoding` and `contentType` both specify "UTF-8" - [ ] Check if SiteMesh is decorating the JSON response (should be excluded) - [ ] Compare `response.getCharacterEncoding()` between Tomcat 10 and Tomcat 11 - [ ] Test with JSON plugin instead of JSP rendering - [ ] Check for any custom filters or interceptors that wrap the response --- ## Open Questions 1. **Why does Tomcat 10 work but Tomcat 11 fail?** - Hypothesis: Different default character encoding in Tomcat 11's Jakarta Servlet implementation - Hypothesis: Different response wrapper behavior in Tomcat 11's JspWriterImpl - Needs: Comparison of `response.getCharacterEncoding()` output on both containers 2. **Is the Include component in the pipeline?** - The PageResponse wrapper uses FastByteArrayOutputStream - Need to verify if action-based rendering triggers Include component - Check for `PageResponse` in thread dumps or debug logs 3. **What is the exact truncation point character?** - Is it a high-ASCII character (0x80-0xFF)? - Is it a multi-byte UTF-8 sequence? - Is it a Unicode character outside the response encoding's range? 4. **Is there a response size pattern?** - Does truncation always occur at the same character position? - Is there a buffer boundary correlation (e.g., 8192 bytes)? - Does the truncation point move if JSON content changes? --- ## Related Research No existing research documents found in `thoughts/` directory. This is the first documentation of this issue. Future research areas: - Tomcat 10 vs Tomcat 11 response handling differences - Jakarta Servlet 6.0 character encoding changes - SiteMesh 3.x buffering behavior with Jakarta EE - Struts response wrapper chain analysis --- ## Conclusion The JSON truncation issue is **NOT caused by the Jakarta EE migration itself**, but rather by a **character encoding mismatch** that triggers documented truncation behavior in `FastByteArrayOutputStream`. This issue exists in both javax and jakarta versions of Struts, but may be **exposed differently** on Tomcat 11 due to: 1. Different default character encodings 2. Different response wrapper implementations 3. Different character encoding libraries **Immediate Action**: The user should switch to the **Struts JSON plugin** for JSON responses, which is the recommended approach and completely avoids the JSP rendering pipeline and its associated encoding issues. **Alternative**: If JSP must be used, ensure: - Consistent UTF-8 encoding throughout the pipeline - SiteMesh excludes JSON endpoints - Diagnostic logging enabled to confirm the root cause
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
