pjfanning opened a new pull request, #1235: URL: https://github.com/apache/pekko-http/pull/1235
### Motivation `PerMessageDeflate` buffers the output of the `Deflater`/`Inflater` in a `ByteArrayOutputStream` and then calls `ByteString.fromArrayUnsafe(output.toByteArray)`. `ByteArrayOutputStream.toByteArray` always copies the buffer, so every compressed or decompressed WebSocket frame pays a full copy of its payload. ### Modification Add an internal `ByteStringOutputStream` (`@InternalApi`, `private[http]`) that extends `ByteArrayOutputStream` and exposes `toByteStringUnsafe`: - when most of the buffer is used, the buffer is wrapped via `ByteString.fromArrayUnsafe(buf, 0, count)` with no copy; - otherwise the bytes are copied to a right-sized array so a large buffer is not retained by a small payload. The stream must not be written to or reused after `toByteStringUnsafe`, which is documented on the method. Both call sites (`inflate` and `deflate` in `PerMessageDeflate`) allocate the stream per call and discard it immediately afterwards. `HeaderCompression` also uses a `ByteArrayOutputStream`, but it reuses one instance across frames via `reset()`, so it is left unchanged here. ### Result No copy of the payload per WebSocket frame when `permessage-deflate` is enabled, and no oversized buffer retained when a frame only fills a small part of it. Note that `toByteStringUnsafe` can return a non-compact `ByteString1`, so a consumer that calls `toArrayUnsafe()` on the result will copy - that is the same single copy as before, just moved, and it is avoided entirely for consumers that do not need an array. ### Tests - `sbt "http-core/testOnly org.apache.pekko.http.impl.util.ByteStringOutputStreamSpec org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec"` - 51 tests succeeded, 0 failed - New `ByteStringOutputStreamSpec` covers the empty, exactly-filled, grown-buffer and small-write-in-a-large-buffer cases, including that the copied result is unaffected by later writes to the stream - `scalafmt --list --mode diff-ref=upstream/main` - no files reported - `sbt headerCreateAll` - used to add the headers for the new files - MiMa not run - the change is confined to `@InternalApi private[http]` code and adds no public API ### References None - the `ByteStringOutputStream` implementation is adapted from Apache Pekko gRPC (Apache License 2.0), https://github.com/apache/pekko-grpc/pull/862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
