[
https://issues.apache.org/jira/browse/THRIFT-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18091965#comment-18091965
]
Jens Geyer edited comment on THRIFT-6069 at 6/26/26 9:57 PM:
-------------------------------------------------------------
Use PyUnicode_AsUTF8AndSize when available, with a fallback for older Python 3
releases, so unicode strings can be encoded without allocating an intermediate
PyBytes object. The const-correct buffer is forwarded through a new
writeString(const char*, int32_t) overload on each protocol so length prefixing
uses the protocol-native encoding (int32 for binary, varint for compact). The
internal writeBuffer helper was made const-correct so the zero-copy path does
not need to const_cast CPython-managed string storage.
Add a fastbinary unicode round-trip test for TApplicationException to cover the
new encode path, and a TSerializer regression test that asserts pure-Python and
accelerated CompactProtocol produce identical bytes for unicode payloads.
Performance (50k iterations, warmed):
|Workload |Baseline|This commit|Speedup|
|-------------------------------|----------|-------------|---------|
|encode simple (30B, 1 string)|0.60 us |0.55 us |1.09x |
|encode 10-string (182B) |1.44 us |1.01 us |1.43x |
|encode complex (395B) |3.02 us |2.56 us |1.18x |
The more string fields a struct has, the larger the gain. Decode is unchanged.
_(Extracted from commit message)_
was (Author: jensg):
Use PyUnicode_AsUTF8AndSize when available, with a fallback for older Python 3
releases, so unicode strings can be encoded without allocating an intermediate
PyBytes object. The const-correct buffer is forwarded through a new
writeString(const char*, int32_t) overload on each protocol so length prefixing
uses the protocol-native encoding (int32 for binary, varint for compact). The
internal writeBuffer helper was made const-correct so the zero-copy path does
not need to const_cast CPython-managed string storage.
Add a fastbinary unicode round-trip test for TApplicationException to cover the
new encode path, and a TSerializer regression test that asserts pure-Python and
accelerated CompactProtocol produce identical bytes for unicode payloads.
Performance (50k iterations, warmed):
| Workload | Baseline | This commit | Speedup |
|-------------------------------|----------|-------------|---------|
| encode simple (30B, 1 string) | 0.60 us | 0.55 us | 1.09x |
| encode 10-string (182B) | 1.44 us | 1.01 us | 1.43x |
| encode complex (395B) | 3.02 us | 2.56 us | 1.18x |
The more string fields a struct has, the larger the gain. Decode is unchanged.
_(Extracted from commit message:)_
> suggestion for a few python perf improvements
> ---------------------------------------------
>
> Key: THRIFT-6069
> URL: https://issues.apache.org/jira/browse/THRIFT-6069
> Project: Thrift
> Issue Type: Improvement
> Reporter: Mark Molinaro
> Priority: Minor
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
