[
https://issues.apache.org/jira/browse/THRIFT-6153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmytro Shteflyuk updated THRIFT-6153:
-------------------------------------
Fix Version/s: 0.25.0
> Reduce native Ruby Compact Protocol varint write overhead
> ---------------------------------------------------------
>
> Key: THRIFT-6153
> URL: https://issues.apache.org/jira/browse/THRIFT-6153
> Project: Thrift
> Issue Type: Bug
> Components: Ruby - Library
> Reporter: Dmytro Shteflyuk
> Assignee: Dmytro Shteflyuk
> Priority: Major
> Fix For: 0.25.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> h3. Problem
> The native Ruby Compact Protocol encoder writes each byte of a multibyte
> varint through the transport separately. Encoding boundary i32 and i64 values
> therefore creates five or ten one-byte Ruby strings and performs five or ten
> dynamic transport calls instead of one buffered call.
> h3. Client impact
> Ruby clients that serialize integer-heavy Compact payloads pay avoidable
> object-allocation and method-dispatch costs. The overhead is most visible for
> small payloads, where protocol framing and scalar writes dominate. The
> encoded wire bytes remain correct.
> h3. Reproduction
> Run the following from an Apache Thrift checkout after building the Ruby
> native extension:
> {code:ruby}
> require "thrift"
> class RecordingTransport < Thrift::BaseTransport
> attr_reader :writes
> def initialize
> @writes = []
> end
> def write(data)
> @writes << data.dup
> end
> end
> def encode(type, value)
> transport = RecordingTransport.new
> protocol = Thrift::CompactProtocol.new(transport)
> protocol.public_send(type, value)
> name = type.to_s.delete_prefix("write_")
> bytes = transport.writes.join.unpack1("H*")
> puts "#{name} writes=#{transport.writes.length} bytes=#{bytes}"
> end
> puts "native=#{Thrift::CompactProtocol.new(RecordingTransport.new).native?}"
> encode(:write_i32, -(2**31))
> encode(:write_i64, -(2**63))
> {code}
> Testing on master commit {{c8f876181c4fa132b5a6b42fdbfe04db06b4eee9}}
> produces:
> {noformat}
> native=true
> i32 writes=5 bytes=ffffffff0f
> i64 writes=10 bytes=ffffffffffffffffff01
> {noformat}
> h3. Expected behavior
> The native Compact Protocol encoder should accumulate each varint in a
> bounded stack buffer and call the transport's write method once. This should
> preserve the exact wire bytes and generic transport behavior while reducing
> per-scalar overhead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
