lizining1231 opened a new issue, #3810:
URL: https://github.com/apache/dubbo-go/issues/3810
### Summary
Allow applications to implement an encoding plugin outside dubbo-go,
register it under a name, and select it through standard client/server
configuration. Adding a format should not require changing a switch inside the
framework.
The proposal introduces two complementary extension points:
| Extension | What it controls | Configuration | Applies to |
| --- | --- | --- | --- |
| Serialization | Conversion between business values and encoded message
bodies | `serialization` | Triple, Dubbo, independent gRPC |
| Remoting codec | Encoding and decoding complete Dubbo request/response
frames | `codec` | Classic Dubbo |
Existing defaults and supported wire formats remain unchanged. This proposal
covers the extension mechanism; a Fory plugin is a separate follow-up.
### Problem
Dubbo-go already encodes and decodes RPC traffic. The missing capability is
a consistent way to plug in implementations through its high-level APIs.
At the baseline before this change:
- Triple exposes a low-level `Codec` interface, but high-level selection is
tied to built-in names. Its non-IDL inner-codec registration is private.
- Dubbo has a remoting codec registry and serializer compatibility APIs, but
lacks a unified registration contract connecting a format name, serialization
ID, construction factory and local configuration.
- Independent gRPC uses grpc-go's process-wide subtype registry, which needs
an explicit policy for configuration conflicts.
As a result, an application cannot consistently introduce a format using
only an external package and ordinary configuration. Existing low-level
registries also do not provide a common contract for configured instance
ownership.
### Proposed Design
#### 1. Register a factory, select a name, bind an instance
Provide a public `serialization` package with the minimal object/message API:
```go
type Codec interface {
Name() string
Marshal(any) ([]byte, error)
Unmarshal([]byte, any) error
}
```
Add `extension.RegisterSerialization` and a `MustRegisterSerialization`
helper. A registration declares its protocol, name, encoding mode, wire
identity and factory. The factory receives local configuration and creates the
runtime instance during initialization, rather than on each RPC.
Keep the existing selection options. For example, after importing a package
that registers `custom-json` for Triple:
```go
// Consumer reference options:
client.WithSerialization("custom-json")
// Provider service options:
server.WithSerialization("custom-json")
server.WithAcceptedSerializations("custom-json")
```
The registry stores factories. Each reference or service binds the instances
and receive policy it needs. Active bindings do not consult a mutable global
codec instance on every request.
#### 2. Keep serialization separate from protocol framing
The RPC path remains protocol-specific, with the selected serializer
handling the message body:
```mermaid
flowchart TB
subgraph Client
A[Business arguments] --> B[Selected serializer]
B --> C[Protocol framing]
end
C --> N[Network]
subgraph Server
D[Protocol parsing] --> E[Receive-policy check]
E --> F[Selected deserializer]
F --> G[Business method]
end
N --> D
```
Responses follow the reverse direction. For wrapped Triple calls, the
response uses the request's accepted inner format. The client validates the
response format before decoding it.
For Triple, the protocol continues to own headers, trailers, compression and
framing. For Classic Dubbo, applications may additionally replace the framing
stage through a named remoting codec factory.
#### 3. Adapt to each protocol's encoding model
| Protocol path | Integration |
| --- | --- |
| Triple message encoding | Select a message codec, such as Protobuf or
ProtoJSON, by its registered wire name. |
| Triple non-IDL object encoding | Encode arguments/results with the plugin
inside the existing Protobuf wrappers. Dispatch by the wrapper's serialization
name. Support both gRPC wire and Triple wire for unary calls. |
| Classic Dubbo body encoding | Use `StreamCodec` for the ordered objects in
one complete body. Select the decoder using the validated serialization ID in
the header. |
| Independent gRPC | Adapt a registered message codec to a stable grpc-go
content subtype. Reject incompatible configuration reuse. |
Dubbo's body is an object stream containing metadata, arguments and
attachments. Independent `Marshal` outputs cannot simply be concatenated to
implement that stream. Add optional encoder/decoder interfaces with per-body
state, explicit completion and cleanup. Decoding must support concrete argument
types without losing integer precision or silently accepting overflow.
For complete Dubbo frames, add `extension.RegisterCodec` with a construction
factory and select it through protocol `codec` configuration. Preserve the
existing `remoting.Codec` interface and default implementation.
#### 4. Define receive policy and resource ownership
- **Explicit receive policy:** `accepted-serializations` restricts inbound
formats. An omitted list preserves the protocol's documented defaults; an
explicit empty list disables business formats. Unknown, disabled or malformed
formats produce errors, without guessing another decoder after failure.
- **Local configuration:** pass independently owned, read-only configuration
snapshots to factories. Remote discovery metadata must not change a local
plugin selection or broaden its receive policy, including after reconnection.
- **Shared lifetime:** support optional `io.Closer`. Release partial
instances on initialization failure. Shared connections hold leases so closing
one connection cannot invalidate another; release resources after the last user
finishes. Active requests and streams must retain their resources.
- **Endpoint isolation:** Triple keeps separate contexts for each
group/version. Incompatible Dubbo services sharing a listener must fail export
rather than silently reuse another service's settings.
- **Process-wide gRPC subtypes:** reuse requires equivalent configuration.
Conflicting configuration or an existing external subtype registration must
fail explicitly. These subtype instances have process lifetime.
### Scope and Compatibility
This change includes the public contracts, factory registration, high-level
selection, protocol adapters, receive policies and lifecycle handling.
Preserve default encodings, existing wrapper schemas, JSON behavior and
legacy registration APIs. Keep Triple's existing `Codec` available as an alias
of the public interface. Preserve optional `MarshalAppend` support and
applicable existing fast paths. Duplicate registrations, invalid capabilities
and unknown configured names should fail before business traffic is served.
The following are outside this proposal:
- Shipping a Fory plugin or adding Fory as a default dependency.
- Changing Triple's HTTP framing, transport or flow control.
- Supporting wrapped-object Triple streaming; streaming uses message codecs.
- Assigning new interoperable Dubbo serialization IDs or changing peer
formats.
- Claiming performance improvements or Java interoperability from Go-only
tests.
A future Fory plugin can use the public extension points. Its type mapping,
schema behavior and actual Go/Java RPC interoperability need separate
validation.
### Acceptance Criteria
1. **External extensibility:** an independent module implements and
registers a plugin using public APIs, selects it through high-level
configuration, and completes a real RPC. Verify all four steps: client
encoding, server decoding, server encoding and client decoding.
2. **Protocol coverage:** cover both Triple unary wires, message streaming,
custom Dubbo stream serialization and remoting framing, and independent gRPC
subtypes. Existing built-in formats continue to work.
3. **Strict dispatch:** reject unknown formats, disabled formats, malformed
payloads and invalid Dubbo IDs. Empty messages must not bypass receive policy.
4. **Typed decoding:** preserve wide integers, slices and structs; reject
overflow and fractional values supplied for integer arguments.
5. **Isolation and reconnection:** distinct local configurations remain
independent, including on a shared address. Reconnection preserves local plugin
selection even when a provider advertises a different format.
6. **Cleanup and recovery:** cover failed factories, partial client/service
initialization, cluster assembly failure, shared connection closure and final
resource release. A failed export followed by a retry must serve RPCs.
7. **Handler replacement:** group/version-specific policies remain
independent; replacing a handler does not interrupt an already executing
request.
8. **Concurrency and compatibility:** verify concurrent calls, byte
ownership, resource closure and legacy API/wire behavior with focused
regression tests.
### Related Code
These links describe the baseline before the proposed extension mechanism:
| Area | Existing limitation |
| --- | --- |
| [Remoting codec
registry](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/remoting/codec.go)
| Instances are registered by protocol; named configured factories are
missing. |
| [Triple codec
interface](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/protocol/triple/triple_protocol/codec.go)
| The general encoding contract lives inside the Triple implementation. |
| [Triple inner-codec
registry](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/protocol/triple/triple_protocol/inner_codec.go)
| Non-IDL inner-codec registration is private. |
| [Triple
client](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/protocol/triple/client.go)
and
[server](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/protocol/triple/server.go)
| High-level selection uses built-in serialization branches. |
| [Dubbo serialization
mapping](https://github.com/apache/dubbo-go/blob/6b04ac04bfc868060995dc9fa4e1dcc5138ad02b/protocol/dubbo/impl/serialization.go)
| Format names, IDs and body implementations need a unified extension
contract. |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]