AlexStocks commented on code in PR #1: URL: https://github.com/apache/dubbo-net/pull/1#discussion_r4434759971
########## DubboService/Native/NativeDubboCodec.cs: ########## @@ -0,0 +1,833 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +using DubboNet.DubboService.DataModle; +using NHessian.IO; +using System; +using System.Buffers.Binary; +using System.Collections; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Text.Json; +using System.Text.Encodings.Web; +using System.Text.Unicode; + +namespace DubboNet.DubboService.Native +{ + /// <summary> + /// 编解码 Dubbo2 16 字节帧头、Hessian2 泛化调用请求和响应。 + /// <para>EN: Encodes and decodes Dubbo2 16-byte frames and Hessian2 generic invocation payloads.</para> + /// </summary> + internal static class NativeDubboCodec + { + internal const int HeaderLength = 16; + internal const int DefaultMaxPayloadLength = 8 * 1024 * 1024; + internal const int MaxNormalizedNodes = 100_000; Review Comment: 🟡 **预算是绝对常数,和载荷大小不匹配:大而合法的响应会被当成攻击** `MaxNormalizedNodes = 100_000` / `MaxNormalizedJsonBytes = 32 MiB` 是固定值,而响应体上限是 8 MiB,两者差了两个数量级: 1. **节点预算**:8 MiB 里可以合法放下约 800 万个小整数(`0x90` 起,每元素 1 字节,`0x58` 数组带长度)。一个 15 万元素的 `List<long>`(载荷约 1.3 MiB,真实 JSON 约 1.2 MiB)在 `74bbca3` 上能正常返回,现在会抛 `InvalidDataException`。RPC 返回一份 ID 列表、指标序列这类响应,15 万条并不罕见。 2. **体积预算**:计费按 `3 + 6 × 字符数` 的最坏转义估算。两个 4 MiB 的 **ASCII** 字符串(合计正好 8 MiB 载荷、真实 JSON 约 8 MiB)会计费 `2 × (3 + 24 MiB) = 48 MiB > 32 MiB`,同样被拒——对 ASCII 来说 6× 是纯高估。 **为什么建议改成按载荷定预算**:放大攻击的特征是「**展开量远大于载荷**」,不是「绝对量大」。合法响应的展开量天然与载荷成正比(每个 wire 节点至少 1 字节),所以: ```csharp long bodyLength = frame.Body?.Length ?? 0; int nodeLimit = Math.Max(MaxNormalizedNodes, (int)Math.Min(int.MaxValue, 2 * bodyLength)); long byteLimit = Math.Max(MaxNormalizedJsonBytes, 8 * bodyLength); ``` - DAG 炸弹仍然被挡住:95 字节载荷 → 节点上限仍是 100,000,2^20 立刻超限,你们那条 `SharedReferenceDagIsStoppedByOneWholeTreeNodeBudget` 原样通过; - 1.3 MiB 的合法列表上限升到约 260 万节点,不会被误杀; - 8 MiB 载荷的体积预算放宽到 64 MiB,足够覆盖最坏转义。 如果维护者希望保留固定常量,那至少把 `100_000` 提到与 8 MiB 载荷相称的量级(比如 2,000,000),并在 README 里明确写成「单响应展开节点上限」,别让用户从「我的响应才 1.5 MB 为什么被拒」去反推。 顺带说明:我不建议为此新增公开配置项——按载荷推导既不需要调参,也不会给调用方多一个踩坑的地方。 ########## DubboService/Native/DubboHessian2Reader.cs: ########## @@ -0,0 +1,563 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +using System; +using System.Buffers.Binary; +using System.Collections.Generic; +using System.IO; +using System.Text; + +namespace DubboNet.DubboService.Native +{ + /// <summary> + /// 将 Hessian2 值读取为动态 .NET 列表和字典,用于 CLR 返回类型未知的 Dubbo 泛化响应。 + /// <para>EN: Reads Hessian2 values into dynamic .NET lists and dictionaries for generic Dubbo responses whose CLR result type is unknown.</para> + /// </summary> + internal sealed class DubboHessian2Reader + { + // Wire nesting and JSON reference-expansion depth are independently bounded + // by the same limit; chunk sequences are concatenation, not nesting. + internal const int MaxNestingDepth = 64; + private int _depth; + private sealed class ClassDefinition + { + public string TypeName { get; set; } + public string[] Fields { get; set; } + } + + private readonly byte[] _buffer; + private readonly List<object> _references = new List<object>(); + private readonly List<string> _typeReferences = new List<string>(); + private readonly List<ClassDefinition> _classDefinitions = new List<ClassDefinition>(); + private int _position; + + /// <summary> + /// 创建读取指定 Hessian2 数据的解析器。 + /// <para>EN: Creates a reader for the supplied Hessian2 payload.</para> + /// </summary> + public DubboHessian2Reader(byte[] buffer) + { + _buffer = buffer ?? throw new ArgumentNullException(nameof(buffer)); + } + + /// <summary> + /// 读取一个 Hessian2 整数。 + /// <para>EN: Reads one Hessian2 integer.</para> + /// </summary> + public int ReadInt() + { + return ReadInt(ReadByte()); + } + + /// <summary> + /// 读取一个 Hessian2 字符串。 + /// <para>EN: Reads one Hessian2 string.</para> + /// </summary> + public string ReadString() + { + return ReadString(ReadByte()); + } + + /// <summary> + /// 读取下一个动态 Hessian2 值。 + /// <para>EN: Reads the next dynamic Hessian2 value.</para> + /// </summary> + public object ReadValue() + { + if (_depth >= MaxNestingDepth) + throw new InvalidDataException($"Hessian2 wire nesting exceeds the limit of {MaxNestingDepth} levels."); + _depth++; + try { return ReadValueCore(); } + finally { _depth--; } + } + + private object ReadValueCore() + { + byte tag = ReadByte(); + if (tag == (byte)'N') return null; + if (tag == (byte)'T') return true; + if (tag == (byte)'F') return false; + + if (IsIntTag(tag)) return ReadInt(tag); + if (IsLongTag(tag)) return ReadLong(tag); + if (IsDoubleTag(tag)) return ReadDouble(tag); + + if (tag == 0x4a) + { + return DateTimeOffset.FromUnixTimeMilliseconds(ReadInt64()).UtcDateTime; + } + if (tag == 0x4b) + { + return DateTimeOffset.FromUnixTimeSeconds((long)ReadInt32() * 60).UtcDateTime; + } + + if (IsStringTag(tag)) return ReadString(tag); + if (IsBinaryTag(tag)) return ReadBinary(tag); + + if (IsListTag(tag)) return ReadList(tag); + if (tag == (byte)'C') return ReadClassDefinitionAndValue(); + if (IsMapOrObjectTag(tag)) return ReadMapOrObject(tag); + + if (tag == 0x51) + { + int reference = ReadInt(); + if (reference < 0 || reference >= _references.Count) + { + throw new InvalidDataException($"Invalid Hessian2 reference index {reference}."); + } + return _references[reference]; + } + + throw new InvalidDataException($"Unsupported Hessian2 tag 0x{tag:x2}."); + } + + private object ReadClassDefinitionAndValue() + { + string typeName = ReadString(); + int fieldCount = ReadInt(); + if (fieldCount < 0) + { + throw new InvalidDataException($"Invalid Hessian2 class field count {fieldCount}."); + } + + string[] fields = new string[fieldCount]; + for (int i = 0; i < fields.Length; i++) + { + fields[i] = ReadString(); + } + _classDefinitions.Add(new ClassDefinition + { + TypeName = typeName, + Fields = fields + }); + return ReadValue(); + } + + private object ReadMapOrObject(byte tag) + { + if (tag == (byte)'H' || tag == (byte)'M') + { + string typeName = tag == (byte)'M' ? ReadType() : null; + Dictionary<object, object> map = NewMap(typeName); + _references.Add(map); + while (PeekByte() != (byte)'Z') + { + object key = ReadValue(); + object value = ReadValue(); + map[key ?? "null"] = value; + } + _position++; + return map; + } + + int definitionIndex = tag == (byte)'O' ? ReadInt() : tag - 0x60; + if (definitionIndex < 0 || definitionIndex >= _classDefinitions.Count) + { + throw new InvalidDataException( + $"Invalid Hessian2 class definition index {definitionIndex}."); + } + + ClassDefinition definition = _classDefinitions[definitionIndex]; + Dictionary<object, object> instance = NewMap(definition.TypeName); + _references.Add(instance); + foreach (string field in definition.Fields) + { + instance[field] = ReadValue(); + } + return instance; + } + + private static Dictionary<object, object> NewMap(string typeName) + { + Dictionary<object, object> result = new Dictionary<object, object>(); + if (!string.IsNullOrWhiteSpace(typeName)) + { + result["class"] = typeName; + } + return result; + } + + private object ReadList(byte tag) + { + bool hasEnd; + int length; + + switch (tag) + { + case 0x55: + _ = ReadType(); + hasEnd = true; + length = -1; + break; + case 0x56: + _ = ReadType(); + hasEnd = false; + length = ReadInt(); + break; + case 0x57: + hasEnd = true; + length = -1; + break; + case 0x58: + hasEnd = false; + length = ReadInt(); + break; + default: + hasEnd = false; + if (tag >= 0x70 && tag <= 0x77) + { + length = tag - 0x70; + _ = ReadType(); + } + else + { + length = tag - 0x78; + } + break; + } + + if (length < -1) + { + throw new InvalidDataException($"Invalid Hessian2 list length {length}."); + } + + List<object> list = length > 0 ? new List<object>(length) : new List<object>(); Review Comment: 🔴 **声明长度直接预分配:6 字节载荷可以要求 16 GB** ```csharp List<object> list = length > 0 ? new List<object>(length) : new List<object>(); ``` `length` 来自 `0x58` 之后的 `ReadInt()`,目前只校验了 `length < -1`。`List<object>(capacity)` 会在读到任何元素**之前**就按容量分配引用数组,所以分配量完全由载荷里的那 4 个字节决定: ``` 58 49 7F FF FF FF -> length = 2,147,483,647 -> new object[2^31] ≈ 16 GB ``` **为什么这条重要**:这正是 `a5d1cbe` 想关闭的那类问题——「几十字节的响应放大成 GB 级内存」。但节点/体积预算是在 `ToJsonCompatible` 里生效的,而这里是**解析期**:预算还没开始跑,内存已经分出去了,所以 100,000 节点和 32 MiB 都拦不住它。响应体 8 MiB 的上限(`NativeDubboConnection.cs:215`)也拦不住,6 字节远在限额内。 `new object[int.MaxValue]` 在任何现实机器上都会失败,所以单次请求大概率先拿到 `OutOfMemoryException`;真正的问题是并发下的中等值——比如 20 个并发请求各声明 1 亿元素(各约 800 MB),足以把进程打进 OOM,而每个请求只花了 6 字节。 **Suggestion**:每个元素在 wire 上至少占 1 字节(`ReadValue` 至少读一个 tag),所以「声明长度 > 剩余字节数」的输入一定是坏的。这个检查精确、零成本,而且不需要新增可配置常量: ```csharp int remaining = _buffer.Length - _position; if (length > remaining) { throw new InvalidDataException( $"Hessian2 list length {length} exceeds the {remaining} remaining payload bytes."); } List<object> list = length > 0 ? new List<object>(length) : new List<object>(); ``` 这样 `length` 天然被限制在 ≤ 8 MiB 之内。 **回归用例建议**(`HessianResponseSafetyTest.cs`,断言是 `InvalidDataException` 而不是 OOM): ```csharp [Fact] public void DeclaredListLengthBeyondPayloadIsRejectedBeforeAllocation() { // 0x58 = list with length, 0x49 = 4-byte int, then int.MaxValue var body = new byte[] { 0x58, 0x49, 0x7F, 0xFF, 0xFF, 0xFF }; Assert.Throws<InvalidDataException>(() => Decode(body)); } ``` ########## DubboService/Native/DubboHessian2Reader.cs: ########## @@ -0,0 +1,563 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +using System; +using System.Buffers.Binary; +using System.Collections.Generic; +using System.IO; +using System.Text; + +namespace DubboNet.DubboService.Native +{ + /// <summary> + /// 将 Hessian2 值读取为动态 .NET 列表和字典,用于 CLR 返回类型未知的 Dubbo 泛化响应。 + /// <para>EN: Reads Hessian2 values into dynamic .NET lists and dictionaries for generic Dubbo responses whose CLR result type is unknown.</para> + /// </summary> + internal sealed class DubboHessian2Reader + { + // Wire nesting and JSON reference-expansion depth are independently bounded + // by the same limit; chunk sequences are concatenation, not nesting. + internal const int MaxNestingDepth = 64; + private int _depth; + private sealed class ClassDefinition + { + public string TypeName { get; set; } + public string[] Fields { get; set; } + } + + private readonly byte[] _buffer; + private readonly List<object> _references = new List<object>(); + private readonly List<string> _typeReferences = new List<string>(); + private readonly List<ClassDefinition> _classDefinitions = new List<ClassDefinition>(); + private int _position; + + /// <summary> + /// 创建读取指定 Hessian2 数据的解析器。 + /// <para>EN: Creates a reader for the supplied Hessian2 payload.</para> + /// </summary> + public DubboHessian2Reader(byte[] buffer) + { + _buffer = buffer ?? throw new ArgumentNullException(nameof(buffer)); + } + + /// <summary> + /// 读取一个 Hessian2 整数。 + /// <para>EN: Reads one Hessian2 integer.</para> + /// </summary> + public int ReadInt() + { + return ReadInt(ReadByte()); + } + + /// <summary> + /// 读取一个 Hessian2 字符串。 + /// <para>EN: Reads one Hessian2 string.</para> + /// </summary> + public string ReadString() + { + return ReadString(ReadByte()); + } + + /// <summary> + /// 读取下一个动态 Hessian2 值。 + /// <para>EN: Reads the next dynamic Hessian2 value.</para> + /// </summary> + public object ReadValue() + { + if (_depth >= MaxNestingDepth) + throw new InvalidDataException($"Hessian2 wire nesting exceeds the limit of {MaxNestingDepth} levels."); + _depth++; + try { return ReadValueCore(); } + finally { _depth--; } + } + + private object ReadValueCore() + { + byte tag = ReadByte(); + if (tag == (byte)'N') return null; + if (tag == (byte)'T') return true; + if (tag == (byte)'F') return false; + + if (IsIntTag(tag)) return ReadInt(tag); + if (IsLongTag(tag)) return ReadLong(tag); + if (IsDoubleTag(tag)) return ReadDouble(tag); + + if (tag == 0x4a) + { + return DateTimeOffset.FromUnixTimeMilliseconds(ReadInt64()).UtcDateTime; + } + if (tag == 0x4b) + { + return DateTimeOffset.FromUnixTimeSeconds((long)ReadInt32() * 60).UtcDateTime; + } + + if (IsStringTag(tag)) return ReadString(tag); + if (IsBinaryTag(tag)) return ReadBinary(tag); + + if (IsListTag(tag)) return ReadList(tag); + if (tag == (byte)'C') return ReadClassDefinitionAndValue(); + if (IsMapOrObjectTag(tag)) return ReadMapOrObject(tag); + + if (tag == 0x51) + { + int reference = ReadInt(); + if (reference < 0 || reference >= _references.Count) + { + throw new InvalidDataException($"Invalid Hessian2 reference index {reference}."); + } + return _references[reference]; + } + + throw new InvalidDataException($"Unsupported Hessian2 tag 0x{tag:x2}."); + } + + private object ReadClassDefinitionAndValue() + { + string typeName = ReadString(); + int fieldCount = ReadInt(); + if (fieldCount < 0) + { + throw new InvalidDataException($"Invalid Hessian2 class field count {fieldCount}."); + } + + string[] fields = new string[fieldCount]; Review Comment: 🔴 **同一根因的第二个位置:class definition 的字段数组** ```csharp int fieldCount = ReadInt(); ... string[] fields = new string[fieldCount]; ``` 只有 `fieldCount < 0` 会被拒绝,上限没有。载荷 `43 00 49 7F FF FF FF`(`'C'` + 空类型名 `00` + `int.MaxValue`,共 7 字节)就会走到 `new string[2,147,483,647]` ≈ 16 GB,同样在读到任何字段名之前完成分配。 和 `ReadList` 那条是同一个根因,**请一起修**:只改列表这条路径,攻击面只是从 `0x58` 换成了 `'C'`。 **Suggestion**:同样用剩余字节做上界(每个字段名至少 1 字节 tag): ```csharp int remaining = _buffer.Length - _position; if (fieldCount > remaining) { throw new InvalidDataException( $"Hessian2 class field count {fieldCount} exceeds the {remaining} remaining payload bytes."); } ``` 顺带一个把这类问题收口的做法:`ReadList` 的 `length` 和这里的 `fieldCount` 都出自 `ReadInt()`,抽成一个 `EnsurePlausibleCount(int count)`(内部比对剩余字节),以后新增任何「声明长度」的 tag 都顺手用上,就不会再有第三处。 回归用例可以和上一条对称:`43 00 49 7F FF FF FF` → `InvalidDataException`。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
