fishy commented on a change in pull request #2469: URL: https://github.com/apache/thrift/pull/2469#discussion_r808568115
########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") Review comment: in the table above they are defined as `le`/`gt`, not `lte`/`gte`. ########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") + 2: i8 Type (validator.in = "[1, 2, 4]") +} +``` +- String/Binary +``` +struct StringDemo{ + 1: string Uninitialized (vt.const = "abc") + 2: string Name (vt.min_size = "6", vt.max_size = "12") + 3: string SomeStuffs (vt.pattern = "[0-9A-Za-z]+") + 4: string DebugInfo (vt.prefix = "[Debug]") + 5: string PanicInfo (vt.contains = "panic") +} +``` +- Bool +``` +struct BoolDemo { + 1: bool AMD (vt.const = "true") +} +``` +- Enum +``` +struct EnumDemo { + 1: Type AddressType (vt.const = "String") + 2: Type ValueType (vt.defined_only = "true") +} +``` +- Set/List +``` +struct SetListDemo { + 1: list<string> Persons (vt.min_size = "5", vt.max_size = "10") + 2: set<double> HealthPoints (vt.elem.gt = "0") +} +``` +- Map +``` +struct MapDemo { + 1: map<i32, string> IdName (vt.min_size = "5", vt.max_size = "10") + 2: map<i32, DemoTestRequest> Requests (vt.no_sparse = "true") Review comment: what does `vt.no_sparse` mean? I don't see it defined anywhere in the proposal. ########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") + 2: i8 Type (validator.in = "[1, 2, 4]") +} +``` +- String/Binary +``` +struct StringDemo{ + 1: string Uninitialized (vt.const = "abc") + 2: string Name (vt.min_size = "6", vt.max_size = "12") + 3: string SomeStuffs (vt.pattern = "[0-9A-Za-z]+") + 4: string DebugInfo (vt.prefix = "[Debug]") + 5: string PanicInfo (vt.contains = "panic") +} +``` +- Bool +``` +struct BoolDemo { + 1: bool AMD (vt.const = "true") +} +``` +- Enum +``` +struct EnumDemo { + 1: Type AddressType (vt.const = "String") + 2: Type ValueType (vt.defined_only = "true") +} +``` +- Set/List +``` +struct SetListDemo { + 1: list<string> Persons (vt.min_size = "5", vt.max_size = "10") + 2: set<double> HealthPoints (vt.elem.gt = "0") +} +``` +- Map +``` +struct MapDemo { + 1: map<i32, string> IdName (vt.min_size = "5", vt.max_size = "10") + 2: map<i32, DemoTestRequest> Requests (vt.no_sparse = "true") + 3: map<i32, double> Some, (vt.key.gt = "0", vt.value.lt = "1000") Review comment: this seems to be the only line in all the examples that has the trailing comma. without validator rules thrift is very lenient on trailing comma, so that's fine, but when we actually have trailing comma, it likely makes more sense to have it after the validator rules? so this line should be this instead: ```suggestion 3: map<i32, double> Some (vt.key.gt = "0", vt.value.lt = "1000"), ``` ########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") + 2: i8 Type (validator.in = "[1, 2, 4]") +} +``` +- String/Binary +``` +struct StringDemo{ + 1: string Uninitialized (vt.const = "abc") + 2: string Name (vt.min_size = "6", vt.max_size = "12") + 3: string SomeStuffs (vt.pattern = "[0-9A-Za-z]+") + 4: string DebugInfo (vt.prefix = "[Debug]") + 5: string PanicInfo (vt.contains = "panic") +} +``` +- Bool +``` +struct BoolDemo { + 1: bool AMD (vt.const = "true") +} +``` +- Enum +``` +struct EnumDemo { + 1: Type AddressType (vt.const = "String") Review comment: I guess here in this example `String` is one of the values of enum `Type`? it helps to add that into the example (add the definition of `Type` here). ########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") + 2: i8 Type (validator.in = "[1, 2, 4]") +} +``` +- String/Binary +``` +struct StringDemo{ + 1: string Uninitialized (vt.const = "abc") + 2: string Name (vt.min_size = "6", vt.max_size = "12") + 3: string SomeStuffs (vt.pattern = "[0-9A-Za-z]+") Review comment: I don't see `pattern` defined in the table. for a regexp there are so many flavors, I'd strongly suggest to either define the flavor here, or just remove it altogether. ########## File path: doc/proposal/thrift-parameter-validation-proposal.md ########## @@ -0,0 +1,180 @@ +# Thrift Parameter Validation Proposal + +> Version 1.1 +> +> Dec 15, 2021 +> +> [email protected], [email protected] + +### 1. Abstract +*** +This document presents a proposed set of annotations to the Thrift IDL. The new annotations will supports parameter validation using build-in or third-party validators. The goal of this proposal is to define sematics and behavior of validation annotations, rather than to discuss their implementation. + +### 2. Background +*** +Parameter validation is a common need for web service. In the past, we usually write our validating logics after a RPC message deserialized by thrift. This ways works flexibly enough but restrict poorly: It is dangerous than service A and service B using the same IDL have two different validating rule, which often misdirects developers. Even if we extract our validating codes to a single module, simple and repeated work (ex. `if xx.Field1 > 1 then ...`) is really disturbing. If we can use build tool to generating codes for simple and unchangeable restraint, the web service will be more robust and developers will benefits from lighter work. +Compared to other IDL, the parameter validation gradually gets strong commutity supports like PGV ([protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate)), benefiting from pb's strong plugin mechanism (lacking official plugin mechanism is one reason for we submit this proposal). Take a long-term view, auto-generated parameter validation may be a step towards code-less web service. + +### 3. Proposal +*** +This proposal includes three part: Validate Annotation Sematics, Validate Rule and Validate Feedback. The first declare how to write a validate annotation, the middle explain how every annotation should behave, the last introduces a mechanism of validating feedback. + +#### 3.1 Validate Annotation Sematics +This sematics uses same rule of [Thrift IDL](https://thrift.apache.org/docs/idl). The validate option only works on struct fields, thus we must start from Field sematics part. +- Field +```apache +Field ::= FieldID? FieldReq? FieldType Identifier ('=' ConstValue)? ValidateAnnotations? ListSeparator? +``` +- ValidateAnnotations +```apache +ValidateAnnotations ::= '(' ValidateRule+ ListSeparator? ')' +``` +- ValidateRule +```apache +ValidateRule := ('validate' | 'vt') Validator+ = '"' ValidatingValue? '"' +``` +- Validator + + Build-in validating logics. See [Supported Validator](#321-supported-validator) part. +```apache +Validator = '.' Identifier +``` +- ValidatingValue +```apache +ValidatingValue := (ToolFunction '(' )? Arguments ‘)’? +``` +- ToolFunction + + Build-in or user-defined tool functions. See [Tool Function](#325-tool-function) part. +```apache +ToolFunction := '@' Identifier +``` +- Arguments +```apache +Arguments := (DynamicValue ListSeparator?)* +``` +- DynamicValue +```apache +DynamicValue := ConstValue | FieldReference +``` +- FieldReference + + See [Field Reference](#324-field-reference) part. +```apache +FieldReference := '$' ReferPath +ReferPath := FieldName?(('['IntConstant']')|('.'Identifier))? +``` +- All other sematics keep same with [standard definition](https://thrift.apache.org/docs/idl) + +### 3.2 Validate Rule +The validate rule is works as a Boolean Expression, and Validator is core logic for one validate rule. Every Validator works like an Operator, calculating the Validating Value and Field Value, and then compare. For example, `gt` (greater than) will compare the right Validating Value with value of the field it belongs to, and return `true` if field value is greater than value or `false` if field value is not. We appoint that: Only if the validate rule returns true, the validated parameter is valid. + +#### 3.2.1 Supported Validator +Below lists the support validators. Value type means the type of validating value, field type means type of validated field. + +| validator | behavior | value type | field type | secodary validator | +| ------------ | -------------------------------- | ------------------------------------ | ---------------------- | ------------------ | +| const | must be constant | string, bool | same with value | - | +| defined_only | must be defined value | enum | enum | - | +| not_nil | must not be empty | "true" | any | - | +| skip | skip validate | "true" | any | - | +| eq | equals to (`==`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| ne | not equals to (`!=`) | i8, i16, i32, i64, f64, string, bool | same with value | - | +| lt | less than (`<`) | i8, i16, i32, i64, f64 | same with value | - | +| le | less equal (`<=`) | i8, i16, i32, i64, f64 | same with value | - | +| gt | greater than (`>`) | i8, i16, i32, i64, f64 | same with value | - | +| ge | greater equal (`>=`) | i8, i16, i32, i64, f64 | same with value | - | +| in | within given container | i8, i16, i32, i64, f64 | same with value | - | +| not_in | not within given container | i8, i16, i32, i64, f64 | same with value | - | +| elem | field's element constraint | any | list, set | support | +| key | field's element key constraint | any | map | support | +| value | field's element value constraint | any | map | support | +| min_size | minimal length | i8, i16, i32, i64 | string, list, set, map | - | +| max_size | maximal length | i8, i16, i32, i64 | string, list, set, map | - | +| prefix | field prefix must be | string | string | - | +| suffix | suffix must be | string | string | - | +| contain | must contain | string | string | - | +| not_contain | must not contain | string | string | - | + +- Secodary validator is a successive validator, usually used at container-type field. See below Set/List/Map examples. + +#### 3.2.2 IDL example +- Number +``` +struct NumericDemo{ + 1: double Value (validator.gte = "1000.1", validator.lte = "10000.1") + 2: i8 Type (validator.in = "[1, 2, 4]") +} +``` +- String/Binary +``` +struct StringDemo{ + 1: string Uninitialized (vt.const = "abc") + 2: string Name (vt.min_size = "6", vt.max_size = "12") + 3: string SomeStuffs (vt.pattern = "[0-9A-Za-z]+") + 4: string DebugInfo (vt.prefix = "[Debug]") + 5: string PanicInfo (vt.contains = "panic") +} +``` +- Bool +``` +struct BoolDemo { + 1: bool AMD (vt.const = "true") +} +``` +- Enum +``` +struct EnumDemo { + 1: Type AddressType (vt.const = "String") Review comment: also according to the table `const` can only be used on string and bool types, while an enum is technically an int type. I also don't see why `in` and `not_in` can only be used for int types but not string/bool types. all in all I still don't see anything `const` can do that `in` cannot so I'd suggest just use `in` and remove `const`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
