TheNeuralBit commented on a change in pull request #12906: URL: https://github.com/apache/beam/pull/12906#discussion_r493105830
########## File path: sdks/python/apache_beam/typehints/schemas.py ########## @@ -17,36 +17,42 @@ """ Support for mapping python types to proto Schemas and back again. -Python Schema -np.int8 <-----> BYTE -np.int16 <-----> INT16 -np.int32 <-----> INT32 -np.int64 <-----> INT64 -int ---/ -np.float32 <-----> FLOAT -np.float64 <-----> DOUBLE -float ---/ -bool <-----> BOOLEAN -Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") -Mapping <-----> MapType -Sequence <-----> ArrayType -NamedTuple <-----> RowType -beam.Row ---/ - -nullable=True on a Beam FieldType is represented in Python by wrapping the -typing in Optional. - -The mappings for STRING and BYTES are different between python 2 and python 3, -because of the changes to str: -py3: -str/unicode <-----> STRING -bytes <-----> BYTES -ByteString ---/ - -py2: -str will be rejected since it is ambiguous. -unicode <-----> STRING -ByteString <-----> BYTES +Imposes a mapping between common Python types and Beam portable schemas +(https://s.apache.org/beam-schemas):: + + Python Schema + np.int8 <-----> BYTE + np.int16 <-----> INT16 + np.int32 <-----> INT32 + np.int64 <-----> INT64 + int ---/ Review comment: It's supposed to indicate that when converting to Beam schemas int will map to INT64, but it's not a two-way mapping. Converting back from Beam schemas will produce np.int64, not int. ########## File path: sdks/python/apache_beam/typehints/schemas.py ########## @@ -17,36 +17,42 @@ """ Support for mapping python types to proto Schemas and back again. -Python Schema -np.int8 <-----> BYTE -np.int16 <-----> INT16 -np.int32 <-----> INT32 -np.int64 <-----> INT64 -int ---/ -np.float32 <-----> FLOAT -np.float64 <-----> DOUBLE -float ---/ -bool <-----> BOOLEAN -Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") -Mapping <-----> MapType -Sequence <-----> ArrayType -NamedTuple <-----> RowType -beam.Row ---/ - -nullable=True on a Beam FieldType is represented in Python by wrapping the -typing in Optional. - -The mappings for STRING and BYTES are different between python 2 and python 3, -because of the changes to str: -py3: -str/unicode <-----> STRING -bytes <-----> BYTES -ByteString ---/ - -py2: -str will be rejected since it is ambiguous. -unicode <-----> STRING -ByteString <-----> BYTES +Imposes a mapping between common Python types and Beam portable schemas +(https://s.apache.org/beam-schemas):: + + Python Schema + np.int8 <-----> BYTE + np.int16 <-----> INT16 + np.int32 <-----> INT32 + np.int64 <-----> INT64 + int ---/ + np.float32 <-----> FLOAT + np.float64 <-----> DOUBLE + float ---/ + bool <-----> BOOLEAN + Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") + Mapping <-----> MapType + Sequence <-----> ArrayType + NamedTuple <-----> RowType + beam.Row ---/ + +:code:`nullable=True` on a Beam :code:`FieldType` is represented in Python by +wrapping the type in :code:`Optional`. + +The mappings for :code:`STRING` and :code:`BYTES` are different between python +2 and python 3, because of the changes to str. + +Python 3.x:: + + str/unicode <-----> STRING + bytes <-----> BYTES + ByteString ---/ + +Python 2.x:: Review comment: Yeah I guess I should just drop all the Python 2 shims now :tada: ########## File path: sdks/python/apache_beam/typehints/schemas.py ########## @@ -17,36 +17,42 @@ """ Support for mapping python types to proto Schemas and back again. -Python Schema -np.int8 <-----> BYTE -np.int16 <-----> INT16 -np.int32 <-----> INT32 -np.int64 <-----> INT64 -int ---/ -np.float32 <-----> FLOAT -np.float64 <-----> DOUBLE -float ---/ -bool <-----> BOOLEAN -Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") -Mapping <-----> MapType -Sequence <-----> ArrayType -NamedTuple <-----> RowType -beam.Row ---/ - -nullable=True on a Beam FieldType is represented in Python by wrapping the -typing in Optional. - -The mappings for STRING and BYTES are different between python 2 and python 3, -because of the changes to str: -py3: -str/unicode <-----> STRING -bytes <-----> BYTES -ByteString ---/ - -py2: -str will be rejected since it is ambiguous. -unicode <-----> STRING -ByteString <-----> BYTES +Imposes a mapping between common Python types and Beam portable schemas +(https://s.apache.org/beam-schemas):: + + Python Schema + np.int8 <-----> BYTE + np.int16 <-----> INT16 + np.int32 <-----> INT32 + np.int64 <-----> INT64 + int ---/ Review comment: I guess they seem pretty equivalent to me, but I see your point that it my ascii art for two arrows merging may not be obvious. I'll change it ########## File path: sdks/python/apache_beam/typehints/schemas.py ########## @@ -17,36 +17,36 @@ """ Support for mapping python types to proto Schemas and back again. -Python Schema -np.int8 <-----> BYTE -np.int16 <-----> INT16 -np.int32 <-----> INT32 -np.int64 <-----> INT64 -int ---/ -np.float32 <-----> FLOAT -np.float64 <-----> DOUBLE -float ---/ -bool <-----> BOOLEAN -Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") -Mapping <-----> MapType -Sequence <-----> ArrayType -NamedTuple <-----> RowType -beam.Row ---/ - -nullable=True on a Beam FieldType is represented in Python by wrapping the -typing in Optional. - -The mappings for STRING and BYTES are different between python 2 and python 3, -because of the changes to str: -py3: -str/unicode <-----> STRING -bytes <-----> BYTES -ByteString ---/ - -py2: -str will be rejected since it is ambiguous. -unicode <-----> STRING -ByteString <-----> BYTES +Imposes a mapping between common Python types and Beam portable schemas +(https://s.apache.org/beam-schemas):: + + Python Schema + np.int8 <-----> BYTE + np.int16 <-----> INT16 + np.int32 <-----> INT32 + np.int64 <-----> INT64 + int ------> INT64 + np.float32 <-----> FLOAT + np.float64 <-----> DOUBLE + float ------> DOUBLE + bool <-----> BOOLEAN + str/unicode <-----> STRING + bytes <-----> BYTES + ByteString ------> BYTES + Timestamp <-----> LogicalType(urn="beam:logical_type:micros_instant:v1") + Mapping <-----> MapType + Sequence <-----> ArrayType + NamedTuple <-----> RowType + beam.Row ------> RowType + +Note that some of these mappings are provided as conveniences, Review comment: Thanks for helping me clarify my documentation :) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
