Re: [I] [Proposal] Decouple logical from physical types [datafusion]

[via GitHub]

findepi commented on issue #11513:
URL: https://github.com/apache/datafusion/issues/11513#issuecomment-2401814374

   > ```rust
   > pub trait LogicalType {
   >     fn native(&self) -> &NativeType;
   >     fn name(&self) -> Option<&str>; // as in 'ARROW:extension:name'.
   > }
   > ```
   
   @notfilippo  This might work. I started similar prototype for 
https://github.com/apache/datafusion/issues/12644.
   
   To reliably encode types in Arrow schema, we would need a structured 
approach, perhaps something like this
   ```rust
   pub trait Type: Sync + Send {
       fn name(&self) -> &Arc<TypeName>;
   ...
   }
   
   #[derive(Debug, Clone, PartialEq, Eq)]
   pub struct TypeName {
       basename: String,
       name: String,
       parameters: Vec<TypeParameter>,
   }
   
   pub type TypeNameRef = Arc<TypeName>;
   
   #[derive(Debug, Clone, PartialEq, Eq)]
   pub enum TypeParameter {
       Type(TypeNameRef),
       Number(i128),
   }
   
   impl TypeName {
       pub fn basename(&self) -> &str {
           &self.basename
       }
       pub fn name(&self) -> &str {
           &self.name
       }
       pub fn parameters(&self) -> &[TypeParameter] {
           &self.parameters
       }
   
       pub fn new(basename: impl Into<String>, parameters: Vec<TypeParameter>) 
-> Self {
           // TODO assert parses, no braces, etc.
           let basename = basename.into();
           let name = FormatName {
               basename: &basename,
               parameters: &parameters,
           }
           .to_string();
           Self {
               basename,
               name,
               parameters,
           }
       }
       pub fn from_basename(basename: impl Into<String>) -> Self {
           Self::new(basename, vec![])
       }
   }
   ```
   
   To allow type-checking ("is int" or "is string") for optimizer rules such as 
`UnwrapCastExprRewriter` we could use `Any`
   ```rust
   pub trait Type: Sync + Send {
       fn as_any(&self) -> &dyn Any;
   ...
   }
   ```
   
   Type should also describe how to compare (native) values of that type, their 
equality, etc.
   (for example, per SQL spec, `timestamp with time zone` has  interesting 
equality defined on part of the value, so bit-by-bit equality cannot be 
assumed).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org