crepererum opened a new issue, #16652:
URL: https://github.com/apache/datafusion/issues/16652

   # What
   Make sure that error types are reasonably small (in bytes).
   
   # Why
   Currently our error types are huge, esp. `DataFusionError` -- which is used 
all over the place -- measures over 100 bytes. For comparison: that's [the size 
of over 4 
`String`s](https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=0ff77c94f91c32282196dde5d99f819c).
 Now allocation-wise, this usually wouldn't be an issue on the cold path, but 
the issue is that large errors also blow up `Result` enums which are allocated 
on stack and within async state machines. So you pay for that on the "happy 
path" as well.
   
   # How
   Place a few careful `Box`es within the respective error variants and remove 
the `#[expect(clippy::large_enum_variant)`. Since the `Box` allocation only 
happens on the error path, this makes the happy path strictly better. And the 
error path is only a bit slower due to that[^1]
   
   
   [^1]: If you look at all the `String`s and `Vec`s in our error types, I 
would argue that we don't optimize for error performance anyways. And it gets 
even worse when the `backtrace` feature is enabled.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org

Reply via email to