Hi Arrow Team,

I'm using Apache Arrow with AWS Lambda Functions.

The primary motivation is AWS Athena's user-defined functions[1].  Those
functions process and return Arrow IPC segments.

* The published Python wheels for Apache Arrow include almost every feature
of Arrow. (Gandiva, Plasma, Flight)
* The 9.0.0 wheel is 32MB compressed and 107MB decompressed.  Lambda
functions are limited to 50MB total compressed and 250MB decompressed.
* The smaller the Lambda function, the faster it runs from a cold start.

Would it be possible to create a new Python package (i.e., "pyarrow-slim")
that would disable some of the functionality but result in smaller python
wheels?

Thank you for considering this,

Rusty

[1] - https://docs.aws.amazon.com/athena/latest/ug/querying-udf.html

Reply via email to