jacopotagliabue commented on issue #8011: URL: https://github.com/apache/iceberg/issues/8011#issuecomment-1627466061
@mr-brobot this is FANTASTIC - I solved my problem with this a while ago by monkeypatching both plan_files and to_arrow ``` DataScan.plan_files = _plan_files DataScan.to_arrow = _to_arrow ``` and basically removing the offending libraries to run in a Lambda - I agree that would be great to have some PyArrow config to just handle this better. I can't check now (not at laptop) but I'm pretty sure plan_files uses ThreadPools and even that is still not allowed in AWS lambda: did you test "live" in lambda or just docker? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
