codenohup opened a new issue, #83: URL: https://github.com/apache/flink-agents/issues/83
### Search before asking - [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar. ### Description During our usage of Pemja, we've encountered a few issues and would like to create this issue to track them. Note: Since flink-agents depends on the version of Pemja included in Flink, we are unable to upgrade Pemja independently. We have to wait for Flink to upgrade it, and then we can benefit from the fix in flink-agents accordingly. **Issue 1: Incorrect reference counting of Python objects in Pemja** Problem: - Pemja incorrectly manages the reference count of Python objects, which may lead to premature object deallocation. Current workaround in flink-agents: - We are using _interpreter.set()_ to manually increase the reference count as a temporary solution. However, this causes these objects to never be freed, potentially leading to memory leaks. Target fixed version: - Pemja 0.5.3 - Flink 1.20.2 **Issue 2: Generator objects are incorrectly converted into PyIterator in Pemja** Problem: - When a Python generator is passed through Pemja, it is mistakenly treated as a _PyIterator_. This causes the call to next(generator) to hang or block indefinitely. Current workaround in flink-agents: - We are wrapping the generator object into a regular Python object to bypass this issue. Target fixed version: - Pemja 0.5.5 - Flink 1.20.2 ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
