GitHub user skrawcz added a comment to the discussion: Parallelization Enhancement Ideas
@mitstake could you provide an example that shows equivalence in Hamilton? That would help me start from a common comparison and reduce my interpretation. I think I'm slowly starting to get it. Other possibly tangiental thoughts: 1. redo parallelization -- we should be able to do nested parallelization if we involve a queue, rather than the current way it is implemented. Would that solve some of this problem? 2. I think recursively with graphs. I think that's what you're getting at with drivers above? I think there could be some more syntactic sugar to achieve what you want. E.g. I think the boilerplate collect could be avoided somehow. One challenge is you can't necessarily resolve the graph in this manner -- I think that's fine, but perhaps there's a way to constrain the path walked... not sure if that makes sense. GitHub link: https://github.com/apache/hamilton/discussions/1412#discussioncomment-15661531 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
