Hans Zeller created TRAFODION-2437: -------------------------------------- Summary: Implement a monitor for communications between ESPs of a fragment Key: TRAFODION-2437 URL: https://issues.apache.org/jira/browse/TRAFODION-2437 Project: Apache Trafodion Issue Type: New Feature Components: sql-exe Reporter: Hans Zeller Assignee: Hans Zeller Priority: Minor
Today, the ESPs that are executing parallel fragment instances can't talk to each other. They all do their work independently and may finish at different times. This works fine for normal parallel algorithms, but it somewhat restricts what we can do in parallel. Sometimes, a communication or maybe just a simple synchronization would be desirable. Examples: - Make all ESPs wait until the last ESP has finished before proceeding. This could create a global blocking operator, for example when we need a blocking operator to prevent the Halloween problem. - Dynamic range-repartitioning: After doing local sorts, negotiate common split boundaries to perform a range-repartitioning, so that we can produce a globally sorted result in parallel. - Communication between parallel instances of TMUDFs. Some TMUDFs may need similar communication, which could allow them to do more things in parallel. When implementing this, we may also need to pay some attention to potential deadlocks, although with blocking operators like sorts there should not be any deadlock issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)