GGraziadei commented on issue #8816: URL: https://github.com/apache/storm/issues/8816#issuecomment-4829705837
I am adding the first proposal for this braimstorming process: instead of a single `JCQueue`, each executor is assigned two separate instances: a data queue and a control queue. Classification: Upon ingestion, the incoming tuple is evaluated by its source stream id. If it belongs to one of the reserved __ system streams, it is routed to the control queue; otherwise, it is directed to the data queue. Prioritization: During every consumption cycle, the consumer drains the control queue first (or applies a heavy processing weight to it) before handling the data queue. This approach creates a physical separation for the control plane, rendering it immune to data-plane backpressure, overflow, and backlogs. This directly achieves our goal, however, it introduces a strict requirement: the ingestion path must read and inspect every tuple's stream prior to enqueuing. ## Pros - Control Plane Isolation: Because the control queue utilizes a dedicated buffer, it cannot be blocked, delayed, or dropped by data backlogs. This guarantees minimal control-plane jitter even under complete data-plane saturation. - Independent Configuration: The control queue can be minimally sized, with a dynamic batch size, and entirely exempt from standard overflow or backpressure (BP) machinery. Meanwhile, the data queue maintains its existing overflow and BP semantics without modification. - Consumption Logic: A "drain control first" policy is highly predictable, simple to reason about, and easy to verify. Given that control traffic is inherently low-volume, the risk of starving the data plane is negligible in practice. - Negligible Cost on Primary Ingress: For remote tuples, deserialization is already completed by DeserializingConnectionCallback before reaching transferLocalBatch (WorkerState.java:240). Consequently, calling getSourceStreamId() for classification is a cheap field access rather than an expensive deserialization step. ## Cons - Overhead on the Hot Path: Inspecting every tuple adds an extra conditional branch and processing step directly on the critical ingestion paths. Additionally, the definition of what constitutes a "control stream" must be strictly centralized to prevent logic drift as new system streams are introduced. - Memory and bookkeeping footprint: Doubling the queue instances per executor doubles the underlying buffers, queue metrics, and the per-producer-thread batcher footprint. - Potential Starvation Risks: While "control first" scheduling is simple, it requires safeguards to ensure a sudden, hypothetical burst of system traffic cannot indefinitely stall data processing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
