[
https://issues.apache.org/jira/browse/IMPALA-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968594#comment-16968594
]
Tim Armstrong commented on IMPALA-9127:
---------------------------------------
Maybe the states would be something like:
* PROBING_HASH_PARTITIONS_NO_BATCH -> hash_partitions are valid and we are
processing probe batches. We do not have a current probe batch.
* PROBING_HASH_PARTITIONS_IN_BATCH -> hash_partitions are valid and we are
processing probe batches. We have a current probe batch.
* OUTPUTTING_UNMATCHED_ROWS -> we have some output rows to emit
* CURRENT_PROBE_COMPLETE -> finished processing the current input probe rows
and any subsequent actions. There may be more spilled partitions.
* OUTPUTTING_NULL_PROBE_ROWS -> processing null_probe_rows_ for output
* OUTPUTTING_NULL_AWARE_PARTITION -> processing null_aware partition for output
* END -> finishing processing all probe input, spilled partitions and any
follow-up work.
> Clean up probe-side state machine in hash join
> ----------------------------------------------
>
> Key: IMPALA-9127
> URL: https://issues.apache.org/jira/browse/IMPALA-9127
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> There's an implicit state machine in the main loop in
> PartitionedHashJoinNode::GetNext()
> https://github.com/apache/impala/blob/eea617b/be/src/exec/partitioned-hash-join-node.cc#L510
> The state is implicitly defined based on the following conditions:
> * !output_build_partitions_.empty() -> "outputting build rows after probing"
> * builder_->null_aware_partition() == NULL -> "eos, because this the
> null-aware partition is processed after all other partitions"
> * null_probe_output_idx_ >= 0 -> "null probe rows being processed"
> * output_null_aware_probe_rows_running_ -> "null-aware partition being
> processed"
> * probe_batch_pos_ != -1 -> "processing probe batch"
> * builder_->num_hash_partitions() != 0 -> "have active hash partitions that
> are being probed"
> * spilled_partitions_.empty() -> "no more spilled partitions"
> I think this would be a lot easier to follow if the state machine was
> explicit and documented, and would make separating out the build side of a
> spilling hash join easier to get right.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
