Light-City opened a new pull request, #37171:
URL: https://github.com/apache/arrow/pull/37171
### Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in
the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your
changes and offer better suggestions for fixes.
-->
We have a scene. There is a plan in pg that looks like the following. For
the Append node, there are two scans in parallel, and then there is a column of
data, but the column names are different. If it is mapped to the arrow schema
It is a different field. For the append node, we will get two batches. The
first batch comes from the first scan, and the second batch comes from the
second scan, but because the two columns are constructed based on the scan The
schema is different, so the final schema of the two batches is different. When
we construct the slot returned by the Append node, we use the schema of the
first batch. When we put the data of the second batch into it, the verification
fails due to inconsistent shcema.
Therefore, the problem is simplified to: For a node, If there are n child
nodes, the schema of the following child nodes must be consistent. If not, the
schema of n-1 child nodes must be the same as the first schema, so there is
logic to rewrite the schema of the batch data.
```
-> Vec Append
-> Vec Seq Scan on public. tenk1
Output: tenk1.unique1
-> Vec Seq Scan on public.tenk1 tenk1_1
Output: tenk1_1.fivethous
```
However, when reading the batch code, there is only the read-only interface
schema(), so here we submit a pr to add and rewrite the schema interface, and
only modify the columns with the same type. If they are not the same, an
invalid modification will be returned.
backgroud: https://github.com/apache/arrow/issues/37170
### What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is
sometimes worth providing a summary of the individual changes in this PR.
-->
- record_batch.h
- record_batch.cc
- record_batch_test.cc
### Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are
they covered by existing tests)?
-->
yes, see record_batch_test.cc.
gtest filter is:
```
TestRecordBatch.RewriteSchema
```
### Are there any user-facing changes?
yes: see background in issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
