[
https://issues.apache.org/jira/browse/HUDI-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qian heng updated HUDI-1462:
----------------------------
Description:
This is my query on a rt view of MOR table：
select count(*) from ***mor_rt where platform = "HYLOOP" and
`_hoodie_commit_time`= '20201210213000';
|_c0 |
|0 |
But it do have records whose `_hoodie_commit_time` equal '20201210213000', it
can be proved by the result of query:
select `_hoodie_commit_time`, count(*) from ***mor_rt where platform = "HYLOOP"
group by `_hoodie_commit_time`;
|_hoodie_commit_time|_c1 |
....
|20201210213000 |16 |
....
I think it may be a bug and is caused by the wrong code logic in
HoodieCombineRealtimeRecordReader .
{code:java}
//代码占位符
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
LOG.info("Reading from record reader");
LOG.info(HoodieRealtimeRecordReaderUtils.arrayWritableToString(value));
return true;
} else if (recordReaders.size() > 0) {
this.currentRecordReader.close();
this.currentRecordReader = recordReaders.remove(0);
return this.currentRecordReader.next(key, value); //TODO: bug? return
this.next(key, value);
} else {
return false;
}
}
{code}
was:
This is my query on a rt view of MOR table：
select count(*) from ***mor_rt where platform = "HYLOOP" and
`_hoodie_commit_time`= '20201210213000';
+------+
| _c0 |
+------+
| 0 |
+------+
But it do have records whose `_hoodie_commit_time` equal '20201210213000', it
can be proved by the result of query:
select `_hoodie_commit_time`, count(*) from ***mor_rt where platform = "HYLOOP"
group by `_hoodie_commit_time`;
+----------------------+------+
| _hoodie_commit_time | _c1 |
+----------------------+------+
....
| 20201210213000 | 16 |
....
I think it may be a bug and is caused by the wrong code logic in
HoodieCombineRealtimeRecordReader.
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
{code:java}
//代码占位符
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
LOG.info("Reading from record reader");
LOG.info(HoodieRealtimeRecordReaderUtils.arrayWritableToString(value));
return true;
} else if (recordReaders.size() > 0) {
this.currentRecordReader.close();
this.currentRecordReader = recordReaders.remove(0);
return this.currentRecordReader.next(key, value); //TODO: bug? return
this.next(key, value);
} else {
return false;
}
}
{code}
> The rt view query returns a wrong result with predicate push down
> -----------------------------------------------------------------
>
> Key: HUDI-1462
> URL: https://issues.apache.org/jira/browse/HUDI-1462
> Project: Apache Hudi
> Issue Type: Bug
> Components: Common Core
> Affects Versions: 0.6.1
> Reporter: qian heng
> Priority: Major
>
> This is my query on a rt view of MOR table：
> select count(*) from ***mor_rt where platform = "HYLOOP" and
> `_hoodie_commit_time`= '20201210213000';
> |_c0 |
> |0 |
> But it do have records whose `_hoodie_commit_time` equal '20201210213000',
> it can be proved by the result of query:
> select `_hoodie_commit_time`, count(*) from ***mor_rt where platform =
> "HYLOOP" group by `_hoodie_commit_time`;
> |_hoodie_commit_time|_c1 |
> ....
> |20201210213000 |16 |
> ....
> I think it may be a bug and is caused by the wrong code logic in
> HoodieCombineRealtimeRecordReader .
> {code:java}
> //代码占位符
> public boolean next(NullWritable key, ArrayWritable value) throws IOException
> {
> if (this.currentRecordReader.next(key, value)) {
> LOG.info("Reading from record reader");
> LOG.info(HoodieRealtimeRecordReaderUtils.arrayWritableToString(value));
> return true;
> } else if (recordReaders.size() > 0) {
> this.currentRecordReader.close();
> this.currentRecordReader = recordReaders.remove(0);
> return this.currentRecordReader.next(key, value); //TODO: bug? return
> this.next(key, value);
> } else {
> return false;
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
