[
https://issues.apache.org/jira/browse/ARROW-10025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörn Horstmann updated ARROW-10025:
-----------------------------------
Description:
The filter kernel accesses the filter array in chunks of u64 and tries to
access elements if the corresponding bit is set. This assumes that bits outside
of the valid len of that array have to be unset. That assumption is currently
not correct for output of the NOT and the IS NULL kernels, which also operate
on and negate a whole chunk of their input arrays.
Testcase (depending on is null kernel from ARROW-10016)
{code:java}
#[test]
fn csv_query_with_is_null_predicate() -> Result<()> {
let mut ctx = ExecutionContext::new();
register_aggregate_csv(&mut ctx)?;
let sql = "SELECT COUNT(1) FROM aggregate_test_100 WHERE c1 IS NULL";
let actual = execute(&mut ctx, sql).join("\n");
let expected = "0".to_string();
assert_eq!(expected, actual);
Ok(())
}
{code}
was:
The filter kernel accesses the filter array in chunks of u64 and tries to
access elements if the corresponding bit is set. This assumes that bits outside
of the valid len of that array have to be unset. That assumption is currently
not correct for output of the NOT and the IS NULL kernels, which also operate
on and negate a whole chunk of their input arrays.
{code:java}
#[test]
fn csv_query_with_is_null_predicate() -> Result<()> {
let mut ctx = ExecutionContext::new();
register_aggregate_csv(&mut ctx)?;
let sql = "SELECT COUNT(1) FROM aggregate_test_100 WHERE c1 IS NULL";
let actual = execute(&mut ctx, sql).join("\n");
let expected = "0".to_string();
assert_eq!(expected, actual);
Ok(())
}
{code}
> [Rust] Filter kernel relies on bits outside of valid len of boolean array
> -------------------------------------------------------------------------
>
> Key: ARROW-10025
> URL: https://issues.apache.org/jira/browse/ARROW-10025
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust, Rust - DataFusion
> Affects Versions: 1.0.1
> Reporter: Jörn Horstmann
> Priority: Major
>
> The filter kernel accesses the filter array in chunks of u64 and tries to
> access elements if the corresponding bit is set. This assumes that bits
> outside of the valid len of that array have to be unset. That assumption is
> currently not correct for output of the NOT and the IS NULL kernels, which
> also operate on and negate a whole chunk of their input arrays.
> Testcase (depending on is null kernel from ARROW-10016)
> {code:java}
> #[test]
> fn csv_query_with_is_null_predicate() -> Result<()> {
> let mut ctx = ExecutionContext::new();
> register_aggregate_csv(&mut ctx)?;
> let sql = "SELECT COUNT(1) FROM aggregate_test_100 WHERE c1 IS NULL";
> let actual = execute(&mut ctx, sql).join("\n");
> let expected = "0".to_string();
> assert_eq!(expected, actual);
> Ok(())
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
