I am having a play with QueryRecord to do some filtering but I have run across
this problem. I have a schema for my records which includes a union type, so
the relevant part of the schema is
{
"type":"record",
"namespace":"blah",
"name":"SimpleTraffic",
"fields":[
{"name":"src_address","type":"string"},
{"name":"flag_s","type":["int","boolean"]}
]
}
This is because I am processing CSV records that look this, where 1 is true and
0 is false.
192.168.0.1,1
Into JSON that looks like this, using a ConvertRecord and an Update Record.
{"src_address":"192.168.0.1","flag_s":true}
Then I create a QueryRecord so I can filter out the cases where the flag is
false. So I use this query.
select * from flowfile where flag_s = true
But I get this error
org.apache.calcite.sql.validate.SqlValidatorException: Cannot apply '=' to
arguments of type '<JAVATYPE(CLASS JAVA.LANG.OBJECT)> = <BOOLEAN>'
Is this because the type is a Union type and the Calcite processor cannot work
out which subtype it should be? Can I do anything to persuade the query to use
an operator or a function on this field to make it usable? I have tried casting
to Boolean or Char but no success. Or do I need to use two separate "before"
and "after" schemas to eliminate the union?
Regards
Steve Hindmarch
