Additional info on this option. Currently, this option is only used for non-aligned format tables where each column value is stored as one cell.
We cannot pushdown preds on aligned format rows as hbase filter since columns within that row are not known to hbase. We have plans to extend that for aligned format table either as a user written custom filter that can interpret aligned row or through a coprocessor. anoop -----Original Message----- From: Eric Owhadi [mailto:[email protected]] Sent: Tuesday, October 27, 2015 10:31 AM To: [email protected] Subject: RE: SERIALIZED option on columns That is great, I was thinking that predicate pushdown would need expensive data conversion on RS side because we could not use byte compares, thanks to this new encoding, this is taken care off. Coolness... -----Original Message----- From: Anoop Sharma [mailto:[email protected]] Sent: Tuesday, October 27, 2015 12:26 PM To: [email protected] Subject: RE: SERIALIZED option on columns it encodes columns before writing them to hbase so they could be used by hbase to compare as bytes. Helps with pushdown of preds. If this option is not specified, then column values are written as it. Default is SERIALIZATION ON. On read, we decode it back to its original value. Same kind of encoding that we do for key columns. Code is in exp/exp_function.cpp where it deals with encoding and decoding. anoop -----Original Message----- From: Eric Owhadi [mailto:[email protected]] Sent: Tuesday, October 27, 2015 10:23 AM To: [email protected] Subject: RE: SERIALIZED option on columns And what is the semantic? Is it just to say that this should not be in aligned format in a mix format mode? Or is there more to it? Eric -----Original Message----- From: Anoop Sharma [mailto:[email protected]] Sent: Tuesday, October 27, 2015 12:17 PM To: [email protected] Subject: RE: SERIALIZED option on columns Yes, it is a new option in trafodion. It was just enabled as default. We can update documentation about it. anoop -----Original Message----- From: Eric Owhadi [mailto:[email protected]] Sent: Tuesday, October 27, 2015 10:15 AM To: [email protected] Subject: SERIALIZED option on columns Hi all, I am trying to understand predicate push down code. There is reference to columns tagged “serialized”. So I went on the SQL reference manual looking for what was the semantic of this tag. Could not find a single hit on “Serialized” keyword. Is that a new feature? Or is the documentation needing an update? I can open a Jira on documentation if needed? Eric
