imay commented on issue #1897: HLL column may store null value URL: https://github.com/apache/incubator-doris/issues/1897#issuecomment-535822148 After I looked into backend code, I found 1. HLL column default column is "0". This should be fixed to no default value for HLL Column 2. InsertStmt consider that this default value can be ingested into Doris. This should be fixed to not allow varchar is casted to HLL column. 3. OLAPTableSink will send string "0" as serialized HLL column, which is invalid. This should be checked and filtered before sending. 4. "0" will serialized into an empty string. This should be serialized to a HLL empty set. 5. empty string is a nullptr when reading HLL column. we will handle nullptr as HLL empty set. 6. nullptr will cause backend process crash. I will fix backend in following ways. 1. I will add simple check for HLL column when data loading, this can avoid dirty data being ingested in our storage. 2. we will add some branch to let dirty data can be handler without causing backend process crash. @HangyuanLiu Can you improve our process in Frontend?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
