[ https://issues.apache.org/jira/browse/IMPALA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-2729: ---------------------------------- Issue Type: New Feature (was: Bug) > Support default values for DECIMAL in Avro. > ------------------------------------------- > > Key: IMPALA-2729 > URL: https://issues.apache.org/jira/browse/IMPALA-2729 > Project: IMPALA > Issue Type: New Feature > Components: Backend > Affects Versions: Impala 2.0, Impala 2.1, Impala 2.2, Impala 2.3.0 > Reporter: Alexander Behm > Priority: Minor > Labels: ramp-up, usability > > Impala may fail to scan Avro files of a table that has default values > specified on DECIMAL columns. > The following criteria must be met for hitting this bug: > 1. The Impala table was created from an Avro schema that has a default value > on a DECIMAL field > 2. There is an Avro file storing data of that table that does not have the > DECIMAL field declared as part of the table schema from 1 > When querying such a table Impala will return the following error > {code} > Field 'decimalFieldName' is missing from file and default values of type > DECIMAL are not yet supported. > {code} > The relevant code can be found in hdfs-avro-scanner.cc: > {code} > Status HdfsAvroScanner::WriteDefaultValue( > SlotDescriptor* slot_desc, avro_datum_t default_value, const char* > field_name) { > if (avro_header_->template_tuple == NULL) { > avro_header_->template_tuple = template_tuple_ != NULL ? > template_tuple_ : > scan_node_->InitEmptyTemplateTuple(*scan_node_->tuple_desc()); > } > switch (default_value->type) { > case AVRO_BOOLEAN: { > // We don't call VerifyTypesMatch() above the switch statement so we > don't want to > // call it in the default case (since we VerifyTypesMatch() can't > handle every type > // either, and we want to return the correct error message). > RETURN_IF_ERROR(VerifyTypesMatch(slot_desc, default_value)); > int8_t v; > if (avro_boolean_get(default_value, &v)) DCHECK(false); > RawValue::Write(&v, avro_header_->template_tuple, slot_desc, NULL); > break; > } > case AVRO_INT32: { > RETURN_IF_ERROR(VerifyTypesMatch(slot_desc, default_value)); > int32_t v; > if (avro_int32_get(default_value, &v)) DCHECK(false); > RawValue::Write(&v, avro_header_->template_tuple, slot_desc, NULL); > break; > } > ... > <--- case for AVRO_DECIMAL not handled > default: > return Status(TErrorCode::AVRO_UNSUPPORTED_DEFAULT_VALUE, field_name, > avro_type_name(default_value->type)); > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org