[
https://issues.apache.org/jira/browse/PARQUET-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16977014#comment-16977014
]
Priyank Bagrecha edited comment on PARQUET-1684 at 11/24/19 5:24 AM:
---------------------------------------------------------------------
i am experiencing this issue even if the field is set to the default value. i
think this is happening because *hasField* will return false even if a field
has been set to default value in proto3.
([https://github.com/protocolbuffers/protobuf/issues/359]) and code uses
[getAllFields|https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/MessageOrBuilder.html#getAllFields--]
which will only return fields which have been set.
{code:java}
@Test
public void testDefaultIntegerValueNullInsteadofZero() {
/*
syntax = proto3;
Message Person {
string id = 1;
int32 age = 2;
}
*/
Descriptors.FieldDescriptor descriptor =
Person.getDescriptor().findFieldByName("age");
Person p1 = Person.newBuilder().setId("1").setAge(0).build();
Assert.assertEquals(false, p1.hasField(descriptor));
Person p2 = Person.newBuilder().setId("2").build();
Assert.assertEquals(false, p2.hasField(descriptor));
}
{code}
was (Author: pbagrecha):
i am experiencing this issue even if the field is set to the default value. i
think this is happening because *hasField* will return false even if a field
has been set to default value in proto3.
(https://github.com/protocolbuffers/protobuf/issues/359)
{code}
@Test
public void testDefaultIntegerValueNullInsteadofZero() {
/*
syntax = proto3;
Message Person {
string id = 1;
int32 age = 2;
}
*/
Descriptors.FieldDescriptor descriptor =
Person.getDescriptor().findFieldByName("age");
Person p1 = Person.newBuilder().setId("1").setAge(0).build();
Assert.assertEquals(false, p1.hasField(descriptor));
Person p2 = Person.newBuilder().setId("2").build();
Assert.assertEquals(false, p2.hasField(descriptor));
}
{code}
> [parquet-protobuf] default protobuf field values are stored as nulls
> --------------------------------------------------------------------
>
> Key: PARQUET-1684
> URL: https://issues.apache.org/jira/browse/PARQUET-1684
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.10.0, 1.11.0
> Reporter: George Haddad
> Priority: Major
> Labels: pull-request-available
>
> When the source is a protobuf3 message, and the target file is Parquet, all
> the default values are stored in the output parquet as `{{null`}} instead of
> the actual type's default value.
> For example, if the field is of type `int32`, `double` or `enum` and it
> hasn't been set, the parquet value is `{{null`}} instead of `0`. When the
> field's type is a `string` that hasn't been set, the parquet value is
> {{`null`}} instead of an empty string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
