Hi Charlie,

Thanks for sharing the template.
Following configurations for UpdateRecord did the flat to nested mapping:

- Replacement Value Strategy: Record Path Value
- Dynamic property: "/phone" = "/"

It maps the flat record into /phone child record.
Fields those are not included in the outgoing schema will be discarded.
But it would work only if nested fields have the same name.

Example result:
[ {
  "person" : "john",
  "gender" : "m",
  "phone" : {
    "phone1" : "123-456-7890",
    "phone1type" : "mobile",
    "phone2" : "234-567-8901",
    "phone2type" : "home"
  }
} ]

If the mapping gets more complicated than that, then I'd go with
ConvertRecord -> JoltTransformJSON.

Hope this helps,
Koji

On Sat, Feb 10, 2018 at 1:31 AM, Charlie Frasure
<charliefras...@gmail.com> wrote:
> Hi,
>
> I'm including a template here that mimics the problem I'm having.  I
> generate a one record csv and try to convert it to a JSON file with a nested
> section for phones.  The phone section of the JSON file converts as null
> instead of having the phone attributes.
> If we can resolve this problem, I would also like to map phone1 and phone2
> attributes to phone[0] ... phone[n].
>
> Can this be completed with the record writers?
>
> On Thu, Feb 8, 2018 at 10:25 PM, Charlie Frasure <charliefras...@gmail.com>
> wrote:
>>
>> Hi,
>>
>> I'm having trouble taking a flat record and building an avro schema that
>> lets me nest it.
>> I found this example schema, but after using a ConvertRecord or
>> UpdateRecord, I receive the equivalent of [{"parent" = null}]
>>
>> {"type": "record", "name": "CustomerInput", "namespace":
>> "org.apache.example", "fields": [
>>         {"name": "id", "type": "string"},
>>         {"name": "companyName", "type": ["null", "string"], "default":
>> null},
>>         {"name": "revenue", "type": ["null", "string"], "default": null},
>>         {"name" : "parent", "type" : [ "null",
>>             {"type" : "record", "name" : "parent", "fields" : [
>>                 {"name" : "name", "type" : ["null", "string"], "default" :
>> null},
>>                 {"name" : "id", "type" : "string"}
>>             ] }
>>         ], "default" : null }
>> ]}
>>
>> This example found online is meant to union a person schema with an
>> address schema, creating an "Address" type, but this resulted in a "Could
>> not parse incoming data" error.
>>
>> [
>>   {"type": "record", "name": "Address", "fields": [
>>       {"name": "streetaddress", "type": "string"},
>>       {"name": "city", "type": "string"}
>>   ]},
>>   {"type": "record", "name": "person", "fields": [
>>       {"name": "firstname","type": "string"},
>>       {"name": "lastname", "type": "string"},
>>       {"name": "address", "type": "Address"}
>>   ]}
>> ]
>>
>> I would also like to be able to include multiples of these sub-records as
>> well, such that I could have 0 to n addresses with distinct address types
>> associated to one person.
>> Is this possible with the record processors, or do I need to focus on Jolt
>> / ConvertAvro?
>>
>> Thanks,
>> Charlie
>>
>>
>

Reply via email to