Re: Validate CSV/Records by name instead of position

2019-09-18 Thread Eric Chaves
Awesome, thanks for the detailed steps!

Em qua, 18 de set de 2019 às 11:41, Jerry Vinokurov 
escreveu:

> This certainly works. You can create a schema registry and define an Avro
> schema listing your fields. Then make sure that when you set up the reader,
> it's configured to read the header so that it knows which fields go where
> in the record, set up the mode of the schema access to read from the
> registry you created, and then set the name of the actual schema, which is
> a property on the schema registry. This will correctly validate your CSV
> for you.
>
> On Wed, Sep 18, 2019 at 9:59 AM Eric Chaves  wrote:
>
>> Hi folks,
>>
>> Is it possible to validate fields/columns in Record or CSV by its name
>> instead of it's position? For example I have a record with two mandatory
>> fields and some optional fields but they may be on different position on
>> each ingested file. Should I use a script or there is already a processor
>> that could help me out with those?
>>
>> Regards,
>>
>
>
> --
> http://www.google.com/profiles/grapesmoker
>


Re: Validate CSV/Records by name instead of position

2019-09-18 Thread Jerry Vinokurov
This certainly works. You can create a schema registry and define an Avro
schema listing your fields. Then make sure that when you set up the reader,
it's configured to read the header so that it knows which fields go where
in the record, set up the mode of the schema access to read from the
registry you created, and then set the name of the actual schema, which is
a property on the schema registry. This will correctly validate your CSV
for you.

On Wed, Sep 18, 2019 at 9:59 AM Eric Chaves  wrote:

> Hi folks,
>
> Is it possible to validate fields/columns in Record or CSV by its name
> instead of it's position? For example I have a record with two mandatory
> fields and some optional fields but they may be on different position on
> each ingested file. Should I use a script or there is already a processor
> that could help me out with those?
>
> Regards,
>


-- 
http://www.google.com/profiles/grapesmoker