Re: PutSQL benchmarking ?

[l vic]

Hi Matt,
No, I don't do record splitting, data looks like {
"attr1":"val1",...[{....}]}
where "parent" data is saved into 1 record in "parent" table and array data
is saved into multiple records in "child" table...
What's "lineage duration"?
Event Duration
< 1ms
Lineage Duration
00:00:00.070

On Wed, Feb 6, 2019 at 2:59 PM Matt Burgess <mattyb...@apache.org> wrote:

> In your flow, what does the data look like? Are you splitting it into
> individual records, then converting to SQL (probably via JSON) and
> calling PutSQL? If so, that's not going to be very performant; the
> PutDatabaseRecord processor combines all that together so you can
> leave your data in its original state (i.e. many records in one flow
> file). For benchmarking PutDatabaseRecord (PDR), you could provide
> sample data via GenerateFlowFile, run a few through PDR, and check the
> provenance events for fields such as durationMillis or calculations
> like (timestampMills - lineageStart).
>
> Regards,
> Matt
>
> On Wed, Feb 6, 2019 at 2:07 PM l vic <lvic4...@gmail.com> wrote:
> >
> > I have performance issues with PutSQL i my flow... Is there some way to
> benchmark time required to write certain number of records to table from
> GenerateFlowFile?
> > Thank you,
>