On Thu, Jan 25, 2024 at 11:25 AM torikoshia <torikos...@oss.nttdata.com> wrote: > > On 2024-01-24 17:05, Masahiko Sawada wrote: > > On Tue, Jan 23, 2024 at 1:02 AM torikoshia <torikos...@oss.nttdata.com> > > wrote: > >> > >> On 2024-01-17 14:47, Masahiko Sawada wrote: > >> > On Wed, Jan 17, 2024 at 2:22 PM torikoshia <torikos...@oss.nttdata.com> > >> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to > >> >> skip malformed data, but there is no way to watch the number of > >> >> skipped > >> >> rows during COPY. > >> >> > >> >> Attached patch adds tuples_skipped to pg_stat_progress_copy, which > >> >> counts the number of skipped tuples because source data is malformed. > >> >> If SAVE_ERROR_TO is not specified, this column remains zero. > >> >> > >> >> The advantage would be that users can quickly notice and stop COPYing > >> >> when there is a larger amount of skipped data than expected, for > >> >> example. > >> >> > >> >> As described in commit log, it is expected to add more choices for > >> >> SAVE_ERROR_TO like 'log' and using such options may enable us to know > >> >> the number of skipped tuples during COPY, but exposed in > >> >> pg_stat_progress_copy would be easier to monitor. > >> >> > >> >> > >> >> What do you think? > >> > > >> > +1 > >> > > >> > The patch is pretty simple. Here is a comment: > >> > > >> > + (if <literal>SAVE_ERROR_TO</literal> is specified, otherwise > >> > zero). > >> > + </para></entry> > >> > + </row> > >> > > >> > To be precise, this counter only advances when a value other than > >> > 'ERROR' is specified to SAVE_ERROR_TO option. > >> > >> Thanks for your comment and review! > >> > >> Updated the patch according to your comment and option name change by > >> b725b7eec. > > > > Thanks! The patch looks good to me. I'm going to push it tomorrow, > > barring any objections. > > Thanks!
Pushed (commit 729439607). Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com
