Hi, On Mon, Oct 5, 2015 at 5:02 AM, Begin Daniel <jfd...@hotmail.com> wrote:
> In order to process a large amount of data I need to run a procedure using > parallel batch processes. > The query I run in each process is expected to ... > > It seems, you are trying to achieve the same, what we did. Find my blog entry <http://manojadinesh.blogspot.com/2015/07/parallel-operations-with-plpgsql_9.html>, which may help you in this scenarios. 1- select a bunch of id (500) in a table (wait4processing > ) containing the list of all records to process > 2- remove selected records from wait4processing table in order to > eliminate duplicate processing > 3- run the procedure (build_contributions_clusters) over the range of > selected ids > > --The query I use: > With ids as( delete from wait4processing where id in( select id from > wait4processing limit 500) returning id) > select build_contributions_clusters(min(id),max(id)) from ids; > > The query runs properly if I send it sequentially (wait for the completion > of the query before sening it again) but it does'nt work when sending > multiple occurrences in parallel. Seems from the results I got that the > first query received by the server runs properly but the following ones try > to process the same first 500 records even if deleted - the > build_contributions_clusters procedure receive NULL values instead of the > following 500 records. > > Since I am almost certain it is the expected behavior, I would like to like > to understand why, and I would also appreciate to see alternative queries > to do the job. > > Thanks :-) > -- Regards, Dinesh manojadinesh.blogspot.com
