When you use vectors it is really buffered-reading and you call it in a
loop. It just lets you process multiple rows at a time rather than one at a
time.
The number of rows returned by a results set may well exceed your vector
size so you should process the data then fetch the next number of rows. You
can, of course, copy the data into another buffer and have one or more
thread process that while you fetch more. You cannot "swap" them though.
Copying the data usually isn't that expensive really.
Processing queries and results sets is generally one of the hardest
techniques to do efficiently. In my experience I have seen it done
inefficiently much of the time, and then they "blame" the database for
being slow. (One particular evil is fetching each value by its column name.
Those who learn to use boost::datetime and its facet for parsing strings of
dates/times make the mistake of creating the facet and locale for every
date/time they parse too).
On Thu, Feb 16, 2012 at 11:26 AM, Simon Walter <[email protected]> wrote:
> On 02/16/2012 06:01 PM, Neil Morgenstern wrote:
> > The issue is that when you bind the result you are actually binding a
> > pointer to a contiguous buffer, not a C++ object.
> > So actually it's &vec[0] that is being bound. If a vector is resized
> > upward then &vec[0] would change
> > and the pointers would no longer be valid.
> >
> > Once the results are read in the vectors can be resized down for you
> > just to inform you how many were actually returned.
> > The fact that you get fewer results than your buffer size doesn't mean
> > there aren't more results, just that there are no
> > more results ready at present. Remember the database doesn't necessarily
> > return all its results at once. Therefore
> > you can resize your vectors at this point but when a vector does a
> > resize down it doesn't reduce the capacity therefore
> > if you resize it back up again the pointers won't actually change. The
> > size is a logical size, i.e. indicates the number of
> > rows populated.
> >
> > Note that a vector must be resized even to write into the memory
> > directly. You can't rely on reserve.
> >
> >
>
> Hi Neil, thanks for explaining that. I thought perhaps SOCI was doing a
> push_back or similar - expensive I guess...
>
> So perhaps I would need to create any of my queries that return a set to
> only return a certain sized result set. I think I can figure out how to
> write the SQL for that, but I was thinking perhaps SOCI already has that
> functionality. I guess that would require some kind of DAL. Seeing as it
> does abstract away certain things like 'prepare', it seems that SOCI
> does have at least some of this functionality.
>
> Is SOCI able to abstract away reading rowsets and provide some kind of
> interface for that?
>
> Or must I craft the SQL and keep reading results until I get back a
> result set smaller than what I initialized the vector to?
>
> Are there any other best practices?
>
> Many thanks,
>
> Simon
>
>
> --
> simonsmicrophone.com
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Soci-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/soci-users
>
------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Soci-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/soci-users
