Pawel Sztromwasser wrote: > Hi all, > > I have been trying to optimize the performance of LowessNormalization > plugin a little bit lately. It is extremely slow on our Postgres > database when using small 'block groups'.
It sounds like you are using an older version of BASE. The performance of the Lowess plug-in for small block group sizes was greatly improved in the 2.5.0 release. The tests showed decrease in execution time to about 10-20% of the original. This was on arrays with 48 blocks and block group size=1. Now the execution time is more or less the same for all values of the block group size parameter. More information about this at: > Anyway, I came across something that surprised me. When using > DynamicResultIterator I noticed that hasNext() does not behave in the > way I expect. After fetching a row by next() method, invoking hasNext() > causes iterator to move to the next row. I checked that on the following > example (it = iterator): The next() method always returns the same object which is just a "proxy" that fetches data from the underlying ResultSet object. > > command | result > ----------------------- > r = it.next() | - > r.getInt(pos) | 324 > it.hasNext() | true > r.getInt(pos) | 325 > ----------------------- > > I looked into the code to check what is going on. > DynamicResultIterator.hasNext() invokes next() method on > java.sql.ResultSet object to check if there are rows to read, but it > does not move the cursor back(ResultSet.previous()). In addition, next() > method uses hasNext() to move the cursor forward (otherwise executing > if(it.hasNext) r=it.next() would skip one row). No rows are skipped. The hasNext() method only moves to the next row if next() has been called in the meantime. It is safe to call hasNext() any number of times. It will just move to the next row the first time. > > Is this how it is meant to be? Yes. > I realize that the usual way of using > hasNext() is in the context like: > > while(it.hasNext) > r=it.next() > // process r > > where this issue is not a problem, but sometimes it is useful to check > if there are next rows of data without actually moving the cursor. One > could also benefit from previous() method in that DynamicResultIterator > class. > There are probably reasons behind this kind of implementation I am not > aware of. Then mentioning in the docs that hasNext moves the pointer to > next result would be nice. The only way to check if there are more rows in a ResultSet is to try to move to the next row. For performance reasons all queries are executed in read-only, forward-only mode so there is no possibility to move backwards. And since the DynamicResultIterator is an java.util.Iterator, we are only interested in moving forwards anyway. You are correct that the documentation could be clearer about the issue that it is actually the hasNext() method that moves to the next row, not the next() method. The next() method at least states that the returned SqlResult is just a pointer to the current row and if results from previous rows are needed then that information needs to be copied by the calling code. This is in itself also a performance optimization since we never make unnecessary copies in the BASE core. /Nicklas ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ basedb-devel mailing list basedb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/basedb-devel
