A really bad N+1 solution
For your enjoyment here is how I solved the N+1 problem before I turned to iBatis. I'm know that I'm opening myself to major code ridicule.
Mark
/** * Translates a List of maps into a normailzed structure. Choose any number * of column names and pass them in as an array. The result will be an object * graph grouped in the same order as the order of the column names passed in.
* Each parent is a map with an element called child that holds a List of * Maps which in turn have their own child list. An added feature is that the * root holds the sum of its children on each level in a key called
* 'numChildLevels*'. Just substitute an integer for the *. * * This is most often used for reports. This allow you to get all the data * with one call and then create the structure you need. This saves
* potentially hundreds of database calls while recursing. */ public static List normalizeListOfMaps(List dataList, String[] colHeaders){ List focus = null; List lastRowVals = new ArrayList(
colHeaders.length); List childList = null; Map currRow = null; Map lastRow = null; Map rootMap = null; int n = 0; String key = null; String curr = null; String last = null;
// The focus holds a reference to the active parent at each level. List[] focusLevel = new List[colHeaders.length]; focusLevel[0] = new ArrayList(); Iterator i = dataList.iterator();
while (i.hasNext()) { currRow= (Map) i.next();
for (n = 0; n colHeaders.length; n++) { key = (String) colHeaders[n];curr = (String) currRow.get(key); if (lastRow != null) last = (String)
lastRow.get(key);
if (last == null || !curr.equals(last)) {
if (n == 0) { rootMap = currRow; } focus = focusLevel[n]; focus.add(currRow); // Keep a sum of all the levels in each root record. Integer sum = (Integer)
rootMap.get(numChildLevels + (n));if (sum != null)rootMap.put(numChildLevels + (n), new Integer(sum.intValue() + 1));elserootMap.put(numChildLevels + (n), new Integer(1));
// When not a leaf node. if (n colHeaders.length - 1) { childList = new ArrayList(); focusLevel[n+1] = childList; currRow.put(child,childList); // Create a copy so it can have its own child pointer.
currRow = new HashMap(currRow); } } } lastRow = currRow; } return focusLevel[0]; }
Re: help on com.ibatis.common.beans.ProbeException
I didn't test it myself but could it be that your parameter map says int but the actual type is Integer?
Mark
On 5/17/05, Binh Tran [EMAIL PROTECTED] wrote:
Hi all,When I tried to execute a procedure to return type NUMBER, I got this errormessage.
[java] Caused by: com.ibatis.common.beans.ProbeException: Could not setproperty 'userId' for com.cellmania.cingular.user.UserParams.Cause:java.lang.IllegalArgumentException: argument type mismatch[java] Caused by:
java.lang.IllegalArgumentException: argument typemismatch[java] atcom.ibatis.common.beans.JavaBeanProbe.setProperty(JavaBeanProbe.java:309)[java] atcom.ibatis.common.beans.JavaBeanProbe.setObject
(JavaBeanProbe.java:229)[java] atcom.ibatis.common.beans.GenericProbe.setObject(GenericProbe.java:73)[java] atcom.ibatis.sqlmap.engine.exchange.ComplexDataExchange.setData(ComplexDataExc
hange.java:109)[java] atcom.ibatis.sqlmap.engine.mapping.parameter.BasicParameterMap.refreshParameterObjectValues(BasicParameterMap.java:140)My XML block istypeAlias alias=UserParams2
type=com.cellmania.cingular.user.UserParams /parameterMap id=registerParams class=userParams2parameter property=userId jdbcType=NUMERIC javaType=int
mode=OUT /parameter property=userName jdbcType=VARCHAR /parameter property=password jdbcType=VARCHAR nullValue= /
parameter property=jobTitle jdbcType=VARCHAR nullValue= /parameter property=level jdbcType=VARCHAR nullValue= /parameter property=firstName jdbcType=VARCHAR nullValue= /
parameter property=lastName jdbcType=VARCHAR nullValue= /parameter property=taxId jdbcType=VARCHAR nullValue= /parameter property=email /
parameter property=workPhone jdbcType=VARCHAR nullValue=/parameter property=fax jdbcType=VARCHAR nullValue=/parameter property=address1 jdbcType=VARCHAR nullValue=/
parameter property=address2 jdbcType=VARCHAR nullValue=/parameter property=city jdbcType=VARCHAR nullValue=/parameter property=state jdbcType=VARCHAR nullValue=/
parameter property=zip jdbcType=VARCHAR nullValue=/parameter property=country jdbcType=VARCHAR nullValue=/parameter property=companyName jdbcType=VARCHAR nullValue= /
parameter property=companyUrl jdbcType=VARCHAR nullValue= //parameterMapprocedure id=register parameterMap=registerParams
cacheModel=UserCache { ? = call cingular.register( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ? ) }/procedureAnd my UserParams block is private IntegeruserId= null;
public void setUserId(Integer userId) { this.userId = userId; } public Integer getUserId() { return userId; }Please help.Binh.
Re: Caching - The same cacheModel with similar queries
Woudn't it be better to use the isNotNull tag to wrap your where statement? The cache should keep each queries resultsseperate since null will hash differently than any value passed in. Mark On 4/18/05, Guido García Bernardo [EMAIL PROTECTED] wrote: Hi,I've two questions regarding with caching.Is caching useful when using two similar queries (the second one being the first one plus a where clause) ? statement id=getAreas resultMap=area_resultcacheModel=areas-cache select * from AREAS /statement statement id=getArea resultMap=area_resultcacheModel=areas-cache select * from AREAS where codigo = #value# /statement cacheModel id=areas-cache readOnly=true type=MEMORY flushInterval hours=24 / property name=reference-type value=WEAK / /cacheModelIs there any problem when using WEAK references? (http://www.mail-archive.com/ibatis-user-java@incubator.apache.org/msg01190.html)Thank you.Guido García Bernardo.
RE: ibatis technology need company references.
I know you're looking for product references but to give you more ammo you should make a hard copy of the Data Mapper Development Guide and bring that to your meetings as well. It is an impressive document. Mark -Original Message- From: Victor Stepanov [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 06, 2005 11:56 AM To: ibatis-user-java@incubator.apache.org Subject: ibatis technology need company references. Hi All. I need to provide references about ibatis and sqlmaps that this product is stable can anybody give some references or where I can found it. Thanks.
Re: Please Help in arguing for iBatis SqlMaps
Wow, I think this is going to be a fun thread. On Apr 7, 2005 1:55 PM, Kaushik Ashodiya [EMAIL PROTECTED] wrote: Hi, I have hard time arguing for iBatis SqlMaps (not because something wrong with SqlMaps). Please give me more reasons so that I can convince them. Their argument: 1. For small project SqlMap is adding unnecessary complexity. 2. Why learn one more tool when you have a simple base data access object that gives you connection, prepared statement and a result and does cleanup? (only when extended class calls cleanup() !!!) 3. Generally open source projects dies shortly and does not have support. 4. Adding many jars of those open source projects make out project more complex and un-maintainable. 5. What if SqlMaps goes out of market? It is fairly new and not hardened. my-peers-arguments Again to re-iterate a point I was trying to make yesterday, architecturally speaking you should always try and keep it simple (not on an individual application level) but as far as possible on an enterprise level. I'm going to ask you the same question that I asked yesterday...what do you think are the advantages/disadvantages of O/R mappings for relatively small applications where persistance is not the deciding criteria (or for that matter not even a criteria)?(I've attached yesterdays email thread for Kevin since he didn't get that one :)) I'll list the disadvantages: 1) You add multiple third party jars (open source or otherwise), which you need to be aware of when building new apps. These include components which the vendor will eventually provide if the market demand is high... 2)You add multiple configurations for each of those products, which you need to understand on how to tune in case there are performance issues. 3) Most of these components seem to be build on the current versions of jdk which take a decent time to hit mainstream commercial product support. 4) If these products are not built to a spec (even though they may end up driving a spec), the migration process becomes an added overhead. Even when they do become a part of the spec, the implementation differences of the vendor of choice may be so different that you pretty much waste any advantage of early adoption. If you are a vendor of a product, an effective early adoption can translate into you either being bought out by the bigger established vendor or you getting to be a key player in that spacebut we don't fall in that category. 5)You need to be able to train existing resources to use the product (unless there is a plan to let go people and get new fresh resources who may have had experience with the adopted product). Now if the existing resources themselves have a discomfort moving over to an object oriented paradigm, putting up them up against the challenges of going thru the learning curve of extra products further lowers the probability of success. 6) You increase the possibility of chaos and reduce flexibility of resource utilization. Besides the six points above, I would not recommend use of products like iBatis in applications at UCOP is because in my opinion the applications that we do or have done are relatively small and O/R mapping for those kinds of apps is probably an overkill (again personal opinion.) To some level Hibernate is a valid candidate for evaluation (taking into account that it is driving the EJB spec). But again, my business acumen says that IBM will definitely take care of that. Now my perpective looking at the example below, one of the key shortcommings that I see is it couples the Display/View with the model (Data Access) Basically what this means is that if you wanted to create the same object tomorrow from not a non database/jndi source, you would be re-writing the logic for creating the same object. Another disadantage that I mentioned yesterday was if your display object gets created from multiple datasources (one db2, one sybase) transaction management becomes an issue How would you use iBatis in a system which does not need persistanc but needs transaction management? Isn't the whole idea of going with some big commercial vendor like IBM to consolidate your infrastructure source so that you dont knock on different doors? Often such a vendor is selected based on how many services required by a business are provided by the vendor. Nothing against Hibernate or iBatis. I'm sure they are great at what they are designed to do. But each one of them doesn;t do all that we probably need (they still need an underlying EJB container.) If thats the case, why would you want to introduce the additional learning curve and the additional layer associated with each product? Do you truely think that it's a necessity? Short of long: I would recommend usage of vendor supported spec based ejb conatiner for applications needing persistance or transaction management. Eventually these will be
RE: Can DAO calls be in a constructor...
Yah. Anad Fida was asking before about how to use queryForMap with column
parameters in a Complex Property. Branon Goodin told him that it wasn't
possible. His suggestion was to query your accounts first and then iterate
through the list using the queryForMap. However in my case if my first
Complex Property result pulls back a tree I thought that it would be a pain
traverse it to decorate it with the maps that I wish Complex Property could
have done it the first place. I don't disagree with 1), I just didn't like
any of the other workarounds I came up with.
Mark
-Original Message-
From: Clinton Begin [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 22, 2005 9:09 PM
To: ibatis-user-java@incubator.apache.org
Subject: Re: Can DAO calls be in a constructor...
Two things:
1) Thats' an absolutely horrible design. Domain classes should not
call or depend on the DAO layer in any way.
2) You may have hit a limitation of the DB2 driver, possibly multiple
statements may not be open at the same time on a single connection.
No matter what, you should probably rethink that design.
Cheers,
Clinton
On Tue, 22 Feb 2005 16:49:31 -0500, Mark Bennett
[EMAIL PROTECTED] wrote:
Can DAO calls be in a constructor if it is an object created by a Complex
Property?
First I know this is risky because you might encounter recursion. But if
no
cycle is created, then is this possible at all?
resultMap id=juvenileResult class=juvenile
result property=someBean.myBean column=MYCOLUMN/
/resultMap
class MyBean {
private static ProfileService profileService =
ProfileService.getInstance();
public MyBean() {
// *** This will cause another iBatis call while other is
still in
progress ***
List list = profileService.getListOfSuff(1);
}
}
Mark
Feb 22, 2005 4:36:41 PM EST Error HTTP
[WebAppServletContext(5788681,NCJOIN,/NCJOIN)] Servlet failed with
Exception
com.ibatis.dao.client.DaoException: Failed to execute queryForObject - id
[getJuvenile], parameterObject [20210]. Cause:
com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in ncjoin/persistence/sqlmapdao/sql/Profile.xml.
--- The error occurred while applying a result map.
--- Check the Profile.juvenileResult.
--- The error happened while setting a property on the result object.
--- Cause: COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver] CLI0125E
Function sequence error. SQLSTATE=HY010
Caused by: COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver] CLI0125E
Function sequence error. SQLSTATE=HY010
Caused by: com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in ncjoin/persistence/sqlmapdao/sql/Profile.xml.
--- The error occurred while applying a result map.
--- Check the Profile.juvenileResult.
--- The error happened while setting a property on the result object.
--- Cause: COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver] CLI0125E
Function sequence error. SQLSTATE=HY010
Caused by: COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver] CLI0125E
Function sequence error. SQLSTATE=HY010
at
com.ibatis.dao.client.template.SqlMapDaoTemplate.queryForObject(SqlMapDaoTem
plate.java:164)
at
ncjoin.persistence.sqlmapdao.ProfileSqlMapDao.getJuvenile(ProfileSqlMapDao.j
ava:22)
at java.lang.reflect.Method.invoke(Native Method)
at com.ibatis.dao.engine.impl.DaoProxy.invoke(DaoProxy.java:72)
at $Proxy141.getJuvenile(Unknown Source)
at
ncjoin.service.ProfileService.getJuvenile(ProfileService.java:36)
at
ncjoin.main.servlet.RequestProcessor.processActionPerform(RequestProcessor.j
ava:54)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
at
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
:263)
at
weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java
:200)
at
weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletCo
ntext.java:2390)
at
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java
:1959)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:137)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:120)
Caused by:
com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in ncjoin/persistence/sqlmapdao/sql/Profile.xml.
--- The error occurred while applying a result map.
--- Check the Profile.juvenileResult.
--- The error happened while setting a property on the result object.
--- Cause: COM.ibm.db2.jdbc.DB2Exception: [IBM][CLI Driver
Re: cacheModel still hitting database
I'll have to check that (I had to run home), thanks. -- Original Message -- Reply-To: ibatis-user-java@incubator.apache.org Date: Mon, 21 Feb 2005 22:32:46 + (GMT) From: Kris A. Jenkins [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] Subject: Re: cacheModel still hitting database To: ibatis-user-java@incubator.apache.org Mark, In your sql-map-config.xml, look at the 'settings' attribute. Do you have 'cacheModelsEnabled=true' set? Kris --- Mark Bennett [EMAIL PROTECTED] wrote: I'm trying to get the cacheModel working, but it keeps hitting the database. This is just an example so please ignore the context. I'm using iBatis-2 and this setup returns the data properly. I get no errors. This is run through a web container and not a unit test so the cache should persist. The query object is an Integer so it should hash the same each time. However, whenever I cange the data in the databse, the change shows up on the page. Any advice? Thanks. Mark cacheModel id=drugScreenCache type=LRU readOnly=true serialize=false flushInterval hours=12/ property name=cache-size value=500/ /cacheModel resultMap id=drugScreenListResult class=drugScreen result property=id column=ACTID/ result property=testDate column=TESTDATE/ /resultMap select id=getDrugScreenListByJuvenile resultMap=drugScreenListResult cacheModel=drugScreenCache SELECT * FROM drugscreen WHERE rsrcid = #value# ORDER by TESTDATE DESC /select ___ ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com
RE: cacheModel still hitting database
Yup, that was it thanks. Hey Steven, this may work for you too. Mark -Original Message- From: Kris A. Jenkins [mailto:[EMAIL PROTECTED] Sent: Monday, February 21, 2005 5:33 PM To: ibatis-user-java@incubator.apache.org Subject: Re: cacheModel still hitting database Mark, In your sql-map-config.xml, look at the 'settings' attribute. Do you have 'cacheModelsEnabled=true' set? Kris --- Mark Bennett [EMAIL PROTECTED] wrote: I'm trying to get the cacheModel working, but it keeps hitting the database. This is just an example so please ignore the context. I'm using iBatis-2 and this setup returns the data properly. I get no errors. This is run through a web container and not a unit test so the cache should persist. The query object is an Integer so it should hash the same each time. However, whenever I cange the data in the databse, the change shows up on the page. Any advice? Thanks. Mark cacheModel id=drugScreenCache type=LRU readOnly=true serialize=false flushInterval hours=12/ property name=cache-size value=500/ /cacheModel resultMap id=drugScreenListResult class=drugScreen result property=id column=ACTID/ result property=testDate column=TESTDATE/ /resultMap select id=getDrugScreenListByJuvenile resultMap=drugScreenListResult cacheModel=drugScreenCache SELECT * FROM drugscreen WHERE rsrcid = #value# ORDER by TESTDATE DESC /select ___ ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com
