Hey, I got the same problem, of slowness while using MYSQL data model, after a small research and looking into mysql's query log, revealed that user - user recommendation just floods the database with thousands and thousands of requests. and thats on small database. for now Im dumbping the database into file, and using filedata model which works much faster
On Wed, Jul 13, 2011 at 10:03 AM, Sean Owen <[email protected]> wrote: > That's too small to be that slow. There are a bunch of ways this could be > slower than it should. > > The DataSource may not matter. What's important is whether it is actually a > pooling DataSource from the container. You may want to check whether it > seems to be reusing connections. > Table indexes, on user ID, item ID and both together as primary key are > important. > You may want a larger pool if you are sending concurrent requests. > Use a caching wrapper around the UserSimilarity. > > But yes loading into memory is going to be much better for you. Use > ReloadFromJDBCDataModel. > > On Wed, Jul 13, 2011 at 6:09 AM, Salil Apte <[email protected]> wrote: > > > Oh yea, at runtime, I'm getting back a BasicDataSource object for my > > DataSource. Is that correct? > > > > On Tue, Jul 12, 2011 at 9:59 PM, Salil Apte <[email protected]> > wrote: > > > So I started actually looking at performance today and it is pretty > > > horrendous. I've got about 61,000 rows in my database which I'm > > > assuming isn't *that* many rows. But recommendations are taking > 20 > > > seconds. Is there some way to ensure pooling is turned on? What else > > > is a big driver for performance? My tables are setup so that I have a > > > multiple index (for uniqueness) for <user_id, item_id> pairs. That > > > way, there cannot be two entries with the same <user_id, item_id>. I'm > > > not sure where to go from here. > > > > > > Thanks for the help! > > > > > > On Tue, Jul 12, 2011 at 12:47 AM, Sean Owen <[email protected]> wrote: > > >> You can ignore it. It just doesn't know for sure you have a pool. > > >> I believe I have even removed this in a recent refactoring. > > >> > > >> On Tue, Jul 12, 2011 at 2:21 AM, Salil Apte <[email protected]> > > wrote: > > >> > > >>> So I keep getting this warning from either Mahout or the server (I'm > > >>> guessing the former): > > >>> > > >>> WARNING: You are not using ConnectionPoolDataSource. Make sure your > > >>> DataSource pools connections to the database itself, or database > > >>> performance will be severely reduced. > > >>> > > >>> I'm not really sure why this is happening. I have the following > > >>> resource in my webapp's context.xml file. Is there anything else I > > >>> need to do enable connection pooling with a JNDI resource? > > >>> > > >>> <Resource name="jdbc/offline-local" auth="Container" > > >>> type="javax.sql.DataSource" username="root" password="" > > >>> driverClassName="com.mysql.jdbc.Driver" > > >>> > > >>> > > > url="jdbc:mysql://localhost:3306/offlinedevel?autoReconnect=true&cachePreparedStatements=true&cachePrepStmts=true&cacheResultSetMetadata=true&alwaysSendSetIsolation=false&elideSetAutoCommits=true" > > >>> validationQuery="select 1" maxActive="16" maxIdle="4" > > >>> removeAbandoned="true" logAbandoned="true" /> > > >>> > > >>> Thanks in advance. > > >>> > > >>> -Salil > > >>> > > >> > > > > > >
