you are alright Paul! One of my queries was hogging the DB. I will improve its logic... I will check the Solr api. Considering the performance terms, the SOLR works better than hibernate?
Thanks Paul, you are the best. 2014-08-22 18:11 GMT-03:00 Paul Libbrecht <[email protected]>: > Danilo, > > have you checked the MySQL process list? > I'd suspect something is hogging. > For search, I'd recommend to leverage solr… but with an amount of > customizations. There are some hooks in the solr-plugin, I believe. > > hope it helps. > > paul > > > On 22 août 2014, at 22:54, Danilo Oliveira <[email protected]> > wrote: > > > Hello Devs, > > > > I am developing an application based on XWiki that is mapping, > connecting, > > relating and graphical disposing movie information in order to make > > possible to the user explore their trailers. > > > > At the beginning with a light data set (<5k movies) the application was > > running well, but today I started to populate my database (MYSQL) and the > > application became unusable, the queries is taking more than 5 minutes to > > complete. Actually, it has more than 15k movies (1 movie = 1 doc) and I > > need to upload more 100k. > > > > I already have checked the cache and performance page but I don't know if > > they[1][2] solve my problem: > > I think that is a architecture challenge. > > > > My AS IS process is: > > -User insert a movie, > > -the application search for the movie and their related films based on > its > > characteristics (a lot of joins and other algorithms) (bottleneck) > > -the application returns the results as a map; > > > > I am wondering if I could use the custom mapping[3] to solve my problem > due > > the fact that the relationship information for each movie, in this first > > moment, don't need to change often. Each movie has X movies related, > sorted > > by similarity. So, I could create some relationship algorithm that will > run > > scheduled ( 1 time by week) and populate this new table .I am thinking to > > use dataframe panda of python to talk directlly with mysql and make data > > analysis, any other suggestion? > > > > So I would create a custom map to my relationship movie class, run the > > algorithm, populate the new table, so my TO BE would be: > > > > TO BE > > -user insert movie info; > > -simple select on the customtable "MoviesRelated"; > > -the application returns the results; > > > > I would appreciate some opinion. Thank you very much. > > > > [1]http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances > > [2]http://extensions.xwiki.org/xwiki/bin/view/Extension/Cache+Module > > [3]http://platform.xwiki.org/xwiki/bin/view/DevGuide/CustomMapping > > > > Danilo > > -- > > Danilo Amaral de Oliveira > > Engenheiro de Computação > > celular (32) 9111 - 6867 > > _______________________________________________ > > devs mailing list > > [email protected] > > http://lists.xwiki.org/mailman/listinfo/devs > > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > -- Danilo Amaral de Oliveira Engenheiro de Computação celular (32) 9111 - 6867 _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
