Probably not very constructive, but would there be a straightforward way of partitioning your instance into two (as a worst case solution)?
On Wed, Oct 12, 2016 at 4:03 PM, Abdul karim Jaafar < [email protected]> wrote: > Dears, > Actually we have a lot of data sets that need different category > combination (attribute), because we need to use filter in reporting (data > visualizer, pivot table) to multi data elements so reliance on event forms > (programs) can't be useful for project like POLIO, AIRI that we work on, > > and we use it for programs that need tracking like health information > system and community health programs > so we have too much categoryoptioncombo, > > also we have in one program over than 1300 dataelement 500 of them in > multi stages that make this program huge, I faced a problem because of this > (more than 1600 column in PostgreSQL) and I fixed it, but analytics after > that take more time. however analytic period problem happen before that. > > Actually I don't know what to do! > > Regadrs, > Abdul karim, > > -----Original Message----- > From: Dhis2-users [mailto:dhis2-users-bounces+abdulkarim.jaafar87= > [email protected]] On Behalf Of Uwe Wahser > Sent: Wednesday, October 12, 2016 4:06 PM > To: Knut Staring <[email protected]>; Jason Pickering < > [email protected]> > Cc: [email protected] > Subject: Re: [Dhis2-users] long period of analytics > > That's a good point, Jason: a few months ago we tried under a similar > hardware setting to run the system with around 350.000 > categoryOptionCombinations. > Although it was possible to create all of the combinations (keeping the > process alive for 5 days), DHIS2 started to behave funny in a lot of ways > (analytics worked, though). > > Better try to split your categories into categoryOptionCombinations and > attributeOptionCombinations. categories where you might add new options go > into categoryOptionCombinations, categories which will certainly not change > (e.g. > age, gender) go into attributeOptionCombinations. That way you split the > possible combinations into two chunks. I our example we have 52,800 > categoryOptionCombinations and 64,800 attributeOptionCombinations (which > runs without a problem), where we'd theoretically have over 3 billion > combinations if they were combined. > > For our next data-cube we might try using anonymous events for these data, > as Jason suggested, which might get rid of the problem of pre-defind > cat-combinations. > > Regards, Uwe > > > Jason Pickering <[email protected]> hat am 12. Oktober 2016 > > um 15:38 > > geschrieben: > > > > > > Just to confirm, > > > > _Categoryoptioncomboname: 437689 rows > > > > You would seem to have an extraordinary number of category combos. Is > > this correct? > > > > On Wed, Oct 12, 2016, 14:35 Knut Staring <[email protected]> wrote: > > > > > Maybe also test an upgrade? > > > > > > On 12 Oct 2016 14:22, "Uwe Wahser" <[email protected]> wrote: > > > > > > Hi Abdul, > > > > > > I saw that you posted this already a few months ago - without > > > success, as it seems. So let me just take a lucky guess, of what > > > else you could try (that's what I'd try): > > > > > > - in catalina.out, can you identify a specific phase of the > > > analytics run where most of the time is spent or is everything > > > taking longer. Maybe with that detail info the DEV team can help you > > > further. > > > - can you try to list the available database indexes in your > > > installation and compare with a fresh install of DHIS2.22 if any are > > > missing? > > > - can you try to rebuild/repair all the indexes in your installation > > > (never tried this in PostgreSQL myself, though) > > > > > > Don't know if this makes sense, but better than nothing :-) > > > > > > Regards, Uwe > > > > > > > > > > Abdul karim Jaafar <[email protected]> hat am 12. > > > > Oktober > > > 2016 um > > > > 12:22 geschrieben: > > > > > > > > > > > > Dears all, > > > > > > > > > > > > > > > > I’ve been facing a problem with analytic period for over than two > > > > months, > > > > > > > > > > > > > > > > And I didn’t find any solution for this problem. > > > > > > > > > > > > > > > > I did a tuning for Postgres and every week I did maintains from > > > > data administration, also there is no problems in data integrity. > > > > > > > > > > > > > > > > But the analytic period is increasing gradually for over than 20 > > > > hours > > > > > > > > > > > > > > > > I don’t know what are thing that increase the period for analytics > > > > table update. > > > > > > > > > > > > > > > > we have: > > > > > > > > Tracked entity instance: 97358 > > > > > > > > Tracked entity data value: 1872790 > > > > > > > > Tracked entity attribute data value: 1217314 > > > > > > > > Data value: 614956 > > > > > > > > > > > > > > > > Also I have these rows in table below: > > > > > > > > _Categoryoptioncomboname: 437689 rows > > > > > > > > Categoryombos_optioncombos: 669471 rows > > > > > > > > > > > > > > > > We did hosting in our own server kindly look for information below: > > > > > > > > DHIS2, Database and hosting information: > > > > > > > > Version: 2.22 > > > > > > > > Build revision: 22090 > > > > > > > > Environment variable:DHIS2_HOME > > > > > > > > External configuration directory:/home/dhis/config > > > > > > > > File store provider: filesystem > > > > > > > > Database type: PostgreSQL > > > > > > > > Database name: dhis2 > > > > > > > > Database user: dhis > > > > > > > > Java opts: -Xmx16000m -Xms8000m > > > > > > > > Java home: /usr/lib/jvm/java-8-oracle/jre > > > > > > > > Java temporary directory: /home/dhis/tomcat-dhis/temp > > > > > > > > Java version: 1.8.0_101 > > > > > > > > Java vendor: Oracle Corporation > > > > > > > > OS name: Linux > > > > > > > > OS architecture: amd64 > > > > > > > > OS version: 4.4.0-34-generic > > > > > > > > Server memory: Mem Total in JVM: 10237 Free in JVM: 8507 Max > > > > Limit: 14222 > > > > > > > > CPU cores:12 > > > > > > > > Users information: > > > > > > > > There are: 63 users > > > > > > > > User role: 13 > > > > > > > > > > > > > > > > I think this volume data not too much to effects the analytics. > > > > > > > > > > > > > > > > Any solutions please? > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > _______________________________________________ > > > > Mailing list: https://launchpad.net/~dhis2-users > > > > Post to : [email protected] > > > > Unsubscribe : https://launchpad.net/~dhis2-users > > > > More help : https://help.launchpad.net/ListHelp > > > > > > _______________________________________________ > > > Mailing list: https://launchpad.net/~dhis2-users > > > Post to : [email protected] > > > Unsubscribe : https://launchpad.net/~dhis2-users > > > More help : https://help.launchpad.net/ListHelp > > > > > > _______________________________________________ > > > Mailing list: https://launchpad.net/~dhis2-users > > > Post to : [email protected] > > > Unsubscribe : https://launchpad.net/~dhis2-users > > > More help : https://help.launchpad.net/ListHelp > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~dhis2-users > Post to : [email protected] > Unsubscribe : https://launchpad.net/~dhis2-users > More help : https://help.launchpad.net/ListHelp > > -- Knut Staring Dept. of Informatics, University of Oslo Norway: +4791880522 Skype: knutstar http://dhis2.org
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-users Post to : [email protected] Unsubscribe : https://launchpad.net/~dhis2-users More help : https://help.launchpad.net/ListHelp
