[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18117: importer fix for periodTypes, reload and attach earlier in the newObject process
revno: 18117
committer: Morten Olav Hansen morte...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 23:35:50 +0700
message:
importer fix for periodTypes, reload and attach earlier in the newObject
process
modified:
dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/DefaultObjectBridge.java
dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/importers/DefaultIdentifiableObjectImporter.java
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/DefaultObjectBridge.java'
--- dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/DefaultObjectBridge.java 2015-01-19 09:19:46 +
+++ dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/DefaultObjectBridge.java 2015-01-23 16:35:50 +
@@ -257,6 +257,7 @@
{
for ( PeriodType periodType : periodStore.getAllPeriodTypes() )
{
+periodType = periodStore.reloadPeriodType( periodType );
periodTypes.add( periodType );
periodTypeMap.put( periodType.getName(), periodType );
}
=== modified file 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/importers/DefaultIdentifiableObjectImporter.java'
--- dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/importers/DefaultIdentifiableObjectImporter.java 2015-01-19 09:19:46 +
+++ dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/metadata/importers/DefaultIdentifiableObjectImporter.java 2015-01-23 16:35:50 +
@@ -337,9 +337,9 @@
log.debug( Trying to save new object = + ImportUtils.getDisplayName( object ) + ( + object.getClass().getSimpleName() + ) +
);
+updatePeriodTypes( object );
objectBridge.saveObject( object, !options.isSharing() );
-updatePeriodTypes( object );
reattachCollectionFields( object, collectionFields, user );
objectBridge.updateObject( object );
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] [Dhis2-users] Date category combination
Hi Robin, you spotted a bug. It's been fixed now in trunk and 2.18. Get it in 45 min. Lars On Fri, Jan 23, 2015 at 3:55 PM, Robin Martens mart...@sher.be wrote: Hi all, I want to delete my analytics tables and start all over. The (only?) way to do this is through the Data Admin - Maintenance app but when clearing the tables nothing happens. All tables still exist in Postgresql and the data remains available in the Pivot tables. We're still in 2.17. Is this a known issue or is there a workaround? Sorry, couldn't test on the demo server as it seems to take forever to try to delete the analytics tables there. Regards, Robin -- *De :* Dhis2-users [dhis2-users-bounces+martens= sher...@lists.launchpad.net] de la part de Guy Ekani [constyek...@yahoo.fr ] *Envoyé :* vendredi 23 janvier 2015 13:10 *À :* Lars Helge Øverland; Enmanuel Toribio *Cc :* dhis2-us...@lists.launchpad.net; dhis2-devs@lists.launchpad.net *Objet :* Re: [Dhis2-users] Date category combination Thank you Lars, I will try it ! But for next features, I think it will be useful to assign a Value Type (number, date, Text, Yes/no, etc...) to a category . Sincerely, = EKANI Guy Le Jeudi 22 janvier 2015 22h47, Lars Helge Øverland larshe...@gmail.com a écrit : Hi Guy, currently the way it works is that a data element can only have one category combination. This means that all fields in the form for that data elements will have the same data type. The best way to achieve the form layout above is to have 2 data elements: - One with numeric value type, and a category with three age groups. - One with date type and default category. Then put these fields into a custom data entry form. regards, Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Fwd: [Bug 1402244] [NEW] event report/visulizer: option set sort order not respocted
Olav, how/where can I reproduce it? This is what I tried (demo): - opened both ER and EV, selected Inpatient.. program, then Mode of discharge - selected a custom order, clicked update, but it worked as expected The demo is now 2.18, but nothing has been changed lately in this regard. Greg, seems adding asc=OUNAME does not have any effect ( https://www.dhis2.org/doc/snapshot/en/developer/html/ch01s22.html) Should be fixed, but please write a new bug report as it's an unrelated issue. On Thu, Jan 22, 2015 at 3:21 PM, Greg Rowles greg.row...@gmail.com wrote: Hi OIav/Devs Is this bug related to my observation: I cannot get event (aggregate) analytics to sort my data by column [OU] alphabetically ASC. Right now data is displayed with no apparent display order... Regards, Greg -- Forwarded message -- From: Olav P 1402...@bugs.launchpad.net Date: Sat, Dec 13, 2014 at 7:41 PM Subject: [Dhis2-devs] [Bug 1402244] [NEW] event report/visulizer: option set sort order not respocted To: dhis2-devs@lists.launchpad.net Public bug reported: Event report/visulizer apps do not respect the sort order that is set for option sets. 2.17. ** Affects: dhis2 Importance: Undecided Status: New -- You received this bug notification because you are a member of DHIS 2 developers, which is subscribed to DHIS. https://bugs.launchpad.net/bugs/1402244 Title: event report/visulizer: option set sort order not respocted Status in DHIS 2: New Bug description: Event report/visulizer apps do not respect the sort order that is set for option sets. 2.17. To manage notifications about this bug go to: https://bugs.launchpad.net/dhis2/+bug/1402244/+subscriptions ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp -- *Health Information Systems Program - South Africa* *- - - - - - - **- - - - - - - **- - - - - - - **- - - - - - - **- - - - - * Mobile :073 246 2992 Landline: 021 554 3130 Fax: 086 733 8432 Skype: gregory_rowles ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18118: Analytics tables. Fixed clean tables function.
revno: 18118
committer: Lars Helge Overland larshe...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 18:23:01 +0100
message:
Analytics tables. Fixed clean tables function.
modified:
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/AnalyticsTableManager.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/AbstractJdbcTableManager.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/DefaultAnalyticsTableService.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/JdbcCompletenessTargetTableManager.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/JdbcEventAnalyticsTableManager.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/JdbcOrgUnitTargetTableManager.java
dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/PartitionUtils.java
dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/ListUtils.java
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataadmin/src/main/java/org/hisp/dhis/dataadmin/action/maintenance/PerformMaintenanceAction.java
dhis-2/dhis-web/dhis-web-maintenance/dhis-web-maintenance-dataadmin/src/main/resources/META-INF/dhis/beans.xml
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/AnalyticsTableManager.java'
--- dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/AnalyticsTableManager.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/AnalyticsTableManager.java 2015-01-23 17:23:01 +
@@ -47,7 +47,7 @@
public static final String EVENT_ANALYTICS_TABLE_NAME = analytics_event;
/**
- * Returns analytics tables which yearly partitions.
+ * Returns analytics tables which are yearly partitions.
*
* @param earliest the start date for the first year to generate table partitions.
* @param latest the end date for the last year to generate table partitions.
@@ -55,6 +55,11 @@
ListAnalyticsTable getTables( Date earliest );
/**
+ * Returns all potential analytics tables which are yearly partitions.
+ */
+ListAnalyticsTable getAllTables();
+
+/**
* Checks if the database content is in valid state for analytics table generation.
* Returns null if valid, a descriptive string if invalid.
*/
=== modified file 'dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/AbstractJdbcTableManager.java'
--- dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/AbstractJdbcTableManager.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/AbstractJdbcTableManager.java 2015-01-23 17:23:01 +
@@ -123,10 +123,20 @@
{
log.info( Get tables using earliest: + earliest );
+return getTables( getDataYears( earliest ) );
+}
+
+@Override
+@Transactional
+public ListAnalyticsTable getAllTables()
+{
+return getTables( ListUtils.getClosedOpenList( 1500, 2100 ) );
+}
+
+private ListAnalyticsTable getTables( ListInteger dataYears )
+{
ListAnalyticsTable tables = new ArrayList();
-ListInteger dataYears = getDataYears( earliest );
-
Collections.sort( dataYears );
String baseName = getTableName();
=== modified file 'dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/DefaultAnalyticsTableService.java'
--- dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/DefaultAnalyticsTableService.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/DefaultAnalyticsTableService.java 2015-01-23 17:23:01 +
@@ -157,7 +157,7 @@
@Override
public void dropTables()
{
-ListAnalyticsTable tables = tableManager.getTables( null );
+ListAnalyticsTable tables = tableManager.getAllTables();
for ( AnalyticsTable table : tables )
{
=== modified file 'dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/JdbcCompletenessTargetTableManager.java'
--- dhis-2/dhis-services/dhis-service-analytics/src/main/java/org/hisp/dhis/analytics/table/JdbcCompletenessTargetTableManager.java
[Dhis2-devs] Error Generating Event Reports
Dear All, I have an instance of DHIS2 version 2.16, which is primarily used for tracker related data on Single event with registration. The issue is after generating analytic tables and view results on Event Reports I receive No data found on both aggregated data as well as individual records. I tested it on 2.17 and 2.18, but same error persists. I checked the catalina.out and it puts out several errors related to uidlevels. Same appear for uidlevel3. In analytic table only uidlevel1 is there, which is also empty. Following is the exact error. column uidlevel2 does not exist The relevant portion of catalina.out is attached herewith. I'd be grateful if you could help me fix this. Thank you -- Pamod * INFO 2015-01-23 22:55:25,382 Query failed, likely because the requested analytics table does not exis t (JdbcEventAnalyticsManager.java [http-bio-8080-exec-4]) org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [select psi, ps, exe cutiondate, longitude, latitude, ouname, oucode, ou,Qa4oueS34QK from analytics_event_2014_ujjobvvaap l where executiondate = '2014-01-01' and executiondate = '2014-12-31' and (uidlevel2 = 'vxQB0NMggC2' )and ps = 'gknkafmeM9y' and Qa4oueS34QK = '1' limit 100 offset 0]; nested exception is org.postgresql .util.PSQLException: ERROR: column uidlevel2 does not exist Position: 201 at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExcept ionTranslator.java:99) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFal lbackSQLExceptionTranslator.java:73) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFal lbackSQLExceptionTranslator.java:81) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFal lbackSQLExceptionTranslator.java:81) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:416) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:471) at org.springframework.jdbc.core.JdbcTemplate.queryForRowSet(JdbcTemplate.java:526) at org.hisp.dhis.analytics.event.data.JdbcEventAnalyticsManager.getEvents(JdbcEventAnalyticsMana ger.java:250) at org.hisp.dhis.analytics.event.data.JdbcEventAnalyticsManager.getEvents(JdbcEventAnalyticsMana ger.java:236) at org.hisp.dhis.analytics.event.data.DefaultEventAnalyticsService.getEvents(DefaultEventAnalyti csService.java:305) at org.hisp.dhis.webapi.controller.EventAnalyticsController.getQueryJson(EventAnalyticsControlle r.java:233) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod .java:221) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandl erMethod.java:137) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAnd Handle(ServletInvocableHandlerMethod.java:110) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHand leMethod(RequestMappingHandlerAdapter.java:777) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInte rnal(RequestMappingHandlerAdapter.java:706) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandle rMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857) at javax.servlet.http.HttpServlet.service(HttpServlet.java:620) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 303) at
Re: [Dhis2-devs] Problems with detailed metadata export..
Hi Bob, I think you are right. I've seen this before, and reducing the number of calls to one usually helps a lot. The problem is that getCurrentUser() is transactional and makes a (cached) db query. The overhead of a huge number of transaction causes things to slow down. If you or someone could write a patch/fix it would be great. Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Problems with detailed metadata export..
Hi Bob In this system, there are only 277 users, but about 18000 data elements. This problem surfaced after the upgrade to 2.17 and after the injection of about 14,000 new data elements into the system. Before, it was fine. Hope it can be solved quickly, as even the loading of data elements in the analysis tools is excruciatingly slow. Best regards, Jason On Fri, Jan 23, 2015 at 1:10 PM, Bob Jolliffe bobjolli...@gmail.com wrote: Hi Jason Just spent a little time looking further at this. I think the stack dump has the answer after all. I believe the inefficiency is coming from the fact that for each object access there is a call to getAccess( T object ) in DefaultAclService. The trouble is this in turn is calling currentUserService.getCurrentUser() each time. You wouldn't think this is such a load but each time I look at a snapshot of the thread running it is busy in that method so its obviously significant. I guess it is doing a string comparison search each time (userService.getUserCredentialsByUsername( username )), so with sufficient number of users in the system its accounting for the bulk of the time - a lot of users multiplied by a lot of dataelements and its going to be more problematic. Do you have a lot of users? The solution has got to be to call currentUserService.getCurrentUser() just once, and then use the getAccess( T object, User user ) method instead. I'm guessing that will reduce the computation load for returning a list of dataelements hugely. The trouble is I am not 100% of where that getAccess call is coming from in this object sharing code. Morten will know better but I guess everything starts in the AbstractCrudController. Given that currentUserService.getCurrentUser() is not such a lightweight operation we clearly need to be much more conservative in how often we use it. Currently it is peppered all ove the place. Can't we do this just once per session? And from there on make sure to use getAccess( T object, User user ) ? I'd bet a pint of best that this would reduce the time for that call to list dataelements by 90%. Bob On 22 January 2015 at 10:58, Bob Jolliffe bobjolli...@gmail.com wrote: Sorry Jason. A quick look through isnt shedding light on me :-( Is this a test environment or did you run that against a running server with other users? There doesn't seem to be any tight loop like we saw with the paging bug, so I guess what we have here is not so much as a logical error as some sort of gross inefficiency somewhere. One thing I notice is that almost all RUNNABLE threads are somehow engaged with operations relating to user service. getCurrentUser, getUserCredentials etc. Given that this is a static snapshot in time of where exactly each thread is at, it seems odd that so many are caught up chasing users instead of doing real work. This might be a red herring or a pointer to that gross inefficiency - it might be that if the user credentials are being retrieved and tested each time the user is touching a metadata object (and it it is maybe touching many of them with this metadata export) that that is leading to excessive activity. Do you have a vast number of users in the system by any chance? On 22 January 2015 at 08:09, Jason Pickering jason.p.picker...@gmail.com wrote: OK, is this better? Stack attached. Tasks: 178 total, 1 running, 177 sleeping, 0 stopped, 0 zombie Cpu(s): 99.1%us, 0.1%sy, 0.0%ni, 0.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st Mem: 16433100k total, 16330904k used, 102196k free,71828k buffers Swap: 262140k total,25108k used, 237032k free, 11411252k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 28091 dhis 20 0 12.0g 3.6g 5916 S 796 23.0 328:48.52 java Command used was curl --verbose http:///dhis/api/dataElements.json?links=falsepaging=false; -u : Regards, Jason On Wed, Jan 21, 2015 at 11:05 PM, Bob Jolliffe bobjolli...@gmail.com wrote: Jason the cpu is not really through the roof here. Can you try another dump (while the cpu is through the roof)? The interesting threads to look at are the http-bio-8080 threads, which are the tomcat worker threads. If the server is not being accessed by dozens of users most of them should be sitting in a WAITING state. So then it is interesting to look at those which are marked as RUNNABLE. These are the ones which are trying as hard as they can to get the attention of the scheduler in order to chew cpu cycles. So if your process has become cpu bound then sometimes its possible to figure out which thread it is and what it is trying to do. This attached stack dump doesn't reveal much unfortunately, but then again the cpu (from top) is showing 62% idle. So try and get the cpu burning up again and grab 2 or 3 jstack samples. If it is cpu bound then it might show. On 21 January 2015 at 19:28, Jason Pickering jason.p.picker...@gmail.com
Re: [Dhis2-devs] Problems with detailed metadata export..
Hi Jason Just spent a little time looking further at this. I think the stack dump has the answer after all. I believe the inefficiency is coming from the fact that for each object access there is a call to getAccess( T object ) in DefaultAclService. The trouble is this in turn is calling currentUserService.getCurrentUser() each time. You wouldn't think this is such a load but each time I look at a snapshot of the thread running it is busy in that method so its obviously significant. I guess it is doing a string comparison search each time (userService.getUserCredentialsByUsername( username )), so with sufficient number of users in the system its accounting for the bulk of the time - a lot of users multiplied by a lot of dataelements and its going to be more problematic. Do you have a lot of users? The solution has got to be to call currentUserService.getCurrentUser() just once, and then use the getAccess( T object, User user ) method instead. I'm guessing that will reduce the computation load for returning a list of dataelements hugely. The trouble is I am not 100% of where that getAccess call is coming from in this object sharing code. Morten will know better but I guess everything starts in the AbstractCrudController. Given that currentUserService.getCurrentUser() is not such a lightweight operation we clearly need to be much more conservative in how often we use it. Currently it is peppered all ove the place. Can't we do this just once per session? And from there on make sure to use getAccess( T object, User user ) ? I'd bet a pint of best that this would reduce the time for that call to list dataelements by 90%. Bob On 22 January 2015 at 10:58, Bob Jolliffe bobjolli...@gmail.com wrote: Sorry Jason. A quick look through isnt shedding light on me :-( Is this a test environment or did you run that against a running server with other users? There doesn't seem to be any tight loop like we saw with the paging bug, so I guess what we have here is not so much as a logical error as some sort of gross inefficiency somewhere. One thing I notice is that almost all RUNNABLE threads are somehow engaged with operations relating to user service. getCurrentUser, getUserCredentials etc. Given that this is a static snapshot in time of where exactly each thread is at, it seems odd that so many are caught up chasing users instead of doing real work. This might be a red herring or a pointer to that gross inefficiency - it might be that if the user credentials are being retrieved and tested each time the user is touching a metadata object (and it it is maybe touching many of them with this metadata export) that that is leading to excessive activity. Do you have a vast number of users in the system by any chance? On 22 January 2015 at 08:09, Jason Pickering jason.p.picker...@gmail.com wrote: OK, is this better? Stack attached. Tasks: 178 total, 1 running, 177 sleeping, 0 stopped, 0 zombie Cpu(s): 99.1%us, 0.1%sy, 0.0%ni, 0.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st Mem: 16433100k total, 16330904k used, 102196k free,71828k buffers Swap: 262140k total,25108k used, 237032k free, 11411252k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 28091 dhis 20 0 12.0g 3.6g 5916 S 796 23.0 328:48.52 java Command used was curl --verbose http:///dhis/api/dataElements.json?links=falsepaging=false; -u : Regards, Jason On Wed, Jan 21, 2015 at 11:05 PM, Bob Jolliffe bobjolli...@gmail.com wrote: Jason the cpu is not really through the roof here. Can you try another dump (while the cpu is through the roof)? The interesting threads to look at are the http-bio-8080 threads, which are the tomcat worker threads. If the server is not being accessed by dozens of users most of them should be sitting in a WAITING state. So then it is interesting to look at those which are marked as RUNNABLE. These are the ones which are trying as hard as they can to get the attention of the scheduler in order to chew cpu cycles. So if your process has become cpu bound then sometimes its possible to figure out which thread it is and what it is trying to do. This attached stack dump doesn't reveal much unfortunately, but then again the cpu (from top) is showing 62% idle. So try and get the cpu burning up again and grab 2 or 3 jstack samples. If it is cpu bound then it might show. On 21 January 2015 at 19:28, Jason Pickering jason.p.picker...@gmail.com wrote: Not really sure what is going on, but here is the stack dump. Maybe it will help. And some more info below. top - 21:26:40 up 477 days, 16:20, 2 users, load average: 2.99, 2.05, 1.03 Tasks: 171 total, 1 running, 170 sleeping, 0 stopped, 0 zombie Cpu(s): 38.3%us, 0.0%sy, 0.0%ni, 61.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.1%st Mem: 16433100k total, 15416512k used, 1016588k free,86276k buffers Swap: 262140k total,18872k
Re: [Dhis2-devs] [Dhis2-users] Date category combination
Thank you Lars, I will try it ! But for next features, I think it will be useful to assign a Value Type (number, date, Text, Yes/no, etc...) to a category . Sincerely, = EKANI Guy Le Jeudi 22 janvier 2015 22h47, Lars Helge Øverland larshe...@gmail.com a écrit : Hi Guy, currently the way it works is that a data element can only have one category combination. This means that all fields in the form for that data elements will have the same data type. The best way to achieve the form layout above is to have 2 data elements: - One with numeric value type, and a category with three age groups.- One with date type and default category. Then put these fields into a custom data entry form. regards, Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Bug 1413990] [NEW] Accented characters don't work in user group name
Public bug reported: When I create user group in 2.18 with an accented character in the name, the accented character gets garbled in the user group listing. To reproduce, go to https://apps.dhis2.org/demo/dhis-web-maintenance- user/getAllUserGroups.action and click Add New. Enter a user group name with an accented character, like Côte d'Ivoire user group. Select at least one user to join the group and hit Add. The user group listing shows something like Côte d'Ivoire user group. ** Affects: dhis2 Importance: Undecided Status: New -- You received this bug notification because you are a member of DHIS 2 developers, which is subscribed to DHIS. https://bugs.launchpad.net/bugs/1413990 Title: Accented characters don't work in user group name Status in DHIS 2: New Bug description: When I create user group in 2.18 with an accented character in the name, the accented character gets garbled in the user group listing. To reproduce, go to https://apps.dhis2.org/demo/dhis-web-maintenance- user/getAllUserGroups.action and click Add New. Enter a user group name with an accented character, like Côte d'Ivoire user group. Select at least one user to join the group and hit Add. The user group listing shows something like Côte d'Ivoire user group. To manage notifications about this bug go to: https://bugs.launchpad.net/dhis2/+bug/1413990/+subscriptions ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18112: minor fix, only call getCurrentUser one time for getAccess in CurrentUserController
revno: 18112
committer: Morten Olav Hansen morte...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 21:23:04 +0700
message:
minor fix, only call getCurrentUser one time for getAccess in
CurrentUserController
modified:
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/user/CurrentUserController.java
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/user/CurrentUserController.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/user/CurrentUserController.java 2015-01-19 08:13:30 +
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/user/CurrentUserController.java 2015-01-23 14:23:04 +
@@ -190,9 +190,9 @@
@RequestMapping( value = /dashboards, produces = { application/json, text/* } )
public void getDashboards( HttpServletResponse response ) throws NotAuthenticatedException, IOException
{
-User currentUser = currentUserService.getCurrentUser();
+User user = currentUserService.getCurrentUser();
-if ( currentUser == null )
+if ( user == null )
{
throw new NotAuthenticatedException();
}
@@ -201,11 +201,11 @@
for ( org.hisp.dhis.dashboard.Dashboard dashboard : dashboards )
{
-dashboard.setAccess( aclService.getAccess( dashboard ) );
+dashboard.setAccess( aclService.getAccess( dashboard, user ) );
for ( DashboardItem dashboardItem : dashboard.getItems() )
{
-dashboardItem.setAccess( aclService.getAccess( dashboardItem ) );
+dashboardItem.setAccess( aclService.getAccess( dashboardItem, user ) );
}
}
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18111: minor fix, only call getCurrentUser one time for addAccessProperties
revno: 18111
committer: Morten Olav Hansen morte...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 21:17:20 +0700
message:
minor fix, only call getCurrentUser one time for addAccessProperties
modified:
dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AbstractCrudController.java
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AbstractCrudController.java'
--- dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AbstractCrudController.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-web/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/AbstractCrudController.java 2015-01-23 14:17:20 +
@@ -58,6 +58,7 @@
import org.hisp.dhis.schema.Schema;
import org.hisp.dhis.schema.SchemaService;
import org.hisp.dhis.user.CurrentUserService;
+import org.hisp.dhis.user.User;
import org.hisp.dhis.webapi.controller.exception.NotFoundException;
import org.hisp.dhis.webapi.service.ContextService;
import org.hisp.dhis.webapi.service.LinkService;
@@ -830,9 +831,11 @@
protected void addAccessProperties( ListT objects )
{
+User user = currentUserService.getCurrentUser();
+
for ( T object : objects )
{
-((BaseIdentifiableObject) object).setAccess( aclService.getAccess( object ) );
+((BaseIdentifiableObject) object).setAccess( aclService.getAccess( object, user ) );
}
}
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Bug 1402244] Re: event report/visulizer: option set sort order not respocted
** Changed in: dhis2 Assignee: (unassigned) = Jan Henrik Øverland (janhenrik-overland) ** Changed in: dhis2 Milestone: None = 2.19 -- You received this bug notification because you are a member of DHIS 2 developers, which is subscribed to DHIS. https://bugs.launchpad.net/bugs/1402244 Title: event report/visulizer: option set sort order not respocted Status in DHIS 2: New Bug description: Event report/visulizer apps do not respect the sort order that is set for option sets. 2.17. To manage notifications about this bug go to: https://bugs.launchpad.net/dhis2/+bug/1402244/+subscriptions ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Problems with detailed metadata export..
I have backportet to 2.17 -- Morten On Fri, Jan 23, 2015 at 9:59 PM, Bob Jolliffe bobjolli...@gmail.com wrote: I guess you will also have to backport to 2.17. On 23 January 2015 at 14:08, Morten Olav Hansen morte...@gmail.com wrote: Yes, we have already fixed this a few places. I'm making a note of it, and will fix it later. -- Morten On Fri, Jan 23, 2015 at 8:58 PM, Lars Helge Øverland larshe...@gmail.com wrote: Hi Bob, I think you are right. I've seen this before, and reducing the number of calls to one usually helps a lot. The problem is that getCurrentUser() is transactional and makes a (cached) db query. The overhead of a huge number of transaction causes things to slow down. If you or someone could write a patch/fix it would be great. Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] [Dhis2-users] Date category combination
Hi all, I want to delete my analytics tables and start all over. The (only?) way to do this is through the Data Admin - Maintenance app but when clearing the tables nothing happens. All tables still exist in Postgresql and the data remains available in the Pivot tables. We're still in 2.17. Is this a known issue or is there a workaround? Sorry, couldn't test on the demo server as it seems to take forever to try to delete the analytics tables there. Regards, Robin De : Dhis2-users [dhis2-users-bounces+martens=sher...@lists.launchpad.net] de la part de Guy Ekani [constyek...@yahoo.fr] Envoyé : vendredi 23 janvier 2015 13:10 À : Lars Helge Øverland; Enmanuel Toribio Cc : dhis2-us...@lists.launchpad.net; dhis2-devs@lists.launchpad.net Objet : Re: [Dhis2-users] Date category combination Thank you Lars, I will try it ! But for next features, I think it will be useful to assign a Value Type (number, date, Text, Yes/no, etc...) to a category . Sincerely, = EKANI Guy Le Jeudi 22 janvier 2015 22h47, Lars Helge Øverland larshe...@gmail.com a écrit : Hi Guy, currently the way it works is that a data element can only have one category combination. This means that all fields in the form for that data elements will have the same data type. The best way to achieve the form layout above is to have 2 data elements: - One with numeric value type, and a category with three age groups. - One with date type and default category. Then put these fields into a custom data entry form. regards, Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18114: set min for name/shortName/description to 1
revno: 18114
committer: Morten Olav Hansen morte...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 22:29:20 +0700
message:
set min for name/shortName/description to 1
modified:
dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseIdentifiableObject.java
dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseNameableObject.java
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseIdentifiableObject.java'
--- dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseIdentifiableObject.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseIdentifiableObject.java 2015-01-23 15:29:20 +
@@ -220,7 +220,7 @@
@JsonProperty
@JacksonXmlProperty( isAttribute = true )
@Description( The name of this Object. Required and unique. )
-@PropertyRange( min = 2 )
+@PropertyRange( min = 1 )
public String getName()
{
return name;
=== modified file 'dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseNameableObject.java'
--- dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseNameableObject.java 2015-01-17 07:41:26 +
+++ dhis-2/dhis-api/src/main/java/org/hisp/dhis/common/BaseNameableObject.java 2015-01-23 15:29:20 +
@@ -165,7 +165,7 @@
@JsonProperty
@JsonView( { ShortNameView.class, DetailedView.class, ExportView.class } )
@JacksonXmlProperty( isAttribute = true )
-@PropertyRange( min = 2 )
+@PropertyRange( min = 1 )
public String getShortName()
{
return shortName;
@@ -180,7 +180,7 @@
@JsonProperty
@JsonView( { DetailedView.class, ExportView.class } )
@JacksonXmlProperty( namespace = DxfNamespaces.DXF_2_0 )
-@PropertyRange( min = 2 )
+@PropertyRange( min = 1 )
public String getDescription()
{
return description;
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
Re: [Dhis2-devs] Problems with detailed metadata export..
I guess you will also have to backport to 2.17. On 23 January 2015 at 14:08, Morten Olav Hansen morte...@gmail.com wrote: Yes, we have already fixed this a few places. I'm making a note of it, and will fix it later. -- Morten On Fri, Jan 23, 2015 at 8:58 PM, Lars Helge Øverland larshe...@gmail.com wrote: Hi Bob, I think you are right. I've seen this before, and reducing the number of calls to one usually helps a lot. The problem is that getCurrentUser() is transactional and makes a (cached) db query. The overhead of a huge number of transaction causes things to slow down. If you or someone could write a patch/fix it would be great. Lars ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18115: GIS, projection fix.
Merge authors:
Jan Henrik Øverland (janhenrik-overland)
revno: 18115 [merge]
committer: Jan Henrik Overland janhenrik.overl...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 17:04:30 +0100
message:
GIS, projection fix.
modified:
dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/plugin.html
dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js
dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/core.js
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/plugin.html'
--- dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/plugin.html 2014-12-24 15:40:47 +
+++ dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/plugin.html 2015-01-23 16:01:30 +
@@ -6,9 +6,6 @@
!--
link rel=stylesheet type=text/css href=//localhost:8080/dhis-web-commons/javascripts/ext/resources/css/ext-all-gray.css/
--
-!--
-link rel=stylesheet type=text/css href=//localhost:8080/dhis-web-commons/javascripts/ext/resources/css/tmp.css/
---
script src=//localhost:8080/dhis-web-commons/javascripts/ext/ext-all.js/script
script src=//maps.google.com/maps/api/js?sensor=false/script
=== modified file 'dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js'
--- dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js 2015-01-21 18:52:56 +
+++ dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js 2015-01-23 16:01:30 +
@@ -2658,8 +2658,11 @@
handler: function() {
var name = nameTextfield.getValue(),
layers = gis.util.map.getRenderedVectorLayers(),
+centerPoint = function() {
+var lonlat = gis.olmap.getCenter();
+return new OpenLayers.Geometry.Point(v.lon, v.lat).transform('EPSG:900913', 'EPSG:4326');
+}(),
layer,
- lonlat = gis.olmap.getCenter(),
views = [],
view,
map;
@@ -2681,7 +2684,7 @@
view.hidden = !layer.visibility;
- // Operand
+ // operand
if (Ext.isArray(view.columns) view.columns.length) {
for (var j = 0; j view.columns.length; j++) {
for (var k = 0, item; k view.columns[j].items.length; k++) {
@@ -2702,8 +2705,8 @@
map = {
name: name,
- longitude: lonlat.lon,
- latitude: lonlat.lat,
+ longitude: centerPoint.x,
+ latitude: centerPoint.y,
zoom: gis.olmap.getZoom(),
mapViews: views,
user: {
=== modified file 'dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/core.js'
--- dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/core.js 2015-01-21 18:52:56 +
+++ dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/core.js 2015-01-23 16:01:30 +
@@ -1156,19 +1156,26 @@
};
afterLoad = function() {
+var lon = parseFloat(gis.map.longitude) || 0,
+lat = parseFloat(gis.map.latitude) || 20,
+zoom = gis.map.zoom || 3;
+
register = [];
+// validate, transform
+if ((lon = -180 lon = 180) (lat = -90 lat = 90)) {
+var p = new OpenLayers.Geometry.Point(lon, lat).transform('EPSG:4326', 'EPSG:900913');
+
+lon = p.x;
+lat = p.y;
+}
+
if (gis.el) {
gis.olmap.zoomToVisibleExtent();
}
else {
-if (gis.map.longitude gis.map.latitude gis.map.zoom) {
- gis.olmap.setCenter(new OpenLayers.LonLat(gis.map.longitude, gis.map.latitude), gis.map.zoom);
-}
-else {
- gis.olmap.zoomToVisibleExtent();
-}
- }
+gis.olmap.setCenter(new OpenLayers.LonLat(lon, lat), zoom);
+}
// interpretation button
if (gis.viewport.shareButton) {
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
[Dhis2-devs] [Branch ~dhis2-devs-core/dhis2/trunk] Rev 18116: Minor
Merge authors:
Jan Henrik Øverland (janhenrik-overland)
revno: 18116 [merge]
committer: Jan Henrik Overland janhenrik.overl...@gmail.com
branch nick: dhis2
timestamp: Fri 2015-01-23 17:12:04 +0100
message:
Minor
modified:
dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js
--
lp:dhis2
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
Your team DHIS 2 developers is subscribed to branch lp:dhis2.
To unsubscribe from this branch go to
https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
=== modified file 'dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js'
--- dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js 2015-01-23 16:01:30 +
+++ dhis-2/dhis-web/dhis-web-apps/src/main/webapp/dhis-web-mapping/scripts/app.js 2015-01-23 16:11:33 +
@@ -2660,7 +2660,7 @@
layers = gis.util.map.getRenderedVectorLayers(),
centerPoint = function() {
var lonlat = gis.olmap.getCenter();
-return new OpenLayers.Geometry.Point(v.lon, v.lat).transform('EPSG:900913', 'EPSG:4326');
+return new OpenLayers.Geometry.Point(lonlat.lon, lonlat.lat).transform('EPSG:900913', 'EPSG:4326');
}(),
layer,
views = [],
___
Mailing list: https://launchpad.net/~dhis2-devs
Post to : dhis2-devs@lists.launchpad.net
Unsubscribe : https://launchpad.net/~dhis2-devs
More help : https://help.launchpad.net/ListHelp
