If sync got turned off somehow and records weren't recorded on parent, then yes, there will be records out of sync and you might need to restore the children from the parent copy. I'd recommend doing that asap.
I see your copied in your logs from the 28th. Are there any rows with "ERROR" or "WARN" on them? Just out of curiosity, what made you choose some of those Objects Not To Sync? Specifically, why do you have org.openmrs.Form/Field/FormField set to not sync? Are you moving those records manually? Are you using formentry? If so, you just need the org.openms.module.formentry package instead. org.openmrs.module.htmlformentry should be ok to sync. Is the cleanup scheduled task still running? Or was it run manually about the time things stopped being recorded? Ben On Thu, Apr 5, 2012 at 4:43 AM, Jim Grace <[email protected]> wrote: > Hi All, > > Thank you Dave, Ben and Mark for the responses so far. If you can help us > further to figure out what's going on, we would be grateful. It has us > stumped. We're doing a production pilot of Sync from our central server to > 2 nearby child sites in the same district, after successful internal > testing. One site is synched through a cell modem, and the notebook running > OpenMRS at the other site is brought back each night to sync through our > LAN. (We expect to deploy both modalities, so we are piloting them both.) > If it goes well, we want to roll out Sync (in stages) to our full network > consisting of child nodes at our district hospitals and grandchild nodes in > the clinics of those districts. We would be replacing Remote Form Entry, > which we are currently using. > > Everything went fine in both directions between our root parent server and > the two production pilot children for over a week. Then the root server > mysteriously stopped entering new sync rows in the database, even though it > continued to be used heavily. We didn't change anything. Now our children > databases are out of sync with the root, and it looks like we have to do a > complete backup/restore to get them synchronized again. We would love to > expand Sync to all our other sites, but not if we might have more such > problems! > > Rebooting the parent server made it start synching again perfectly (except > that the databases are now out of sync from during the failure period -- so > any new data synched from server to child blocks the queue if it is > dependent on data that missed being synched.) I don't expect there are any > configuration problems since we never changed the configuration between > working - not working - working again. We didn't change UUIDs, > configuration, not a single thing. > > Here is the configuration we have from parent to child (taken from parent > server) and child to parent (taken from child server): > > [From Parent server] Objects NOT to Send/Export: > org.openmrs.Form > org.openmrs.module.reporting > org.openmrs.scheduler.TaskDefinition > org.openmrs.Cohort > org.openmrs.scheduler > org.openmrs.FormField > org.openmrs.hl7 > org.openmrs.Field > org.openmrs.reporting > org.openmrs.logic.token.TokenRegistration > org.openmrs.GlobalProperty > org.openmrs.module.formentry > org.openmrs.module.patientflags > org.openmrs.module.dataintegrity > org.openmrs.module.logic > org.openmrs.api.db.SerializedObject > org.openmrs.notification.Alert > org.openmrs.logic > > [From Parent server] Objects NOT to Receive/Import: > org.openmrs.Form > org.openmrs.module.reporting > org.openmrs.scheduler.TaskDefinition > org.openmrs.Cohort > org.openmrs.module.htmlformentry > org.openmrs.scheduler > org.openmrs.FormField > org.openmrs.hl7 > org.openmrs.Field > org.openmrs.reporting > org.openmrs.logic.token.TokenRegistration > org.openmrs.GlobalProperty > org.openmrs.Concept > org.openmrs.module.formentry > org.openmrs.module.patientflags > org.openmrs.module.dataintegrity > org.openmrs.module.logic > org.openmrs.api.db.SerializedObject > org.openmrs.notification.Alert > org.openmrs.logic > > [From Child server] Objects NOT to Send/Export: > org.openmrs.Field > org.openmrs.Concept > org.openmrs.api.db.SerializedObject > org.openmrs.scheduler > org.openmrs.module.logic > org.openmrs.FormField > org.openmrs.Form > org.openmrs.logic > org.openmrs.module.reporting > org.openmrs.GlobalProperty > org.openmrs.api.db.SerializedObject > org.openmrs.module.htmlformentry > org.openmrs.Cohort > org.openmrs.module.formentry > org.openmrs.logic.token.TokenRegistration > org.openmrs.scheduler.TaskDefinition > org.openmrs.reporting > org.openmrs.hl7 > org.openmrs.notification.Alert > > [From Child server] Objects NOT to Receive/Import: > org.openmrs.Field > org.openmrs.api.db.SerializedObject > org.openmrs.scheduler > org.openmrs.module.logic > org.openmrs.FormField > org.openmrs.Form > org.openmrs.logic > org.openmrs.module.dataintergrity > org.openmrs.module.patientflags > org.openmrs.module.reporting > org.openmrs.GlobalProperty > org.openmrs.api.db.SerializedObject > org.openmrs.Cohort > org.openmrs.module.formentry > org.openmrs.logic.token.TokenRegistration > org.openmrs.scheduler.TaskDefinition > org.openmrs.reporting > org.openmrs.hl7 > org.openmrs.notification.Alert > > We can see new sync_record rows created on our parent system until one > with record_id 23722, timestamped 2012-03-28 09:28:20. Then there were no > more sync_record rows created until after we rebooted. The following > catalina.out entries from our parent server seem to show the last two > parent sync_records 23721 and 23722 before they stopped, followed by a > successful sync operation from the child server Rabuor: > > INFO - LoggingAdvice.invoke(109) |2012-03-28 09:29:01,720| In method > SyncService.updateSyncRecord. Arguments: SyncRecord=SyncRecord(23721) > contains org.openmrs.Privilege, > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,720| Exiting method > updateSyncRecord > INFO - LoggingAdvice.invoke(109) |2012-03-28 09:29:01,722| In method > SyncService.updateSyncRecord. Arguments: SyncRecord=SyncRecord(23722) > contains org.openmrs.Privilege, > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,722| Exiting method > updateSyncRecord > INFO - LoggingAdvice.invoke(109) |2012-03-28 09:29:01,724| In method > SyncService.saveRemoteServer. Arguments: RemoteServer=RemoteServer(1): > Rabuor, > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,726| Exiting method > saveRemoteServer > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,847| Exiting method > saveEncounter > INFO - LoggingAdvice.invoke(109) |2012-03-28 09:29:01,848| In method > HL7Service.encounterCreated. Arguments: Encounter=Encounter: [1171134 Tue > Mar 27 00:00:00 EAT 2012 ADULTRETURN null 51216 Revised Adult Follow-Up > Form num Obs: [Obs #51749135, Obs #51749164, Obs #51749129, Obs #51749108, > Obs #51749111, Obs #51749131, Obs #51749130, Obs #51749125, Obs #51749124, > Obs #51749115, Obs #51749126, Obs #51749121, Obs #51749123, Obs #51749119, > Obs #51749153, Obs #51749122, Obs #51749148, Obs #51749149, Obs #51749150, > Obs #51749151, Obs #51749147, Obs #51749140, Obs #51749141] num Orders: 0 > ], > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,849| Exiting method > encounterCreated > INFO - LoggingAdvice.invoke(134) |2012-03-28 09:29:01,849| Exiting method > processHL7Message > > I've looked through catalina.out after this point, but can find no clue as > to why the sync_record rows stopped being recorded. Is there something in > particular I should look for? > > Meanwhile on the child servers, new sync_records continued to be entered, > and they were all synchronized successfully with the parent. It was only > the flow from parent to both children that was interrupted because no new > sync_records were being generated by the parent. > > When we realized that nothing was moving from parent to child, we checked > sync_record and sync_server_record tables and noticed that no new records > were being generated by the parent. We verified that all child encounters > were present on the parent, but no parent encounters had been transferred > to the children after the last sync_record. We restarted the OpenMRS Ubuntu > server, and the sync_records started being added again. There has been no > problem since in the last few days, except new records that were dependent > on unsynchronized objects have also caused problems. New sync_record are > now being added again on the parent, starting from just after the system > reboot. > > We would greatly appreciate any further advice you can give us to > understand this problem. > > Thanks, > Jim > > --------------------------------- > > > At 11:32 PM 02-04-12, Ben Wolfe wrote: > Also see > https://wiki.openmrs.org/display/docs/Sync+Module+Technical+Overview for > a description of the tables. > > To add to Dave's questions: > Did you set the "do not sync" objects to anything likw * or org.openmrs.* > per chance? > > Ben > > On Mon, Apr 2, 2012 at 4:16 PM, Dave Thomas <[email protected]> wrote: Hi. > If you're not seeing any new records in sync_server_record on the parent, > you're in trouble. This is the table that is used to keep track of what > records have propagated to what child servers. Without rows being inserted > into this table, no data will ever propagate to the children. In your sync > overview page on the parent, can you actually see the child servers, and > are they configured correctly? Did you change the server uuids at all, > either on the children or on the parent? On the parent, the state field > in the sync_record table isn't important. Its the state field in the > sync_server_record that governs propagation. > > d > > > On Mon, Apr 2, 2012 at 8:05 AM, Erick Mugoma <[email protected] > > wrote: Hi Ben and Mark Am just wondering what may or may not be > happening with the sync_server_record and sync_record tables on the > central/parent server. Plse help me understand something, I expect that > every time a new record is created on the parent, the record details are > logged to the above tables and you can even use these tables to track if > the record was committed, sent, new or failed. Why is it that when I do a > query to check if there any records in the sync_server_record table I can't > see any record for example I should be able to see today's records > (02-04-12) since the clean -up was last done on 31 march 2012. Any I deas > on what is not Happenning? On the parent sync_record table, the state > field is not being updated. The state column value is 'NEW' for all the > records. I expect that the state should change once the record is committed > to all the children. Any reason why this is not so? When I look at the > same table on the child servers, the state column value is 'COMMITTED' > Looking > at the History Changes on the UI What I see are old changes like last > updated is 28-03-2012. That is last week wed when we have new records being > entered even today (02-04-12) and should be able to see them under history > changes. Any Explanation why I can't see the newest/latest updates. > > Thanks > > Erick > > > ------------------------------ > Click here to > unsubscribe<[email protected]?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > > > ------------------------------ > Click here to > unsubscribe<[email protected]?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > ------------------------------ > Click here to > unsubscribe<[email protected]?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > _________________________________________ To unsubscribe from OpenMRS Developers' mailing list, send an e-mail to [email protected] with "SIGNOFF openmrs-devel-l" in the body (not the subject) of your e-mail. [mailto:[email protected]?body=SIGNOFF%20openmrs-devel-l]
