I have a customer who is also interested... Hans
On Tue, 2009-10-27 at 08:58 +0100, Jacques Le Roux wrote: > Hi Deyan, Pankaj, > > Could you share your work done so far with the community ? > I agree that we shouyd try to use and maintain EntitySync. > But for the volume reason Deyan outlined and in the meantime I'm sure his > solution would be appreciated too! > I suggest to create Jira or/and wiki pages. I'm ready to help on this. > > Thanks > > Jacques > > From: "Jacques Le Roux" <[email protected]> > > Thanks for comment Deyan, > > > > I will try Pankaj Jain's tips in a 1st time > > > > Jacques > > > > From: "Deyan Tsvetanov" <[email protected]> > > Hi Jacques, > > > > no, I didn't work on fixing the entity sync issues. > > Instead I did my own sync tool using db triggers and stored procedures. > > > > The entity sync could be fixed, but it would require like two - three > > weeks of work. > > I am sorry I didn't post any details, i was kind of busy because of a > > missed deadline :) > > > > The entity sync is kind of useless in this state imho. > > Reasons: > > > > 1) Does not do any fail over - if the sync process is interrupted for > > any reason - like network outage, or ofbiz restart, or system freeze > > it will be considered as a success sync and won't continue from the > > point it has failed. Could be fixed though :) > > > > 2) Pretty slow. The entities are transfered over the network using RMI. > > For transferring 5000 product records ( along with their relations - > > product_price, product_assoc, etc ) > > it took over 2 hours using pretty good hard and 50 mbits network > > connection. > > > > 3) The example of entities to sync in the sync howto notes will never > > work. There will be records with duplicate PKs :) > > > > The solution for those issues imho are: > > 1) Refactor a bit the entity sync code so the client remembers the last > > successful sync state. This would fix the fail over issues. currently > > the server ( MCS ) remembers the > > sync status for each client. > > > > 2) Don't use RMI :) It would be much faster if the entities that are > > going to be transfered in order to get synced are serialized to XML for > > example and > > transferred over the network in a text format. RMI is easy to use but > > has bad performance. Just to give you an example: > > about 5000 products ( which is probably about 20 000 entities in total > > to pull ) takes 2 hours on a 50 mbits connection. What if we had a > > pretty huge supermarket > > which has 100 000 sales per day ( this means 100 000 sales orders , > > average 50 order_items per order, 100 000 payments, 100 000 invoices, > > 100 000 invoice items ). > > It would require an entire night to sync - if we are lucky :) > > > > 3) I can provide the complete list of tables needed to get synced for a > > POS solution, including fix of the issue with the duplicate PKs :) > > > > However for our current project we've used triggers which create a > > transaction log for push on the POS databases and a transaction log for > > pull on the MCS instance. > > Pulling those 5000 products along with all the related records takes > > about 20 secs on the same hardware. I've configured a crontab job which > > invoces the sync postgre stored procedures > > every 5 minutes and it works perfectly. A status table is maintained on > > the MCS so we can check regularly if a POS hasn't synced for let's say 1 > > hour - this means either network outage or some other problem. > > > > Yes, I know this solution is DB dependent but if the ofbiz sync > > framework issues are fixed and we use the list of tables to sync i've > > managed to create something useful could come out of it :) > > > > So - i hope my e-mail answers your question, if you want to know more - > > don't hesitate :) > > > > Cheers, > > Deyan > > > > -----Original Message----- > > From: Jacques Le Roux <[email protected]> > > Reply-to: "Jacques Le Roux" <[email protected]> > > To: Deyan Tsvetanov <[email protected]>, > > [email protected] > > Subject: Re: EntitySync issues & suggestions ( was EntitySync RMI > > error ) > > Date: Mon, 26 Oct 2009 08:34:51 +0100 > > > > > > Hi Deyan, > > > > Did you work on this ? I will need to use EntitySync soon and I know you > > are not the 1st to complain about it (Si Chen did for instance). On the > > other hand, some seems to have used it successfully... > > > > Thanks > > > > Jacques > > ----- Original Message ----- > > From: Deyan Tsvetanov > > To: Jacques Le Roux ; [email protected] > > Sent: Sunday, October 04, 2009 10:57 AM > > Subject: EntitySync issues & suggestions ( was EntitySync RMI > > error ) > > > > > > > > Hi Jacques and list, > > > > After playing for almost 2 weeks with the entity sync feature > > I've come up with the conclusion that it still needs > > some work to get ready for production environment. Currently it > > is unreliable, and useless :( > > > > - There is a bug in the design. The MCS remembers the sync > > status and currently reached sync timestamp for each client. > > This is not a good approach imho. Why ? > > The client asks the server: have you got something for me ? > > Next: The MCS checks the last successful sync timestamp for that > > particular client and selects the entities that should be passed > > to the client. After successfully transferring the entities to > > the client over RMI the MCS assumes that the sync is successful > > and updates the last successful sync timestamp. > > > > The problem is that the client does not confirm the success of > > the operation. If for some reason the client is unable to update > > the entities ( either shutdown, > > power failure, DB error or whatever ) the MCS isn't aware of > > that and the next sync will not include the missed entities. > > There is no way to fix the skipped > > entities but re-creating the client's database from zero. > > > > That is the main issue actually. It could be easily solved > > however. The most easy ways are: > > 1) The client will remember the sync timestamp and will include > > it as an argument when asking the server for pull sync. This way > > the client will > > update the last successful sync timestamp after storing each > > entity. The server does not have to remember anything regarding > > the client's sync status. > > > > 2) After each sync session the server will wait for confirmation > > from the client before updating the last successful sync > > timestamp. > > If not updated than the sync process is considered to be > > unsuccessful and the timestamp will not be updated. > > > > I personally prefer approach 1) as in case of network error, > > power failure or whatever error the sync process will continue > > from the place it has stopped. > > > > > > - There are some swallowed exceptions. If a network problem > > occurs during the sync process the server's status remains "sync > > running" and the client is > > not able to sync anymore until the status is reset by an > > operator. > > > > - The last issue is the performance. Using RMI is an easy > > approach from developer's perspective but it is very very slow. > > I had a database with 5000 products. The initial sync took about > > 2 hours. The hardware is pretty good, the network connection > > between the MCS and the client ofbiz > > is about 50 mbits. What if we have 100 000 sales per day ? The > > sync process will take probably the entire night :) > > Here I would suggest transferring GenericEntity instances over > > RMI to be removed. Instead a regular XML could be used. Either > > over RMI or SOAP - does not matter. > > > > So in general the picture would be: > > > > 1) Server does not remember any timestamp for any client. > > 2) Client calls the pull entity sync RMI ( or soap ) method and > > passes the entity sync ID and the timestamp of the last > > successful sync as arguments. Including the timezone of > > course :) > > 3) The server generates an XML file for all the entities in that > > entity sync group using the timestamp provided by the client. > > 4) Client starts updating the entities described in the received > > XML line by line. After each entity is updated successfully the > > timestamp is also updated ( on the client side - the > > server does not remember any client timestamps ). > > 5) If a failure occurs in the middle of the update process what > > we'll have is a sync process completed to like 50%. With the > > correct timestamp. When the next sync > > job starts the client will use the timestamp and the server will > > generate a new XML from the place the sync has failed. Of course > > an optimization could be used - > > the XML will be stored in a new db table so the client can > > continue storing the entities without asking the server to > > generate something he has already done. > > > > Some improvements that could be also useful: > > 1) The client ofbiz instances have sequence-id-prefix. So when > > the server is accepting push syncs it could also check the > > prefixes of the PKs and disallow entities without the correct > > prefix in the primary key. That's just a cheap insurance against > > errors :) We don't want store A to make sales on behalf of store > > B just because somebody made an error while configuring the > > pos ofbiz instance. > > > > 2) A time synchronization mechanism could be also implemented. > > In general when syncing there is a requirement that the server's > > and client's clocks are in sync ( ntp ). We don't want > > to set the system clock in ofbiz, but we could implement a check > > if the clocks are in sync and refuse to start the > > synchronization in certain cases. > > > > 3) A new column could be added to all the entities: > > isFromEntitySync : flag. This way we could avoid cases in which > > the client is trying to push records to the server which were > > actually pulled by the server. Yes, I know the entity's created > > timestamp should help us avoid such cases, but when the sync > > process is interrupted things like that could happen. > > > > Otherwise for our current project we've implemented a DB level > > synchronization - using triggers for INSERT, UPDATE and DELETE > > and a stored procedure which syncs the clients with a MCS. > > It works much ( tens of times ) faster than ofbiz entity sync . > > But it is DB dependent. So after getting comments and > > suggestions to this e-mail I would like to invest some time in > > fixing all those issues with the ofbiz entity sync and switch to > > it when it's ready :) > > > > Thanks for your time, > > Deyan > > > > -----Original Message----- > > From: Jacques Le Roux <[email protected]> > > Reply-to: "Jacques Le Roux" <[email protected]> > > To: [email protected] > > Subject: Re: EntitySync RMI error > > Date: Sun, 27 Sep 2009 17:02:57 +0200 > > > > > > Hi Deyan, > > > > It would be very valuable if you could post an article on wiki about > > your experience with this. Don't worry about where to > > put it, I > > will eventually take care to put it under FAQ... > > > > TIA > > > > Jacques > > > > From: "Deyan Tsvetanov" <[email protected]> > > >I found an issue though: > > > > > > If the connection gets dropped during sync : > > > - the client ( POS ) prints connection reset by peer > > > - the server ( MCS ) sync status remains to running. > > > - the next sync does not start because MCS complains there is > > another > > > sync running already. > > > > > > "Reset run status" from webtools -> entity sync status > > > helps. > > > > > > > > > I'll investigate further and log a bug, although the solution seems > > > pretty simple - looks like a swallowed IOException, > > > which should be handled by re-setting the sync status. > > > > > > CHeers, > > > DEyan > > > > > > -----Original Message----- > > > From: Deyan Tsvetanov <[email protected]> > > > Reply-to: [email protected] > > > To: [email protected] > > > Subject: Re: EntitySync RMI error > > > Date: Sat, 26 Sep 2009 11:22:55 +0300 > > > > > > > > > Got it ! > > > > > > Seems that the previous exception was coming from the MCS ( server > > ) . > > > When I stopped the server I started getting the exception I kind > > of like > > > more :) > > > > > > > > > Message: Exception calling remote pull and report EntitySync > > service > > > with name: remotePullAndReportEntitySyncDataRmi; > > org.ofbiz.service.Gene > > > ricServiceException: RMI Error (Connection refused to host: > > > 192.168.1.100; nested exception is: > > > java.net.ConnectException: Connection refused: connect) > > > > > > So the problem was at the server side - RMIIF env var in > > startofbiz.sh > > > wasn't set , so ofbiz was trying to get RMI host IP by resolving > > the > > > hostname . > > > > > > The misleading things for me was that: > > > 1) no any sign in the MCS logs that somebody is trying to connect > > and > > > the MCS itself can not connect to its own RMI registry. > > > 2) no any difference in the exceptions text on the POS ( client ) > > side: > > > the 2 exceptions ( local one - can not connect to 192.168.1.100 > > and the > > > remote one - can not connect to 127.0.0.1 ) > > > look the same :) > > > > > > So issue is solved, > > > sorry for bothering :) > > > > > > -- deyan > > > > > > -----Original Message----- > > > From: Deyan Tsvetanov <[email protected]> > > > Reply-to: [email protected] > > > To: [email protected] > > > Subject: Re: EntitySync RMI error > > > Date: Sat, 26 Sep 2009 09:52:55 +0300 > > > > > > > > > During startup I get: > > > > > > 2009-09-26 08:46:31,905 (default-invoker-Thread-10) [ > > > AbstractEngine.java:73 :INFO ] Loaded Service Locations : > > > [main-rmi=rmi://127.0.0. > > > 1:1099/RMIDispatcher, > > > main-http=http://127.0.0.1:8080/webtools/control/httpService, > > > entity-sync-rmi=rmi://192.168.1.100:1099/RMIDispatcher, > > > > > entity-sync-http=http://192.168.1.100:8080/webtools/control/httpService, > > > rita-rmi=rmi://127.0.0.1:1099/RMIDispatcher, eedcc-test=http://127. > > > 0.0.1:8080/webtools/control/httpService] > > > > > > entity-sync-rmi seems to be ok ... > > > > > > > > > > > > > > > -----Original Message----- > > > From: Deyan Tsvetanov <[email protected]> > > > Reply-to: [email protected] > > > To: [email protected] > > > Subject: EntitySync RMI error > > > Date: Sat, 26 Sep 2009 09:22:23 +0300 > > > > > > > > > Hi guys, > > > > > > I'm trying to configure RMI entity sync. I'm following > > > http://docs.ofbiz.org/display/OFBIZ/Sync+Setup+Notes+and+Example > > > > > > What I've done so far: > > > > > > 1) entity-sync-rmi to rmi://192.168.1.100:1099/RMIDispatcher > > > > > > 2) set RMIIF=-Djava.rmi.server.hostname=127.0.0.1 > > > ( as per the example ). > > > > > > 3) I've imported the entity sync groups, SandJobs, etc. > > > > > > However when the sync starts ( on the POS instance ) I get the > > following > > > error: > > > > > > Exception calling remote pull and report EntitySync service with > > name: > > > remotePullAndReportEntitySyncDataRmi; > > org.ofbiz.service.GenericServic > > > eException: RMI Invocation Error (Connection refused to host: > > 127.0.0.1; > > > nested exception is: > > > java.net.ConnectException: Connection refused: connect) > > > Exception: org.ofbiz.service.GenericServiceException > > > Message: RMI Invocation Error (Connection refused to host: > > 127.0.0.1; > > > nested exception is: > > > java.net.ConnectException: Connection refused: connect) > > > ---- cause > > > > > --------------------------------------------------------------------- > > > Exception: java.rmi.ConnectException > > > Message: Connection refused to host: 127.0.0.1; nested exception > > is: > > > java.net.ConnectException: Connection refused: connect > > > ---- cause > > > > > --------------------------------------------------------------------- > > > Exception: java.net.ConnectException > > > Message: Connection refused: connect > > > ---- stack trace > > > --------------------------------------------------------------- > > > > > > > > > It insists connecting to 127.0.0.1 no matter what I type in > > > serviceengine.xml. > > > Any help would be appreciated :) > > > > > > Thanks in advance, > > > Deyan > > > > > > > > > > > > > -- Antwebsystems.com: Quality OFBiz services for competitive rates
