This code is wrong. As you can see variables "pushed" and "currentPushTry" are not thread safe in the first method. That might cause the errors.
On Mon, Oct 12, 2015 at 12:23 PM, Aiyadurai Rajeevan <[email protected]> wrote: > Hi All, > > I have been testing the below code for synchronizing the critical > resource(remote repo URL), But still some of the pushes are failed. > > public boolean push(String remoteRepoUrl, String pushBranch, File repoFile) > throws RepositoryMgtException { > try { > Git gitRepo = getGitRepository(remoteRepoUrl, repoFile); > log.info("Remote repo URL================>>>>>"+remoteRepoUrl); > > > Iterable<PushResult> pushResults = gitRepo.push() > .setRemote(remoteRepoUrl) > .setRefSpecs(new RefSpec("refs/heads/" + pushBranch)) > .setCredentialsProvider(getCredentialsProvider()) > .call(); > // we need to verify if git push was successful. Here we can check > RemoteRefUpdate status is rejected or not. > boolean pushed = true; > int currentPushTry=1; > for (PushResult pushResult : pushResults) { > if (pushResult.getRemoteUpdates().size() > 0) { > Collection<RemoteRefUpdate> refUpdates = > pushResult.getRemoteUpdates(); > if (refUpdates != null && refUpdates.size() > 0) { > log.warn("RefUpdates size======>>>>>>"+refUpdates.size()); > for (RemoteRefUpdate refUpdate : refUpdates) { > if (refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_OTHER_REASON || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_NODELETE || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_NONFASTFORWARD || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_REMOTE_CHANGED) { > log.info("Remote > name================>>>>>>"+refUpdate.getRemoteName()); > pushed = false; > > //pushed=pushFailed(remoteRepoUrl,pushBranch,repoFile); > log.warn("Failed to push artifacts on repo, > So called pushFailed method" + remoteRepoUrl + " due to " + > refUpdate.getStatus() + " Message " + > refUpdate.getMessage()+" Application"+repoFile); > break; > > } > } > } > } > } > int maxPushTryCount = > Integer.parseInt(AppFactoryUtil.getAppfactoryConfiguration().getFirstProperty(AppFactoryConstants.S2GIT_PUSH_RETRY_COUNT)); > long retrySleepTime = > Long.parseLong(AppFactoryUtil.getAppfactoryConfiguration().getFirstProperty(AppFactoryConstants.S2GIT_PUSH_RETRY_SLEEP_TIME)); > log.info("Max push try > count========================>>>>"+maxPushTryCount); > log.info("Retry sleep > time=============================>>>"+retrySleepTime); > synchronized (remoteRepoUrl) { > while (currentPushTry < maxPushTryCount && !pushed) { > //log.info("====================Thread name in Re > try:========"+Thread.currentThread().getName()); > Thread.sleep(retrySleepTime * currentPushTry); //Pause for > 10*retrycount seconds > pushed = pushFailed(remoteRepoUrl, pushBranch, repoFile, > currentPushTry); > currentPushTry++; > } > } > return pushed; > > > } catch (RepositoryMgtException e) { > String msg = > "Error while pushing : " + pushBranch + " due to " + > e.getMessage() + " from RepositoryMgtException"; > log.error(msg, e); > throw new RepositoryMgtException(msg, e); > } catch (GitAPIException e) { > String msg = > "Error while pushing : " + pushBranch + " due to " + > e.getMessage() + " from GitAPIException"; > log.error(msg, e); > throw new RepositoryMgtException(msg, e); > } catch (InterruptedException e) { > String msg = > "Thread is interupted due to " + > e.getMessage(); > log.error(msg, e); > return false; > } catch (AppFactoryException e) { > e.printStackTrace(); > return false; > } > > } > public boolean pushFailed(String remoteRepoUrl, String pushBranch, File > repoFile,int currentPushTry) throws RepositoryMgtException, GitAPIException { > log.warn("Re trying " + currentPushTry + " time, for " + repoFile); > boolean pushed; > log.info("Lock obtained for remote repo URL: " + remoteRepoUrl); > Git gitRepo = getGitRepository(remoteRepoUrl, repoFile); > > > gitRepo.pull().setRebase(true).setCredentialsProvider(getCredentialsProvider()).call(); > Iterable<PushResult> pushResults = gitRepo.push() > .setRemote(remoteRepoUrl) > .setRefSpecs(new RefSpec("refs/heads/" + pushBranch)) > .setCredentialsProvider(getCredentialsProvider()) > .call(); > // we need to verify if git push was successful. Here we can > check RemoteRefUpdate status is rejected or not. > pushed = true; > for (PushResult pushResult : pushResults) { > if (pushResult.getRemoteUpdates().size() > 0) { > Collection<RemoteRefUpdate> refUpdates = > pushResult.getRemoteUpdates(); > if (refUpdates != null && refUpdates.size() > 0) { > log.warn("RefUpdates size======>>>>>>" + > refUpdates.size()); > for (RemoteRefUpdate refUpdate : refUpdates) { > if (refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_OTHER_REASON || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_NODELETE || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_NONFASTFORWARD || > refUpdate.getStatus() == > RemoteRefUpdate.Status.REJECTED_REMOTE_CHANGED) { > log.info("Remote name================>>>>>>" > + refUpdate.getRemoteName()); > pushed = false; > log.warn("Failed to push artifacts on repo:" > + remoteRepoUrl + " due to " + > refUpdate.getStatus() + " Message " + > refUpdate.getMessage()+" Application"+repoFile); > break; > > } > } > } > } > } > return pushed; > > } > > > > Thanks & Regards, > S.A.Rajeevan > Software Engineer WSO2 Inc > E-Mail: [email protected] | Mobile : +94776411636 > > On Fri, Oct 9, 2015 at 12:40 PM, Aiyadurai Rajeevan <[email protected]> > wrote: > >> Hi All, >> >> I did 10 round of load test, below are the findings, >> >> Successfull out of 15 app creation requestSuccessfull out of 10 app >> creation request12101510131014101310141010991011101310 >> Since we have implemented a retry + sleep(if the push failed it will >> retry 3 time max with a sleep time, *sleeptime = 10 sec*retrycount* ) >> scenario along with the above explained approach, In the above testing the >> failures occurred at the end of last retry. So we thought of having this >> retry count and sleep time in appcatory.xml as configurable and go with >> this. Here, This sleep time will affect the failed application only and >> other applications will not be affected by this sleep time. >> >> Any views or optimal suggestions please? >> >> Thanks & Regards, >> S.A.Rajeevan >> Software Engineer WSO2 Inc >> E-Mail: [email protected] | Mobile : +94776411636 >> >> On Fri, Oct 9, 2015 at 10:58 AM, Aiyadurai Rajeevan <[email protected]> >> wrote: >> >>> Hi All, >>> >>> I have been doing a load test on WSO2 AppFactory where I'm trying to >>> create 15 apps in 15 threads with a rampup period of 10sec.(There is not if >>> we increase the ramup period) >>> >>> I was able to create ~5 apps and the rest app creation failed due to >>> s2git repo not synced for the next application push. So we implemented a >>> pull and rebase before push if its failed in the first push try, With this >>> solution we could able to create minimum of 10 application in each try. The >>> rest of the apps are failing because of below fact. >>> >>> Say 11th application being pushed(not commited), at the same time 12th >>> application get pull of the tenant repo space, So it will not have 11th >>> application's update. Now 11th push has been completed and now 12th >>> application push failing as it doesn't have the updated tenant repo space. >>> >>> So I tried to synchronized the remote repo URL and noticed the success >>> application creation went down further (6 or 7 application success). Anyhow >>> synchronizing is not a good approach as this will reduce the performance. >>> >>> Any suggestions to overcome this issue would be much appreciated. >>> >>> Thanks & Regards, >>> S.A.Rajeevan >>> Software Engineer WSO2 Inc >>> E-Mail: [email protected] | Mobile : +94776411636 >>> >> >> > -- Best Regards Samith Dassanayake Software Engineer | Cloud TG WSO2, Inc. | http://wso2.com lean. enterprise. middleware Mobile : +947 76207351 Blog : buddycode.blogspot.com
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
