"waiting for repository to have its cache updated" going on forever

We had this script working a week ago, but suddenly we found something was wrong. The enclosed image is from the Timeline view. The log-text is long, and repeatedly updated with the same contents. We suspect the line “Waiting for repository ‘Hjalp’ to have its cache updated” points to the problem. We have tried all sorts of solutions, including resetting the repository. Nothing seems to help. Could you give us a clue?

/Anders

Hi Anders

What build of Continua are you using, and what is hte repository type? We did make some improvements to the locking around repository caches recently which sped up builds starting (removed some locking that was overkill).

Hi Vincent,

I just downloaded and installed last version (1.5.1.117) yesterday, but it didn’t help. This is a SVN repository.

/Anders

Hi Anders,

We are not experiencing this issue here so will need some further information to diagnose.

Are there any errors in the event log? Can you enable debug logging, restart the Continua Server sevice, run this configuration again, then send us a copy of the debug log?

If you could also download a diagnostics report using the link at the top of the Events log page, this help us to further troubleshoot this issue.



I’m having the same issue. Where it used to work, it’s now just checking for the repository and after 10 minutes or so I break it off.

ContinuaCI-diagnostics-report-2015-04-07_13-22-31.zip (16.131 KB)

Hi Dennis

I think we need to see a debug log to figure out what is going on. Please enable debug logging : http://wiki.finalbuilder.com/display/continua/Debug+Logging

Restart the service and start a build. When it get’s stuck checking the repository, stop the service and send us the log (email support ).

Hi,

This issue is still present in Continua 1.6.0.132. We hit it twice today, but we don’t have debugging enabled, unfortunately. In both cases there is a ton of messages in timeline (10k+) as seen on initial screenshot and the only way to stop this is too kill a hanging hg process on server, which blocks related repo. The following error can be found in log and is likely related:

"Build Queue

Error updating cache for repository SVNDev to changeset revision 36926. Details: Running C:\Program Files\VSoft Technologies\ContinuaCI\Server\hg\hg.exe with arguments “commit -m “Automated commit by Continua CI from ‘SVNDev’ repository for revision ‘36926’. Comment: Automated tag for 5.01.0102.” --addremove -X .svn -X **.svn -R C:\Continua\Rc\d199e2b3 --config ui.ignore= --config ui.username=Continua --noninteractive” on server failed with return code 255 and error output: "skipping unreadable ignore file ‘C:\Continua\Rc\d199e2b3’: Permission denied
waiting for lock on repository C:\Continua\Rc\d199e2b3 held by ':'
abort: repository C:\Continua\Rc\d199e2b3: timed out waiting for lock held by :"

Regards,
Ilya

We are still looking into this. We have not been able to reproduce the problem here (on multiple test servers), and we didn’t receive a debug log from Dennis so it’s difficult to pinpoint what the cause is. I’ve been trawling through the code to see if I could find the cause but nothing so far.

Are you able to make it happen at will, or at least after a few attempts? If so then it is definately worth enabling debug logging (don’t forget to turn it off once you have seen the error).

Hi Vincent,

The issue seems to be gone for now and I can’t reproduce it at will. But this is not the first time that I encounter it while adding new configuration.
In this case we did add a new remote agent that day (and it takes about an hour to clone a repo for the first run), so I believe it has something to do with repo being locked for cloning while server tries to update main repo with latest changes.

Thanks,
Ilya