we are observing a high load of more than 60% CPU usage on the Agent Service. There’s a regular break when the cpu usage goes down to approx. 2-3% for about five seconds, before it will go up again. There are no active builds running.
We noticed build times going up about 2-3 weeks ago. The server got restarted, trying to solve the issue. As we usually install the latest version of Continua it could be version 1.9.2.523 introducing the problems to us. If not caused by any other issue.
The processes which run on the agent when no build is running are property collection, workspace cleanup, and repository synchronisation.
We did recently change the repository synchronisation to start as soon as a changeset is detected rather than waiting for the build to start executing, but this should mostly be running under a Mercurial ‘hg’ process.
If you have recently changed any of your cleanup settings then there may be a backlog of workspace cleanup running. If so, then you should see plenty of disk activity too.
Nothing has changed with property collection for a long while, but something may have changed on the agent machine which means there is an additional property that it is having trouble collecting.
We’re only seeing an average of less than 0.1 CPU cycles on our agent services, so we’ll need some additional information to diagnose this.
Can you enable debug logging on the agent? After restarting the service, wait until you see the high CPU usage, leave running for a few minutes, then send the debug log to support at finalbuilder.com. Ensure that you note the time that the high CPU usage started, so we can match this against the corresponding log entries.
Version 1.9.2.554 includes several improvements to agent CPU usage during property collection. Please let us know whether the fixes the issue you are seeing.
thanks for the latest update. I confirm cpu usage went down. The process will remain silent for approximately 20-25 seconds before it goes up to 45% for about 4 seconds. That is much of an improvement. Still our buildserver is extremely slow. I’ve not been able to figure out yet, what is going on here. But this (last) issue does not seem to be related to Continua.
However, i had an issue with the agent service. After installing Delphi 10.4.2 a build would not start, because of the following situation:
Stage awaiting agent
No agents are currently available to execute the stage.
computer: Initial agent property collection is not complete
After restarting the agent service it did start though. I have never seen such an issue before.
It’s normal to see an “initial agent property collection is not complete” message for 30 seconds or even up to a minute, if a build is started, just after an agent has started, but either before the agent has registered with the server or finished collecting properties.
How long did you wait before restarting the agent service? If longer than a minute, then check if there were any errors in the Continua CI event log or the Windows event log on the agent.
After another report of the “initial agent property collection is not complete” issue we were able to reproduce. The issue was happening after restarting the server service and required that the agent service was restarted, or a property was changed on the agent, before the build could continue. The next version will include a fix.
glad you were able to reproduce the issue. As for our slow buildserver…it’s gone. It runs at normal speed again. Though, i can not tell why. Maybe that property collection kept the server busy initially. As always, thanks a lot for your support.
