Hello,
We are trying out ContinuaCI in one of our projects and our experience was good until we hit a strange problem while creating another configuration based on branch-aware svn repository. So far we had 2 working configs with their own repositories: one builds from trunk, and another allows to run a build from a branch (thus it monitors branches with default /[^/]+/ pattern).
Now, we’ve trying to create a config that will monitor tags (with a branch pattern) and will allow to select one and run a build from it. We’ve added a new branch-aware repo, that monitor top-level tags with /[^/]+/ pattern, but the problem is that it’s unable to initialize properly. It take about 30 mins to “Gather changesets” and then turns to “Ready” state, but configuration is not showing any branches to select from.
It seems to be an internal OutOfMemory error, as it shown in EventLog and same info is in debug log:
Message: Array dimensions exceeded supported range.
Stack Trace: at System.String.SplitInternal(Char[] separator, Int32 count, StringSplitOptions options)
at Continua.Modules.Builds.Repositories.Subversion.SVNList(String workingFolder, String relativeUrl, Boolean recursive, Boolean useCredentials, Boolean allowRetry, Boolean retrying)
at Continua.Modules.Builds.Repositories.Subversion.IdentifyBranches(String workingFolder) …>
From what I understand there is recursive svn list command that will gather all filenames from pattern path, and there are literally millions of those in out tag subfolder, but less that 50 actual ‘tags’ here. This is 1.5.1.49 win64 server on win server with 8GB ram and 4,5Gb free physical memory was free right before update task was started. The question is: are we configuring that pattern correct? And is there a way to configure top-level branch pattern without spending 30 mins on each update?
Thanks you,
DNS
Hi,
Thank you for reporting this bug. There's a couple of issues here:
Firstly, the .Net Split function is running out of memory due to the size of the string. We've now replaced this with a custom function which does not have this issue.
Secondly, it takes a long time to run svn list with the recursive option on repositories with a very large number of files. You've not provided full details of your repository settings so it's not clear what you are setting as your Branches Path. Have you set it to "/" instead of "/branches/" so that you can include both /branches/* and /tags/* as branches? If so ,we recommend that you use the settings on the Tags tabs for this instead - this will still allow you to build on tags.
If this is not the case and it is still taking a long time when using "/branches/" as the Branch Path, then the problem here is that our branch pattern is designed to allow matching branches at any level e.g. /branches/dev/version1. We need to check the pattern against every folder to ensure we get all possible matches. Unfortunately there is not a svn command to list directories only so we need to get all files too, hence the long list. Note however this is only a problem when initialising the repository, any additional branches are picked up from the list of modified folders with each new changeset.
As most people will only use top-level branches folders e.g. /branches/dev but not /branches/dev/version1, we have now added a new "Match branch pattern against top-level folders only" option for Subversion repositories. This will switch off the recursive option when running svn list, making this part of the initialisation a lot faster.
We've uploaded a new version with these updates to the Continua CI download page
Hi,
Thank you for fixing this issue so fast! In our case we have a pretty large repo: tags subpath has millions of files, but at the same time, we only use top-level tags, so non-recursive list helps a lot.
Regards,
DNS