Hi,
we are using submodules with our Git repositories. We use tags in the submodule to identify the commit, that is used for the build. Sometimes it happens that a commit to the submodule is pushed to the server, but the tag for it gets created and pushed thereafter. That change is not reflected in the file “_CI_ChangesetDetails.xml”. The file still contains the SHA1 hash, rather than the tag.
Thomas
Hi Thomas,
It’s so long since we looked into git submodules, so we’ll need some time to set up an suitable test environment and work through this. From an initial look through the code, I can see that we get the current submodule details just after a new commit is detected on the main repository. We never subsequently fetch any new tags from the submodule. We look into this further tomorrow.
Hi Thomas,
Can you describe the order of your commits to the submodule and main repository in more detail - maybe with a list of example git command lines? Continua CI, will only pick up changes to the submodule when they have been committed to the main repository. If you then tag the submodule, you would need to make a new commit on the main repository to pick up the tag.
Hi Dave,
for example:
The super project references the submodule by a hash. The tag is just an additional information. Thus there is nothing to commit to the super project. It makes no sense to produce an artifical change to the super project to produce a commit that will eventually trigger Continua to update its caches.
A workaround is to reset the repository. Actually this demonstrates that it is an Continua issue.
Since there is no change in the super project, Continua does not detect the additional tag of the submodule. Effectively it is the information written to the file “_CI_ChangesetDetails,.xml” which does not include the tag reference. Everything else works correctly. Thus checking for submodules explicitely is required only when the option is checked to write the details file.
Hi Thomas,
The issue is with timing. When Continua CI detects a new git commit, it then checks out the commit to the cache folder, updates any submodules, creates the changeset details file using “git submodule status” to get the submodule details, and then commits everything to the repository cache. At this point, a build can be triggered using the files in the repository cache, before any tag is added to the commit in the submodule. The Continua CI changeset is basically a representation of the state of the repository commit at the time it is stored in the repository cache.
We could wait for a while, before committing to the repository cache, to see if any tags are added to the commit - but how long should we wait? We don’t want to hold up the build. Alternatively, we could update an existing repository cache entry (and changeset) when we detect a tag change, but it may be too late - the build may have already been triggered.
If “Tag Changes” is set to “Detect And List As New Changeset”, Continua CI currently creates a new entry in the repository cache and a new virtual changeset in the database when a tag change is detected on the main repository. A build can then be triggered on the “tag change” changeset. We could do a similar thing with new tags detected on submodules. We’ll look into this option, but don’t expect anything until after the holidays.
Hi Dave,
your suggestion is interesting. I want to add some remarks though. As i understand, a tag change in a submodule would trigger any configuration for which repositories exist, that use that submodule. Therefore it is important, that the change is relevant only, if the submodules HASH is referenced by the detected tag.
Also there is an implication with the UI. How is it is possible to recognize, that a virtual changeset was created (and thus triggering a build) by a tag change of a submodule?
Please note, that we have configured all of our repositories to poll “never”, but be notified by TFS Webhooks.
Thomas
Yes, of course, we would intend tags added to submodules to work the same way as tags added to the main repository work currently.
We don’t currently, display “tag change” changesets differently - you can usually tell by the commit message. We will be looking into adding an indicator of some sort to them in due course.
Continua CI would need to know when to query for a change on the submodule. So you would need to set up a “code pushed” web hook on the submodule repository for each main repository in Continua CI which references that submodule.
Hi Dave,
my understanding is that this issue is fixed with version 1.9.1.567. However it’s not working for me. My xml file does not reflect the tag change. I also tried to manually refresh Continua’s repo cache.
Any ideas?
Hi Thomas,
We added a function to run the git fetch tags --prune command in each submodule folder, before we run git submodule status --recursive to get the submodule details. This is to ensure that the latest submodule tags are retrieved when a commit is detected on the main repository.
We haven’t yet added any code to deal with the scenario where a tag is added to a existing commit after the cache as been updated - this would involve a significant amount of work.
When you reset the repository, the latest changeset for each branch should be updated. Additionally, if the “Load any older changesets which have been tagged” option is ticked, the tagged changesets should be updated. These changesets should include the latest tag information for each submodule at the time of the update.
If this is not the case, then we would be interested in seeing a debug log recorded during the time period when the repository is reset, with details of changeset that is not being updated as expected.