Hi there, running Continua 1.8.0.243, our unit tests (MSTest) sometimes fail, with an error message never seen before for other Projects/Configurations. Does this sound familiar? Looking for a solution here, or otherwise, could this be a defect? (Quick scanning release notes of 1.8.1 does not seem to resolve something in this direction.)
An error occurred while parsing unit test results output.
Exception: SteppingException Message: An error occurred while parsing report file: C:\CI_WS\Ws\17007\Source\TCXPlatform\01.Sources\UnitTest\TcEngineer\2016-09-19_07_29_34.trx Stack Trace: at Continua.Modules.Builds.Actions.UnitTest.UnitTestParserHelperBase.Parse(IBuildSteppingContext context) at Continua.Modules.Builds.Actions.UnitTestAction.OnInterpretResult(IBuildSteppingContext context, Int32 exitCode, String consoleOutput) Exception: Exception Message: Report File Name: C:\CI_WS\Ws\17007\Source\TCXPlatform\01.Sources\UnitTest\TcEngineer\2016-09-19_07_29_34.trx does not exist Stack Trace: at Continua.Modules.Builds.Actions.UnitTest.MSTestParserHelper.ParseReport(IBuildSteppingContext context) at Continua.Modules.Builds.Actions.UnitTest.UnitTestParserHelperBase.Parse(IBuildSteppingContext context)
This error message is reported if MSTest does not produce a results file. The file name for the results file is passed to MSTest with the command line parameter /resultsfile. Can you look at the command line in the build log and check that the /resultsfile parameter matches the file name in the error message. Also check if there are any prior errors reported in the MSTest output in the build log.
further investigations tells that, when such an error occurs, all unit tests could not be found:
[code]Loading Source/Repository/Project.testsettings…Loading C:\CI_AWS\Ws\17391\Source\Repository\Project.vsmdi…Starting execution…Test TestLegacyGetIOConnections6 cannot be found.Test CreateCANOpenModuleListEntryTest2 cannot be found.Test TestAssignDefinition3 cannot be found…[/code]
We could not find any other MSTest errors, but we’re gonna check if we can get some additional info with verbose logging (if such a thing exists within MSTest).
Meanwhile, just checking with you if this sounds familiar. The project we’re building was previously being build with Finalbuilder Server 6/7 and has run without any similar unit test errors for years. Basically we transferred the msbuild and mstest action from FB to Continua, fetching source code from the same repository but underneath the infra has of course changed (other build server running build/test, etc.).
It looks like the test assemblies are not being copied to the output directory when running the build. This could be because they are not set to “copy to local” in the project? Another possibility is that the path for the assembly resolution directory (or another directory) in your testsettings file is incorrect.
thanks. You’re close: - We’re building the test solution directly into our source tree (e.g. $Source.Repository$\Solution\bin) - We’re building the solution + test solution in Stage 1, but invoke MSTest in Stage 2. - Sometimes, the build output of the test solution is not available in Stage 2, hence, makes sense it cannot fint the tests and then does not generate a report
Turns out that the test always succeeds when the first stage and second stage (Build and unittest) are performed by the same agent. Sometimes, that agent is allocated to perform another build (after it completed the first stage), and the unit test stage is then performed by a different agent. In such a case, this second agent does not have a sourcetree with the build output $Source.Repository$ is “clean”, and then cannot find the tests.
As far as I know Coninua syncs the $Workspace$\Output dir by design (stage rules), but will never preserve the $Source$ dir. At least it is not transferred between agents, by designs, which make totally sence. But what seems to happen is that the $Source$ dir is being preserved between stages for that specific Agent, which is new for me. I am aware of the Agent Cleanup rules (last step in the configuration design) and that by default it is being cleared when the build completed. It in fact says build, however, I always interpreted this as - $Source$ is being kept during a stage, but once the OUtput dir has been synced with the server, the agent will cleanup directly.
Could you explain the designed mechanism? It’s not a big deal anyway, but just want to learn how it works, so we can learn how to use it properly.
In my opinion, the build should always have failed and the source dir should always be cleared when a new stage starts. As of how it appears to work right now, for builds relying on output that is incorrectly “persisted” in the sources directory, its faulty design is being masked until the second stage is run on a different agent, which may happen months or years later when for instance you scale up your environment or when particular agents are used more extensively.
Great that you’ve worked out what is going on with the unit tests.
Yes, you’re right - the workspace does not get cleaned up at the end of each stage only at the end of each build. So if the same agent is used for a stage then the previous stages files will be available in the workspace.
I don’t know what the original reasons were for this design. If you only have one agent or restrict the build to one agent using agent requirements or shared resources, this could be a bonus as it means you don’t need to transfer files from the agent and to the server and back between stages. I agreed however that it introduces inconsistant behaviour and should not really be the default.
The workspace rules currently support the >> operator which cleans up the destination directory before copying the files. According the help text on the Stage Options dialog you can use the >> operator with repository rules. But this is not true - the >> operator was never implemented for repository rules.
We’ll add this operator so that there is more control over this process and we’re thinking that we’ll also make this the default for all server to agent rules in new stages.