I noticed that you have Warnings and Error tracking, and you guys are probably already ruminating over something along these lines or maybe even working towards it.
I'd like some way to be able to drill down more into the failure data, over time, to be able to look for various trends. This will mean expanding the data set and providing ways for projects/users to compartmentalize the streams of data being processed. Specifically the ability to annotate targets and configurations, incremental vs complete builds, etc.
For example, a solution might have 3 targets (presentationLibrary, dataLibrary, application) and 4 configurations (debug, profiling, testing and public).
Incremental builds would be accompanied by a contributor identity and a fileset that denotes which files were changed. This data could be used to determine that specific files are regularly associated with failures and therefore need review. Or, nefariously, a managed could determine that every time David Glenn checks something in, the build is broken for 3 hours afterwards and decide it's time for his annual review...
And perhaps, maybe, a way to make the build process Issue/Ticket number aware. Being able to see how many build failures relate to Ticket #2217 might help you determine that it's going to run long or that it's badly specc'd or that you need to assign it to a different coder.
Being able to determine which targets or configurations are regularly experiencing failures would also be useful because it might highlight a specific API or featureset that is problematic and needs a revisit. Better to spend a day fixing "uiDisplayCustomerInformation" than spending 15 minutes every time one of the other coders tries to add some new data.
If you could also keep the errors/warnings it might be possible to determine recurring issues like - say - a bad macro or that nobody understands what "enableDontDisableVerticalSyncWhenSet" means, especially since it takes a range of values between 10 and 90 and everyone wants to assign it TRUE or FALSE.
Obviously this is beginning to leave the domain of 'Build" and enter the domain of Continuous Integration/Code Management, with natural follow ons like expansion of the "Foundation" concept to other bug track systems (e.g trac/bugzilla).
That's an awful lot for one post, I have a hunch you guys are already contemplating "FinalIntegrator" (quick, TM it :) ) but incase you're wondering "where do we start" I thought I'd hurl that out there.
- Oliver