NCrunch 3.2.0.3 Visual Studio 2008-2017
NCrunch 3.2.0.3 Visual Studio 2008-2017 | 28 Mb
It intelligently runs automated tests so that you don't have to, and gives you a huge amount of useful information about your tested code, such as code coverage and performance metrics, inline in your IDE while you type.
Automatic Concurrent Testing
A normal cycle of test driven development makes you stop to run your tests so often that it's just plain painful. Think about the steps you usually go through:
Write the test
Stop and run the test
Write the code under test
Stop and run tests
Refactor the code under test
Stop and run tests
Drink some coffee and repeat
NCrunch takes away all the pain and leaves a warm happy feeling behind. So you end up with:
Write the test
Write the code under test
Refactor the code under test
Be happy, drink some coffee and repeat!
Code Coverage
NCrunch collects test coverage for your code while it runs your tests.
This is shown next to your code in coloured markers showing which lines the tests touched, with marker colours indicating pass or fail status.
You can also navigate to any covering tests from any line of code, making it easy to see which tests you might impact with a change.
Full code coverage metrics are also available for your entire solution, allowing you to see where your code coverage is heavy and where it's light.
Performance Metrics
NCrunch profiles your tests during their execution to pick up the execution time of every line of code under test.
Metrics are shown inline conveniently with a tooltip, and 'hot spots' are shown with special colouring on the code coverage markers.
Inline Exception Details
The stack traces of exceptions thrown from your tests are processed by NCrunch and projected over the code coverage markers.
This makes it really easy to spot where your tests went wrong, without the information getting in your way.
No matter where you are in your source code, you'll be able to analyse problems quickly and without fuss.
Intelligent Test Execution
NCrunch tracks all sorts of interesting statistics about your tests, and it uses this information in the most intelligent way possible.
Tests that you have recently impacted with your code changes are highly prioritised for execution.
NCrunch uses a powerful weighting system designed to give you the most important feedback as fast as possible.
Distributed Processing
NCrunch can offload build and test work to other computers for processing.
Tasks are cleanly farmed out to any number of connected machines, forming grids to execute tests.
Grid servers can be shared between developers allowing teams to pool their resources.
Grids can even be scaled into the cloud to maximise testing throughput.
Distributed processing with NCrunch is highly effective, allowing concurrent execution of dozens or possibly even hundreds of tests at any one time.
Changes in v3.2.0.3
V3 contains three new major feature areas: VS2017 support, A new configuration system, and new test impact detection.
Visual Studio 2017 RC Support
V3 introduces an installer targeted to the new VS2017 RC.
NCrunch now also has support for the new version of MSTest, including its assembly-level test categories and row tests.
VS2017 saw the introduction of the first publicly available version of Microsoft's tooling for .NET Core, so integration with .NET Core is now possible and is a work in progress. At the moment I hope to have something available for this in January.
Of course, VS2017 hasn't actually been released yet, so things may still change before it hits RTM. Any breaking changes made in VS2017 will be addressed in future builds of V3.
New Configuration System
Since the first days of NCrunch, it's been acknowledged that NCrunch's configuration system has been inadequate. I've been constantly surprised at how many different ways people use the product, and this made it impossible to choose a configuration structure that would work for everyone. Some people like to share their ignored tests, while others use them as part of their individual workflow. Some people want granular control over settings at project level, while others just want big broad solution-wide settings. The list goes on.
V3 adds tremendous flexibility to the configuration system. Because flexibility also adds complexity, most of the new settings features are hidden behind a 'simple mode' option that is enabled by default. This is mostly because I'd expect new users of the software to quickly be overwhelmed by the new system and most of its options aren't very relevant when you're using NCrunch for the first time. Turning off the 'simple mode' option in the Configuration Window will open up the power of the new system. If you don't like the new system or you don't need it, you can just keep 'simple mode' on forever and NCrunch will behave in the old familiar way.
Settings As Data
Internally in the engine, NCrunch settings are now stored as data rather than static elements in the software. This makes it possible for settings to be added by modules (an important consideration for the upcoming NCrunch SDK/API), while also giving much more consistent handling of settings regardless of their level in the structure (i.e. global, solution, project, etc). All settings are now stored in XML readable form. Binary storage of settings or user preferences is now a thing of the past. The settings files themselves are designed for easy VCS merging of changes. The engine is also designed to preserve declared settings it doesn't understand, so the new format should be both forwards and backwards compatible between future versions of NCrunch.
Separation Of User/Shared Settings
One of the biggest problems with the old configuration system was that it forced everyone into storing settings in either user or shared files as determined by the settings themselves. V3 does away with this. The new Configuration Window has separate nodes for declared user ("my") and shared settings. The new 'Share tests ignored by NCrunch' and 'Share metrics excluded by NCrunch' settings control where NCrunch should share the data associated with the ignored tests and metrics, which is also stored in the settings files.
Inheritance
NCrunch now has an inheritance structure that determines which value is applicable for a setting at a given point. This makes it possible for you to define settings at a higher level than they normally sit (i.e. project build settings can be declared at solution or global level). This also has consideration for user/shared settings, so you could have a setting that defaults to one value for your team, while overriding this just for yourself.
The new configuration system introduces some big changes to NCrunch's engine modes. Previously, engine modes have been fixed contexts that could only control the behaviour of the engine in a few select areas. Now, they have the capability to override almost all global and solution-level settings within NCrunch. You can now create engine modes that override UI and performance settings, or completely change the workflow of the engine.
Here are a few ideas for engine modes that can now be created under the new system:
A mode that ramps up the number of processing threads, increases the number of pooled processes and turns on parallel execution to push the engine to its limit. This could be used to run through lots of tests quickly (AFK processing!).
A mode that uses the Custom environment variables setting to set an environment variable in every build and test process. You can then have custom code in your tests that considers this variable, implementing different testing behaviour dependent on the current engine mode.
You can change the default engine modes so that they don't just change the behaviour of the engine, but also adjust the NCrunch UI to make the mode more obvious. For example, I'm finding it quite useful to change the coverage marker shape when I'm running tests manually so that I don't forget to switch back to an automatic mode later.
Of course, the existing functionality of NCrunch V2 can still be applied by engine modes. You can still use them to target specific tests for automatic execution or metrics analysis. Even more than before, it's now possible to get a very different experience from NCrunch with just the flick of a switch.
Improved Test Impact Detection
NCrunch V3 introduces significant improvements to its detection of impacted tests.
Prior to V3, NCrunch's impact detection has been very rudimentary. Impacted tests were detected only at text-level, where any test with coverage markers on a changed source file was considered to be impacted.
By default, impact detection is now performed through an extensive IL-level comparison between changed assemblies. This means that you won't see tests considered as impacted due to comment changes anymore. The analysis is now also much better targeted, so you shouldn't see tests become impacted if they don't touch the methods you've changed. The analysis takes into consideration a wide range of edge cases to try and give a highly accurate picture of which tests are likely to be affected by changes. The goal of the new system is to make it possible to make heavier use of NCrunch's 'Run impacted tests automatically, others manually' engine mode, thereby reducing the resource consumption of NCrunch as a whole.
Impact detection data also carries across sessions, so NCrunch can figure out which tests were impacted by changes made to the solution while Visual Studio was closed. This makes it possible to use the impact detection to target tests when using the NCrunch console tool. So the 'Run impacted tests automatically, others manually' can now be used in your CI system, though this should be used with caution as no impact analysis is 100% accurate.
NCrunch will now also track which files are accessed by tests under execution. Files that are found to correspond with files transferred to the NCrunch workspace will be considered as 'covered' by tests. When these files are changed, the tests will be accordingly marked as impacted. This new behaviour is controlled using the Track file dependencies configuration setting.
NCrunch's impact detection behaviour is controlled by the newly introduced Impact detection mode setting. The old behaviour is still available as this may still have some advantages over the new mode in terms of a marginal engine performance improvement.
V3 contains three new major feature areas: VS2017 support, A new configuration system, and new test impact detection.
Visual Studio 2017 RC Support
V3 introduces an installer targeted to the new VS2017 RC.
NCrunch now also has support for the new version of MSTest, including its assembly-level test categories and row tests.
VS2017 saw the introduction of the first publicly available version of Microsoft's tooling for .NET Core, so integration with .NET Core is now possible and is a work in progress. At the moment I hope to have something available for this in January.
Of course, VS2017 hasn't actually been released yet, so things may still change before it hits RTM. Any breaking changes made in VS2017 will be addressed in future builds of V3.
New Configuration System
Since the first days of NCrunch, it's been acknowledged that NCrunch's configuration system has been inadequate. I've been constantly surprised at how many different ways people use the product, and this made it impossible to choose a configuration structure that would work for everyone. Some people like to share their ignored tests, while others use them as part of their individual workflow. Some people want granular control over settings at project level, while others just want big broad solution-wide settings. The list goes on.
V3 adds tremendous flexibility to the configuration system. Because flexibility also adds complexity, most of the new settings features are hidden behind a 'simple mode' option that is enabled by default. This is mostly because I'd expect new users of the software to quickly be overwhelmed by the new system and most of its options aren't very relevant when you're using NCrunch for the first time. Turning off the 'simple mode' option in the Configuration Window will open up the power of the new system. If you don't like the new system or you don't need it, you can just keep 'simple mode' on forever and NCrunch will behave in the old familiar way.
Settings As Data
Internally in the engine, NCrunch settings are now stored as data rather than static elements in the software. This makes it possible for settings to be added by modules (an important consideration for the upcoming NCrunch SDK/API), while also giving much more consistent handling of settings regardless of their level in the structure (i.e. global, solution, project, etc). All settings are now stored in XML readable form. Binary storage of settings or user preferences is now a thing of the past. The settings files themselves are designed for easy VCS merging of changes. The engine is also designed to preserve declared settings it doesn't understand, so the new format should be both forwards and backwards compatible between future versions of NCrunch.
Separation Of User/Shared Settings
One of the biggest problems with the old configuration system was that it forced everyone into storing settings in either user or shared files as determined by the settings themselves. V3 does away with this. The new Configuration Window has separate nodes for declared user ("my") and shared settings. The new 'Share tests ignored by NCrunch' and 'Share metrics excluded by NCrunch' settings control where NCrunch should share the data associated with the ignored tests and metrics, which is also stored in the settings files.
Inheritance
NCrunch now has an inheritance structure that determines which value is applicable for a setting at a given point. This makes it possible for you to define settings at a higher level than they normally sit (i.e. project build settings can be declared at solution or global level). This also has consideration for user/shared settings, so you could have a setting that defaults to one value for your team, while overriding this just for yourself.
The new configuration system introduces some big changes to NCrunch's engine modes. Previously, engine modes have been fixed contexts that could only control the behaviour of the engine in a few select areas. Now, they have the capability to override almost all global and solution-level settings within NCrunch. You can now create engine modes that override UI and performance settings, or completely change the workflow of the engine.
Here are a few ideas for engine modes that can now be created under the new system:
A mode that ramps up the number of processing threads, increases the number of pooled processes and turns on parallel execution to push the engine to its limit. This could be used to run through lots of tests quickly (AFK processing!).
A mode that uses the Custom environment variables setting to set an environment variable in every build and test process. You can then have custom code in your tests that considers this variable, implementing different testing behaviour dependent on the current engine mode.
You can change the default engine modes so that they don't just change the behaviour of the engine, but also adjust the NCrunch UI to make the mode more obvious. For example, I'm finding it quite useful to change the coverage marker shape when I'm running tests manually so that I don't forget to switch back to an automatic mode later.
Of course, the existing functionality of NCrunch V2 can still be applied by engine modes. You can still use them to target specific tests for automatic execution or metrics analysis. Even more than before, it's now possible to get a very different experience from NCrunch with just the flick of a switch.
Improved Test Impact Detection
NCrunch V3 introduces significant improvements to its detection of impacted tests.
Prior to V3, NCrunch's impact detection has been very rudimentary. Impacted tests were detected only at text-level, where any test with coverage markers on a changed source file was considered to be impacted.
By default, impact detection is now performed through an extensive IL-level comparison between changed assemblies. This means that you won't see tests considered as impacted due to comment changes anymore. The analysis is now also much better targeted, so you shouldn't see tests become impacted if they don't touch the methods you've changed. The analysis takes into consideration a wide range of edge cases to try and give a highly accurate picture of which tests are likely to be affected by changes. The goal of the new system is to make it possible to make heavier use of NCrunch's 'Run impacted tests automatically, others manually' engine mode, thereby reducing the resource consumption of NCrunch as a whole.
Impact detection data also carries across sessions, so NCrunch can figure out which tests were impacted by changes made to the solution while Visual Studio was closed. This makes it possible to use the impact detection to target tests when using the NCrunch console tool. So the 'Run impacted tests automatically, others manually' can now be used in your CI system, though this should be used with caution as no impact analysis is 100% accurate.
NCrunch will now also track which files are accessed by tests under execution. Files that are found to correspond with files transferred to the NCrunch workspace will be considered as 'covered' by tests. When these files are changed, the tests will be accordingly marked as impacted. This new behaviour is controlled using the Track file dependencies configuration setting.
NCrunch's impact detection behaviour is controlled by the newly introduced Impact detection mode setting. The old behaviour is still available as this may still have some advantages over the new mode in terms of a marginal engine performance improvement.
[/b]
[b]
Only for V.I.P Warning! You are not allowed to view this text.