This is a topic I’ve wanted to better understand for a while and never had time to investigate. Also it has been in my backlog to write as a post for a weeks now, and I finally made to writing it.
Some of the questions many developers ask daily are – Why is my Visual Studio taking so much memory? Why did I get Out Of Memory exception? Task manager shows I have enough free memory.
A developer is normally concerned with these questions when they are running on 32bit OS. It is even likelier problem when you are using lots of plugins/additions to your Visual Studio or solutions with many projects or big projects.
The goal of this post is to explain the topic to some degree (I am not a OS/memory expert) and offer a tool and guidance to analyzing why your Visual Studio is eating as much memory as it is. In the end I will offer some recommendations that could help you and that have helped me.
Also note that 64bit OS changes the game a lot and I’ve not heard memory/Out Of Memory problems from a 64bit development environment.
So what's the problem?
Well, the real problem is that Visual Studio (devenv.exe) loads a huge amount binaries (VS components, extensions, user dll’s built by the IDE, …) into its virtual address space. Back when Visual Studio was created such components were smaller and there weren't so many (and although we had less memory and needed less memory, the 32bit OS provided us with a 2GB virtual address space), so the solution of loading everything into the same process virtual address space made sense. Over time however the amount of components, IDE binaries, etc has increased and the size of those components has increased, and developers running on 32bit environment are experiencing more and more out of memory or performance degradation issues. With the current model, just too much is loaded into one process address space.
The following picture is a partial from a tool called VMMap (intro follows) and the purple is the color of image type items in the devenv address space ordered by the size. This is only a fraction of the list, however you may notice this is already consuming > 150MB address space.
The tool and how to analyze?
A month ago I was experiencing a growing frequency of Out Of Memory exceptions and I really wanted to understand why. What was causing it and why suddenly VS ate more than 1GB of memory. The tool is from SysInternals (they have a bunch of clever tools) and is called VMMap. It is a clever tool that gives you a picture of what is taking up the memory in a running process.
Description from VMMap help:
VMMap is a process virtual and physical memory analysis utility. It shows a breakdown of a process's committed virtual memory types as well as the amount of physical memory (working set) assigned by the operating system to those types. Besides graphical representations of memory usage, VMMap also shows summary information and a detailed process memory map. Powerful filtering, refresh and snapshot comparison capabilities allow you to identify the sources of process memory usage and the memory cost of application features.
The tool gives you a quick overview and detail grid of each item in the address space. The following is an overview of my biggest memory usage with no solution loaded but only VS and addins such as Resharper, VSCommands, Productivity Tools, TFS Power Tools (so basically a new VS opened).
The key things I read from this are (and they are marked with red boxes):
- Committed – the total usage of virtual memory by process by type (described with color scheme).
- Private Bytes – committed private virtual memory (backed by paging file and charged against system commit limit)
- Working Set – committed virtual memory that is in physical memory and owned by process
- What is the type of biggest consumer of memory (virtual, private, working set)? - Image type or “.exe”/”.dll”
- How much free address space the process has.
Additional to this, you can see the growth or decrease in the process memory usage by using the devenv tool a bit and then refreshing the data in VMMap. VMMap will keep statistical information and offer a graph the stats.
Please note that the image above doesn’t match the initial overview listing. Its taken from a devenv run where I had been working and refreshed the VMMap regularily.
One approach for analyzing your Visual Studio
The following would be a good way to compare a Visual Studio where all extensions have been loaded to a Visual Studio where no extensions have been loaded.
1. Run Visual Studio in SafeMode (no extensions loaded) and see what is the memory consumption then.
2. Run Visual Studio in normal mode and see what is the memory consumption then.
Recommendations to improve Out Of Memory Exceptions on 32bit OS and VS
1. Disable and do not use so many extensions. (im just putting this out there, im sure most add-ins that have been installed are actually valuable and required for you)
2. Sometimes it is not viable to just disable extensions (e.g. they offer critical productivity). Tools such as Resharper or CodeRush that tend to be more memory extensive, also have ways to optimize where the tool is being used, so make sure you configure the extensions to work optimally for you solutions/projects.
3. Create several solution files and distribute the projects between those solution files to support the team to work with the source code that they normally want to change together. Don’t create and work with one huge solution file which has all the 10…N projects loaded into it and hope that Visual Studio will work with it. You will optimize the performance by limiting the amount of projects/files loaded at once into Visual Studio.
One more option to optimize the work here is to unload the projects that you do not want to change, restart and the next time you open solution Visual Studio should remember those and not load the projects.
4. The fourth option is to increase the amount of virtual memory a process running in user mode can have. By default 32bit Windows OS limits it to 2GB and the other 2GB is reserved for System. However, you can modify this and have 3GB for user and 1GB for system. Please note that if you have lots of services/process running under system, those might starve now!
However in my development machine and in many of my colleagues machines we have applied this change and it has had a positive effect of being able to work with solution files that use to throw Out Of Memory exception before. And we have not notices any system processes starving. In reality on a development machine, there shouldn’t really be so many system services that would require more than 1GB address space.
To increase user mode virtual memory address space to 3GB In Windows Vista/Windows 7 run the following command in a CMD that is running in admin mode:
- bcdedit /set IncreaseUserVa 3072
3072 is the maximum value. You can set it to less that 3072 as well, e.g. 2560 (2,5GB).
You will need to restart the computer for the change to take effect.
You can revert the change using the following command:
- bcdedit /deletevalue IncreaseUserVa
You will need to restart the computer for the change to take effect.
In Windows XP, you can do this with the /3GB switch. More information on following MSDN article.
5. Migrate to 64bit OS (again im just putting this out there). This is not solving the problem, that VS is using lots of memory. That can be better affected by the previous points (solution, project file modifications, addins, extensions etc)
I'm finding two main values for myself from this tool.
Firstly, I can see what is consuming all the virtual address space in the process and based on that possibly I can resolve by not using some add-ins. At least I know what is causing the issue – pure usage, or too many add-ins or something else. The question is – why am I running out of memory or why am I using so much memory.
Secondly (and not so related to Visual Studio), I can image using VMMap tool to analyze my own product services and applications while they are running and analyzing what is consuming all the memory. This is another tool in the arsenal. There are others profilers that offer also information about memory usage and consumption in application, but most of them don’t show you e.g. how much binaries take compared to heap.
In the end …
In the end I will leave you with a picture from my development environment after I applied the change to increase user virtual address space to 3GB.
What do you see from this? Do you read and notice the effect of the change?
I would also like to thank Nathan Halstead from Microsoft Visual Studio Platform team for answering my questions and providing me valuable information that helped me write this post.