It won’t be the first, and for sure not the last. Redgate tool belt has a set of tools which come in handy when your at the edge of giving up. I have used the database refactor tool, to make nightmares go away, and schema migrations horrors are in the past. Have read of memory leaks on blogs but had yet faced the challenge of tracking one.
Over the last month, I have seen my windows service which starts and runs threads to handle different jobs increase in memory till 900MB in a period of two weeks. Restarting the service fixed the issue but i had failed to find the reason why.
I should say, I tried a couple of solutions including live debugging on the production server and my dev machine. Each day I tried a “trick” but zilch. Until it hit me, Redgate has a performance and memory profiler. I last used Redgate Performance profiler to hunt down beasts ages ago, in any case the tools are just click, wait, results and aaaaaahhhh ok, fix that. Performance profiler didn’t get me much. Ran the memory profiler for 3 hours taking snapshots every 30 minutes. Luckily by just looking at the results it was obvious who the culprit was. Googled LocalDataStoreElement, and the results where in my favor. In the code I was storing thread specific data using Thread.AllocateDataSlot etc, and from
http://support.microsoft.com/kb/2540745, the issue occurs because the Thread.AllocateDataSlot method in the .NET Framework 4 creates a memory leak. Sigh!!!. Now how to fix it? Below I quote from the microsoft site:-
A supported hotfix is now available from Microsoft. However, it is intended to correct only the problem that this article describes. Apply it only to systems that are experiencing this specific problem.
To resolve this problem, contact Microsoft Customer Support Services to obtain the hotfix. For a complete list of Microsoft Customer Support Services telephone numbers and information about support costs, visit the following Microsoft website:
Note In special cases, charges that are ordinarily incurred for support calls may be canceled if a Microsoft Support Professional determines that a specific update will resolve your problem. The usual support costs will apply to additional support questions and issues that do not qualify for the specific update in question.
Great that I have the code and I can apply the recommended fix for switching from Thread.AllocateDataSlot to ThreadLocal<T>. For minimal changes to code, I made use of ThreadLocal<IDictionary<string, object>>. I do appreciate the memory profiler tool for working the charms, if it wasn’t for the scary price tag, I would purchase the tool after the trial ends.