Deadlocks

Deadlocks occur when two threads hold locks and one of them tries to grab the lock from the other thread. Imagine that thread A has a method foo() that holds lock A. foo() calls bar(), which holds lock B. If bar() also calls foo() in some circumstances, it is possible that some thread will be in foo() (holding lock A) and call into bar(). If another thread is already in bar() then the first thread will block while it waits for lock B. Meanwhile, if the second thread tries to call into foo(), it will also block while it waits for lock A. As a result, both threads will block indefinitely. This is bad.

To help prevent deadlocks, Mozilla debug builds have special automatic deadlock detection when nsAutoLock is used. This code keeps track of all lock chains, so it can determine whether any chains cross each other as in the above example. This doesn’t mean that a deadlock will occur, it just means that it could occur.

Because deadlocks are one of the most difficult problems to debug (particularly on a remote user’s machine) and it’s very hard to anticipate if a risk of deadlock will lead to an actual deadlock, we should avoid these cases by fixing any cases where a “potential deadlock” warning occurs. The patch in https://bugzilla.mozilla.org/show_bug.cgi?id=322959 is very useful because it provides stack traces that make it far easier to figure out which locks are causing the potential deadlock. When NS_TRACE_MALLOC_XXX is defined, Firefox runs much slower, so it only makes sense to turn this on (and rebuild XPCOM) when you’re debugging potential deadlocks.

The best guideline is to avoid calling into “foreign” (i.e. external to current class) code while a lock is active. Always use nsAutoLock, and enclose it in braces to isolate the code that needs the lock (e.g. protecting a local data member). If you absolutely must call into foreign code for some reason, make sure that that code doesn’t use any locks. If for some reason even this isn’t possible, definitely make sure that the foreign code doesn’t call back into code that activates the original lock while it has a lock active.