This reverts commit 6939a5aca7. Daniel Vetter supplied a set of fixes for all the module unload bugs he could trigger on his machines, so let the fun recommence!
6939a5aca7