Refer to Microsoft's knowledge base article: KB 811833 - The effects of enabling the "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing...". Excerpt:
Microsoft .NET Framework applications such as Microsoft ASP.NET only allow for using algorithm implementations that are certified by NIST to be FIPS 140 compliant. Specifically, the only cryptographic algorithm classes that can be instantiated are those that implement FIPS-compliant algorithms. The names of these classes end in"CryptoServiceProvider" or "Cng". Any attempt to create an instance of other cryptographic algorithm classes, such as classes with names ending in"Managed", cause an InvalidOperationException exception to occur. Additionally, any attempt to create an instance of a cryptographic algorithm that is not FIPS compliant, such as MD5, also causes an InvalidOperationException exception.
By default, ASP.NET wants to use algorithms that are incompatible with FIPS compliance. To solve your problem, you'll need to change ASP.NET configuration to use a compatible algorithm instead. Refer to machineKey Element on how to configure another algorithm.
This MSDN forum post suggests 3DES ought to be compatible... although the machineKey documentation listed previously does say about AES (the default in ASP.NET 4.0) "This algorithm is compliant with the United States Federal Information Processing Standards (FIPS)".
I've also heard that having debug="true" in your web.config may cause the error.