Running in an Azure Worker Role, with mulitple thread access to the cache writes, the MemoryCache fails to evict cache items when memory size limits are reached. If the worker role routine that uses the cache is called into from a WPF test harness, and the WPF Garbage Collection has concurrent GC turned off, then the evictions fire. If concurrent GC is on, evictions don't fire.
Interestingly, running the worker role directly using dev fabric, setting the GC concurrency has no effect. However, a periodic GC.Collect() and GC.WaitForPendingFinalizers() will trigger the evictions.
Please see the StackOverflow discussion for further details: http://stackoverflow.com/questions/5655439/net-4-memorycache-leaks-with-concurrent-garbage-collection