A 32-bit signed-int constant (negative) is returned from a function as unsigned-int, and assigned to a signed-int variable. This signed variable is then compared to the original signed constant. In x64 release builds the variable is zero extended and the constant is sign extended causing them to not match.
Both numbers should be treated as 32-bit values, not 64-bit values. Sign or zero extending an int to 64-bits is only legal if it does not affect the results. Therefore this is a code-gen bug. I'm not sure why it doesn't happen more often as the circumstances seem quite common and the repro is quite stable.