Home Dashboard Directory Help
Search

Unit test with Assert.AreEqual(2.3, double.NaN, 0.1) passes by omjbe


Status: 

Active


4
0
Sign in
to vote
Type: Bug
ID: 762286
Opened: 9/12/2012 2:04:24 AM
Access Restriction: Public
0
Workaround(s)
view
0
User(s) can reproduce this bug

Description

A unit test with Assert.AreEqual(2.3, double.NaN, 0.1) passes but it should fail because 2.3 is not equal to double.NaN.

I believe that it should be handled by the AreEqual method because the behaviour is not consistent:
1. Assert.AreEqual(2.3, double.NaN) // => fails
2. Assert.AreEqual(2.3, double.NaN, 0.1) // => pass
Details
Sign in to post a comment.
Posted by omjbe on 6/27/2014 at 1:04 AM
Thank you for your suggestions. The described workaround helps.

Unfortunately, this workaround is not practicable in my situation. In our company multiple development teams are affected by this bug. We are not allowed to manipulate our development machines this way.

We have still a lot VS2010 projects with unit tests and we have to guarantee that they continue to run on all our development machines.

Would it be possible to provide a fix in VS2013 that allows to run it “side-by-side” with VS2010 without these side-effects?
Posted by Microsoft on 6/27/2014 at 12:35 AM
Hi omjbe,

Can you try this:
1) Back up this DLL somewhere (just in case in you need it - so that you can GAC this DLL again):
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.UnitTestFramework\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll        

2) After backing this up, Delete this DLL from the above location.

3) Then, your project should pick up the only existing UnitTestFramework DLL of 2013.
Even in 2010, it should pick up the 2013 one. you can check whether this works for you.
Posted by omjbe on 6/25/2014 at 2:44 AM
How to reproduce: I believe it is important to install VS2010 first on a clean PC and then install VS2013.
Posted by omjbe on 6/25/2014 at 2:41 AM
Thank you for the ideas. Unfortunately, none of them helped.
1. A repair of VS2013 did not help. However, my colleagues have the same issue on their PCs too.
2. I have tried out the csproj snippet but it did not help.

VS2013 loads the UnitTestFramework 10.0.0.0 from VS2010 instead the one from VS2013. Here is the assembly that the CLR loads for this unit test:

Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.UnitTestFramework\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll        
Yes    
No
Skipped loading symbols.        
42    
10.00.30319.1    
2010-03-18 13:27    
5A620000-5A653000    
[5148] vstest.executionengine.x86.exe    
[2] UnitTestAdapter: Running test
Posted by Microsoft on 6/25/2014 at 1:17 AM
Hi omjbe,

Can you please try this workaround:

In your VS 2013 Unit Test Project, open the "CSPROJ" file and edit it manually. In there, you will see the reference section where you will see these lines:

    <When Condition="('$(VisualStudioVersion)' == '10.0' or '$(VisualStudioVersion)' == '') and '$(TargetFrameworkVersion)' == 'v3.5'">
     <ItemGroup>
        <Reference Include="Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" />
     </ItemGroup>
    </When>
    <Otherwise>
     <ItemGroup>
        <Reference Include="Microsoft.VisualStudio.QualityTools.UnitTestFramework" />
     </ItemGroup>
    </Otherwise>

As you can see, "When" section corresponds to VS 2010 whereas the else section is for everything else. Due to version agnostic reference of other section, may be - the project is picking up 10.1 version ( of VS 2010 ) incorrectly. Please change this reference in "Otherwise" Section to "10.0" i.e. fully qualified name. So, your csproj should look like this:
    <When Condition="('$(VisualStudioVersion)' == '10.0' or '$(VisualStudioVersion)' == '') and '$(TargetFrameworkVersion)' == 'v3.5'">
     <ItemGroup>
        <Reference Include="Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" />
     </ItemGroup>
    </When>
    <Otherwise>
     <ItemGroup>
        <Reference Include="Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL" />
     </ItemGroup>
     </ItemGroup>
    </Otherwise>

Please try this and tell us if this helped.
Posted by Microsoft on 6/24/2014 at 11:12 AM
Hi,

We've tried reproducing this even with a side by side setup and still are not able to, we humbly recommend you try repairing your VS 2013-Update2 installation and try verifying if that helps solve this issue.
Else please do revert back to us for further assistance.

Thanks,
Posted by omjbe on 6/18/2014 at 4:13 AM
I have changed the state of this bug back to open because the bug is not entirely fixed (see comments below). Why have you closed this bug again?
Posted by omjbe on 6/1/2014 at 10:11 PM
Why do you change the field "Resolution" from "Not Reproducible" to "Fixed"? It is not fixed in a way that it works when VS2010 is installed in parallel to VS2013.
Posted by omjbe on 4/14/2014 at 2:10 AM
Unfortunately, I have not found a workaround. Since the Visual Studio versions 2010, 2012 and 2013 comes with different UnitTestFramework.dll assemblies that all have the same version number 10.0.0.0, I am not able to refer a specific one. It seems to be random (or some internal CLR logic) which one will be loaded if different Visual Studio versions are installed.
Posted by omjbe on 4/14/2014 at 1:26 AM
I have found the reason. On my PC I have installed VS 2010 and VS 2013. The runtime loads the UnitTestFramework.dll from Visual Studio 2010 although the unit test project was created with VS 2013 and targeted for .NET 4.5.1:

Microsoft.VisualStudio.QualityTools.UnitTestFramework.dllC:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.UnitTestFramework\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll         Yes    No    Skipped loading symbols.        42    10.00.30319.1    2010-03-18 13:27    5A620000-5A653000    [5148] vstest.executionengine.x86.exe    [2] UnitTestAdapter: Running test

In my opinion this behavior is dangerous. This create errors (like this bug) which are very hard to track down.
Posted by Microsoft on 4/13/2014 at 10:25 PM
This actually show up as failed for me as expected (attaching a screenshot). My colleague will help you investigate this further.

Thanks,
Vinay
Posted by omjbe on 4/1/2014 at 8:38 AM
I have attached a unit test project which shows this issue. It was created with VS 2013 Update 1.
Posted by Microsoft on 4/1/2014 at 2:23 AM
omjbe,
We see that this has been fixed in Visual Studio 2013. Can you please post your code sample to help us repro.

Thanks,
Vinay
Posted by omjbe on 1/24/2014 at 2:18 AM
Unfortunately, this is not true. I’m using Visual Studio 2013 Ultimate (English) and I see still this bug.
Posted by Microsoft on 1/23/2014 at 9:18 PM
This inconsistency has now been fixed in Visual Studio 2013.
            Assert.AreEqual(2.3, double.NaN, 0.1); => fails
            Assert.AreEqual(2.3, double.NaN); => fails

Thanks,
Vinay
Visual Studio Team
Posted by CornedBee on 12/14/2012 at 2:51 AM
> The behavior is different because there is calculation of delta where one of the operands is NaN.
> This is the standard .NET behavior and we don't want to special case for NaN in Assert.AreEqual.

Bullshit!

This has absolutely nothing to do with the "standard .Net behavior" and everything with your broken implementation of Assert.AreEqual.

That you have closed this bug as "by design" is a travesty and seriously hurts my trust in the reliability of .Net as a platform.
Posted by Microsoft on 10/1/2012 at 2:43 AM
Hi,

The behavior is different because there is calculation of delta where one of the operands is NaN. This is the standard .NET behavior and we don't want to special case for NaN in Assert.AreEqual.

Mathew Aniyan
Program Manager - Visual Studio ALM
Posted by omjbe on 9/24/2012 at 11:32 PM
I believe that it should be handled by the AreEqual method because the behaviour is not consistent:
1. Assert.AreEqual(2.3, double.NaN) // => fails
2. Assert.AreEqual(2.3, double.NaN, 0.1) // => pass

Why should the AreEqual method that allows me to define a delta value have a different behaviour to the one without a delta?

We expected a different behaviour of the AreEqual method and thus missed some serious bugs within our unit tests.
Posted by Microsoft on 9/17/2012 at 11:32 PM
Hi omjbe,

Thanks for using VS and posting your query/feedback.

Due to .net behavior, operations involving double.NaN results in double.NaN and logical operation are evaluated to false.
Since we are not handling special case of double.NaN, this is expected behavior of Assert.AreEqual when double.NaN is involved.

Regards,
Visual Studio Product Team
Posted by Microsoft on 9/12/2012 at 9:15 PM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.
Posted by Microsoft on 9/12/2012 at 2:50 AM
Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)
Sign in to post a workaround.
File Name Submitted By Submitted On File Size  
AssertAreEqualTest.zip 4/1/2014 15 KB
Connect965142.png 4/13/2014 21 KB
AssertAreEqual-Screenshot.png 4/14/2014 32 KB