Bad value in EDI causing a crash when executing MASKMOVDQU

Bad value in EDI causing a crash when executing MASKMOVDQU by Luc Lapierre

Closed

as Fixed

Type:	Bug
ID:	646911
Opened:	2/24/2011 11:07:48 AM
Access Restriction:	Public

Workaround(s)

view

User(s) can reproduce this bug

The C++ compiler apparently doesn't take into account the implicit dependency between _mm_maskmoveu_si128 and the EDI register (which must contain the store address). The disassembly shows that the initialization of EDI is placed after the actual MASKMOVDQU instruction. Attempting to write at the invalid address in EDI causes a crash.

Details (expand)

Visual Studio 2010 SP1 Beta

Reliability

Microsoft 32-bit C/C++ Optimizing Compiler Version 16.00.31118.01 for 80x86

C:\>cl /O2 /Oy- mm_maskmoveu_si128.cpp
C:\>mm_maskmoveu_si128.exe

Note: without /O2 /Oy-, the compiled executable doesn't crash.

English

Windows XP

English

Access violation.

No crash & expecting the initialization of EDI to precede MASKMOVDQU in the disassembly.

File Name	Submitted By	Submitted On	File Size
mm_maskmoveu_si128.cpp	Luc Lapierre	2/24/2011	2 KB

Posted by Haohua Xie on 4/10/2013 at 3:09 AM

This is a troublesome issue to crash the app. a workaround is to set the EDI register to proper value using the inline asm

Posted by Microsoft on 2/24/2011 at 6:57 PM

Thank you for submitting feedback on Visual Studio 2010 and .NET Framework. Your issue has been routed to the appropriate VS development team for review. We will contact you if we require any additional information.

Posted by Microsoft on 2/24/2011 at 11:14 AM

Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)

Posted by Haohua Xie on 4/10/2013 at 3:08 AM

#define FIX_MASKMOVDQU_EDI(p) __asm{ mov edi, dword ptr [p]}
FIX_MASKMOVDQU_EDI(pU);
_mm_maskmoveu_si128(iU[0], RGB2YUVStoreMask, (char *)pU);