Home Dashboard Directory Help
Search

Callee disassembly expects address which caller is not providing in x64 mode by Ctrue


Status: 

Closed
 as Fixed Help for as Fixed


1
0
Sign in
to vote
Type: Bug
ID: 674672
Opened: 6/8/2011 9:57:07 AM
Access Restriction: Public
Moderator Decision: Sent to Engineering Team for consideration
0
Workaround(s)
view
0
User(s) can reproduce this bug

Description

I am getting an illegal memory access error in my VS-compiled application.

I have a solution which starts with a C# WPF .exe which loads an interim wrapper c++ .dll, which calls a legacy, unmanaged, mfc-extension c++ dll. Inside the c++ dll, there are two files MyType.h and MyType.cpp. Mytype.h defines the following type:

class __declspec(dllexport) MyType {
public:
union {
struct {
unsigned int Id : 23;
unsigned int Flag : 1;
unsigned int Type : 4;
unsigned int Unused : 4; /* 32 bits total */
};

unsigned int All_Bits; /* Full 32 bits of MyType */
};

MyType Is_Inst();

/* There are some more function definitions here, but no other
variables, aside from some statically defined ones. */
};

The problem arises only now, when we are trying to convert our legacy code to being x64 compatible. In x64 mode when returning Is_Inst(), the disassembly does the following:

// MyType MyType::Is_Inst () {
// uchar Bar=0;
// MyType Foo={0};
mov dword ptr [rdx],0 /* %rdx is 0x17, from a prev fn call. */

// return Foo;
mov rax,rdx
// }
ret

As you can see, Is_Inst expects %rdx to have an address in it, but the caller does not ever load an address into %rdx. Instead %rdx just has junk in it from a previous call to a function which required two parameters. Both the caller and the callee are in the MyType.cpp file.

I have already discussed the problem with others on StackOverflow and on MSDN, so there is more information there as to what I have done to try to solve the issue:
http://stackoverflow.com/questions/6258052/illegal-read-write-error-when-making-legacy-code-x64-compliant
http://social.msdn.microsoft.com/Forums/en/vcgeneral/thread/25a967bc-6d6d-4c79-b973-596bb6b39258

Additionally, I've tried to force both functions to pass by reference by changing the size of the structure to being 40 bits long. (e.g., I added an extra field to the MyType union, "long long Junk: 40;")
Details
Sign in to post a comment.
Posted by Microsoft on 8/18/2011 at 3:38 PM
Hi,
    According to your comment on Stackoverflow, you can workaround the issue by adding a dummy parameter to Is_Inst. This reminds me of another connect bug: http://connect.microsoft.com/VisualStudio/feedback/details/661294/wrong-this-pointer-when-using-templates-in-c-and-targeting-x64-platform
    We have recently fixed that issue.
    If you believe they are different issues, feel free to reactivate the bug and provide a repro so that I can take a further look into it.

Xiang Fan
Visual C++ Team
Posted by MS-Moderator07 [Feedback Moderator] on 6/8/2011 at 6:44 PM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.

Posted by MS-Moderator07 [Feedback Moderator] on 6/8/2011 at 6:44 PM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.

Posted by MS-Moderator01 on 6/8/2011 at 10:54 AM
Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)
Sign in to post a workaround.