Search

No way to determine internal offset used by MemoryMappedViewAccessor - Makes SafeMemoryMappedViewHandle property unusable by James Chaldecott

Active

5
0
Sign in
to vote
Type: Bug
ID: 537635
Opened: 3/1/2010 7:37:30 AM
Access Restriction: Public
1
Workaround(s)
3
User(s) can reproduce this bug
MemoryMappedViewAccessor has a SafeMemoryMappedViewHandle property, which returns the ViewHandle being used internally by the MemoryMappedView, but does not have any property to return the offset being used by the MemoryMappedView.

As the MemoryMappedView is page aligning the offset requested in MemoryMappedFile.CreateViewAccessor(offset,size) it is impossible to use the SafeMemoryMappedViewHandle for anything useful without knowing the offset.

Note that what we actually want to do is use the AcquirePointer(ref byte* pointer) method to allow some fast pointer based (possibly unmanaged) code to run. We're OK with the pointer being page aligned, but it must be possible to find out what the offset from the originally requested address is.
Details (expand)

Product Language

English

Version

Visual Studio 2010 Release Candidate

Operating System

Windows 7

Operating System Language

English

Steps to Reproduce

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO.MemoryMappedFiles;

namespace MmfOffsetFail
{
    class Program
    {
        static void Main(string[] args)
        {
            var mmf = MemoryMappedFile.CreateNew("TestMMF", 128000);

            using (var view = mmf.CreateViewAccessor())
            {
                for (int i = 0; i < 32000; i++)
                {
                    view.Write(i * 4, i);
                }
            }

            ShowViewContents(mmf, 0);
            ShowViewContents(mmf, 100);
            ShowViewContents(mmf, 4096);
            ShowViewContents(mmf, 16384);
            ShowViewContents(mmf, 24000);

            Console.ReadLine();
        }

        private static void ShowViewContents(MemoryMappedFile mmf, int viewOffset)
        {
            using (MemoryMappedViewAccessor view = mmf.CreateViewAccessor(viewOffset * 4, 512))
            {
                Console.WriteLine("Reading from MemoryMappedViewAccessor with offset = {0} ({1} bytes)", viewOffset, viewOffset * 4);
                var viewHandle = view.SafeMemoryMappedViewHandle;
                for (int i = 0; i < 5; i++)
                {
                    // Really need to apply an offset when using viewHandle here, but there's no way to know what offset is being applied by the view.
                    Console.WriteLine("Offset {0} : MemoryMappedViewAccessor => {1}, SafeMemoryMappedViewHandle => {2}", i, view.ReadInt32(i * 4), viewHandle.Read<Int32>((ulong)i * 4));
                }
            }
        }
    }
}

Actual Results

Reading from MemoryMappedViewAccessor with offset = 0 (0 bytes)
Offset 0 : MemoryMappedViewAccessor => 0, SafeMemoryMappedViewHandle => 0
Offset 1 : MemoryMappedViewAccessor => 1, SafeMemoryMappedViewHandle => 1
Offset 2 : MemoryMappedViewAccessor => 2, SafeMemoryMappedViewHandle => 2
Offset 3 : MemoryMappedViewAccessor => 3, SafeMemoryMappedViewHandle => 3
Offset 4 : MemoryMappedViewAccessor => 4, SafeMemoryMappedViewHandle => 4
Reading from MemoryMappedViewAccessor with offset = 100 (400 bytes)
Offset 0 : MemoryMappedViewAccessor => 100, SafeMemoryMappedViewHandle => 0
Offset 1 : MemoryMappedViewAccessor => 101, SafeMemoryMappedViewHandle => 1
Offset 2 : MemoryMappedViewAccessor => 102, SafeMemoryMappedViewHandle => 2
Offset 3 : MemoryMappedViewAccessor => 103, SafeMemoryMappedViewHandle => 3
Offset 4 : MemoryMappedViewAccessor => 104, SafeMemoryMappedViewHandle => 4
Reading from MemoryMappedViewAccessor with offset = 4096 (16384 bytes)
Offset 0 : MemoryMappedViewAccessor => 4096, SafeMemoryMappedViewHandle => 0
Offset 1 : MemoryMappedViewAccessor => 4097, SafeMemoryMappedViewHandle => 1
Offset 2 : MemoryMappedViewAccessor => 4098, SafeMemoryMappedViewHandle => 2
Offset 3 : MemoryMappedViewAccessor => 4099, SafeMemoryMappedViewHandle => 3
Offset 4 : MemoryMappedViewAccessor => 4100, SafeMemoryMappedViewHandle => 4
Reading from MemoryMappedViewAccessor with offset = 16384 (65536 bytes)
Offset 0 : MemoryMappedViewAccessor => 16384, SafeMemoryMappedViewHandle => 16384
Offset 1 : MemoryMappedViewAccessor => 16385, SafeMemoryMappedViewHandle => 16385
Offset 2 : MemoryMappedViewAccessor => 16386, SafeMemoryMappedViewHandle => 16386
Offset 3 : MemoryMappedViewAccessor => 16387, SafeMemoryMappedViewHandle => 16387
Offset 4 : MemoryMappedViewAccessor => 16388, SafeMemoryMappedViewHandle => 16388
Reading from MemoryMappedViewAccessor with offset = 24000 (96000 bytes)
Offset 0 : MemoryMappedViewAccessor => 24000, SafeMemoryMappedViewHandle => 16384
Offset 1 : MemoryMappedViewAccessor => 24001, SafeMemoryMappedViewHandle => 16385
Offset 2 : MemoryMappedViewAccessor => 24002, SafeMemoryMappedViewHandle => 16386
Offset 3 : MemoryMappedViewAccessor => 24003, SafeMemoryMappedViewHandle => 16387
Offset 4 : MemoryMappedViewAccessor => 24004, SafeMemoryMappedViewHandle => 16388

Expected Results

Reading from MemoryMappedViewAccessor with offset = 0 (0 bytes)
Offset 0 : MemoryMappedViewAccessor => 0, SafeMemoryMappedViewHandle => 0
Offset 1 : MemoryMappedViewAccessor => 1, SafeMemoryMappedViewHandle => 1
Offset 2 : MemoryMappedViewAccessor => 2, SafeMemoryMappedViewHandle => 2
Offset 3 : MemoryMappedViewAccessor => 3, SafeMemoryMappedViewHandle => 3
Offset 4 : MemoryMappedViewAccessor => 4, SafeMemoryMappedViewHandle => 4
Reading from MemoryMappedViewAccessor with offset = 100 (400 bytes)
Offset 0 : MemoryMappedViewAccessor => 100, SafeMemoryMappedViewHandle => 100
Offset 1 : MemoryMappedViewAccessor => 101, SafeMemoryMappedViewHandle => 101
Offset 2 : MemoryMappedViewAccessor => 102, SafeMemoryMappedViewHandle => 102
Offset 3 : MemoryMappedViewAccessor => 103, SafeMemoryMappedViewHandle => 103
Offset 4 : MemoryMappedViewAccessor => 104, SafeMemoryMappedViewHandle => 104
Reading from MemoryMappedViewAccessor with offset = 4096 (16384 bytes)
Offset 0 : MemoryMappedViewAccessor => 4096, SafeMemoryMappedViewHandle => 4096
Offset 1 : MemoryMappedViewAccessor => 4097, SafeMemoryMappedViewHandle => 4097
Offset 2 : MemoryMappedViewAccessor => 4098, SafeMemoryMappedViewHandle => 4098
Offset 3 : MemoryMappedViewAccessor => 4099, SafeMemoryMappedViewHandle => 4099
Offset 4 : MemoryMappedViewAccessor => 4100, SafeMemoryMappedViewHandle => 4100
Reading from MemoryMappedViewAccessor with offset = 16384 (65536 bytes)
Offset 0 : MemoryMappedViewAccessor => 16384, SafeMemoryMappedViewHandle => 16384
Offset 1 : MemoryMappedViewAccessor => 16385, SafeMemoryMappedViewHandle => 16385
Offset 2 : MemoryMappedViewAccessor => 16386, SafeMemoryMappedViewHandle => 16386
Offset 3 : MemoryMappedViewAccessor => 16387, SafeMemoryMappedViewHandle => 16387
Offset 4 : MemoryMappedViewAccessor => 16388, SafeMemoryMappedViewHandle => 16388
Reading from MemoryMappedViewAccessor with offset = 24000 (96000 bytes)
Offset 0 : MemoryMappedViewAccessor => 24000, SafeMemoryMappedViewHandle => 24000
Offset 1 : MemoryMappedViewAccessor => 24001, SafeMemoryMappedViewHandle => 24001
Offset 2 : MemoryMappedViewAccessor => 24002, SafeMemoryMappedViewHandle => 24002
Offset 3 : MemoryMappedViewAccessor => 24003, SafeMemoryMappedViewHandle => 24003
Offset 4 : MemoryMappedViewAccessor => 24004, SafeMemoryMappedViewHandle => 24004
      You can indicate your satisfaction with how Microsoft handled this issue by completing this quick 3 question survey. [Details]

 

File Attachments
0 attachments
Sign in to post a comment.
Posted by Max Charp on 2/26/2012 at 10:18 AM
Temporary solution:

    public unsafe static class Helper
    {
        static SYSTEM_INFO info;

        static Helper()
        {
            GetSystemInfo(ref info);
        }

        public static byte* Pointer(this MemoryMappedViewAccessor acc, int offset)
        {
            var num = offset % info.dwAllocationGranularity;
            
            byte* tmp_ptr = null;

            RuntimeHelpers.PrepareConstrainedRegions();

            acc.SafeMemoryMappedViewHandle.AcquirePointer(ref tmp_ptr);

            tmp_ptr += num;

            return tmp_ptr;
        }

        [DllImport("kernel32.dll", SetLastError = true)]
        internal static extern void GetSystemInfo(ref SYSTEM_INFO lpSystemInfo);

        internal struct SYSTEM_INFO
        {
            internal int dwOemId;
            internal int dwPageSize;
            internal IntPtr lpMinimumApplicationAddress;
            internal IntPtr lpMaximumApplicationAddress;
            internal IntPtr dwActiveProcessorMask;
            internal int dwNumberOfProcessors;
            internal int dwProcessorType;
            internal int dwAllocationGranularity;
            internal short wProcessorLevel;
            internal short wProcessorRevision;
        }
    }

Use
            MemoryMappedFile memoryFile; /...

            var offset = 100;
            var acc = memoryFile.CreateViewAccessor(offset, 16, MemoryMappedFileAccess.ReadWrite);
            var ptr = acc.Pointer(offset);
Posted by Max Charp on 2/24/2012 at 11:36 AM
        public static int* Pointer(this MemoryMappedViewAccessor acc)
        {
            var tmp_ptr = (byte*)IntPtr.Zero;

            acc.SafeMemoryMappedViewHandle.AcquirePointer(ref tmp_ptr);

            return (int*)tmp_ptr;
        }

        public Writer GetWriter()
        {
            var offset = 0;
            var acc = memoryFile.CreateViewAccessor(offset, PageSize, MemoryMappedFileAccess.ReadWrite);
            var ptr = acc.Pointer();
            
            var offset2 = 16;
            var acc2 = memoryFile.CreateViewAccessor(offset2, PageSize, MemoryMappedFileAccess.ReadWrite);
            var ptr2 = acc2.Pointer();

            *ptr = 12345;

            if (*ptr == *ptr2)
            {
                // It's always true!!!!!!!!!!!!!!!!
                System.Diagnostics.Debugger.Break();
            }

            return new Writer((byte*)ptr, PageSize, acc, storage);
        }
Posted by James Chaldecott on 9/27/2011 at 1:35 AM
Any chance of getting this fixed in .Net 4.5?
Posted by Andy Bird on 11/19/2010 at 4:41 AM
This is also an issue for us. We have a similar use case to the original description from James. As James notes, when a view accessor is created with a non-zero offse, the AcquirePointer method appears to return a pointer to a page aligned address rather than to the start of the view so for the AcquirePointer method to be of any use in this scenario, it is necessary to know the appropriate offset to apply to the pointer (which in general is a function of both the offset applied when creating the view and the page size / alignment - ie. it's not trivial to handle this in the client code).

Ideally, the AcquirePointer method would apply the offset before returning the pointer, although since this would now be a breaking change I guess this may not be possible. Although there could be a flag added to the accessor to enable this feature for new code.

Next best would be a way to retrieve a value from the accessor that provides the appropriate offset value to be applied to the pointer returned by the AcquirePointer method. This would need to take account of the offset passed when creating the view accessor and the page alignment.

At a minimum the behaviour should be clearly documented - at present there is nothing in the documentation to indicate the the pointer returned by AcquirePointer will in general not take account of the offset passed when creating the view accessor and is therefore only really usable when the view is created with a zero offset.

Has anyone found a decent workaround for this?
Posted by David Christensen1 on 8/24/2010 at 8:38 AM
The information is available in MemoryMappedViewAccessor.m_view.PointerOffset
Unfortunately m_view is private; I can see why you might want to keep that private, but having a PointerOffset on the Accessor would be most useful
Posted by James Chaldecott on 3/23/2010 at 7:27 AM
Is there any way to get progress notifications on this? RTM must be getting pretty close, and this is a serious issue for us.
Posted by Microsoft on 3/1/2010 at 10:50 PM
Thanks for your feedback. We were able to reproduce the issue you are seeing. We are routing this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.
Posted by Microsoft on 3/1/2010 at 7:03 PM
Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)
Sign in to post a workaround.
Posted by Max Charp on 2/26/2012 at 10:21 AM
Temporary solution:

    public unsafe static class Helper
    {
        static SYSTEM_INFO info;

        static Helper()
        {
            GetSystemInfo(ref info);
        }

        public static byte* Pointer(this MemoryMappedViewAccessor acc, int offset)
        {
            var num = offset % info.dwAllocationGranularity;
            
            byte* tmp_ptr = null;

            RuntimeHelpers.PrepareConstrainedRegions();

            acc.SafeMemoryMappedViewHandle.AcquirePointer(ref tmp_ptr);

            tmp_ptr += num;

            return tmp_ptr;
        }

        [DllImport("kernel32.dll", SetLastError = true)]
        internal static extern void GetSystemInfo(ref SYSTEM_INFO lpSystemInfo);

        internal struct SYSTEM_INFO
        {
            internal int dwOemId;
            internal int dwPageSize;
            internal IntPtr lpMinimumApplicationAddress;
            internal IntPtr lpMaximumApplicationAddress;
            internal IntPtr dwActiveProcessorMask;
            internal int dwNumberOfProcessors;
            internal int dwProcessorType;
            internal int dwAllocationGranularity;
            internal short wProcessorLevel;
            internal short wProcessorRevision;
        }
    }

Use
            MemoryMappedFile memoryFile; /...

            var offset = 100;
            var acc = memoryFile.CreateViewAccessor(offset, 16, MemoryMappedFileAccess.ReadWrite);
            var ptr = acc.Pointer(offset);