Slow compilation of 'large' amounts of data (objects/arrays) with references

Slow compilation of 'large' amounts of data (objects/arrays) with references by OReubens

Closed

as Fixed

Type:	Bug
ID:	694100
Opened:	10/10/2011 8:54:18 AM
Access Restriction:	Public
Moderator Decision:	Sent to Engineering Team for consideration

Workaround(s)

view

User(s) can reproduce this bug

I have an app that requires some extensive tables where elements of tables refer to elements in other tables.

The VS2010 C++ (reproducible in VS2008) compiler seems to have difficulty compiling sources with large amounts of data declarations (variables) when referring to other variables depending on how you're doing it.

Details (expand)

Visual Studio 2010 SP1

Create a standard C++ console app through the wizzard.

Substiture code below
[code]

// RefTest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"

// Class A
class A
{
public:
    A(int a) : a(a) {}

    int a;
};

// Make a couple instances of A
const A a1(1);
const A a2(2);
const A a3(3);
const A a4(4);

// Class B, has a reference/pointer to A.

// Pick one of the following methods
//#define USEREF
//#define USEPTR
//#define USEREFARRAY
//#define USEPTRARRAY_NOCTOR
#define USEPTR_NOCTOR

#ifdef USEREF
    class B
    {
    public:
        B(const A& refA) : refA(refA) {}

        const A& refA;
    };

    #define MAKE_B2(num, a)    const B b##num(a);
    #define MAKE_B(num, a)    MAKE_B2(num,a)
#endif

#ifdef USEPTR
    class B
    {
    public:
        B(const A* ptrA) : ptrA(ptrA) {}

        const A* ptrA;
    };

    #define MAKE_B2(num, a)    const B b##num(&a);
    #define MAKE_B(num, a)    MAKE_B2(num,a)
#endif

#ifdef USEREFARRAY
    class B
    {
    public:
        B(const A& refA) : refA(refA) {}

        const A& refA;
    };

    #define MAKE_B(num, a)    B(a),
#endif

#ifdef USEPTRARRAY_NOCTOR
    class B
    {
    public:
        const A* ptrA;
    };

    #define MAKE_B(num, a)    { &a, },
#endif

#ifdef USEPTR_NOCTOR
    class B
    {
    public:
        const A* ptrA;
    };

    #define MAKE_B2(num, a)    const B b##num = { &a };
    #define MAKE_B(num, a)    MAKE_B2(num,a)
#endif

// Generate B instances

#ifdef USEREFARRAY
const B ab[] = {
#endif
#ifdef USEPTRARRAY_NOCTOR
const B ab[] = {
#endif
MAKE_B(__LINE__, a1)
MAKE_B(__LINE__, a2)
MAKE_B(__LINE__, a3)
MAKE_B(__LINE__, a4)
// Copy more data here...
#ifdef USEREFARRAY
};
#endif
#ifdef USEPTRARRAY_NOCTOR
};
#endif

int _tmain(int argc, _TCHAR* argv[])
{
    return 0;
}

[/code]

Via copy/paste copy the 4 MAKE_B(__LINE__, a1)... lines until source is 20000 lines long.

The code essentially has 5 methods to achieve the same functional effect which can be selected via the USEREF/USEPTR/USEREFARRAY/USEPTRARRAY_NOCTOR/USEPTR_NOCTOR define. instances of B referring to instances of A.

Compile with Link time code generation, and /OPT:REF in release build.

English

Windows Vista

English

compilation with the USEREF, USEPTR and USEREFARRAY define take well over a minute to compile. Compilation time does not seem to be linear to the amount of lines of data (MAKE_B but seem to increase exponentially)

the USEREF method is what I have/would like in the actual app. And is reproduced here.
USEPTR was a test to see if reference/pointer is at fault, it does not appear to be
USEREFARRAY was me expecting the time taken caused by the size/lookups in the compile/link time symbol tables and trying to do the same as USEREF without the extra symbols.

USEPTRARRAY_NOCTOR was essentially the method being used before I added the references to other tables. Linking to other tables was done through lookups in code. The refactoring to linked data is an optimisation to avoid lookups in key area's.

USEPTR_NOCTOR was an attempt to gauge the impact on compile/link symboltables. It is slightly slower than the previous one, but barely noticable.

I would expect compile times of all 5 methods to be considerably closer together since essentially they all generate (or should) the exact same code.

I can technically use the USEPTR_NOCTOR methodology, but this is annoying in several areas of the code as it requires me to mix '.' and '->' on member access, and makes some of the templates we had planned impossible/harder. I would much prefer a method where I can use references.

0 attachments

Posted by Microsoft on 5/3/2012 at 10:26 AM

The fix is available in the Visual Studio 11 Beta.

ian Bearman
Principal Software Developer
Visual C++ Code Generation and Optimization

Posted by OReubens on 5/3/2012 at 5:43 AM

Since submitting this issue, the codebase has now progressed to a degree that I've exhausted the workarounds and messy cludges around not being able to efficiently have const structures have references to other const structures.

Can you elaborate on the "future release of the compiler" you speak of ?

Does this mean this fix is available in the Visual Studio 11 Beta ? Or that it will be available in the VS11 Release ?

Posted by Microsoft on 11/21/2011 at 11:27 AM

Thank you for reporting this. I was able to confirm this compiler performance issue. The compiler takes approximately 41 seconds to compile your sample test case on my workstation which is significantly longer than i would expect. I traced the issue to an operation with an O(n^2) running time where n is the number of overlapping symbols. Potentially, this could be replaced with a constant time look-up which would greatly improve your scenario with negligible cost to the other scenarios. To set expectations, I prototyped this solution and was able to get the compile time down to about 8 seconds (a 5x speed up). A complete solution will address this issue in a future release of the compiler.

Thanks,
ian Bearman
Visual C++ Code Generation and Optimization Team

Posted by MS-Moderator09 [Feedback Moderator] on 10/10/2011 at 10:54 PM

Thank you for submitting feedback on Visual Studio 2010 and .NET Framework. Your issue has been routed to the appropriate VS development team for review. We will contact you if we require any additional information.

Posted by MS-Moderator01 on 10/10/2011 at 9:41 AM

Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)