Home Dashboard Directory Help
Search

Relax the limitation of active buffers in SSIS by Daniel Cai - KingswaySoft


Status: 

Active


3
0
Sign in
to vote
Type: Suggestion
ID: 870447
Opened: 5/9/2014 2:52:09 PM
Access Restriction: Public
0
Workaround(s)
view

Description

Based on our testing and what we read on Internet, there is a limitation on how many active buffers SSIS can have at the same time. The current default is 5, and there isn't a way to overwrite or change it. This has significantly limited the scalability of SSIS Platform.

For instance, when we do integration for web service based interfaces (such as Microsoft Dynamics CRM, Dynamics NAV, Dynamics AX, etc.), the performance is much slower when compared to database technology. The most common way to improve the data load performance is to use multi-threading. With SSIS, we use BDD component to achieve this.

However with the limitation of active buffers that is currently imposed by the platform, we can have at most 5 active buffers which means at most 5 concurrent threads, this is not good enough. When we can write integration using C# programming language (or VB.NET) ourselves, it is easy to spin up 16 threads without seeing performance degradation (by working with Dynamics CRM web service interfaces). When using BDD component, even we have 10 branches (or 20, no matter how many you have), we can only see at most 5 concurrent requests from IIS side, we are not able to take full advantage of SSIS BDD component.

Details
Sign in to post a comment.
Sign in to post a workaround.