SSIS: Package Parts - by Jamie Thomson

Status : 

  Won't Fix<br /><br />
		Due to several factors the product team decided to focus its efforts on other items.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.

Sign in
to vote
ID 338609 Comments
Status Closed Workarounds
Type Suggestion Repros 1
Opened 4/16/2008 10:29:03 PM
Access Restriction Public


Today in SSIS we build, distribute and execute these things called packages. Packages are fairly large, verbose, non-descript BLOBs and that has some rather negative implications such as:

-Changing anything in the package, even something as small as moving a container one pixel to the left, causes the package to get checked out (assuming you have your packages under source code control)
-Support for multiple-developer teams is limited. If you have two people building dataflows then they have to be working on separate packages.
-The only method of reusing tasks is copy-and-paste
-Comparing two packages to check for differences and hence merging them together is not possible
Sign in to post a comment.
Posted by Jamie Thomson on 5/14/2012 at 6:51 AM
I have re-opened this here:
Posted by BlueRaja on 8/31/2010 at 11:33 AM
How is this for an argument to change your judgement:
Posted by Microsoft on 8/10/2010 at 5:00 PM
Thank you for contacting us. After analyzing this issue and prioritizing it in the context of our current development plans, we have found regretfully that it does not meet the bar to be included in an upcoming product release. If you feel that you could provide us with more data to change that judgment, we welcome any further information. Your feedback continues to be invaluable to us and we take all your communication seriously. Please continue to contact us with any questions, suggestions and issues as they arise.

     The SSIS Team
Posted by DocWabaki on 4/1/2010 at 10:28 AM
Microsoft really needs to take a look at this issue. SSIS is not code in the normal sense. They need to seperate all the user settings and other nonsense stored in the DTSX file into a different file. Then, it could be merged like code and changes could be evaluated like code. Today, looking at the impact of a single change set is way too complex. If you have multiple developers working on code, as we do, then managing code promotion can become a convoluted nightmare. We have developed tools in Team Foundation Server to indicate when we have conflicting change sets, some ready for promotion and others not. The lack of concise change sets means we need to manually adjust the code as we got through branch promotion. It should be like any other language where we can merge the code and pick up the changes we need.
Posted by Chris Gomez on 10/9/2008 at 6:08 AM
Consider that not all teams can use MSFT based source solutions. Our department uses CVS. We've had no problems writing .NET solutions of all kinds with Visual Studio.NET to 2005 to 2008. But with SSIS, we've found all the same issues, not to mention it is difficult for the merge tool to try and reconcile changes to packages (even if they are superficial like move a task one pixel over).

There are far too many instances where the solution is: "Well, just get a clean copy of the tip (HEAD in CVS) and rebuild. Make sure it works now, and then make your change." It happens so often as to really cramp productivity and start encouraging too much ownership of items (well, let's let Jack make that change because he's been working on that package recently).
Posted by erasmus_ssis on 9/25/2008 at 2:44 AM
I'm with Jamie on this, all the issues identified are really a pain in the neck on a daily basis. would definitely love to see these resolved in the next release.
Posted by Gorm Braarvig on 6/1/2008 at 3:39 PM
I agree, but I would do it slightly differently:
I would like the ability to "link" (like in linking .obj-files) packages ran from a parent-package so that the packages (ran with execute package) would appear as groups in the parent package.
This way unit-testing would be feasible, package would stay small, connections and configuration could be reused.
This should be easier to implement.
Posted by David R Buckingham on 5/9/2008 at 11:20 AM
I strongly support this notion. I would go even further and request that the designer display attributes be removed from the programmatic files. This would allow someone to rearrange the flow _visually_ for their own viewing pleasure without necessitating what appears to be a code change from source control's perspective. It may dramatically increase the number of files associated with a given project, but if you are already keeping the database schema objects in their own files, it's not that different from what many SQL developers are used to.
Posted by Michael Hotek on 5/8/2008 at 8:13 PM
I would second this. Yes, you can break large, complex processes into multiple packages and then build an overall package to run a set of packages. But, this is a bit of a kludge. Comparing packages is impossible, because even a tiny change causes almost everything to be rewritten and moved around in the file. I'm not all that much for breaking this stuff out into separate files, because with a complicated package you can quickly have dozens or hundreds of tiny files. I would like to see components treated like parts. Each part having a defined section within the package file so that you could compare them across changes. It would also allow multiple people to work on various pieces of the same package without interfering with each other.
Posted by MatthewRoche on 4/27/2008 at 4:16 PM
I just want to add my enthusiastic support for this idea. I regularly tell my audiences at conferences and users groups that "SSIS packages are code" in an attempt to make them realize that the best practices that they have learned over the years apply to SSIS development. Use source control, have a documented and repeatable deployment process, comment, test, you name it.

But the lack of reuse in SSIS packages means that while the practices still apply, the tools often do not exist to make them easy to use. In this regard, using SSIS is not unlike giving up .NET and moving back to the days of VB5. Sure, you had inheritance, but it was interface inheritance only, and any code reuse was something of a kludge. I'd love to see SSIS take this big step forward...
Posted by Jamie Thomson on 4/21/2008 at 1:03 PM
good stuff. Thank you Mr Noor!
Posted by Microsoft on 4/21/2008 at 12:05 PM
Hi Jamie,
Thanks for these (and other) great comments on the real-world of etl development practices. After SQL 2008 we're going to be taking a hard look at these scenarios to see how to better support them, and using your comments in that process.