Home Dashboard Directory Help
Search

memory consumption alot higher on x64 for XslCompiledTransform.Transform then on x86 by FZB


Status: 

Active


23
0
Sign in
to vote
Type: Bug
ID: 508748
Opened: 11/6/2009 6:28:35 AM
Access Restriction: Public
3
Workaround(s)
view
18
User(s) can reproduce this bug

Description

the XslCompiledTransform.Transform function uses alot more memory running on a x64 system vs running on x86. the more xpath is used in the xslt, the more significant the memory usage becomes, for the attached xml and xslt, its 30 mb mem used with x86, but ~400mb on x64. i attached the cs file as well as sample xml and xslt files.

use the sample like "ConsoleApplication1.exe C:\test\transform.xslt" and have all files in c:\test"

if you watch the task manager performance tab youll see the mem usage, the more spatch * = xpath) you use, the worse it gets
Details
Sign in to post a comment.
Posted by Andrew Mayorov on 10/22/2012 at 7:13 AM
No matter if XSLT didn't take the world over in the past decade, it is very important language that allows for such transformation scenarios that are not possible with Razor, for example. So I strongly hope that this issue will be fixed soon. After all if it is caused by JIT "peculiarities" then this problem could also be seen in other components of the framework.
Posted by MattDuguidNZ on 9/4/2012 at 7:14 PM
Given this bug was first reported over 3 years ago now, can we please have an update on where Microsoft are with either a concrete date for a fix or suitable work around? This issue is severely affecting our production environment.
Posted by FZB on 9/28/2011 at 2:53 AM
the bug is still around in .net 4.5 dev preview
Posted by Rockets_2011 on 6/14/2011 at 3:34 PM
We are still seeing this issue. Has there been a fix for this yet or any suggested work around?
Posted by Microsoft on 5/26/2010 at 9:14 AM
Unfortunately, we've known about this issue for quite some time. It's caused by 2 things: Really terrible IL generated by the XSLT compiler, and some algorithms in the 64 bit JIT that have polynomial scaling. We're actually working on porting the 32 bit JIT compiler to x64, but that won't see the light of day until the _next_ side-by-side release of the runtime (as in "2.0 & 4.0 run side-by-side, but 3.0/3.5/3.5SP1 were 'in-place' releases). I've switched this over to a 'suggestion' so I can keep it attached to the JIT-throughput work item to make sure this is fixed when the newly ported JIT is ready to ship.
Posted by Microsoft on 1/27/2010 at 8:58 AM
The root of the problem is related to two things: First, the x64 JIT compiler has a few algorithms that are quadratically scaling. One of them is the debug info generator, unfortunately. So for very large methods, it really gets out of control. The second problem is the implementation of xslt. It appears (and I don't know xslt at all) that our xslt generator produces simplistic methods that cause problems like this to arise. A more intelligent XSLT compiler would likely break them up into smaller chunks. I'll see if I can track down who owns that particular piece of code. The other functional area that I know issues exist like this is our Regular Expression compiler, for virtually identical reasons. The RegEx engine produces very brain-dead code that for specific types of RegEx's result in incredibly large methods. I don't have any particular example, and I know that it's not particularly common, but it's been something that we've seen a number of times since the 64 bit framework first released in 2005. Again, I wish I had a better response, but all I can say right now is "we're working on it" :-(
Posted by mpb on 12/2/2009 at 4:11 PM
a) you should mark this class in framework as not usable or carefully usable under x64 so that the developer will also know it.
b) as a programmer _unfortunately_ i do know some problems are harder to fix then others - _unfortunately_ my customers ususally dont care about that too much, they just want me to produce a working solution

thats annoying. so i don't see any chance to move some of our applications to x64.
because of heavily using xslt/xml with hence high memory usage they would really benefit from x64.
i had such great experience with x86 .net. now i am frustrated.

what does this bug mean for other code? can it be affected too?
are there other known bugs that make use of x64 framework dangerous?
is there any workarround possible? XslTransform is not an option because its a) deprecated and b) there is no support for msxsl:script and other things.
Posted by FZB on 12/2/2009 at 3:21 PM
this isnt ment to be jumping the messanger, not ment as personal attack or something, but this is a bit of a problem. on the one hand, the current generation of ms product become 64bit only (exchange, sharepoint w2k8 r2), on the other hand we are stuck with a function that make using the framework in the 64 bit world impossible.
i am aware there is the reality of schedules, but considering going full out on 64 bit as microsoft, not having this considered for 4.0 (_eventually_ suggest it might be around even longer) seems a bit of a letdown.
in regards of known issue, i didnt find any public info on it, xslttransform is marked deprecated, suggesting to use xsltcompiledtransform.
anyhow, thanks for the info
Posted by Microsoft on 12/2/2009 at 12:47 PM
This issue is a) known, and b) not really trivial to address. It's a design issue with the 64 bit JIT. We're in the early stages of replacing our 64-bit JIT implementation, so it will _eventually_ get address, but not in the CLR 4.0 timeframe, unfortunately.
Posted by FZB on 11/13/2009 at 12:08 AM
thanks, thats great news
Posted by Microsoft on 11/12/2009 at 1:22 AM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.

Thank you
Posted by Microsoft on 11/9/2009 at 12:55 AM
Thank you for your feedback, We are currently reviewing the issue you have submitted.
Posted by mpb on 11/6/2009 at 2:29 PM
Thats interesting. With complex xslt (50kb) and debug enabled in the XslCompiledTransform constructor the memory raises several GB freezing my machine completely. Without debug its about 600 MB during transformation and 200 MB after transform. Memory get not released during GC.Collect.
This would make XslCompiledTransform unusable especialy in web services or sql server on x64 machines maybe blocking the whole server.

Running that in x86 mode i can't reproduce that problem as mentioned in the bug report.

Posted by FZB on 11/6/2009 at 6:33 AM
forgot to mention in the report, the behavior shows on more then just w2k3 and is in framework 2 to 4 (tested 2, 3.5 sp1 and 4 b1)
Sign in to post a workaround.
Posted by Max Toro on 1/25/2013 at 8:14 PM
For ASP.NET the workaround is to 'enable 32-bit' on the IIS application pool.
Posted by Zach Parton on 7/13/2011 at 7:26 AM
Another workaround is to ditch the internal Microsoft XSLT transform completely. We are looking at one of Altova's libraries at the moment. Its cheap and hopefully does not turn a web service on a 64 bit windows box into a ram pig just trying to execute XSLT. Who would have imagined that in 2011 we would not be able to do this without consuming 6 GB of ram?
Posted by Zach Parton on 7/13/2011 at 7:20 AM
Since Microsoft has not figured out in 3 years how to create a < 400MB image from a 20K XSLT compiled transform script we have been surviving by setting the assemblies that are implementing any XSLT transformation to 32 bit. This is pretty pathetic that this has not been addressed. Balmer really shit the bed on this one.
File Name Submitted By Submitted On File Size  
transform.xslt (restricted) 11/6/2009 -
Test.xml (restricted) 11/6/2009 -
Program.cs (restricted) 11/6/2009 -