Directory.EnumerateDirectory etc unusable due to frequent UnauthorizedAccessExceptions (even runas Administrator) - by Schneider

Status : 

 


49
0
Sign in
to vote
ID 512171 Comments
Status Active Workarounds
Type Bug Repros 27
Opened 11/19/2009 5:03:53 AM
Access Restriction Public

Description

When running as non-administrator, the new set of File system Enumeration APIs cannot be used to recursively enumerate files or folders in a reliable way.

These are all affected:
Directory.EnumerateDirectories
DirectoryInfo.EnumerateDirectories
Directory.EnumerateFiles
DirectoryInfo.EnumerateFiles
Directory.EnumerateFileSystemEntries
DirectoryInfo.EnumerateFileSystemInfos

By design it appears these file system enumerators throw exceptions when they encounter an inaccessible folder, instead of just ignoring it and moving on. This exception causes the enumeration to cease.

This may be due to a limitation of the Win32 API, but something here seems wrong. For example, a non-administrator can enumerate the file system by doing dir /s at the command line, so we should surely be able to do the same from .NET code?
Sign in to post a comment.
Posted by Ian Kennedy on 9/3/2014 at 10:33 AM
Amazing. Still an issue after more than four years.
Posted by lennartb- on 10/18/2013 at 4:15 AM
Still an issue in .Net 4.5
Posted by Xied75 on 11/30/2012 at 8:21 AM
Still get this on Windows Azure Worker Role on Windows Server 2012.
Posted by PCTO on 9/6/2011 at 3:35 PM
Yes, the best solution is an option that allows the developer to choose if he or she wishes the enumeration to continue if an authorization exception is encountered.

As is the methods are completely unprofessional and unusable in a production application, and should either be adjusted, or deprecated and replaced. The primary concern is that they essentially deny access to discovering files that the current user DOES have permission to access by short-circuiting just because a single file or folder exists to which that user does not have access. The user should have an easy means for enumerating all of the files in the file system to which he/she has permissions, regardless of any files that may exist to which they do not.

This is possible with the windows API, and currently the only viable solution to this issue is to write your own recursion using pinvoke calls, which is not at all ideal. It is my opinion that this is the single most poorly implemented piece of functionality in the entire framework.
Posted by Richard Deeming on 8/22/2011 at 9:35 AM
This problem also applies to junction points.

For example, attempt to enumerate the directories in your profile folder. You will get:
Access to the path '%USERPROFILE%\AppData\Local\Application Data\' is denied.

In this case, you don't want to enumerate the junction point, since it points back to the parent directory, so you'd end up with infinite recursion.
Posted by Ramon Barreda on 1/6/2011 at 11:18 AM
Please include a fix with Visual Studio 2010 SP1 as these are very powerful functions, and not yet fixed in Visual Studio SP1 Beta.

The workarounds suggested are not acceptable, as both take 5 to 6 minutes to iterate just through my users folder with > 25,000 files. (Even using Parallel programming)

Right now there are two choices for me:

Either write very complex methods for “IEnumerable-based methods” allowing to catch SecurityExceptons, and then process those “Access Denied” Folders using Shell API calls.
(almost done)

Or use only Shell API calls (FindFirstFile & FindNextFile). Results return in about 1 minute.
(currently used by my backup and file organizer applications)

Justin Van Patten suggested a “new SearchOption - continue on errors”, but there would be no way to know which files or folders were denied access. So full iteration of files and folders would not be possible.

I think possible solutions could be: Callback to catch the exception(s) thrown by the “IEnumerable-based methods” without breaking the loop, or perhaps a Security parameter that allowed the iteration of the whole tree. (Overloaded methods with extra parameters).

Thank You.
Posted by Daniel J. Wojcik on 12/23/2010 at 11:12 AM
The workaround pointed by Frank Dzaebel works.
What he didn't mention is that it is 15 times as slow. (2000 files/second vs 120 files/second on my machine)

But, not much choice in the matter, is there.
Posted by Schneider on 4/13/2010 at 3:14 AM
Thanks for response.

Shame it didn't make it into .net 4 but I understand the complexity of planning this stuff.

That said it seems a bit of an obvious oversight not to have this as a basic user story - somebody trying to write a basic 'explorer' style application that asynchronously enumerates through the file system.

Your comment about SearchOptions is a very interesting one from a design perspective. Regarding principle of least surprise I think most ppl would be surprised that you could not enumerate any directory containing any trivial permission violation. Hence having to explicitly pass a parameter is odd. Changing it now of course could potentially be a breaking change? Lastly, semantically I'm not sure "errors" is the right description for this - I dont consider it an error if I cant enumerate a file due to permissions. As I said using dir *.* /s does not throw "errors". Anyway, just food for thought.

Hope to see this fixed in SP1.

Thanks, Schneider
Posted by Microsoft on 3/31/2010 at 5:12 PM
Hi Schneider1,

Thanks for reporting this. One of the goals of the new IEnumerable-based methods on Directory (EnumerateFiles, EnumerateDirectories, and EnumerateFileSystemEntries) was to have consistent behavior with the older array-based methods (GetFiles, GetDirectories, and GetFileSystemEntries), to make it easy to move from the old methods to the new ones. The behavior you're seeing is consistent with the older methods.

That said, we agree that it'd be useful to be able to more easily enumerate all files/directories on the file system that an app can access. Unfortunately, we weren't able to address this for .NET 4, but we are considering addressing this in the next version of .NET. One way we might address this is by providing a new SearchOption value to "continue on errors". Therefore, I'm going to keep this suggestion open for tracking.

Regards,

Justin Van Patten
Program Manager
CLR Base Class Libraries
Posted by Microsoft on 11/20/2009 at 8:05 PM
Thank you for your feedback, We are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)
Posted by Schneider on 11/19/2009 at 5:15 AM
In fact, even when running as an administrator the example code stops because the entire file system is enumerated. On my computer it stops on this:

UnAuthDir: Access to the path 'c:\Documents and Settings\' is denied.