Home Dashboard Directory Help
Search

Executing commands which require quotes and variables is practically impossible by Joel Bennett


Status: 

Closed
 as Fixed Help for as Fixed


108
1
Sign in
to vote
Type: Bug
ID: 376207
Opened: 10/17/2008 12:10:35 PM
Access Restriction: Public
8
Workaround(s)
view
24
User(s) can reproduce this bug

Description

It's FAR TOO HARD to pass parameters to applications which require quoted strings. I asked this question in IRC with a "roomful" of PowerShell experts, and it took hour for someone to figure out a way (I originally started to post here that it is simply not possible).

This completely breaks PowerShell's ability to serve as a general purpose shell, because we can't do simple things like executing sqlcmd. The number one job of a command shell should be running command-line applications...

As an example, trying to use SqlCmd from SQL Server 2008, there is a -v parameter which takes a series of name:value parameters. If the value has spaces in it, you must quote it. For instance, this will work in cmd.exe (it doesn't do anything, just exits silently):
        sqlcmd -v user="Joel Bennett" -Q "select '$(user)' as UserName"

But in PowerShell, it has to be:
        sqlcmd -v user=`"Joel Bennett`" -Q 'select ''$(user)'' as UserName'

Notice that you have to escape things two different ways here? How many people are going to figure that out?

But it gets worse if you need to user a variable (eg, $Env:FullName):
$Env:FullName="Joel Bennett"

So you can execute it in CMD:
     sqlcmd -v user="%FullName%" -Q "select '$(user)' as UserName"

But how many PowerShell users are going to figure out that this (sortof) works:
     sqlcmd -v user=($env:FullName) -Q 'select ''$(user)'' as UserName'

What appears to be happening is that "user=($env:FullName)" is treated as a two part array: "user=" and "Joel Bennett" ... so the engine inserts a space after the "user=" (which I can't get rid of by setting $ofs, oddly) and then inexplicably and magically chooses to wrap quotes around the "Joel Bennett" ....

The problem is, as you can see from these examples, there is no single way to write a command line to invoke this application correctly, so even after you master all 4 or 5 different ways of quoting and escaping things, you're still guessing as to which will work when ... or, you can just shell out to cmd, and be done with it.
Details
Sign in to post a comment.
Posted by Douglas McCormick on 2/8/2014 at 1:42 PM
I have to agree with all the other comments/posts below.........This is complete garbage Microsoft, really? Powershell is that dumb and you consider this as "working as designed"?
How in the world would this bug be considered closed in any definition-- I'm ready to take a hammer to PS--- I'm looking to wrap nearly everything into cygwin instead-- at least there's enough people and thought leadership to fix a bug/issue instead of conveniently calling it closed.

This should be addressed properly and patched up for v2 and above

Posted by Keith Hill MVP on 2/3/2014 at 9:18 PM
>> --% operator presented as a fix to this issue is totally inadequate for any but the most trivial of use cases.

Huh? This is for way more than trivial use cases. It essentially puts the PowerShell parsing into a cmd.exe equivalent mode (ie dumb mode) e.g.:

MSBuild /t:Publish --% /p:TargetDatabaseName="MyDatabase";TargetConnectionString="Data Source=.\;Integrated Security=True" /p:SqlPublishProfilePath="Deploy.publish.xml" Database.sqlproj

Or how about this case:

Set-Content EchoTest.cmd "Echo %1"
$test="Hello There"
$env:test = $test
.\echotest.cmd --% %test%

Outputs:
Hello

Or how about this case:

$env:SiteName = $SiteName
$env:SiteManagementUrl = $SiteManagementUrl
MSDeploy.exe --% -verb:delete -verbose -dest:contentPath="%SiteName%/App_Offline.htm",computerName="%SiteManagementUrl%"

If you understand how --% works, it isn't very difficult to get your exe parameters to work just as they did in cmd.exe.
Posted by stangm on 11/15/2013 at 7:46 AM
Just to add, this is not fixed in powershell 3. The automatic entry of quotes when spaces are in the variable is major problem. Example:

Set-Content EchoTest.cmd "Echo %1"
$test="Hello There"
.\echotest.cmd $test

Expected Output:
Hello

Actual Output:
"Hello There"

Note the quotes

Now try
Set-Content EchoTest.cmd "Echo %1"
$test="Hello"
.\echotest.cmd $test

Expected Output
Hello

Actual Output:
Hello

Note no quotes

The moment you put a space in a string, Powershell "helpfully" adds quotes to it when passing to the command.... Why?




Posted by Morgan Larosa on 9/3/2013 at 11:14 PM
Just adding my voice to the chorus of "this isn't actually fixed" - the --% operator presented as a fix to this issue is totally inadequate for any but the most trivial of use cases.

The most common complaint that I've seen in relation to this issue is Web Deploy: if I have, for example, a runCommand provider issuing a command with parameters, there's no way for me to successfully run the msdeploy command line from Powershell if I want to generate any of the arguments using variable interpolation or string formatting.

The evidence is all here in the previous comments and the thousands of forum threads, blog posts, StackOverflow questions, etc. - this issue is not fixed, and to mark it as such is disingenuous and disappointing from the Powershell team.
Posted by Mr Konrad Sikorski on 7/31/2013 at 5:14 AM
I cannot find out the correct way to invoke something like this:

MSBuild /t:Publish /p:TargetDatabaseName="MyDatabase";TargetConnectionString="Data Source=.\;Integrated Security=True" /p:SqlPublishProfilePath="Deploy.publish.xml" "Database.sqlproj"

I am using this command:

$DatabaseConnectionString = "Data Source=.\;Integrated Security=True"
MSBuild /t:Publish "/p:TargetDatabaseName=`"$DatabaseName`";TargetConnectionString=`"$DatabaseConnectionString`"" /p:SqlPublishProfilePath=`"Deploy.publish.xml`" `"Database.sqlproj`"

and it doesn't work. I have spend 4 hours trying many solutions without success. :/
Posted by Mike Cheel on 7/18/2013 at 11:03 AM
Until you can call a powershell script with spaces in the parameter variables this is not fixed at all. It is a shame I have to call a batch file from powershell just to call msdeploy.exe.
Posted by ColinBowern on 6/24/2013 at 10:10 PM
Here is an example of something that I can't do in PowerShell 3.0:

MSDeploy.exe -verb:delete -verbose -dest:contentPath='$SiteName/App_Offline.htm',computerName='$SiteManagementUrl'

This attempt fails:

& $MSDeploy -verb:delete -verbose $("-dest:contentPath='{0}',computerName='{1}'" -f $ContentPath, $SiteManagementEndpoint)

with:

MSDeploy.exe : Error: Unrecognized argument '"-dest:contentPath='My Site/App_Offline.htm',computerName='https://webnode:8172/MSDeploy.axd?Site=My+Site'"'. All arguments must begin with "-".
Posted by emission on 12/10/2012 at 2:38 AM
Closed?

Can MS please describe:

1. What you mean by closed.
2. What's so difficult... This seems like shell language 101.
Posted by matthickford on 6/6/2012 at 8:07 AM
"Closed as fix"?

If so, PLEASE link the fix!!!!!!
Posted by AussieJohn on 6/1/2012 at 12:12 AM
If we could call PowerShell directly from SQL Server (SQL Connect 303417) that would help a fair bit with SQL PowerShell quoting issues when going in that direction. To have to call xp_cmdshell (so the old CMD shell) and then PowerShell adds to quoting problems.

In the PowerShell to SQL direction, an easy-to-fix sqlcmd example is the server name. In CMD shell
sqlcmd -S machine\instance -E
works. In PowerShell you have to do
sqlcmd -S 'machine\instance' -E
which may inhibit some DBAs from using PowerShell.
Posted by Microsoft on 12/9/2011 at 4:24 PM
Using the LiteralPath parameter is the pattern we've established for dealing with special characters. We've added LiteralPath to a bunch of cmdlets in PowerShell 3.0. We've also addressed a number of one-off cases in our backtick and square bracket handling. We realize we've not done here, so please continue to file additional specific high-impact cases so we can continue to address them in future releases.
Posted by enndub on 7/15/2011 at 5:04 PM
You can escape double quotes for command line arguments with a backslash.

PS C:\Documents and Settings\Nick> echo '"hello"'
"hello"
PS C:\Documents and Settings\Nick> echo.exe '"hello"'
hello
PS C:\Documents and Settings\Nick> echo.exe '\"hello\"'
"hello"

Much easier than most of the workarounds here, but it would still be nice to get this bug fixed.

As long as we pass powershell a properly escaped string, it should automatically take care of whatever escaping is necessary for however it executes command line programs.
Posted by Softlion on 12/14/2010 at 10:22 PM
The first bug is:

$toto = "toto"
$test = ($toto,'-value='+$toto)
$test

Result by Powershell v2:
toto
-value=
toto

The correct result should be:

toto
-value=toto


The second bug is:

# Needs poweshell community extentions installed
Import-Module Pscx
$toto = "toto"
$test = ($toto,"-value=`"$toto`"")
Start-Process EchoArgs.exe -Wait -NoNewWindow -ArgumentList $test


Result by Powershell v2:

Arg 0 is <toto>
Arg 1 is <-value=toto>

The correct result should be:

Arg 0 is <toto>
Arg 1 is <-value="toto">
Posted by Oisín Grehan on 8/17/2010 at 7:51 AM
MSDeploy.exe (web deployment tool) is practically unusable in powershell because of the hellish nature of this problem.
Posted by Microsoft Connect on 1/29/2010 at 1:55 PM
Your bug has not been fixed in PowerShell 2.0. However we have kept the bug active for next version
Posted by Joel 'Jaykul' Bennett on 11/4/2009 at 4:35 PM
Check the workarounds, that "Invoke-Legacy" is called Start-Process ;-)
Posted by Charlie Russel - MVP on 11/4/2009 at 3:45 PM
Sorry, fat fingers gave me double quotes in that comment when it should have been single quotes. But still lame. ;)
Posted by Charlie Russel - MVP on 11/4/2009 at 3:41 PM
Interesting suggestion from Kirk. I like it. Especially the alias. ;)

My current workaround is:
cmd /c "whatever it is I need to invoke"

Which is really stupid and lame, but PoSH doesn't always make it easy to avoid this. If our goal is to get rid of the need for CMD, then we need a way to more transparently handle this sort of scenario.
Posted by Kirk Munro on 11/4/2009 at 2:01 PM
Maybe this could be solved in PowerShell v3 with a new cmdlet, Invoke-LegacyCommand, that takes a script block with whatever legacy commands you want to invoke. Then it could simply parse that script block with specific rules for legacy commands (maybe evaluating variables inside double-quotes but leaving the rest as is), invoking the legacy commands correctly without a lot of quoting horseplay.

And you could give it a default alias of "cmd".... :)
Sign in to post a workaround.
Posted by BrettySpaghetti on 1/14/2014 at 7:23 AM
Hi,

The following workaround has worked for me:

$variableWithSpace = "String with spaces"
$variableQuotedSpaces = '"{0}"' -f "$variableWithSpace"

Now variableQuotedSpaces equals "String with spaces" including the quotation marks.

Hope it helps
Posted by stangm on 11/15/2013 at 8:02 AM
ignore below, added before adequate testing
Posted by stangm on 11/15/2013 at 7:59 AM
So here is a silly workaround for the add quotes problem (see my comment on the comments page:

Set-Content EchoTest.cmd "Echo %1"
$space=" "
$test="Hello$spaceThere"
.\echotest.cmd $test

Expected Output
Hello
Actual Output
Hello

Posted by Unregistered User on 9/12/2010 at 12:21 PM
To execute in PS we need scape $ not use second way
sqlcmd -v user=`"Joel Bennett`" -Q "select '`$(user)' as UserName"

Or
PS C:\test> $CmdLine="sqlcmd -v user=`"Joel Bennett`" -Q `"select '`$(user)' as UserName`""
PS C:\test> $CmdLine|cmd

Posted by Martin Zugec on 4/7/2010 at 7:58 AM
Instead of playing with ", ', '', ''''', """"" I personally prefer to use [Char]34, respectively $([Char]34).

Even though it sometimes takes longer to type (I use constant $QUOTE), you will get what you expect.
Posted by Tom Holt on 12/21/2009 at 2:36 PM
I gave up on any cmdlet provided by MS. Instead I use the Start method in the framework as follows:

[System.Diagnostics.Process]::Start("Cmd", "/c anything you want to put here with embedded quotes, and variables")

Works every time that I have used it.
Posted by Keith Hill MVP on 11/4/2009 at 7:28 PM
Start-Process doesn't work in this case e.g.:

PS> Microsoft.PowerShell.Management\start-process echoargs -Wait -NoNewWindow -Arg @'
>> -v -source:conn="Data Source=pubs" user="Joel Bennett" -Q "select '$(user)' as UserName"
>> '@
>>
Arg 0 is <-v>
Arg 1 is <-source:conn=Data Source=pubs>
Arg 2 is <user=Joel Bennett>
Arg 3 is <-Q>
Arg 4 is <select '$(user)' as UserName>

Note that arg 1 is missing the double quotes around "Data Source=pubs" and "Joel Bennett".
Posted by Joel 'Jaykul' Bennett on 11/4/2009 at 4:35 PM
In PowerShell 2, you can Use Start-Process:

Microsoft.PowerShell.Management\Start-Process sqlcmd -Arg @'
-v user="Joel Bennett" -Q "select '$(user)' as UserName"
'@ -Wait -NoNewWindow

It works perfectly, afaict (try it with the PSCX EchoArgs.exe).