Home Dashboard Directory Help
Search

Receiving binary data from PSRemoting is extremely slow by Altrive


Status: 

Active


1
0
Sign in
to vote
Type: Bug
ID: 801989
Opened: 9/23/2013 12:28:51 PM
Access Restriction: Public
0
Workaround(s)
view
1
User(s) can reproduce this bug

Description

When transfer binary data over PSRemoting. Receiveing binary data(byte[]) from PSSession is extremely slow and too much memory used .
(16 seconds to receive 256KB data and use about 250MB memory consumption in my environment(Windows 8/PowerShell v3))

If convert data manually to Base64 text before transfer. (Set $useBase64=$true in sample code). This problem is not happened.

I've confirmed that byte array transfer over PSRemoting is supported by following document .
http://msdn.microsoft.com/en-us/library/dd340926.aspx

It' seems, there is a problem in byte[] serializer/deserializer in PSRemoting.

TestCode(Revised: Initial commit has wrong transfer rate bps calculation )
---------------------------
function Main
{
    Clear-Host

    #Create PSSession
    $session = New-PSSession -ComputerName localhost

    #Set Test Configuration
    $bufferSize = 256KB
    $useBase64 = $false
    [GC]::GetTotalMemory($true) > $null
    
    Write-Host "Test Send-Data"
    Write-Host "-----------------------------------------------------------------"
    $sw = [Diagnostics.stopwatch]::StartNew()
    Send-Data -Session $session -BufferSize $bufferSize -UseBase64:$useBase64
    $elapsed = $sw.Elapsed
    Write-Host ("Elapsed: {0}[ms]" -f $elapsed.TotalMilliseconds)
    Write-Host ("Rate : {0:F2}[Mbps]" -f ($bufferSize / 1MB / $elapsed.TotalSeconds * 8))
    Write-Host ("Total Memory(Before GC): {0,-4}[MB]" -f ([GC]::GetTotalMemory($false) / 1MB))
    Write-Host ("Total Memory(After GC): {0,-4}[MB]" -f ([GC]::GetTotalMemory($true) / 1MB))

    Write-Host ""
    
    Write-Host "Test Receive-Data"
    Write-Host "-----------------------------------------------------------------"
    $sw = [Diagnostics.stopwatch]::StartNew()
    Receive-Data -Session $session -BufferSize $bufferSize -UseBase64:$useBase64
    $elapsed = $sw.Elapsed
    Write-Host ("Elapsed: {0}[ms]" -f $elapsed.TotalMilliseconds)
    Write-Host ("Rate : {0:F2}[Mbps]" -f ($bufferSize / 1MB / $elapsed.TotalSeconds * 8))
    Write-Host ("Total Memory(Before GC): {0,-4}[MB]" -f ([GC]::GetTotalMemory($false) / 1MB))
    Write-Host ("Total Memory(After GC): {0,-4}[MB]" -f ([GC]::GetTotalMemory($true) / 1MB))

    #Cleanup
    $session | Remove-PSSession
}

function Send-Data($Session, [long] $BufferSize, [switch] $UseBase64)
{
    Write-Host ("Create byte array(Size={0}KB)" -f ($BufferSize / 1KB)) -ForegroundColor Gray
    $buffer = New-Object byte[]($BufferSize)

    Write-Host "Set random data to suppress data compresion in PSRemoting" -ForegroundColor Gray
    $rand = New-Object Random
    $rand.NextBytes($buffer)

    if ($UseBase64){
        Write-Host "Start data transfer(Base64)..." -ForegroundColor Gray
        $buffer = [Convert]::ToBase64String($buffer)
    }
    else {
        Write-Host "Start data transfer(Byte[])..." -ForegroundColor Gray
    }

    $result = Invoke-Command -Session $Session -ScriptBlock {
        $data = $using:buffer
    }
    Write-Host "Complete data transfer..." -ForegroundColor Gray
}

function Receive-Data($Session, [long] $BufferSize, [switch] $UseBase64)
{
    $result = Invoke-Command -Session $Session -ScriptBlock {
        Write-Host ("Create byte array(Size={0}KB)" -f ($using:BufferSize / 1KB)) -ForegroundColor Gray
        $buffer = New-Object byte[]($using:BufferSize)

        Write-Host "Set random data to suppress data compresion in PSRemoting" -ForegroundColor Gray
        $rand = New-Object Random
        $rand.NextBytes($buffer)

        if ($using:UseBase64){
            Write-Host "Start data transfer(Base64)..." -ForegroundColor Gray
            return [Convert]::ToBase64String($buffer)
        }
        else {
            Write-Host "Start data transfer(Byte[])..." -ForegroundColor Gray
            return $buffer
        }
    }
    Write-Host "Complete data transfer..." -ForegroundColor Gray
}

#Execute
. Main
Details
Sign in to post a comment.
Posted by guitarrapc on 9/23/2013 at 1:28 PM
retry code with as Rate for Mbps was worng.
Also confirm it can reproduce for both localhost and Remote PC within LAN. (In AWS VPC, it was about same rate score as localhost)

if $useBase64 = $false :
Test Send-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Byte[])...
Complete data transfer...
Elapsed: 123.4679[ms]
Rate : 16.20[Mbps]
Total Memory(Before GC): 38.0815200805664[MB]
Total Memory(After GC): 34.6247100830078[MB]

Test Receive-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Byte[])...
Complete data transfer...
Elapsed: 18528.9313[ms]
Rate : 0.11[Mbps]
Total Memory(Before GC): 286.395645141602[MB]
Total Memory(After GC): 34.6760101318359[MB]




if $useBase64 = $true :
Test Send-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Base64)...
Complete data transfer...
Elapsed: 122.2188[ms]
Rate : 16.36[Mbps]
Total Memory(Before GC): 34.7945098876953[MB]
Total Memory(After GC): 30.9473495483398[MB]

Test Receive-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Base64)...
Complete data transfer...
Elapsed: 118.9348[ms]
Rate : 16.82[Mbps]
Total Memory(Before GC): 37.422492980957[MB]
Total Memory(After GC): 31.0038299560547[MB]
Posted by guitarrapc on 9/23/2013 at 12:45 PM
It can reproduced with PowerShell V3.0 on Windows Server 2012 R2.

if $useBase64 = $false :
Test Send-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Byte[])...
Complete data transfer...
Elapsed: 121[ms]
Rate : 0.27[Mbps]
Total Memory(Before GC): 24.6865768432617[MB]
Total Memory(After GC): 21.6620559692383[MB]

Test Receive-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Byte[])...
Complete data transfer...
Elapsed: 18467[ms]
Rate : 0.00[Mbps]
Total Memory(Before GC): 277.08805847168[MB]
Total Memory(After GC): 21.6804428100586[MB]




if $useBase64 = $true :
Test Send-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Base64)...
Complete data transfer...
Elapsed: 131[ms]
Rate : 0.25[Mbps]
Total Memory(Before GC): 25.4661331176758[MB]
Total Memory(After GC): 21.0235366821289[MB]

Test Receive-Data
-----------------------------------------------------------------
Create byte array(Size=256KB)
Set random data to suppress data compresion in PSRemoting
Start data transfer(Base64)...
Complete data transfer...
Elapsed: 123[ms]
Rate : 0.27[Mbps]
Total Memory(Before GC): 27.3952560424805[MB]
Total Memory(After GC): 21.0812835693359[MB]
Sign in to post a workaround.