Speeding Up Background Jobs

by Jun 30, 2014

Background jobs can be a great thing to speed up scripts because they can do things in parallel. However, background jobs only work well if the code you run does not produce large amounts of data – because transporting back the data via XML serialization often takes more time than you can save by executing things in parallel.

Fortunately, you control how much data background jobs return. Have a look.

This is a piece of code where three tasks ($code1-3) run in parallel. Two runs in background jobs, and one uses the foreground PowerShell.

$start = Get-Date

$code1 = { Get-Hotfix }
$code2 = { Get-ChildItem $env:windir\system32\*.dll }
$code3 = { Get-Content -Path C:\Windows\WindowsUpdate.log }

$job1 = Start-Job -ScriptBlock $code1 
$job2 = Start-Job -ScriptBlock $code2
$result3 = & $code3 

$alljobs = Wait-Job $job1, $job2 

Remove-Job -Job $alljobs
$result1, $result2 = Receive-Job $alljobs

$end = Get-Date
$timespan = $end - $start
$seconds = $timespan.TotalSeconds
Write-Host "This took me $seconds seconds."

This takes about half a minute. When you run all three jobs sequentially and do not use background jobs, they just take 5 seconds.

$start = Get-Date

$result1 = Get-Hotfix 
$result2 = Get-ChildItem $env:windir\system32\*.dll 
$result3 = Get-Content -Path C:\Windows\WindowsUpdate.log 

$end = Get-Date
$timespan = $end - $start
$seconds = $timespan.TotalSeconds
Write-Host "This took me $seconds seconds."

So background jobs really have made the code more complex and increased the script runtime.
Only when you start and optimize the return data will background jobs become useful. The fewer data they emit, the better.

$start = Get-Date

$code1 = { Get-Hotfix | Select-Object -ExpandProperty HotfixID }
$code2 = { Get-Content -Path C:\Windows\WindowsUpdate.log | Where-Object { $_ -like '*successfully installed*' }}
$code3 = { Get-ChildItem $env:windir\system32\*.dll | Select-Object -ExpandProperty Name }

$job1 = Start-Job -ScriptBlock $code1 
$job2 = Start-Job -ScriptBlock $code2
$result3 = & $code3 

$alljobs = Wait-Job $job1, $job2 

Remove-Job -Job $alljobs
$result1, $result2 = Receive-Job $alljob 

This time, the background jobs only return as much data as is really needed, and the job with the most output is moved to the foreground PowerShell. This will cut down execution time considerably.

Generally, background jobs work best when they simply do something (for example a configuration) but do not return anything or return just a small amount of data.

Twitter This Tip! ReTweet this Tip!