Enable Real-Time Streaming with Foreach

by Jan 4, 2019

Classic foreach loops are the fastest loop available but they come with a severe limitation. Foreach loops do not support streaming. You need to always wait for the entire foreach loop to finish before you can start processing the results.

Here are some example illustrating this. With the code below, you have to wait a long time until you “see” the results:

$result = foreach ($item in $elements)
{
    "processing $item"
    # simulate some work and delay
    Start-Sleep -Milliseconds 50
}

$result | Out-GridView 

You cannot pipe the results directly. The following code produces a syntax error:

$elements = 1..100

Foreach ($item in $elements)
{
    "processing $item"
    # simulate some work and delay
    Start-Sleep -Milliseconds 50
} | Out-GridView 

You could use $() to enable piping, but you would still have to wait for the loop to finish before the results are piped in one big chunk:

$elements = 1..100

$(foreach ($item in $elements)
{
    "processing $item"
    # simulate some work and delay
    Start-Sleep -Milliseconds 50
}) | Out-GridView 

Here is a little-known trick that adds real-time streaming to foreach: simply use a script block!

$elements = 1..100

& { foreach ($item in $elements)
{
    "processing $item"
    # simulate some work and delay
    Start-Sleep -Milliseconds 50
}} | Out-GridView 

Now you “see” the results as they are produced, and enjoy real-time streaming.

Of course, you could as well abandon foreach in the first place, and use the ForEach-Object pipeline cmdlet instead:

$elements = 1..100

$elements | ForEach-Object {
    $item = $_
    
    "processing $item"
    # simulate some work and delay
    Start-Sleep -Milliseconds 50
} | Out-GridView

However, ForEach-Object is much slower than the foreach keyword, and there are cases where you simply can’t use ForEach-Object. For example, in a lot of database code, your code needs to check for end-of-file flags and reads records one by one which exclude ForEach-Object as an option.


psconf.eu – PowerShell Conference EU 2019 – June 4-7, Hannover Germany – visit www.psconf.eu There aren’t too many trainings around for experienced PowerShell scripters where you really still learn something new. But there’s one place you don’t want to miss: PowerShell Conference EU – with 40 renown international speakers including PowerShell team members and MVPs, plus 350 professional and creative PowerShell scripters. Registration is open at www.psconf.eu, and the full 3-track 4-days agenda becomes available soon. Once a year it’s just a smart move to come together, update know-how, learn about security and mitigations, and bring home fresh ideas and authoritative guidance. We’d sure love to see and hear from you!

Twitter This Tip! ReTweet this Tip!