Downloading Information from Internet (Part 4)

by Apr 18, 2018

In the previous tip we explained how you can use Invoke-WebRequest to download data from webpages, and for example retrieve excuses from a webpage that serves random excuses. However, when you try this, you might get back always the same excuse (or data).

$url = 'http://pages.cs.wisc.edu/~ballard/bofh/bofhserver.pl' $page = Invoke-WebRequest -Uri $url -UseBasicParsing $content = $page.Content $pattern = '(?s)<br><font size\s?=\s?"\+2">(.+)</font' if ($page.Content -match $pattern) { $matches[1] } 

Most likely, this happens when you are behind a proxy server that cached the website information. To work around this, simply append the URL with a random parameter like so:

$url = "http://pages.cs.wisc.edu/~ballard/bofh/bofhserver.pl?$(Get-Random)" $page = Invoke-WebRequest -Uri $url -UseBasicParsing $content = $page.Content $pattern = '(?s)<br><font size\s?=\s?"\+2">(.+)</font' if ($page.Content -match $pattern) { $matches[1] } 

Twitter This Tip! ReTweet this Tip!