Beginning with PowerShell 3.0, the cmdlet Invoke-WebRequest can download web page content quite easily. This would scrape all links from www.powertheshell.com for example:
#requires -Version 3 $url = 'http://www.powertheshell.com' $page = Invoke-WebRequest -Uri $url $page.Links
You can also access the raw HTML content like this:
#requires -Version 3 $url = 'http://www.powertheshell.com' $page = Invoke-WebRequest -Uri $url $page.RawContent
When you do this with other URLs, you may occasionally see security dialogs popping up, asking for permission to store cookies. To suppress these dialogs and run the command unattended, make sure you add the –UseBasicParsing parameter.
