Accessing the Internet from PowerShell: Net.WebClient

The first thing we need to understand in order to make PowerShell talk with web content, is a .Net framework object, WebClient. WebClient is found in the System.Net namespace and provides us, the humble PowerShell developer with a simplified method for accessing HTTP/HTTPS/FTP content. Yes, you did just read that correctly, not only will our code be able to make use of HTTP(S) based content but it will also allow us to hit FTP Servers as well!

So lets get a PowerShell window open, and get an instance of the WebClient object. Simply make a new-object of type Net.WebClient and put it in a variable!

Congratulations, you have completed the first step.

Now lets take a look at what the WebClient offers us:

WOAH, that is quite a few methods and properties there.

For now, as I am just introducing the WebClient to you, lets just try to download a single page. Lets put the URL we want into a variable, and then call DownloadString, with a parameter of the URL variable:

The output, if everything has worked and your internet is running, will start with something like:

What the hell? that doesn’t look like Google.com does in Internet Explorer! Yes, that's because we will be seeing the un-rendered HTML, so things will be a little less friendly.

That is it for todays lesson, try downloading some pages yourself and see how you go. The next lesson will cover setting up some optional settings for WebClient such as controlling proxy server access and other context information.

Accessing the Internet from PowerShell

I had many potential titles for this series of posts, as yes, this is the first of a series. The aim of the series is to slowly guide you through interacting with web pages from within PowerShell, extending to downloading and uploading files, posting data to forms and interacting with WebDAV. My aim is to start by talking about each individual task, accessing a page, posting/uploading, download files, etc; cover off some of the issues we might encounter or what things to be on the look out for when working within a corporate environment (authentication and proxy servers) and finally bring everything together into 1) a working PasteBin scrapper that will look for the occurrences of certain works and notify us and 2) a PowerShell module which will allow us to quickly perform all of the above mention web/internet related tasks. I will provide a tonne of examples along the way to assist in helping you understand all of the different concepts that are involved.

Change to code content

I hated how code was being displayed on my site, so you may notice I have started to change it.

All code snippets will now be stored in pastebin.org, and then displayed using embedded links.

Ping Monitoring

I made a script to ping an address and alert me if it doesn't succeed. As an added bonus, it will also alert if the response time becomes to large. Once again, this is based on the usual template.

Enjoy.

Restart stopped services which are set to automatic

This script is handy for your Exchange 2010 deployments, it will find any automatic service which is stopped and start it (optional) and let you know that it happened. This is similar to a previous scripts I have posted.

As usual it uses my usual template.

Why you reboot?

I wanted to know when and by what/who/why servers rebooted. This is just a simple script, and you configure the it to run at start up in Windows task scheduler.

XML + PowerShell = Really neat config files

There are often times where you need to store data within a file to make your script work, examples include:

Complex configuration information
Historical information
Complex data storage
There are a number of ways you can address this when writing a normal .Net application, but the majority of these can easily be used within Powershell. These options include simple text files, SQL databases and what I want to talk about today, XML files!

Using XML files is very simple in PowerShell, most of the hard work will be performed by the Powershell engine and the .Net libraries it is built upon.

Example XML File

Reading data within an XML file
To read an XML file into a variable, simply

Now if we wanted to access all of the XML elements under the "Folders" element, all we would have to do is:

But if there are no "folder" elements below "folders", you will have a nice error message when you go to run that foreach loop! You can resolve this by testing if there are elements under the "folders" element by:

if the above returns false, then there are no elements sitting under "folders". Note that this command is CASE SENSITIVE!!! folders and Folders are not the same. BEWARE!

This is enough for today, and should allow you to understand some of the scripts I will be posting up soon.

Server Up Time Monitoring

These days, servers, even ones running Windows, can be up for a very long time. I recently had a request where they wanted an alert generated if a server had be up for longer than a time period.

I modify the script that can be found here, http://pshscripts.blogspot.com.au/2009/01/get-uptimeps1.html, to fit my standard template and just my preferred style.