Automatic Web Page Use With .NET

          Automatic Web Page Use With .NET

Using web browser macros, the need to get more control over what is happening during the automated use of a web page becomes increasingly obvious. After trying a number of possible solutions, C# and .NET provided the best option. Find out how.

There are lots of reasons why you might want to automate your interaction with a web page. In this project the example, and this is a real example, is of an ISP providing a monthly bandwidth allowance. The problem is keeping track of how much data has been used via the ISP's website. Checking it manually is a time- consuming chore. Automating the job is made more difficult by the need to log on, navigate to the correct page, and then extract the data. While this can be done using browser macro language, it doesn't provide an easy way to process the retrieved data. 

Next we need a makeBrowser method:

private void makeBrowser()
{
 webBrowser1 = new System.Windows.
                         Forms.WebBrowser();
 webBrowser1.Name = "webBrowser1";
}

While we don't actually need a method to show the browser it is invaluable during debugging:

private void showBrowserInForm()
{
 webBrowser1.Location =
         new System.Drawing.Point(12, 300);
 webBrowser1.MinimumSize =
         new  System.Drawing.Size(20, 20);
 webBrowser1.Size =
         new System.Drawing.Size(783, 338);
 webBrowser1.TabIndex = 1;
 Controls.Add(webBrowser1);
}

This adds the control to the current form. We could have used a new form if you wanted to make the browser pop-up separately from the application.

Multipage

The first thing we have to figure out is how to navigate to a page and respond when the page is loaded. Navigating to a page is just a matter of setting the URL property to a valid URL - finding out when the page is fully loaded is a little more difficult. When the document is loaded it triggers a DocumentCompleted event. The problem is that this event is triggered for each frame loaded. This means that you could get multiple DocumentCompleted events per page load.  The solution to this problem is to test the ReadyState and make sure that it is set to Complete before processing the page. 

void webBrowser1_DocumentCompleted(
   object sender,  
   WebBrowserDocumentCompletedEventArgs e)
{
 if ( webBrowser1.ReadyState!=
     WebBrowserReadyState.Complete) return;