C# code to download webpage

By codegambit52 on Oct 11, 2013

This code helps you in downloading any web page.

using System;
using System.Net;
using System.IO;

public class Test
    public static void Main (string[] args)
        if (args == null || args.Length == 0)
            throw new ApplicationException ("bestfishingrodsreview.com"); // Replace it with the URL that you want to download
        WebClient client = new WebClient ();

// Add user agent header if the URL contains query.

        client.Headers.Add ("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");

        Stream data = client.OpenRead (args[0]);
        StreamReader reader = new StreamReader (data);
        string s = reader.ReadToEnd ();
        Console.WriteLine (s);
        data.Close ();
        reader.Close ();


Sign in to comment.
Sorasyn   -  Oct 11, 2013

Wouldn't it be best to throw an NullReferenceException or an ArgumentNullException instead of an ApplicationException with the only helpful data being a website URL?

 throw new ArgumentNullException("Command line arguments expected, but not found."); 
codegambit52  -  Oct 11, 2013

@SunnyD I think u r rite. Let me write an updated version of this snippet. Thanks for pointing it out.

Sorasyn  -  Oct 11, 2013

To be clear, does this just grab the page's resulting HTML code and print it onto the console in a giant glob of text and symbols? What is the purpose of doing so?

You could also rewrite the StreamReader code to use a "using" code block, which would forego the need to close both data, and reader, as they will both be closed automatically.

using (StreamReader sr = new StreamReader(client.OpenRead(args[0]))) {
   while (!sr.EndOfStream)
Sign in to comment

Are you sure you want to unfollow this person?
Are you sure you want to delete this?
Click "Unsubscribe" to stop receiving notices pertaining to this post.
Click "Subscribe" to resume notices pertaining to this post.