E:\web\learnaspcom\htdocs\freebook\learn\ubtoc.xml LearnAsp.com - ASP ASP.net Free Lessons
Search Search

#1 worldwide
FREE Coding Lessons
since 1996
   THE BEST WAY to learn ASP & Asp.net!
Advertise Here!
click for details 		Credits Host:
DiscountASP.net 		Server Admin:
The "Team" 		Contact Info.
Charles M. Carroll 		<Asp.net blog>
<personal site>
xxx

User Controls For Scraping websites
by Charles Carroll and Steve Smith

This page demonstrates a very useful in the real-world a User Control to encapsulate grabbing the resultant string a website would generate if invoked from a browser.

Simple page:

   filename=/experiments/usercontrols/scraper-test.aspx

<Test Script Below>

 

<%@ Register TagPrefix="LearnAsp" TagName="Scraper" Src="Scraper.ascx" %>
<html><head>
<title>Sample of Scraper</title>
</head>
<body bgcolor="#FFFFFF">
<LearnAsp:Scraper theURL="http://www.cnn.com" htmlencode="true" runat="server"/>
</body></html>

The User Control:

filename=/experiments/usercontrols/scraper.ascx

 

<script language="VB" runat="server">
public htmlencode as boolean = false
public theUrl as string
Sub Page_Load(Src As Object, E As EventArgs) 
    Try
        Dim objResponse As System.Net.WebResponse
        Dim objRequest As System.Net.WebRequest
        objRequest = System.Net.HttpWebRequest.Create(theurl)
        objResponse = objRequest.GetResponse()
        Dim sr As new system.io.StreamReader(objResponse.GetResponseStream())
        If htmlencode THEN
            results.text=server.HTMLencode(sr.ReadToEnd())
        Else
            results.text=sr.ReadToEnd()
        end if
    Catch ex As Exception
        results.text=ex.Message
    End Try
End Sub
</script>
<asp:literal id="results"  runat="server" />
There are many worthy charities!!. But perhaps help starving children in Africa or South America AND help Charles too. a $5 tip buys him lunch at McDonalds, a $20 tip buys his kid Hitoshi a new computer game, a $39 tip buys his daughter Michiko a few nice outfits. See our donor list.