Howdy! I still need help in this week's project. Here below is the description. Using Python Eclipse 3.5

Summary:

There are two ways you can use the Google search engine: 1) using the Google Search API. 2) construct an HTTP requrest and send it to the Google server. In this assignment, we will use the second method.  write a program that constructs a http request with search keywords that the user provides and send a search query to the Google search engine.  Parse the result of the search query and extract all the links in the response which uses http.

Description:

The program must prompt the user to enter a word or words of interest to perform a Google search.  Then, it constructs the http request in the form of a URL and requsts the result from the Google server. The Google server sends the HTML formatted response in the form of binary data. The program must decode the binary data to string format and parse the HTML content and extract the links which start with ‘http’ in the response.

The basic form of a search query is:

http://www.google.com/search?q=<search words separated by +> (Links to an external site.)

for example ,if the user enters the search words “ucla extension”, the search query looks like:

http://www.google.com/search?q=ucal+extension (Links to an external site.)

Since we are not using the Google APIs, the Google server may block your request unless it is coming from a valid user agent (i.e. browser). Use the sample code below to form the request object. The first argument to the “Request” construtor is the URL that you need to construct as described above.

user_agent = 'Mozilla/5.0'

   

headers={'User-Agent':user_agent,}

   

request= Request(url,None,headers)

response = urlopen(request)

 

For the second part of the program, define a class “MyHTMLParser” that parses the html response and extract the links that start with ‘http’. Use the example code provided in the module presentation slides. You must use regular expression to find a matching links that start with ‘http’.

This is an example of what the program output would look like if you search for “ucla extension” keyword:

http://www.uclaextension.edu/str/HowtoEnroll.aspx#Concurrent

http://www.ucla.edu/admission/extension-enrollment

http://webcache.googleusercontent.com/search?q=cache:OVUo66xNi78J:www.ucla.edu/admission/extension-enrollment+&cd=10&hl=en&ct=clnk&gl=us

http://www.ucla.edu/academics/continuing-education

http://webcache.googleusercontent.com/search?q=cache:5aQmeiLNRFoJ:www.ucla.edu/academics/continuing-education+&cd=11&hl=en&ct=clnk&gl=us

http://www.yelp.com/biz/ucla-extension-los-angeles-6

http://webcache.googleusercontent.com/search?q=cache:04zPJLOjhOQJ:www.yelp.com/biz/ucla-extension-los-angeles-6+&cd=12&hl=en&ct=clnk&gl=us

http://en.wikipedia.org/wiki/UCLA_Extension

http://www.facebook.com/UCLAExtensionEntertainmentStudies

http://www.facebook.com/UCLAExtensionEntertainmentStudies

http://www.google.com/shopping?hl=en&tab=wf&ei=nj7mVqu9PI--jwPx4JKADw&ved=0EKkuCAwoDA