20.5 WWW Page Retrieval: Using the HTTP Agent

The example program in this section demonstrates how to fetch a World Wide Web (WWW) page using the HTTP agent class.

HTTP is the native protocol of the WWW. It is a stateless client-server protocol: the client requests a document from the server using its Universal Resource Locator (URL), the server transmits the document to the client, and then the server shuts down the connection.

An HTTP URL has the following format:

http://<hostname>[:<port>]/[<path>]

where:

<hostname> is the host name (or IP address) of the server;

<port> is the port number for the server, which defaults to 80;

<path> is the path to the document. An empty path returns the root document.

20.5.1 Code and Explanation

Here is a portion of code followed by comments explaining key lines.

RWCString urlpath;
// ... construct urlpath

RWURL url(urlpath);
cout << "connecting to '" << url.host() << "'..." << endl;     // 1
RWIHttpAgent httpAgent(url_.host(), RWIHttpVersion_1_0());     // 2

RWCSocketPortal portal = httpAgent.get(urlpath);               // 3
RWPortalIStream strm(portal);                                  // 4

RWCString packet;
while (1) {
  packet.readLine(strm);                                       // 5
  if (packet.isNull()) break;
  cout << packet << "\n";
}

`//1`	Reports the HTTP server information to standard output. Note that an HTTP URL is stored in *url, which is an instance of the `int` library's RWURL* convenience class. This class encapsulates the parsing and component management of URL information. Here, the `host()` method of *RWURL* returns the host portion of the URL.
`//2`	Constructs an *RWIHttpAgent* object that connects to an HTTP/1.0 compliant server.
`//3`	Executes the `get()` method to retrieve the indented document, and redeems the *RWSocketPortal* from the returned *RWIOUResult*. Note that this function call may block.
`//4`	Builds an input stream from the socket portal.
`//5`	Reads the document's data from the socket portal.

20.5.2 Further Discussion

The RWURL class is provided in the int library as a convenience class. It recognizes some popular URL schemes and parses out their various parts via its methods. See the entry for class RWURL in the Tools.h++ Professional Class Reference for further information.