This module is the portable HTTP layer, so a caller sends and receives HTTP messages through the same types on every supported platform. The unit of work is a message, which is a header and an optional body. A Request is the message a client sends and a server receives, and a Reply is the message a server sends and a client receives. Both are Message types that share the header-and-body model and differ in the start line.
A Client is an HTTP user agent that holds one request and one reply. There is no separate connect step, because the client opens a TCP connection to its Endpoint when a send needs one. A persistent connection is a keep-alive header that the server also permits, and pipelining reuses that connection. After a 401 reply, Authenticator prepares the request so it can be sent again.
A Server listens on an Endpoint and does not implement resources itself. A Servlet maps an incoming request to a Service and may attach an Authorizer. The service is a factory for Responder objects, and each responder handles one request and writes one reply.
HTTP I/O takes part in the System event-loop model. Asynchronous work attaches a client or a server with setActive() so an EventLoop can monitor the connection, but the loop does not own the HTTP object: the code that creates it keeps it alive while an operation is still waiting on the loop. The same objects also have blocking send and receive operations, though asynchronous operation is the usual path.
Endpoints and TCP belong to Pt::Net. This module names a host and a port with Endpoint and listens or connects through that address. HTTPS is the same client and server API after setSecure() has been given a Pt::Ssl::Context; certificate and handshake details live in the SSL module.
A protocol error that is specific to HTTP is an HttpError, which is an I/O error. A local address that is already occupied still throws AddressInUse from the listen that uses it.
An HTTP connection can be upgraded. WebSocket is the framed IODevice that follows a WebSocket handshake, which the client opens and the server accepts through a WebSocketService and an IOStream taken from the upgraded connection.
The rest of this chapter is the message model, then the client, then the server, then the WebSocket upgrade.
This chapter covers:
An HTTP message is a header and an optional body. The header is a list of name and value fields together with the HTTP version, and the body is a byte stream. Message holds both. Request and Reply are messages: a request adds the method, the URL and the query string, while a reply adds the status code and the status text.
Callers do not normally construct these types. A Client owns the request it will send and the reply it receives, and a Responder is given the server-side request and reply for one exchange, so the same header and body operations apply on both sides.
MessageHeader stores fields in a fixed buffer. set() replaces a field, add() appends another value for the same name, and get() and has() look up a name. begin() and end() walk the fields. Keep-alive, chunked transfer coding, content length and Upgrade are derived from those fields rather than being a second header model.
body() is a bidirectional stream on the message. Write to it to fill a request or a reply that will be sent, and read from it when a receive step has delivered body bytes. discard() drops buffered body data. available() and pending() report how much can be read or still has to be written.
Asynchronous send and receive move a message in steps, and each completed step returns a MessageProgress value. header() is true when the header of that message is available, body() is true when body bytes were processed, and finished() is true when the message is complete. A short message often sets all three on one step, while a long body, or a send that only partly left the socket, needs another begin and end pair. Progress can also be empty, meaning the step made I/O progress without exposing a header or a body yet, in which case the next begin call continues the same message.
HttpError is the HTTP-specific I/O error. Credential is a user name and password that client authentication and server authorization both use, though they do not use the same authenticator type.
MessageHeader is the field list and HTTP version of a message. It is the header half of Message, and through that of Request and Reply. Callers do not usually construct a header on its own; they use header() on the message the client or responder already holds.
Fields are names and values. set() replaces the value for a name, add() appends another value for the same name, and remove() deletes it. get() returns the value or a null pointer, has() reports presence, and isSet() tests that a name has a particular value. begin() and end() walk the fields as Field values, each with a name and a value pointer into the header's buffer.
Keep-alive, chunked transfer coding, content length and Upgrade are not a second header model: they are derived from those fields. isKeepAlive(), isChunked(), contentLength() and isUpgrade() read them, while setKeepAlive() and setUpgrade() write the corresponding fields. HTTP version is versionMajor() and versionMinor(), set together with setVersion().
The header has a fixed size, so the caller must not rely on filling it beyond that limit. clear() empties the fields so the same header can be used for another message.
Message is the header-and-body object that Request and Reply extend. The header is header(), and the body is body(), an iostream. Write the body before sending, and read it when progress reports that body bytes are available.
Callers do not construct a Message. A Client owns the request and reply, and a Responder is passed the server-side pair. The connection that carries the message is connection(), which is the HTTP connection the client or server already opened.
available() is how many body bytes can be read without blocking, and pending() is how many body bytes still have to be written. discard() drops the buffered body so the stream can be reused for another message.
MessageProgress is the result of one begin/end step on a message. The client returns it from endSend() and endReceive(), and the same flags describe how far a request or a reply has moved.
The flags are independent and can be combined. header() means the header of that message is now available to read or has been sent. body() means body bytes were processed on this step. finished() means the message is complete, so the caller must not begin the same send or receive again. trailer() means trailer fields were processed.
A short reply often arrives in one receive step, in which case header, body and finished are all true. A large body, or a socket that accepted only part of a write, returns finished as false, and the caller calls begin again. It is also possible that I/O made progress without a header or body becoming visible yet, so none of the three is true; begin again until finished.
For a chunked send, finished means the current chunk has left the socket, not that the whole request is complete. The completion flag on the next beginSend() decides whether another chunk follows.
Request is the Message a client sends and a server receives. It adds the request line: method, URL and query string. The default method is GET. setUrl() sets the resource path, setMethod() sets the verb, and setQParams() sets the query without the leading question mark. Header fields and the body are the inherited Message operations.
On the client, the request is Client::request(). Fill it before beginSend() or beginReceive(), including keep-alive or content headers that the exchange needs. On the server, the responder receives it in onBeginRequest() and onReadRequest(), where the header is already parsed and the body arrives in chunks.
clear() resets URL, method, query and the inherited header and body so the same request object can be filled for another exchange. Send and receive on Request itself are used by the connection; the client API is Client::beginSend() and Client::beginReceive().
Reply is the Message a server sends and a client receives. It adds the status line: statusCode() and statusText(), set together with setStatus(). The default is 200 OK. The nested StatusCode enumeration names the codes the API uses in replies it generates, but setStatus() accepts any code.
On the client, the reply is Client::reply() after a receive step has made it available. Read body() when progress reports body bytes, and read the status when progress reports the header. On the server, the responder writes the reply and starts sending it with beginSend(). The completion flag is true when this is the last chunk of the body, and false when onWriteReply() should run again for more. Calling beginSend(true) finishes the reply and releases the responder.
clear() resets status, header and body so the same reply object can be used for another exchange.
Client is the HTTP user agent. It holds one Request and one Reply, so the usual work is to fill the request, send it, and read the reply. The client opens a TCP connection to its host when a send needs one; there is no separate connect method.
The host is an Endpoint passed to a constructor or to setHost(). Asynchronous work needs an EventLoop, passed to a constructor or to setActive(), but the loop does not own the client. HTTPS is the same send and receive path after setSecure() has been given a Pt::Ssl::Context.
The request is request(). Set the URL, the method, query parameters and header fields before the send starts. The default method is GET. Write the request body with request().body(). The reply is reply() after a receive step has made it available.
Asynchronous receive is beginReceive() and endReceive(), and replyReceived() is emitted when a step has completed. endReceive() returns MessageProgress; if the reply is not finished, call beginReceive() again. Asynchronous send is beginSend() and endSend(), with requestSent() as the matching signal. A single beginReceive() is enough when the request is already complete and the caller only waits for the reply. Send first when the request has a body, when the body is chunked, or when several requests are pipelined.
A keep-alive header on the request asks for a persistent connection, which the server may still close. Pipelining needs that persistent connection, because several requests are sent before the matching replies are received. close() ends the connection. Leave it open only while the next request will reuse it; a later send on a closed or timed-out connection opens a new one.
send() and receive() are the blocking forms of the same exchange, and they complete the request or the reply on the calling thread.
When a reply is 401, Authenticator complements the request from realm credentials so the client can send it again. Basic authentication is built in, and other Authentication methods can be added.
Client is the HTTP user agent in the client model. It holds one Request and one Reply. Fill the request, send it, and read the reply. The client opens a TCP connection to its host when a send needs one, so there is no separate connect method. The host is an Endpoint passed to a constructor or to setHost().
Asynchronous work needs an EventLoop, passed to a constructor or to setActive(). The loop does not own the client; keep the client alive while an operation is still waiting on the loop. setTimeout() bounds I/O. send() and receive() are the blocking forms of the same exchange.
The request is request(). Set the URL, the method, query parameters and header fields before the send starts, and write the body with request().body(). The default method is GET. The reply is reply() after a receive step has made it available.
Asynchronous receive is beginReceive() and endReceive(). replyReceived() is emitted when a step has completed. The slot calls endReceive(), which returns MessageProgress. If the header is available, the status can be read; if the body is available, it can be read from reply().body(); if the reply is not finished, beginReceive() continues the same reply. A short reply often completes in one step.
The example is an asynchronous GET. The client is constructed with the loop and the host, the request URL is set, and beginReceive() starts the exchange. The slot ends each receive step and exits the loop when the reply is finished.
Pipelining sends several requests before receiving the matching replies, which needs a persistent connection. Set the keep-alive header on the request, connect requestSent() as well as replyReceived(), and start with beginSend() rather than beginReceive(). The send slot calls endSend(); if that send is not finished, beginSend() continues it, and if it is finished, the next request can be filled and sent. When no further request will be pipelined, beginReceive() starts reading the replies. Identify each reply by order or by application state, because Reply does not store the request URL.
A chunked request body is sent with beginSend(false) until the last chunk, so the completion flag is false while more body data will be written. endSend() reports whether the current chunk has left the socket, not whether the whole request is complete. When a chunk has finished and more data remains, write it to request().body() and call beginSend(false) again. When no more chunks remain, beginReceive() finishes the request correctly. To pipeline another chunked request after this one, call beginSend(true) so the request body is terminated.
A keep-alive header on the request asks for a persistent connection, which pipelining needs and which the server may still close. close() ends the connection. Leave it open only while the next request will reuse it; otherwise the server keep-alive timeout may close it before the client is used again. A later send on a closed or timed-out connection opens a new one.
setSecure() assigns a Pt::Ssl::Context so further connections are HTTPS. setPeerName() sets the name expected in the peer certificate. Send and receive are otherwise unchanged. Certificate and handshake details live in the SSL module.
Authenticator prepares a request after a 401 reply so the client can send it again. It is not the server-side authorizer. Store credentials per realm with setCredential(), then call authenticate() with the rejected request and the 401 reply. The method writes the authentication headers onto the request and returns true when that is possible. It returns false when no credentials are available for the realm, or when the challenge cannot be met.
Basic authentication is registered by default. Other Authentication methods can be added with addAuthentication(). The authenticator does not send the request; the caller sends it again through the client.
The example stores a realm credential and applies it to a request that just received 401.
Server is the listening HTTP service. It binds a local Endpoint and accepts connections, but it does not implement a resource. Incoming requests are mapped by servlets. Each Servlet combines a mapping rule, a Service, and an optional Authorizer. addServlet() registers the servlet, and the first servlet that maps the request handles it.
A service is a factory: onGetResponder() creates a Responder for the request, and onReleaseResponder() destroys it after the reply has been sent or the exchange has failed. BasicService is that factory for a responder type, with an allocator. Use a custom Service when the responder type depends on the request, or when responders are pooled.
A responder handles one exchange, and the server calls it in order. onBeginRequest() runs when the request header is available, onReadRequest() runs for each chunk of the request body, onBeginReply() runs when the request is complete and the reply should start, and onWriteReply() runs when a previous beginSend(false) needs another chunk of the reply body. The responder finishes the reply with Reply::beginSend() and the completion flag set to true. It may send that finished reply from an earlier callback, in which case remaining callbacks are skipped and the rest of the request is ignored.
All server I/O is asynchronous, so the server needs an EventLoop, passed to a constructor or to setActive(). listen() binds the local endpoint, and a second listen replaces the previous binding. setSecure() restricts the server to HTTPS. setMaxThreads() bounds worker threads, and timeouts together with setMaxRequestSize() bound idle connections and request size.
MapUrl maps one exact URL, and MapAny maps every request. A custom servlet implements onRequest() and returns true when its service should run. Several servlets may share one service, so the same resource can appear under more than one name.
Authorizer is the server-side access check. It is attached to a servlet, not to the server as a whole, and the same authorizer may be shared. BasicAuthorizer implements HTTP Basic authentication and asks a derived class whether the credentials are granted. Authorization may complete immediately or through an asynchronous Authorization object.
The example is the listening server and a URL mapping. The service is a BasicService for a responder type that the next sections implement, and the loop runs the server.
Server is the listening HTTP service in the server model. It binds a local Endpoint and accepts connections, but it does not implement a resource. Incoming requests are mapped by servlets that have been added with addServlet(). The first servlet that maps the request handles it. removeServlet() unregisters a servlet.
All server I/O is asynchronous, so the server needs an EventLoop, passed to a constructor or to setActive(). The loop does not own the server. listen() binds the local endpoint, optionally with TcpServerOptions, and a second listen replaces the previous binding. cancel() stops accepting and pending work.
setSecure() assigns a Pt::Ssl::Context so the server accepts only HTTPS connections. Further use of servlets and responders is unchanged. Certificate and handshake details live in the SSL module.
setMaxThreads() bounds worker threads. setTimeout() and setKeepAliveTimeout() bound idle I/O and persistent connections, and setMaxRequestSize() rejects a request whose size exceeds that limit.
Servlet is the mapping rule in the server model. It combines a Service with an optional Authorizer and decides whether an incoming request belongs to that service. The server calls isMapped(), which forwards to onRequest(), and the first servlet that returns true handles the request.
Construct it with a service, or with a service and an authorizer. Several servlets may share one service, so the same resource can appear under more than one name, and they may share one authorizer. service() and authorizer() return those pointers.
MapUrl maps one exact URL. MapAny maps every request. A custom servlet implements onRequest() and returns true when its service should run. Do not reimplement MapUrl unless the mapping rule is actually different.
setShutdown() marks the servlet so it stops taking new work, and isIdle() is true when no exchange is using it. detach() unregisters it from the server that holds it.
The example is the usual mapping: one URL, one service. The optional authorizer is passed as a third argument.
Service is the responder factory in the server model. The server does not construct responders itself: when a servlet maps a request, the service's getResponder() calls onGetResponder(), and when the reply has been sent or the exchange has failed, releaseResponder() calls onReleaseResponder(). The service must remain alive while any responder it created is still in use.
BasicService is that factory for a single responder type, using an allocator that defaults to new and delete. Use a custom Service when the responder type depends on the request headers, or when responders are pooled.
upgradeRequested() is emitted when a request asks to upgrade the connection. The signal provides the accepted IOStream and the value of the Upgrade header. That stream is the upgraded connection, not the HTTP message body.
The example is the usual factory: a BasicService for a responder type. The equivalent hand-written service implements onGetResponder() and onReleaseResponder() in the same way.
Responder is the per-exchange handler in the server model. A Service creates it for a mapped request and releases it when the reply has been sent. Derive from it and implement the four callbacks, which the server calls in order.
onBeginRequest() runs when the request header is available, so the responder can inspect fields and prepare the reply. onReadRequest() runs for each chunk of the request body, and may be called more than once. onBeginReply() runs when the request is complete and the reply should start. onWriteReply() runs when a previous Reply::beginSend(false) needs another chunk of the reply body, which is how a large reply is written in chunked encoding.
Finish the reply with Reply::beginSend() and the completion flag set to true. That call may be made from an earlier callback, in which case the remaining callbacks are skipped and the rest of the request is ignored. service() is the service that created this responder.
The example is a responder that ignores the request body and writes a fixed reply. beginSend(true) completes the reply in onBeginReply(), so onWriteReply() stays empty.
Authorizer is the access check in the server model. It is attached to a Servlet, not to the server as a whole, and the same authorizer may be shared by several servlets. It is not the client-side Authenticator. The realm is passed to the constructor and returned by realm().
beginAuthorize() starts the check for a request and reply. When the result is already known, granted is set and a null pointer is returned. When the check needs I/O, an Authorization object is returned; endAuthorization() completes it, and cancelAuthorization() aborts it. The authorizer releases that object in onReleaseAuthorization().
BasicAuthorizer implements HTTP Basic authentication. A derived class implements onAuthorizeCredentials() and either sets granted immediately or returns an Authorization for later completion. BasicUserListAuthorizer is that check against an in-memory user list.
The example grants or denies access from credentials in one step, so it returns a null pointer and does not need to release an authorization object.
WebSocket is an HTTP upgrade. The HTTP exchange performs a handshake, and after the handshake the same TCP connection carries framed messages instead of request and reply messages. WebSocket is the IODevice for that framed stream: read() and write() transfer payload bytes, while the frame type is separate. setSendFrame() selects the opcode for the next send, and receiveFrame() reports the opcode of the frame that was last received.
On the client, beginConnect() sends the handshake to a ws:// URL. connected() is emitted when the attempt finishes, and endConnect() completes it and throws if the handshake failed. The socket must be attached to an event loop before the connect starts.
On the server, WebSocketService is an HTTP Service, mapped with a servlet like any other service. Its responder answers a WebSocket Upgrade request with 101 Switching Protocols, or with 404 when the request is not a WebSocket upgrade. After a successful upgrade the connection is an IOStream, and WebSocket::accept() takes that stream and becomes the framed device.
IOStream is the upgraded connection, not the HTTP message body. Service::upgradeRequested() reports that stream and the Upgrade header value to a service that handles upgrades itself.
Ping and pong are control frames. sendPingFrame() writes a ping, and after a ping is received sendPongFrame() writes the matching pong. Text and binary frames are the data payload. Unknown is the unset frame type.
The example is a client handshake. The slot completes the connect and then writes through the inherited device operation.
WebSocket is the IODevice that follows a WebSocket handshake. After the handshake the same TCP connection carries framed messages instead of HTTP request and reply messages. read() and write() transfer payload bytes. The frame type is separate: setSendFrame() selects the opcode for the next send, and receiveFrame() reports the opcode of the frame that was last received.
On the client, attach the socket to an event loop, connect connected(), and call beginConnect() with a ws:// URL. endConnect() completes the handshake and throws if it failed. On the server, accept() takes the IOStream of an upgraded connection, which a WebSocketService handshake produces.
Ping and pong are control frames. sendPingFrame() writes a ping, and after a ping is received sendPongFrame() writes the matching pong. Text and binary are the data payload. Unknown is the unset frame type.