twisted.web.client

module documentation

(source)

HTTP client.

Class	`Agent`	`Agent` is a very basic HTTP client. It supports HTTP and HTTPS scheme URIs.
Class	`BrowserLikePolicyForHTTPS`	SSL connection creator for web clients.
Class	`BrowserLikeRedirectAgent`	An `Agent` wrapper which handles HTTP redirects in the same fashion as web browsers.
Class	`ContentDecoderAgent`	An `Agent` wrapper to handle encoded content.
Class	`CookieAgent`	`CookieAgent` extends the basic `Agent` to add RFC-compliant handling of HTTP cookies. Cookies are written to and extracted from a `cookielib.CookieJar` instance.
Class	`FileBodyProducer`	`FileBodyProducer` produces bytes from an input file object incrementally and writes them to a consumer.
Class	`GzipDecoder`	A wrapper for a `Response` instance which handles gzip'ed body.
Class	`HostnameCachingHTTPSPolicy`	IPolicyForHTTPS that wraps a `IPolicyForHTTPS` and caches the created `IOpenSSLClientConnectionCreator`.
Class	`HTTPClientFactory`	Download a given URL.
Class	`HTTPConnectionPool`	A pool of persistent HTTP connections.
Class	`HTTPDownloader`	Download to a file.
Class	`HTTPPageDownloader`	Undocumented
Class	`HTTPPageGetter`	Gets a resource via HTTP, then quits.
Class	`PartialDownloadError`	Page was only partially downloaded, we got disconnected in middle.
Class	`ProxyAgent`	An HTTP agent able to cross HTTP proxies.
Class	`RedirectAgent`	An `Agent` wrapper which handles HTTP redirects.
Class	`RequestGenerationFailed`	There was an error while creating the bytes which make up a request.
Class	`RequestTransmissionFailed`	There was an error while sending the bytes which make up a request.
Class	`Response`	A `Response` instance describes an HTTP response received from an HTTP server.
Class	`ResponseDone`	`ResponseDone` may be passed to `IProtocol.connectionLost` on the protocol passed to `Response.deliverBody` and indicates that the entire response has been delivered.
Class	`ResponseFailed`	`ResponseFailed` indicates that all of the response to a request was not received for some reason.
Class	`ResponseNeverReceived`	A `ResponseFailed` that knows no response bytes at all have been received.
Class	`URI`	A URI object.
Class	`WebClientContextFactory`	This class is deprecated. Please simply use `Agent` as-is, or if you want to customize something, use `BrowserLikePolicyForHTTPS`.
Function	`downloadPage`	Download a web page to a file.
Function	`getPage`	Download a web page as a string.
Function	`readBody`	Get the body of an `IResponse` and return it as a byte string.
Function	`urlunparse`	Undocumented
Constant	`SSL`	Undocumented
Class	`_AgentBase`	Base class offering common facilities for `Agent`-type classes.
Class	`_ContextFactoryWithContext`	A `_ContextFactoryWithContext` is like a `twisted.internet.ssl.ContextFactory` with a pre-created context.
Class	`_DeprecatedToCurrentPolicyForHTTPS`	Adapt a web context factory to a normal context factory.
Class	`_FakeUrllib2Request`	A fake `urllib2.Request` object for `cookielib` to work with.
Class	`_FakeUrllib2Response`	A fake `urllib2.Response` object for `cookielib` to work with.
Class	`_GzipProtocol`	A `Protocol` implementation which wraps another one, transparently decompressing received data.
Class	`_HTTP11ClientFactory`	A factory for `HTTP11ClientProtocol`, used by `HTTPConnectionPool`.
Class	`_ReadBodyProtocol`	Protocol that collects data sent to it.
Class	`_RetryingHTTP11ClientProtocol`	A wrapper for `HTTP11ClientProtocol` that automatically retries requests.
Class	`_StandardEndpointFactory`	Standard HTTP endpoint destinations - TCP for HTTP, TCP+TLS for HTTPS.
Function	`_deprecateGetPageClasses`	Mark the protocols and factories associated with `getPage` and `downloadPage` as deprecated.
Function	`_makeGetterFactory`	Create and connect an HTTP page getting factory.
Function	`_requireSSL`	The decorated method requires pyOpenSSL to be present, or it raises `NotImplementedError`.
Function	`_urljoin`	No summary
Constant	`_GETPAGE_REPLACEMENT_TEXT`	Undocumented

def downloadPage(url, file, contextFactory=None, *args, **kwargs): (source)

downloadPage was deprecated in Twisted 16.7.0; please use ZopeInterfaceAttribute 'twisted.web.client._GETPAGE_REPLACEMENT_TEXT' instead.

Download a web page to a file.

Parameters
url	Undocumented
file	path to file on filesystem, or file-like object. See HTTPDownloader to see what extra args can be passed.
contextFactory	Undocumented
*args	Undocumented
**kwargs	Undocumented

def getPage(url, contextFactory=None, *args, **kwargs): (source)

getPage was deprecated in Twisted 16.7.0; please use ZopeInterfaceAttribute 'twisted.web.client._GETPAGE_REPLACEMENT_TEXT' instead.

Download a web page as a string.

Download a page. Return a deferred, which will callback with a page (as a string) or errback with a description of the error.

See HTTPClientFactory to see what extra arguments can be passed.

def readBody(response): (source)

Get the body of an IResponse and return it as a byte string.

This is a helper function for clients that don't want to incrementally receive the body of an HTTP response.

Parameters
response:`IResponse` provider	The HTTP response for which the body will be read.
Returns
A `Deferred` which will fire with the body of the response. Cancelling it will close the connection to the server immediately.

def urlunparse(parts): (source)

Undocumented

SSL = (source)

Undocumented

Value

None

def _deprecateGetPageClasses(): (source)

Mark the protocols and factories associated with getPage and downloadPage as deprecated.

def _makeGetterFactory(url, factoryFactory, contextFactory=None, *args, **kwargs): (source)

Create and connect an HTTP page getting factory.

Any additional positional or keyword arguments are used when calling factoryFactory.

Parameters
url	Undocumented
factoryFactory	Factory factory that is called with `url`, `args` and `kwargs` to produce the getter
contextFactory	Context factory to use when creating a secure connection, defaulting to `None`
*args	Undocumented
**kwargs	Undocumented
Returns
The factory created by `factoryFactory`

def _requireSSL(decoratee): (source)

The decorated method requires pyOpenSSL to be present, or it raises NotImplementedError.

Parameters
decoratee:`callable`	A function which requires pyOpenSSL.
Returns
`callable`	A function which raises `NotImplementedError` if pyOpenSSL is not installed; otherwise, if it is installed, simply return `decoratee`.

def _urljoin(base, url): (source)

Construct a full ("absolute") URL by combining a "base URL" with another URL. Informally, this uses components of the base URL, in particular the addressing scheme, the network location and (part of) the path, to provide missing components in the relative URL.

Additionally, the fragment identifier is preserved according to the HTTP 1.1 bis draft.

Parameters
base:`bytes`	Base URL.
url:`bytes`	URL to combine with `base`.
Returns
An absolute URL resulting from the combination of `base` and `url`.
See Also
`urllib.parse.urljoin()`
https://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-7.1.2

_GETPAGE_REPLACEMENT_TEXT: str = (source)

Undocumented

Value

'https://pypi.org/project/treq/ or twisted.web.client.Agent'