grab.transport

Module Contents

Classes

Urllib3Transport

Grab network transport based on urllib3 library.

Functions

assemble(→ tuple[str, dict[str, Any], bytes | None])

Attributes

URL_DATA_METHODS

grab.transport.URL_DATA_METHODS

grab.transport.assemble(req: grab.request.Request, cookiejar: http.cookiejar.CookieJar) → tuple[str, dict[str, Any], bytes | None]

class grab.transport.Urllib3Transport

Bases: grab.base_transport.BaseTransport

Grab network transport based on urllib3 library.

__getstate__() → dict[str, Any]

__setstate__(state: collections.abc.Mapping[str, Any]) → None

build_pool() → urllib3.PoolManager

reset() → None

wrap_transport_error() → collections.abc.Generator[None, None, None]

select_pool_for_request(req: grab.request.Request) → PoolManager | ProxyManager | SOCKSProxyManager

request(req: grab.request.Request, cookiejar: http.cookiejar.CookieJar) → None

read_with_timeout(req: grab.request.Request) → bytes

get_response_header_items() → list[tuple[str, Any]]

Return current response headers as items.

This funciton is required to isolated smalles part of untyped code and hide it from mypy

prepare_response(req: grab.request.Request, *, document_class: type[grab.document.Document] = Document) → grab.document.Document

Prepare response, duh.

This methed is called after network request is completed hence the “self._request” is not None.

Good to know: on python3 urllib3 headers are converted to str type using latin encoding.

extract_cookiejar(req: grab.request.Request) → http.cookiejar.CookieJar