grab.base

Module Contents

Classes

Grab

class grab.base.Grab(transport: None | BaseTransport | type[BaseTransport] = None, **kwargs: Any)

property doc: None | Document

__slots__ = ['proxylist', 'config', 'transport', 'request_method', 'cookies', 'meta', '_doc']

document_class :type[grab.document.Document]

clonable_attributes = ['proxylist']

process_transport_option(transport: None | BaseTransport | type[BaseTransport], default_transport: type[grab.base_transport.BaseTransport]) → grab.base_transport.BaseTransport

clone(**kwargs: Any) → Grab

Create clone of Grab instance.

Cloned instance will have the same state: cookies, referrer, response document data

Parameters

\**kwargs – overrides settings of cloned grab instance

dump_config() → collections.abc.MutableMapping[str, Any]

Make clone of current config.

load_config(config: grab.types.GrabConfig) → None

Configure grab instance with external config object.

setup(**kwargs: Any) → None

Set up Grab instance configuration.

prepare_request() → grab.request.Request

Configure all things to make real network request.

This method is called before doing real request via transport extension.

create_request_from_config(config: collections.abc.MutableMapping[str, Any]) → grab.request.Request

sync_cookie_manager_with_request_cookies(cookies: collections.abc.Mapping[str, Any], request_url: str) → None

log_request(req: grab.request.Request, extra: str = '') → None

Send request details to logging system.

find_redirect_url(doc: grab.document.Document) → None | str

request(url: None | str = None, **kwargs: Any) → grab.document.Document

Perform network request.

You can specify grab settings in **kwargs. Any keyword argument will be passed to self.config.

Returns: Document objects.

submit(make_request: bool = True, **kwargs: Any) → None | Document

Submit current form.

Parameters

make_request – if False then grab instance will be configured with form post data but request will not be performed

For details see Document.submit() method

process_request_result(req: grab.request.Request) → grab.document.Document

Process result of real request performed via transport extension.

reset_temporary_options() → None

change_proxy(random: bool = True) → None

Set random proxy from proxylist.

classmethod common_headers() → dict[str, str]

Build headers which sends typical browser.

make_url_absolute(url: str, resolve_base: bool = False) → str

Make url absolute using previous request url as base url.

clear_cookies() → None

Clear all remembered cookies.

__getstate__() → dict[str, Any]

__setstate__(state: collections.abc.Mapping[str, Any]) → None