API reference

Sync API

Synchronous Zyte API client.

api_key is your Zyte API key. If not specified, it is read from the ZYTE_API_KEY environment variable. See API key.

Alternatively, you can set an Ethereum private key through eth_key to use Ethereum for payments. If not specified, it is read from the ZYTE_API_ETH_KEY environment variable. See x402.

api_url is the Zyte API base URL. If set to None, it defaults to "https://api.zyte.com/v1/". If using an Ethereum private key, e.g. through eth_key or through the ZYTE_API_ETH_KEY environment variable, None results in "https://api-x402.zyte.com/v1/" instead.

n_conn is the maximum number of concurrent requests to use. See Optimization.

retrying is the retry policy for requests. Defaults to zyte_api_retrying.

user_agent is the user agent string reported to Zyte API. Defaults to python-zyte-api/<VERSION>.

trust_env controls whether aiohttp honors environment-based network settings (e.g. HTTP_PROXY and HTTPS_PROXY). Defaults to False.

Tip

To change the User-Agent header sent to a target website, use customHttpRequestHeaders instead.

get(query: dict[str, Any], *, endpoint: str = 'extract', session: ClientSession | None = None, handle_retries: bool = True, retrying: AsyncRetrying | None = None) → dict[str, Any][source]

Send query to Zyte API and return the result.

endpoint is the Zyte API endpoint path relative to the client object api_url.

session is the network session to use. Consider using session() instead of this parameter.

handle_retries determines whether or not a retry policy should be used.

retrying is the retry policy to use, provided handle_retries is True. If not specified, the default retry policy is used.

iter(queries: list[dict[str, Any]], *, endpoint: str = 'extract', session: ClientSession | None = None, handle_retries: bool = True, retrying: AsyncRetrying | None = None) → Generator[dict[str, Any] | Exception, None, None][source]

Send multiple queries to Zyte API in parallel and iterate over their results as they come.

The number of queries can exceed the n_conn parameter set on the client object. Extra queries will be queued, there will be only up to n_conn requests being processed in parallel at a time.

Results may come an a different order from the original list of queries. You can use echoData to attach metadata to queries, and later use that metadata to restore their original order.

When exceptions occur, they are yielded, not raised.

The remaining parameters work the same as in get().

session(**kwargs: Any) → _Session[source]

Context manager to create a session.

A session is an object that has the same API as the client object, except:

get() and iter() do not have a session parameter, the session creates an aiohttp.ClientSession object and passes it to get() and iter() automatically.
It does not have a session() method.

Using the same aiohttp.ClientSession object for all Zyte API requests improves performance by keeping a pool of reusable connections to Zyte API.

The aiohttp.ClientSession object is created with sane defaults for Zyte API, but you can use kwargs to pass additional parameters to aiohttp.ClientSession and even override those sane defaults.

You do not need to use session() as a context manager as long as you call close() on the object it returns when you are done:

session = client.session()
try:
    ...
finally:
    session.close()

Async API

Asynchronous Zyte API client.

Parameters work the same as for ZyteAPI.

async get(query: dict[str, Any], *, endpoint: str = 'extract', session: ClientSession | None = None, handle_retries: bool = True, retrying: AsyncRetrying | None = None) → dict[str, Any][source]: Asynchronous equivalent to ZyteAPI.get().

iter(queries: list[dict[str, Any]], *, endpoint: str = 'extract', session: aiohttp.ClientSession | None = None, handle_retries: bool = True, retrying: AsyncRetrying | None = None) → Iterator[_ResponseFuture][source]: Asynchronous equivalent to ZyteAPI.iter().

Note

Yielded futures, when awaited, do raise their exceptions, instead of only returning them.

session(**kwargs: Any) → _AsyncSession[source]

Asynchronous equivalent to ZyteAPI.session().

You do not need to use session() as an async context manager as long as you await close() on the object it returns when you are done:

session = client.session()
try:
    ...
finally:
    await session.close()

Retries

zyte_api_retrying: Default retry policy.

aggressive_retrying: Aggresive retry policy.

class RetryFactory[source]

Factory class that builds the tenacity.AsyncRetrying object that defines the default retry policy.

To create a custom retry policy, you can subclass this factory class, modify it as needed, and then call build() on your subclass to get the corresponding tenacity.AsyncRetrying object.

For example, to double the number of attempts for download errors and the time network errors are retried:

from zyte_api import (
    RetryFactory,
    stop_after_uninterrupted_delay,
    stop_on_download_error,
)


class CustomRetryFactory(RetryFactory):
    network_error_stop = stop_after_uninterrupted_delay(30 * 60)
    download_error_stop = stop_on_download_error(max_total=8, max_permanent=4)


CUSTOM_RETRY_POLICY = CustomRetryFactory().build()

class AggressiveRetryFactory[source]

Factory class that builds the tenacity.AsyncRetrying object that defines the aggressive retry policy.

To create a custom retry policy, you can subclass this factory class, modify it as needed, and then call build() on your subclass to get the corresponding tenacity.AsyncRetrying object.

For example, to double the maximum number of attempts for all error responses and double the time network errors are retried:

from zyte_api import (
    AggressiveRetryFactory,
    stop_after_uninterrupted_delay,
    stop_on_count,
    stop_on_download_error,
)


class CustomRetryFactory(AggressiveRetryFactory):
    download_error_stop = stop_on_download_error(max_total=16, max_permanent=8)
    network_error_stop = stop_after_uninterrupted_delay(30 * 60)
    undocumented_error_stop = stop_on_count(8)


CUSTOM_RETRY_POLICY = CustomRetryFactory().build()

Errors

exception RequestError(*args: Any, **kwargs: Any)[source]

Exception raised upon receiving a rate-limiting or unsuccessful response from Zyte API.

property parsed: ParsedError: Response as a ParsedError object.

query: dict[str, Any]

Query sent to Zyte API.

May be slightly different from the input query due to pre-processing logic on the client side.

request_id: str | None: Request ID.

response_content: bytes | None: Response body.

class ParsedError(response_body: bytes, data: dict | None, parse_error: str | None)[source]

Parsed error response body from Zyte API.

data: dict | None

JSON-decoded response body.

If None, parse_error indicates the reason.

classmethod from_body(response_body: bytes) → ParsedError[source]: Return a ParsedError object built out of the specified error response body.

parse_error: str | None: If data is None, this indicates whether the reason is that response_body is not valid JSON ("bad_json") or that it is not a JSON object ("bad_format").

response_body: bytes: Raw response body from Zyte API.

property type: str | None: ID of the error type, e.g. "/limits/over-user-limit" or "/download/temporary-error".