API Reference#

Public package interface for souppot HTML fetching helpers.

souppot.cold_soup(url, check_errors=False)#

Fetch a URL with requests and parse HTML responses.

Parameters:

  • url (str | None) – URL to fetch. None and blank strings are treated as missing.

  • check_errors (bool) – If true, call raise_for_status() before normal status handling.

Return type:

BeautifulSoup | Response | None

Returns:

BeautifulSoup for 200 responses with a text/html content type, the raw requests.Response for other 200 responses, and None for missing URLs or non-200 responses.

Raises:

requests.HTTPError – If check_errors is true and the response status is an HTTP error.

souppot.hot_pot(url, dest, referer=None, timeout_ms=60000)#

Download a URL with Playwright’s request context and save it to disk.

Parameters:

  • url (str | None) – URL to download. None and blank strings raise ValueError.

  • dest (str | Path) – Destination file path. Parent directories are created if needed.

  • referer (str | None) – Optional Referer header to send with the request.

  • timeout_ms (int) – Playwright request timeout in milliseconds.

Return type:

Path

Returns:

The destination path as a Path.

Raises:

  • ValueError – If url is missing.

  • playwright.sync_api.Error – If Playwright cannot complete the request.

souppot.hot_soup(url, wait_seconds=3, wait_selector=None)#

Render a URL with Playwright Chromium and parse the final DOM.

Parameters:

  • url (str | None) – URL to render. None and blank strings are treated as missing.

  • wait_seconds (float) – Seconds to sleep after domcontentloaded when no wait_selector is provided. When waiting for a selector, this is converted to the selector timeout with a minimum of 1000 ms.

  • wait_selector (str | None) – Optional CSS selector to wait for before parsing. If the selector times out, the currently rendered DOM is parsed anyway.

Return type:

BeautifulSoup | None

Returns:

BeautifulSoup for the rendered page, or None for missing URLs or Playwright errors.