Skip to content

Proxies

The httpcore package provides support for HTTP proxies, using either "HTTP Forwarding" or "HTTP Tunnelling". Forwarding is a proxy mechanism for sending requests to http URLs via an intermediate proxy. Tunnelling is a proxy mechanism for sending requests to https URLs via an intermediate proxy.

Sending requests via a proxy is very similar to sending requests using a standard connection pool:

import httpcore

proxy = httpcore.HTTPProxy(proxy_url="http://127.0.0.1:8080/")
r = proxy.request("GET", "https://www.example.com/")

print(r)
# <Response [200]>

You can test the httpcore proxy support, using the Python proxy.py tool:

$ pip install proxy.py
$ proxy --hostname 127.0.0.1 --port 8080

Requests will automatically use either forwarding or tunnelling, depending on if the scheme is http or https.

Authentication

Proxy authentication can be included in the initial configuration:

import httpcore

# A `Proxy-Authorization` header will be included on the initial proxy connection.
proxy = httpcore.HTTPProxy(
    proxy_url="http://127.0.0.1:8080/",
    proxy_auth=("<username>", "<password>")
)

Custom headers can also be included:

import httpcore
import base64

# Construct and include a `Proxy-Authorization` header.
auth = base64.b64encode(b"<username>:<password>")
proxy = httpcore.HTTPProxy(
    proxy_url="http://127.0.0.1:8080/",
    proxy_headers={"Proxy-Authorization": b"Basic " + auth}
)

Proxy SSL

The httpcore package also supports HTTPS proxies for http and https destinations.

HTTPS proxies can be used in the same way that HTTP proxies are.

proxy = httpcore.HTTPProxy(proxy_url="https://127.0.0.1:8080/")

Also, when using HTTPS proxies, you may need to configure the SSL context, which you can do with the proxy_ssl_context argument.

import ssl
import httpcore

proxy_ssl_context = ssl.create_default_context()
proxy_ssl_context.check_hostname = False

proxy = httpcore.HTTPProxy('https://127.0.0.1:8080/', proxy_ssl_context=proxy_ssl_context)

It is important to note that the ssl_context argument is always used for the remote connection, and the proxy_ssl_context argument is always used for the proxy connection.

HTTP Versions

If you use proxies, keep in mind that the httpcore package only supports proxies to HTTP/1.1 servers.

SOCKS proxy support

The httpcore package also supports proxies using the SOCKS5 protocol.

Make sure to install the optional dependancy using pip install 'httpcore[socks]'.

The SOCKSProxy class should be using instead of a standard connection pool:

import httpcore

# Note that the SOCKS port is 1080.
proxy = httpcore.SOCKSProxy(proxy_url="socks5://127.0.0.1:1080/")
r = proxy.request("GET", "https://www.example.com/")

Authentication via SOCKS is also supported:

import httpcore

proxy = httpcore.SOCKSProxy(
    proxy_url="socks5://127.0.0.1:8080/",
    proxy_auth=("<username>", "<password>")
)
r = proxy.request("GET", "https://www.example.com/")

Reference

httpcore.HTTPProxy

A connection pool that sends requests via an HTTP proxy.

__init__(self, proxy_url, proxy_auth=None, proxy_headers=None, ssl_context=None, proxy_ssl_context=None, max_connections=10, max_keepalive_connections=None, keepalive_expiry=None, http1=True, http2=False, retries=0, local_address=None, uds=None, network_backend=None, socket_options=None) special

A connection pool for making HTTP requests.

Parameters:

Name Type Description Default
proxy_url Union[httpcore.URL, bytes, str]

The URL to use when connecting to the proxy server. For example "http://127.0.0.1:8080/".

required
proxy_auth Optional[Tuple[Union[bytes, str], Union[bytes, str]]]

Any proxy authentication as a two-tuple of (username, password). May be either bytes or ascii-only str.

None
proxy_headers Union[Mapping[Union[bytes, str], Union[bytes, str]], Sequence[Tuple[Union[bytes, str], Union[bytes, str]]]]

Any HTTP headers to use for the proxy requests. For example {"Proxy-Authorization": "Basic <username>:<password>"}.

None
ssl_context Optional[ssl.SSLContext]

An SSL context to use for verifying connections. If not specified, the default httpcore.default_ssl_context() will be used.

None
proxy_ssl_context Optional[ssl.SSLContext]

The same as ssl_context, but for a proxy server rather than a remote origin.

None
max_connections Optional[int]

The maximum number of concurrent HTTP connections that the pool should allow. Any attempt to send a request on a pool that would exceed this amount will block until a connection is available.

10
max_keepalive_connections Optional[int]

The maximum number of idle HTTP connections that will be maintained in the pool.

None
keepalive_expiry Optional[float]

The duration in seconds that an idle HTTP connection may be maintained for before being expired from the pool.

None
http1 bool

A boolean indicating if HTTP/1.1 requests should be supported by the connection pool. Defaults to True.

True
http2 bool

A boolean indicating if HTTP/2 requests should be supported by the connection pool. Defaults to False.

False
retries int

The maximum number of retries when trying to establish a connection.

0
local_address Optional[str]

Local address to connect from. Can also be used to connect using a particular address family. Using local_address="0.0.0.0" will connect using an AF_INET address (IPv4), while using local_address="::" will connect using an AF_INET6 address (IPv6).

None
uds Optional[str]

Path to a Unix Domain Socket to use instead of TCP sockets.

None
network_backend Optional[httpcore.NetworkBackend]

A backend instance to use for handling network I/O.

None
Source code in httpcore/__init__.py
def __init__(
    self,
    proxy_url: Union[URL, bytes, str],
    proxy_auth: Optional[Tuple[Union[bytes, str], Union[bytes, str]]] = None,
    proxy_headers: Union[HeadersAsMapping, HeadersAsSequence, None] = None,
    ssl_context: Optional[ssl.SSLContext] = None,
    proxy_ssl_context: Optional[ssl.SSLContext] = None,
    max_connections: Optional[int] = 10,
    max_keepalive_connections: Optional[int] = None,
    keepalive_expiry: Optional[float] = None,
    http1: bool = True,
    http2: bool = False,
    retries: int = 0,
    local_address: Optional[str] = None,
    uds: Optional[str] = None,
    network_backend: Optional[NetworkBackend] = None,
    socket_options: Optional[Iterable[SOCKET_OPTION]] = None,
) -> None:
    """
    A connection pool for making HTTP requests.

    Parameters:
        proxy_url: The URL to use when connecting to the proxy server.
            For example `"http://127.0.0.1:8080/"`.
        proxy_auth: Any proxy authentication as a two-tuple of
            (username, password). May be either bytes or ascii-only str.
        proxy_headers: Any HTTP headers to use for the proxy requests.
            For example `{"Proxy-Authorization": "Basic <username>:<password>"}`.
        ssl_context: An SSL context to use for verifying connections.
            If not specified, the default `httpcore.default_ssl_context()`
            will be used.
        proxy_ssl_context: The same as `ssl_context`, but for a proxy server rather than a remote origin.
        max_connections: The maximum number of concurrent HTTP connections that
            the pool should allow. Any attempt to send a request on a pool that
            would exceed this amount will block until a connection is available.
        max_keepalive_connections: The maximum number of idle HTTP connections
            that will be maintained in the pool.
        keepalive_expiry: The duration in seconds that an idle HTTP connection
            may be maintained for before being expired from the pool.
        http1: A boolean indicating if HTTP/1.1 requests should be supported
            by the connection pool. Defaults to True.
        http2: A boolean indicating if HTTP/2 requests should be supported by
            the connection pool. Defaults to False.
        retries: The maximum number of retries when trying to establish
            a connection.
        local_address: Local address to connect from. Can also be used to
            connect using a particular address family. Using
            `local_address="0.0.0.0"` will connect using an `AF_INET` address
            (IPv4), while using `local_address="::"` will connect using an
            `AF_INET6` address (IPv6).
        uds: Path to a Unix Domain Socket to use instead of TCP sockets.
        network_backend: A backend instance to use for handling network I/O.
    """
    super().__init__(
        ssl_context=ssl_context,
        max_connections=max_connections,
        max_keepalive_connections=max_keepalive_connections,
        keepalive_expiry=keepalive_expiry,
        http1=http1,
        http2=http2,
        network_backend=network_backend,
        retries=retries,
        local_address=local_address,
        uds=uds,
        socket_options=socket_options,
    )

    self._proxy_url = enforce_url(proxy_url, name="proxy_url")
    if (
        self._proxy_url.scheme == b"http" and proxy_ssl_context is not None
    ):  # pragma: no cover
        raise RuntimeError(
            "The `proxy_ssl_context` argument is not allowed for the http scheme"
        )

    self._ssl_context = ssl_context
    self._proxy_ssl_context = proxy_ssl_context
    self._proxy_headers = enforce_headers(proxy_headers, name="proxy_headers")
    if proxy_auth is not None:
        username = enforce_bytes(proxy_auth[0], name="proxy_auth")
        password = enforce_bytes(proxy_auth[1], name="proxy_auth")
        authorization = build_auth_header(username, password)
        self._proxy_headers = [
            (b"Proxy-Authorization", authorization)
        ] + self._proxy_headers