Proxies
The httpcore
package provides support for HTTP proxies, using either "HTTP Forwarding" or "HTTP Tunnelling". Forwarding is a proxy mechanism for sending requests to http
URLs via an intermediate proxy. Tunnelling is a proxy mechanism for sending requests to https
URLs via an intermediate proxy.
Sending requests via a proxy is very similar to sending requests using a standard connection pool:
import httpcore
proxy = httpcore.HTTPProxy(proxy_url="http://127.0.0.1:8080/")
r = proxy.request("GET", "https://www.example.com/")
print(r)
# <Response [200]>
You can test the httpcore
proxy support, using the Python proxy.py
tool:
$ pip install proxy.py
$ proxy --hostname 127.0.0.1 --port 8080
Requests will automatically use either forwarding or tunnelling, depending on if the scheme is http
or https
.
Authentication
Proxy authentication can be included in the initial configuration:
import httpcore
# A `Proxy-Authorization` header will be included on the initial proxy connection.
proxy = httpcore.HTTPProxy(
proxy_url="http://127.0.0.1:8080/",
proxy_auth=("<username>", "<password>")
)
Custom headers can also be included:
import httpcore
import base64
# Construct and include a `Proxy-Authorization` header.
auth = base64.b64encode(b"<username>:<password>")
proxy = httpcore.HTTPProxy(
proxy_url="http://127.0.0.1:8080/",
proxy_headers={"Proxy-Authorization": b"Basic " + auth}
)
Proxy SSL and HTTP Versions
Proxy support currently only allows for HTTP/1.1 connections to the proxy, and does not currently support SSL proxy connections, which require HTTPS-in-HTTPS,
SOCKS proxy support
The httpcore
package also supports proxies using the SOCKS5 protocol.
Make sure to install the optional dependancy using pip install httpcore[socks]
.
The SOCKSProxy
class should be using instead of a standard connection pool:
import httpcore
# Note that the SOCKS port is 1080.
proxy = httpcore.SOCKSProxy(proxy_url="socks5://127.0.0.1:1080/")
r = proxy.request("GET", "https://www.example.com/")
Authentication via SOCKS is also supported:
import httpcore
proxy = httpcore.SOCKSProxy(
proxy_url="socks5://127.0.0.1:8080/",
proxy_auth=("<username>", "<password>")
)
r = proxy.request("GET", "https://www.example.com/")
Reference
httpcore.HTTPProxy
A connection pool that sends requests via an HTTP proxy.
__init__(self, proxy_url, proxy_auth=None, proxy_headers=None, ssl_context=None, max_connections=10, max_keepalive_connections=None, keepalive_expiry=None, http1=True, http2=False, retries=0, local_address=None, uds=None, network_backend=None, socket_options=None)
special
A connection pool for making HTTP requests.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
proxy_url |
Union[httpcore.URL, bytes, str] |
The URL to use when connecting to the proxy server.
For example |
required |
proxy_auth |
Optional[Tuple[Union[bytes, str], Union[bytes, str]]] |
Any proxy authentication as a two-tuple of (username, password). May be either bytes or ascii-only str. |
None |
proxy_headers |
Union[Mapping[Union[bytes, str], Union[bytes, str]], Sequence[Tuple[Union[bytes, str], Union[bytes, str]]]] |
Any HTTP headers to use for the proxy requests.
For example |
None |
ssl_context |
Optional[ssl.SSLContext] |
An SSL context to use for verifying connections.
If not specified, the default |
None |
max_connections |
Optional[int] |
The maximum number of concurrent HTTP connections that the pool should allow. Any attempt to send a request on a pool that would exceed this amount will block until a connection is available. |
10 |
max_keepalive_connections |
Optional[int] |
The maximum number of idle HTTP connections that will be maintained in the pool. |
None |
keepalive_expiry |
Optional[float] |
The duration in seconds that an idle HTTP connection may be maintained for before being expired from the pool. |
None |
http1 |
bool |
A boolean indicating if HTTP/1.1 requests should be supported by the connection pool. Defaults to True. |
True |
http2 |
bool |
A boolean indicating if HTTP/2 requests should be supported by the connection pool. Defaults to False. |
False |
retries |
int |
The maximum number of retries when trying to establish a connection. |
0 |
local_address |
Optional[str] |
Local address to connect from. Can also be used to
connect using a particular address family. Using
|
None |
uds |
Optional[str] |
Path to a Unix Domain Socket to use instead of TCP sockets. |
None |
network_backend |
Optional[httpcore.backends.base.NetworkBackend] |
A backend instance to use for handling network I/O. |
None |