���Z@ � & @ s- d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m
Z
d d l m Z m
Z
m Z m Z d d l m Z d d l m Z m Z d d l m Z d d l m Z m Z m Z d d
l Td d l m Z m Z m Z m Z m Z d d
l Td d l m Z m Z m! Z! d d
l" m# Z# d d
l Td d l m$ Z$ d d l% m& Z& d d l' m( Z( m) Z) m* Z* m+ Z+ m, Z, m- Z- d d l m. Z. m/ Z/ m0 Z0 d d l1 m2 Z2 m3 Z3 m4 Z4 d d l5 m6 Z6 d d l7 m8 Z8 m9 Z9 d d l: m; Z; e j< e j< e j< d d d f Z< d d Z= Gd d � d � Z> Gd d � d e
� Z? Gd d � d e? � Z@ Gd d! � d! e? � ZA Gd" d# � d# � ZB d$ d d% d d& d d' d d( d d) d d* d d+ d, d- d. d/ e jC d0 d d1 d d2 d3 d4 d d5 d d6 d, d7 d d8 d d9 d � ZD d S):zHTTP Client for asyncio.� N)� Coroutine)� CIMultiDict� MultiDict�MultiDictProxy�istr)�URL� )�client_exceptions�
client_reqrep)� connector)�hdrs�http�payload)�*)� ClientError�
ClientOSError�
InvalidURL�ServerTimeoutError�WSServerHandshakeError)�
ClientRequest�ClientResponse�_merge_ssl_params)�ClientWebSocketResponse)� TCPConnector)� CookieJar)�PY_36� CeilTimeout�
TimeoutHandle�proxies_from_env�sentinel�strip_auth_from_url)�WS_KEY�WebSocketReader�WebSocketWriter)�WSHandshakeError�
ws_ext_gen� ws_ext_parse)�FlowControlDataQueue)�tcp_cork� tcp_nodelay)�Trace�
ClientSessionr �request� �< c , s� e Z d Z d Z e d d d d d d d d d
d d d
d d d d d d d d g � Z d Z d Z d Z d d d d d d d d d d d d d e j
d e d e d! e
d" e j d# d d$ d d% d& d' e d( d d) d d* d& d+ d d, d- � Z d. d/ � Z � f d0 d1 � Z e d2 d3 � Z d4 d5 � Z d6 d d7 d d8 d d d d d d d d9 d d: d; d< d d= d d> d& d? d d@ d dA d dB e dC d dD d dE d dF d dG d dH d dI dJ � Z dK f dB dL dM d dN d dO d dP d d d dQ d d d d@ d dA d dF d dC d dD d dE d dG d d< dR dS dT � Z dK f dB dL dM d dN d dO d dP d d d dQ d d d d@ d dA d dF d dC d dD d dE d dG d d< dR dU dV � Z dW dX � Z d9 d dY dZ � Z d9 d d[ d\ � Z d9 d& d] d^ � Z d7 d d_ d` � Z d7 d da db � Z d7 d dc dd � Z de df � Z! dg dh � Z" e# di dj � � Z$ e# dk dl � � Z% e# dm dn � � Z& e# do dp � � Z' e# dq dr � � Z( ds dt � Z) du dv � Z* dw dx � Z+ dy dz � Z, d{ d| � Z- � S)}r+ z/First-class interface for making HTTP requests.�_source_traceback�
_connector�requote_redirect_url�_loop� _cookie_jar�_connector_owner�
_default_auth�_version�_json_serialize�
_read_timeout�
_conn_timeout�_raise_for_status�_auto_decompress�
_trust_env�_default_headers�_skip_auto_headers�_request_class�_response_class�_ws_response_class�_trace_configsNTr �loop�cookies�headers�skip_auto_headers�auth�json_serialize�
request_class�response_class�ws_response_class�version�
cookie_jar�connector_owner�raise_for_statusF� read_timeout� conn_timeout�auto_decompress� trust_env�
trace_configsc C sY d } | d k r< | d k r* | j } n d } t j � } | d k rW t d | � } | j | k rr t d � � | | _ | j � r� t j t j d � � | _
| r| j � rt j
d d d �d | d
d i } | j
d k r� | j
| d <| j | � | d k r t d | � } | | _ | d k rC| j j | � | | _ |
| _ | | _ | | _ | | _ | t k r�| n t | _ | | _ | | _ | | _ | | _ | r�t | � } n t � } | | _ | d k rt d
d � | D� � | _! n t � | _! | | _" | | _# |
| _$ | p1g | _% x | j% D] } | j& � qAWd S)NFTrC z0Session and connector has to use same event loopr zGCreating a client session outside of coroutine is a very dangerous idea�
stacklevel� �client_session�messagez.Creating a client session outside of coroutine�source_tracebackc S s g | ] } t | � � q S� )r )�.0�irZ rZ �
/client.py�
<listcomp>� s z*ClientSession.__init__.<locals>.<listcomp>)'r2 �asyncioZget_event_loopr � RuntimeErrorZ get_debug� traceback�
extract_stack�sys� _getframer/ Z
is_running�warnings�warn�call_exception_handlerr r3 �update_cookiesr0 r4 r5 r6 r7 r �DEFAULT_TIMEOUTr8 r9 r: r; r< r r= � frozensetr> r? r@ rA rB Zfreeze)�selfr rC rD rE rF rG rH rI rJ rK rL rM rN rO rP rQ rR rS rT Z
implicit_loop�context� trace_configrZ rZ r] �__init__A sh
zClientSession.__init__c C s& t j d j | j � t d d �d S)Nz6Inheritance class {} from ClientSession is discouragedrU rV )re rf �format�__name__�DeprecationWarning)�clsrZ rZ r] �__init_subclass__� s z ClientSession.__init_subclass__c sE | | j k r. t j d j | � t d d �t � j | | � d S)Nz8Setting custom ClientSession.{} attribute is discouragedrU rV )�ATTRSre rf ro rq �super� __setattr__)rk �name�val)� __class__rZ r] rv � s zClientSession.__setattr__c C s� | j s~ t r d | i } n i } | j d j | � t | � d | d d i } | j d k rn | j | d <| j j | � d S)N�sourcez Unclosed client session {!r}rW rX zUnclosed client sessionrY )�closedr rf ro �ResourceWarningr/ r2 rg )rk � _warnings�kwargsrl rZ rZ r] �__del__� s
zClientSession.__del__c K s t | j | | | � � S)zPerform HTTP request.)�_RequestContextManager�_request)rk �method�urlr~ rZ rZ r] r, � s zClientSession.request�params�data�json�allow_redirects�
max_redirects�
�compress�chunked� expect100�read_until_eof�proxy�
proxy_auth�timeout�
verify_ssl� fingerprint� ssl_context�ssl�
proxy_headers�trace_request_ctxc , F � s � j r t d � � t | | | | � } | d k rQ | d k rQ t d � � n$ | d k ru t j | d � j �} t | t � r� | d k r� t j
d t � d } g } � j } � j
| � } � j
| � } y t | � } Wn t k
rt | � � Yn Xt � j � } | d k rDx! | D] } | j t | � � q'W| d k r�y t | � } Wn t k
r�t | � � Yn Xt � j | t k r�| n � j � } | j � } � � f d d � � j D� } x& | D] } | j | | | � Id Hq�W| j � }! y�|! ��x�t | � \ } }" | r?|" r?t d � � | d k rQ|" } | d k rf� j } | d k r�| d k r�t j | k r�t d � � | j d � } � j! j" | � }# | d k r�t | � } nK � j# r x? t$ � j% � D]. \ }$ }% |$ | j& k r�|% j' } |% j( } Pq�W� j) | | d
| d | d | d
| d |# d | d | d | d | d |
d � j d � j* d | d | d |! d � d � j+ d | d | �}& yA t, � j- d � j ��" � j. j/ |& d | �Id H}' Wd QRXWn@ t0 j1 k
r0}( z t2 d j3 | � � |( � WYd d }( ~( Xn Xt4 |' j5 d � t6 |' j5 d � yX |& j7 |' � }) y |) j |' | � Id HWn) t8 k
r�|) j9 � |' j9 � � Yn XWnK t: k
r�� Yn7 t; k
r�}( z t< |( j= � |( � WYd d }( ~( Xn X� j! j> |) j? |) j@ � |) jA d, k r| rx) | D]! } | jB | | | |) � Id Hq,W| d& 7} | jC |) � |
r�| |
k r�|) j9 � Pn
|) jD � |) jA d# k r�|) jE t jF k s�|) jA d- k r|) jE t jG k rt jH } d } | jI t jJ � r| jK t jJ � |) jL jI t jM � p,|) jL jI t jN � }* |* d k r<Py t |* d' � jO �}* Wn t k
rvt |* � � Yn X|* j& }$ |$ d. k r�|) j9 � t d+ � � n |$ s�| jP |* � }* | jQ � |* jQ � k r�d } | jK t j d � |* } d } |) jD � qPqWWd QRX� jR r#|) jS � | d k r^|) jT d k rT|) jT jU | jV � n
| jV � tW | � |) _X x) | D]! } | jY | | | |) � Id HqtW|) SWnt t8 k
r}+ zT | j9 � | r�| jV � d } x) | D]! } | jZ | | | |+ � Id Hq�W� WYd d }+ ~+ Xn Xd S)/NzSession is closedz9data and json parameters can not be used at the same time�dumpsz Chunk size is deprecated #1615r c s. g | ]$ } t � | | j d � � � � q S)r� )r* Ztrace_config_ctx)r[ rm )rk r� rZ r] r^ � s z*ClientSession._request.<locals>.<listcomp>z<Cannot combine AUTH argument with credentials encoded in URLzTCannot combine AUTHORIZATION header with AUTH argument or credentials encoded in URLr� rE rF r� rD rG rL r� r� r� rC rJ r� r� �timer�sessionrR r� r� �tracesz Connection timeout to host {0}TF�- �. �/ �3 �4 r �encodedr
�https� z"Can redirect only to http or https)r� r� r� r� r� )r� r� )r
r� r� )[r{ r` r �
ValueErrorr Z JsonPayloadr7 �
isinstance�boolre rf rq r6 �_prepare_headersr r �setr> �addr r r2 r r8 �startrB Zsend_request_startr� r r5 r Z
AUTHORIZATIONZ
with_fragmentr3 Zfilter_cookiesr< r �items�schemer� r� r? r@ r; r r9 r0 �connectr_ � TimeoutErrorr ro r) � transportr( �send�
BaseException�closer �OSErrorr �argsrh rD r� �statusZsend_request_redirect�append�releaser� � METH_HEAD� METH_POST�METH_GET�getZCONTENT_LENGTH�poprE ZLOCATIONZURIr1 �join�originr: rO �
connectionZ add_callback�cancel�tupleZ_historyZsend_request_endZsend_request_exception),rk r� r� r� r� r� rE rF rG r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� Z redirects�historyrL Z skip_headersr\ Ztm�handler� Ztracer� Z
auth_from_urlrD r� Z
proxy_infoZreq�conn�exc�respZr_url�erZ )rk r� r] r� � sV
"
%
zClientSession._request� protocolsg $@�receive_timeout� autoclose�autoping� heartbeatr� r c % C sy t | j | d | d | d | d | d | d | d | d | d |
d
| d | d |
d
| d | d | d | d | �� S)z Initiate websocket connection.r� r� r� r� r� r� rG r� rE r� r� r� r� r� r� r� r� )�_WSRequestContextManager� _ws_connect)rk r� r� r� r� r� r� r� rG r� rE r� r� r� r� r� r� r� r� rZ rZ r] �
ws_connect� s& zClientSession.ws_connectc # % � s� |
d k r t � }
t j t j t j t j t j d i } x0 | j � D]" \ } } | |
k rI | |
| <qI Wt j t j
d � � } | j � |
t j <| r� d j
| � |
t j <| d k r� | |
t j <| r� t d | � } | |
t j <t |
| | | � }
| j | d |
d d d | d | d
| d |
d | �Id H} y�| j d
k r�t | j | j d d d | j d | j �� | j j t j d � j � d k r�t | j | j d d d | j d | j �� | j j t j d � j � d k r%t | j | j d d d | j d | j �� | j j t j d � } t j t j | t � j � � j � } | | k r�t | j | j d d d | j d | j �� d } | rt j | j k rd d � | j t j j d � D� } x! | D] } | | k r�| } Pq�Wd } | r�| j j t j � } | r�y t | � \ } } Wq�t! k
r�} z9 t | j | j d | j" d d | j d | j �� WYd d } ~ Xq�Xn d } d } | j# j$ } | j# j% } t& | d d# d | j' �}! | j( t) |! � |! � t* | d � t+ | | d d d | d | �}" Wn t, k
rG| j- � � Yn> X| j. |! |" | | | | | | j' d | d! | d | d" | �Sd S)$NZ13� �,r� rE r� FrG r� r� r� r� �e rX zInvalid response status�coder� Z websocketzInvalid upgrade headerZupgradezInvalid connection headerzInvalid challenge responsec S s g | ] } | j � � q SrZ )�strip)r[ �protorZ rZ r] r^ C s z-ClientSession._ws_connect.<locals>.<listcomp>r �limitrV rC TZuse_mask�
notakeoverr� r� Zclient_notakeoveri )/r r ZUPGRADEZ WEBSOCKETZ
CONNECTIONZSEC_WEBSOCKET_VERSIONr� �base64Z b64encode�os�urandom�decodeZSEC_WEBSOCKET_KEYr� ZSEC_WEBSOCKET_PROTOCOLZORIGINr% ZSEC_WEBSOCKET_EXTENSIONSr r� r� r Z request_infor� rE �lowerZSEC_WEBSOCKET_ACCEPT�hashlib�sha1r! �digest�splitr& r$ r� r� �protocolr� r' r2 Z
set_parserr" r) r# r� r� rA )#rk r� r� r� r� r� r� r� rG r� rE r� r� r� r� r� r� r� r� Zdefault_headers�key�valueZsec_keyZextstrr� �matchr� Zresp_protocolsr� r� Z
compress_hdrsr� r� �reader�writerrZ rZ r] r� � s�
$ $ "
"
zClientSession._ws_connectc C s� t | j � } | r� t | t t f � s6 t | � } t � } xP | j � D]B \ } } | | k rw | j | | � qL | | | <| j | � qL W| S)z= Add default headers and transform it to CIMultiDict
)r r= r� r r r� r� r� )rk rE �resultZ added_namesr� r� rZ rZ r] r� w s
z ClientSession._prepare_headersc K s" t | j t j | d | | �� S)zPerform HTTP GET request.r� )r� r� r r� )rk r� r� r~ rZ rZ r] r� � s zClientSession.getc K s" t | j t j | d | | �� S)z Perform HTTP OPTIONS request.r� )r� r� r Z METH_OPTIONS)rk r� r� r~ rZ rZ r] �options� s zClientSession.optionsc K s" t | j t j | d | | �� S)zPerform HTTP HEAD request.r� )r� r� r r� )rk r� r� r~ rZ rZ r] �head� s zClientSession.headc K s" t | j t j | d | | �� S)zPerform HTTP POST request.r� )r� r� r r� )rk r� r� r~ rZ rZ r] �post� s zClientSession.postc K s" t | j t j | d | | �� S)zPerform HTTP PUT request.r� )r� r� r ZMETH_PUT)rk r� r� r~ rZ rZ r] �put� s zClientSession.putc K s" t | j t j | d | | �� S)zPerform HTTP PATCH request.r� )r� r� r Z
METH_PATCH)rk r� r� r~ rZ rZ r] �patch� s zClientSession.patchc K s t | j t j | | � � S)z Perform HTTP DELETE request.)r� r� r Z METH_DELETE)rk r� r~ rZ rZ r] �delete� s zClientSession.deletec � s, | j s( | j r | j j � d | _ d S)zMClose underlying connector.
Release all acquired resources.
N)r{ r4 r0 r� )rk rZ rZ r] r� � s
zClientSession.closec C s | j d k p | j j S)z@Is client session closed.
A readonly property.
N)r0 r{ )rk rZ rZ r] r{ � s zClientSession.closedc C s | j S)z(Connector instance used for the session.)r0 )rk rZ rZ r] r � s zClientSession.connectorc C s | j S)zThe session cookies.)r3 )rk rZ rZ r] rM � s zClientSession.cookie_jarc C s | j S)z"The session HTTP protocol version.)r6 )rk rZ rZ r] rL � s zClientSession.versionc C s | j S)zSession's loop.)r2 )rk rZ rZ r] rC � s zClientSession.loopc C s
d | _ d S)zwDetach connector from session without closing the former.
Session is switched to closed state anyway.
N)r0 )rk rZ rZ r] �detach� s zClientSession.detachc C s t d � � d S)NzUse async with instead)� TypeError)rk rZ rZ r] � __enter__� s zClientSession.__enter__c C s d S)NrZ )rk �exc_type�exc_val�exc_tbrZ rZ r] �__exit__� s zClientSession.__exit__c � s | S)NrZ )rk rZ rZ r] �
__aenter__� s zClientSession.__aenter__c � s | j � Id Hd S)N)r� )rk r� r� r� rZ rZ r] � __aexit__� s zClientSession.__aexit__).rp �
__module__� __qualname__�__doc__rj rt r/ r0 r1 r� r� r r r r
�
HttpVersion11r rn rs rv re r r, r� r� r� r� r� r� r� r� r� r� r� r� �propertyr{ r rM rL rC r� r� r� r� r� rZ rZ )ry r] r+ / s� H ��
c @ sj e Z d Z d Z d d � Z d d � Z d d � Z d d
� Z d d � Z d
d � Z d d � Z
d S)�_BaseRequestContextManager�_coro�_respc C s
| | _ d S)N)r� )rk �cororZ rZ r] rn � s z#_BaseRequestContextManager.__init__c C s | j j | � S)N)r� r� )rk �argrZ rZ r] r� � s z _BaseRequestContextManager.sendc C s | j j | � S)N)r� �throw)rk r� rZ rZ r] r � s z _BaseRequestContextManager.throwc C s
| j j � S)N)r� r� )rk rZ rZ r] r� s z _BaseRequestContextManager.closec C s | j j � } | S)N)r� � __await__)rk ZretrZ rZ r] r s z$_BaseRequestContextManager.__await__c C s
| j � S)N)r )rk rZ rZ r] �__iter__ s z#_BaseRequestContextManager.__iter__c � s | j Id H| _ | j S)N)r� r� )rk rZ rZ r] r�
s z%_BaseRequestContextManager.__aenter__N)r� r� ) rp r� r� � __slots__rn r� r r� r r r� rZ rZ rZ r] r� � s r� c @ s e Z d Z d d � Z d S)r� c � s | j j � d S)N)r� r� )rk r� r� �tbrZ rZ r] r� s z _RequestContextManager.__aexit__N)rp r� r� r� rZ rZ rZ r] r� s r� c @ s e Z d Z d d � Z d S)r� c � s | j j � Id Hd S)N)r� r� )rk r� r� r rZ rZ r] r� s z"_WSRequestContextManager.__aexit__N)rp r� r� r� rZ rZ rZ r] r� s r� c @ s: e Z d Z d Z d d � Z d d � Z d d � Z d
S) � _SessionRequestContextManagerr� r� �_sessionc C s | | _ d | _ | | _ d S)N)r� r� r )rk r� r� rZ rZ r] rn " s z&_SessionRequestContextManager.__init__c � s | j Id H| _ | j S)N)r� r� )rk rZ rZ r] r� ' s z(_SessionRequestContextManager.__aenter__c � s# | j j � | j j � Id Hd S)N)r� r� r )rk r� r� r� rZ rZ r] r� + s
z'_SessionRequestContextManager.__aexit__N)r� r� r )rp r� r� r rn r� r� rZ rZ rZ r] r s r r� r� r� rE rF rD rG r� Tr� r� rL r� r� r� Fr rC r� r� r� c C s� d } | d k r- d } t d | d d � } t d | d | d | d | d | � } t | j | | d
| d | d | d
| d | d | d | d |
d | d |
d | d | d | d | �| � S)aR Constructs and sends a request. Returns response object.
method - HTTP method
url - request url
params - (optional) Dictionary or bytes to be sent in the query
string of the new request
data - (optional) Dictionary, bytes, or file-like object to
send in the body of the request
json - (optional) Any json compatibile python object
headers - (optional) Dictionary of HTTP Headers to send with
the request
cookies - (optional) Dict object to send with the request
auth - (optional) BasicAuth named tuple represent HTTP Basic Auth
auth - aiohttp.helpers.BasicAuth
allow_redirects - (optional) If set to False, do not follow
redirects
version - Request HTTP version.
compress - Set to True if request has to be compressed
with deflate encoding.
chunked - Set to chunk size for chunked transfer encoding.
expect100 - Expect 100-continue response from server.
connector - BaseConnector sub-class instance to support
connection pooling.
read_until_eof - Read response until eof if response
does not have Content-Length header.
loop - Optional event loop.
Usage::
>>> import aiohttp
>>> resp = await aiohttp.request('GET', 'http://python.org/')
>>> resp
<ClientResponse(python.org/) [200]>
>>> data = await resp.read()
FNTrC Z force_closerD rL r rN r� r� r� rE rF rG r� r� r� r� r� r� r� r� )r r+ r r� )r� r� r� r� r� rE rF rD rG r� r� rL r� r� r� r rC r� r� r� rN r� rZ rZ r] r, 0 s0 3 )Er� r_ r� r� r� r� rc ra re �collections.abcr Z multidictr r r r Zyarlr r� r r
r Z
connector_modr r
r r r r r r r r r Z client_wsr r Z cookiejarr Zhelpersr r r r r r r! r" r# Zhttp_websocketr$ r% r& Zstreamsr' Z tcp_helpersr( r) Ztracingr* �__all__ri r+ r� r� r� r r� r, rZ rZ rZ r] �<module> sr "
(
.
� � �