���Z�o � @ s d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m
Z
m Z m Z d d l
m Z m Z m Z d d l m Z d d l Z d d l m Z m Z m Z m Z d d l m Z d d l m Z m Z m Z m Z m Z d d
l! m" Z" m# Z# m$ Z$ m% Z% m& Z& m' Z' d d l( m) Z) d d l m* Z* m+ Z+ m, Z, m- Z- m. Z. m/ Z/ d d
l m0 Z0 m1 Z1 m2 Z2 m3 Z3 d d l4 m5 Z5 d d l6 m7 Z7 y d d l8 Z8 Wn e9 k
r�d Z8 Yn Xy d d l: Z; Wn e9 k
rd d l; Z; Yn Xd d d d f Z< e j= d d d d � Gd d � d � � Z> e j= d d d d � Gd d � d � � Z? Gd d � d � Z@ e8 d k r�e8 jA eB e@ eC d � f ZD n eC d � ZD d d � ZE e d d d d g � ZF Gd! d � d � ZG Gd" d � d e+ � ZH d S)#� N)�
namedtuple)�md5�sha1�sha256)� CookieError�Morsel� SimpleCookie)�MappingProxyType)� CIMultiDict�CIMultiDictProxy� MultiDict�MultiDictProxy)�URL� )�hdrs�helpers�http� multipart�payload)�ClientConnectionError�
ClientOSError�ClientResponseError�ContentTypeError�
InvalidURL�ServerFingerprintMismatch)�FormData)�PY_36� HeadersMixin� TimerNoop�noop�reify�
set_result)�SERVER_SOFTWARE�
HttpVersion10�
HttpVersion11� StreamWriter)�
client_logger)� StreamReader�
ClientRequest�ClientResponse� RequestInfo� Fingerprint�frozenT�slotsc @ sF e Z d Z e j d e � Z e j d e � Z e j d e � Z d S)�ContentDisposition�typeN)
�__name__�
__module__� __qualname__�attr�ib�strr/ r �
parameters�filename� r8 r8 �/client_reqrep.pyr. * s r. c @ sF e Z d Z e j d e � Z e j d e � Z e j d e � Z
d S)r* r/ N) r0 r1 r2 r3 r4 r �urlr5 �methodr �headersr8 r8 r8 r9 r* 1 s c @ sR e Z d Z d e d e d e i Z d d � Z e d d � � Z d d � Z
d
S) r+ � � � c C sm t | � } | j j | � } | s3 t d � � n$ | t k sK | t k rW t d � � | | _ | | _ d S)Nz fingerprint has invalid lengthz8md5 and sha1 are insecure and not supported. Use sha256.)�len�HASHFUNC_BY_DIGESTLEN�get�
ValueErrorr r � _hashfunc� _fingerprint)�self� fingerprintZ digestlenZhashfuncr8 r8 r9 �__init__? s zFingerprint.__init__c C s | j S)N)rE )rF r8 r8 r9 rG J s zFingerprint.fingerprintc C s� | j d � s d S| j d � } | j d d � } | j | � j � } | | j k r� | j d � ^ } } } t | j | | | � � d S)NZ
sslcontextZ
ssl_objectZ binary_formTZpeername)Zget_extra_infoZ getpeercertrD �digestrE r )rF � transportZsslobjZcertZgot�host�port�_r8 r8 r9 �checkN s zFingerprint.checkN) r0 r1 r2 r r r rA rH �propertyrG rN r8 r8 r8 r9 r+ 8 s c C s� | d k rJ | rJ t j d t d d �| d k rD t d � � n d } | d k r� t j d t d d �| d k r� t d � � n | } | d k r� t j d t d d �| d k r� t d � � n t | � } t | t � s� t d j | � � � | S) Nz/verify_ssl is deprecated, use ssl=False instead�
stacklevel� zNverify_ssl, ssl_context, fingerprint and ssl parameters are mutually exclusiveFz2ssl_context is deprecated, use ssl=context insteadzCfingerprint is deprecated, use ssl=Fingerprint(fingerprint) insteadzFssl should be SSLContext, bool, Fingerprint or None, got {!r} instead.) �warnings�warn�DeprecationWarningrC r+ �
isinstance�SSL_ALLOWED_TYPES� TypeError�format)�sslZ
verify_sslZ ssl_contextrG r8 r8 r9 �_merge_ssl_params` s2 rZ �
ConnectionKeyrK rL rY c ( @ sM e Z d Z e j e j e j e j h Z e j e j
e j h Z e j
e � j
e j h � Z e j d e j d i Z d Z d Z d Z d Z d Z d Z d d d d d e � d d d d d
d d e j d d d
d d d d d d d d d d d d d d d d d d d d d d d � Z d d � Z e d d � � Z e d d! � � Z e d" d# � � Z! e d$ d% � � Z" e d& d' � � Z# d( d) � Z$ d* d+ � Z% d, d- � Z& d. d/ � Z' d0 d1 � Z( d2 d3 � Z) d4 d5 � Z* d6 d7 � Z+ d8 d9 � Z, d d: d; � Z- d< d= � Z. d> d? � Z/ d@ dA � Z0 dB dC � Z1 dD dE � Z2 dF dG � Z3 d S)Hr( z*/*z
gzip, deflate� N�paramsr< �skip_auto_headers�data�cookies�auth�version�compress�chunked� expect100F�loop�response_class�proxy�
proxy_auth�timer�session�auto_decompressTrY �
proxy_headersc C s |
d k r t j � }
t | t � s3 t | � � t | t t d � f � sZ t | � � | | _ | r� t | j � } | j | � } | j
| j � | j | � } | j d � | _ | | _
| j � | _ | | _ |
| _ |
| _ d | _ | p� t | _ | d k r| n t � | _ | | _ | | _ |
j � rZt j t j d � � | _ | j | � | j! | � | j" | � | j# | � | j$ | � | j% | � | j& | � | j' | | | � | j( | � | s�| j | j) k r�| j* � | j+ | � d S)Nr ),�asyncioZget_event_looprU r �AssertionErrorr/ �_sessionr ZqueryZ
with_query�extendZ
with_fragmentr: � original_url�upperr; rd rc rf �lengthr) rg r �_timer�_auto_decompress�_ssl� get_debug� traceback�
extract_stack�sys� _getframe�_source_traceback�update_version� update_host�update_headers�update_auto_headers�update_cookies�update_content_encoding� update_auth� update_proxy�update_body_from_data� GET_METHODS�update_transfer_encoding�update_expect_continue)rF r; r: r] r< r^ r_ r` ra rb rc rd re rf rg rh ri rj rk rl rY rm �qZurl2r8 r8 r9 rH � sF '
zClientRequest.__init__c C s | j j d k S)N�https�wss)r� r� )r: �scheme)rF r8 r8 r9 �is_ssl� s zClientRequest.is_sslc C s | j S)N)rw )rF r8 r8 r9 rY � s zClientRequest.sslc C s t | j | j | j � � S)N)r[ rK rL r� )rF r8 r8 r9 �connection_key� s z ClientRequest.connection_keyc C s
| j j S)N)r: rK )rF r8 r8 r9 rK � s zClientRequest.hostc C s
| j j S)N)r: rL )rF r8 r8 r9 rL � s zClientRequest.portc C s t | j | j | j � S)N)r* r: r; r< )rF r8 r8 r9 � request_info� s zClientRequest.request_infoc C sM | j s t | � � | j | j } } | rI t j | | p@ d � | _ d S)z8Update destination host, port and connection type (ssl).� N)rK r �user�passwordr � BasicAuthra )rF r: Zusernamer� r8 r8 r9 r � s
zClientRequest.update_hostc C s� t | t � r d d � | j d d � D� } y$ t | d � t | d � f } Wn* t k
r~ t d j | � � d � Yn X| | _ d S)zdConvert request version to two elements tuple.
parser HTTP version '1.1' => (1, 1)
c S s g | ] } | j � � q Sr8 )�strip)�.0�lr8 r8 r9 �
<listcomp>� s z0ClientRequest.update_version.<locals>.<listcomp>�.r r z%Can not parse http version number: {}N)rU r5 �split�intrC rX rb )rF rb �vr8 r8 r9 r~ � s $
z ClientRequest.update_versionc C sd t � | _ | r` t | t t t f � r6 | j � } x' | D] \ } } | j j | | � q= Wd S)zUpdate request headers.N)r
r< rU �dictr
r �items�add)rF r< �key�valuer8 r8 r9 r� s z ClientRequest.update_headersc C s� t d d � t | � D� � | _ | j j � } | j | j � x< | j j � D]+ \ } } | | k rQ | j j | | � qQ Wt j
| k r� | j j } | j j
� s� | d t | j j � 7} | | j t j
<t j | k r� t | j t j <d S)Nc s s | ] } | d f Vq d S)Nr8 )r� �hdrr8 r8 r9 � <genexpr> s z4ClientRequest.update_auto_headers.<locals>.<genexpr>�:)r
�sortedr^ r< �copyrq �DEFAULT_HEADERSr� r� r ZHOSTr: �raw_hostZis_default_portr5 rL Z
USER_AGENTr" )rF r^ Z used_headersr� �valZnetlocr8 r8 r9 r� s z!ClientRequest.update_auto_headersc C s� | s
d St � } t j | j k rQ | j | j j t j d � � | j t j =xt | j � D]f \ } } t | t � r� | j | j t � � } | j
| j | j | j � | | | <q^ | | | <q^ W| j
d d d d � j � | j t j <d S)z Update request cookies header.Nr� �header�sep�;)r r ZCOOKIEr< �loadrB r� rU r r� �setr� Z coded_value�outputr� )rF r` �c�namer� Zmrsl_valr8 r8 r9 r� $ s
z ClientRequest.update_cookiesc C s� | s
d S| j j t j d � j � } | rF | j r� t d � � n@ | j r� t | j t � sj d | _ | j | j t j <d | _ d S)z Set request content encoding.Nr� z9compress can not be set if Content-Encoding header is setZdeflateT)
r< rB r ZCONTENT_ENCODING�lowerrc rC rU r5 rd )rF r_ �encr8 r8 r9 r� 9 s z%ClientRequest.update_content_encodingc C s� | j j t j d � j � } d | k rB | j r� t d � � nk | j r| t j | j k ri t d � � d | j t j <n1 t j | j k r� t t | j
� � | j t j <d S)z!Analyze transfer-encoding header.r� rd zDchunked can not be set if "Transfer-Encoding: chunked" header is setz6chunked can not be set if Content-Length header is setN) r< rB r ZTRANSFER_ENCODINGr� rd rC �CONTENT_LENGTHr5 r@ �body)rF Zter8 r8 r9 r� J s z&ClientRequest.update_transfer_encodingc C s] | d k r | j } | d k r% d St | t j � sC t d � � | j � | j t j <d S)zSet basic auth.Nz%BasicAuth() tuple is required instead) ra rU r r� rW �encoder< r Z
AUTHORIZATION)rF ra r8 r8 r9 r� _ s zClientRequest.update_authc C sR | s
d St | t � r" | � } y t j j | d d �} Wn$ t j k
rd t | � � } Yn X| | _ | j s� t j | j
k r� | j } | d k r� d | _ n( t j | j
k r� t | � | j
t j <t j
| j
k r t j
| j k r | j | j
t j
<| j
rNx9 | j
j � D]( \ } } | | j
k r"| | j
| <q"Wd S)NZ dispositionT)rU r r ZPAYLOAD_REGISTRYrB � LookupErrorr� rd r r� r< �sizer5 � CONTENT_TYPEr^ � content_typer� )rF r� r� r� r� r8 r8 r9 r� k s. z#ClientRequest.update_body_from_datac C s_ | r d | j t j <n* | j j t j d � j � d k rC d } | r[ | j j � | _ d S)Nz 100-continuer� T)r< r ZEXPECTrB r� rf Z
create_future� _continue)rF �expectr8 r8 r9 r� � s $z$ClientRequest.update_expect_continuec C sf | r" | j d k r" t d � � | rG t | t j � rG t d � � | | _ | | _ | | _ d S)Nr z Only http proxies are supportedz,proxy_auth must be None or BasicAuth() tuple)r� rC rU r r� rh ri rm )rF rh ri rm r8 r8 r9 r� � s zClientRequest.update_proxyc C sk | j t k r d S| j t k rH | j j t j � d k rA d Sd Sn | j j t j � d k rg d Sd S)NFz
keep-aliveT�close)rb r# r< rB r �
CONNECTION)rF r8 r8 r9 �
keep_alive� s zClientRequest.keep_alivec � s� | j d k r* | j � Id H| j Id Hzcy� t | j t j � r] | j j | � Id HnH t | j t t f � r� | j f | _ x | j D] } | j | � q� W| j � Id HWn� t
k
r} z@ t | j d | j
� } | | _ | | _ | j j | � WYd d } ~ Xnu t j k
rW} z | j sE| j j | � WYd d } ~ Xn5 t k
r�} z | j j | � WYd d } ~ Xn XWd d | _ Xd S)z-Support coroutines that yields bytes objects.Nz!Can not write request body for %s)r� ZdrainrU r� r ZPayload�write�bytes� bytearrayZ write_eof�OSErrorr �errnor: � __context__� __cause__�protocol�
set_exceptionrn �CancelledError�closed� Exception�_writer)rF �writer�conn�chunk�exc�new_excr8 r8 r9 � write_bytes� s0 " "'zClientRequest.write_bytesc
C sA | j t j k r3 d j | j j | j j � } nT | j r[ | j � r[ t | j � } n, | j j
} | j j r� | d | j j 7} t | j
| j | j � } | j r� | j | j � | j d k r� | j � | j | j k rt j | j k rt j | j k rd | j t j <| j j t j � } | sn| j � rY| j t k rnd } n | j t k rnd } | d k r�| | j t j <d j | j | | j � } | j | | j � | j j | j | | � � | _! | j" | j | j# d | j! d | j$ d | j% d
| j& d | j' �| _( | j( j) | j | j* � | j( S) Nz{}:{}�?zapplication/octet-streamz
keep-aliver� z {0} {1} HTTP/{2[0]}.{2[1]}
r� � continue100rj r� rl )+r; r Z METH_CONNECTrX r: r� rL rh r� r5 Zraw_pathZraw_query_stringr% r� rJ rf rc Zenable_compressionrd Zenable_chunking� POST_METHODSr� r^ r< rB r� r� rb r# r$ Z
write_headersZ create_taskr� r� rg rr r� ru r� rv �response�
_post_initrp )rF r� �pathr� �
connectionZ status_liner8 r8 r9 �send� sH !
! zClientRequest.sendc � s0 | j d k r, z | j Id HWd d | _ Xd S)N)r� )rF r8 r8 r9 r� s zClientRequest.closec C s8 | j d k r4 | j j � s+ | j j � d | _ d S)N)r� rf � is_closed�cancel)rF r8 r8 r9 � terminate s
zClientRequest.terminate)4r0 r1 r2 r ZMETH_GETZ METH_HEADZ METH_OPTIONSZ
METH_TRACEr� Z
METH_PATCHZ METH_POSTZMETH_PUTr� �unionZ METH_DELETEZ ALL_METHODSZACCEPTZACCEPT_ENCODINGr� r� ra r� rg r� r� � frozensetr r$ rH r� rO rY r� rK rL r� r r~ r� r� r� r� r� r� r� r� r� r� r� r� r� r� r8 r8 r8 r9 r( � s\ *
$ ! 9 c @ s e Z d Z d Z d Z d Z d Z d Z d Z d Z e
Z d Z d Z
d Z d Z d Z d d d d d d d d d d d d � Z e d
d � � Z e d d
� � Z e d d � � Z e d d � � Z e d d � � Z e d d � � Z d d � Z e d d � Z d d � Z e d d � � Z e d d � � Z d d! d" � Z d# d$ � Z! e d% d& � � Z" d' d( � Z# d) d* � Z$ d+ d, � Z% d- d. � Z& d/ d0 � Z' d1 d2 � Z( d3 d4 � Z) d5 d6 � Z* d d7 d8 d9 � Z+ d: d d; e, j- d< d= d> d? � Z, d@ dA � Z. dB dC � Z/ d S)Dr) NTr� r� rj r� rl c C s� t | t � s t � | | _ d | _ t � | _ | | _ d | _ | | _ | | _
d | _ f | _ | | _
| d k r� | n t � | _ | | _ i | _ d S)NT)rU r ro r; r< r r` �_url�_contentr� r� �_closed�_history�
_request_infor ru rv �_cache)rF r; r: r� r� rj r� rl r8 r8 r9 rH , s zClientResponse.__init__c C s | j S)N)r� )rF r8 r8 r9 r: @ s zClientResponse.urlc C s t j d t d d �| j S)NzDeprecated, use .url #1654rP � )rR rS rT r� )rF r8 r8 r9 �url_objD s zClientResponse.url_objc C s
| j j S)N)r� rK )rF r8 r8 r9 rK J s zClientResponse.hostc C s | j S)N)r< )rF r8 r8 r9 �_headersN s zClientResponse._headersc C s | j S)N)r� )rF r8 r8 r9 r� R s zClientResponse.request_infoc C se | j j t j � } | d k r% d St j | � \ } } t | � } t j | � } t | | | � S)N) r� rB r ZCONTENT_DISPOSITIONr Zparse_content_dispositionr Z content_disposition_filenamer. )rF �rawZdisposition_typer] r7 r8 r8 r9 �content_dispositionV s z"ClientResponse.content_dispositionc C s= | | _ | | _ | j � r9 t j t j d � � | _ d S)Nr )�_looprp rx ry rz r{ r| r} )rF rf rk r8 r8 r9 r� ` s zClientResponse._post_initc C s� | j d k r d S| j r d S| j d k r� | j j � | j � | j j � r� t rj d | i } n i } | j d j | � t | � d | d d i } | j
r� | j
| d <| j j | � d S)N�sourcezUnclosed response {!r}Zclient_response�messagezUnclosed responseZsource_traceback) r� r� � _connection�release�_cleanup_writerrx r rS rX �ResourceWarningr} Zcall_exception_handler)rF � _warnings�kwargs�contextr8 r8 r9 �__del__f s&
zClientResponse.__del__c C s� t j � } t | j � } | j rE | j j d d � j d � } n | j } t d j | | j | � d | �t | j
d | �| j � S)N�ascii�backslashreplacez <ClientResponse({}) [{} {}]>�file) �io�StringIOr5 r: �reasonr� �decode�printrX �statusr< �getvalue)rF �outZascii_encodable_urlZascii_encodable_reasonr8 r8 r9 �__repr__~ s zClientResponse.__repr__c C s | j S)N)r� )rF r8 r8 r9 r� � s zClientResponse.connectionc C s | j S)z2A sequence of of responses, if redirects occurred.)r� )rF r8 r8 r9 �history� s zClientResponse.historyFc � s d | _ | j | _ | | _ | j j d | j d | j j � d k d | d | j � | j �� x� y | j j � Id H\ } } Wn[ t
j k
r� } z8 t | j
| j d | j d | j d
| j �| � WYd d } ~ Xn X| j d k s| j d k s| j d
k rP| j d k re t | j d � d | _ qe WWd QRX| j | j � | j | _ | j | _ | j | _ t | j � | _ t | j � | _ | | _ xl | j j t j f � D]R } y | j j! | � Wq�t" k
r
} z t# j$ d | � WYd d } ~ Xq�Xq�W| S)zStart response processing.Frj Z skip_payload�head�read_until_eofrl N�coder� r< �d �� �e Tz!Can not load response cookies: %s)%r� r� Z _protocolr� Zset_response_paramsru r; r� rv �readr ZHttpProcessingErrorr r� r r r� r< r� r! Zon_eof�
_response_eofrb r� r� r �tuple� raw_headers�contentZgetallr Z
SET_COOKIEr` r� r r& �warning)rF r� r r� r r� r� r8 r8 r9 �start� sJ
+ !zClientResponse.startc C sn | j r
d S| j d k rW | j j d k rA | j j j rA d S| j j � d | _ d | _ | j � d S)NT)r� r� r� Zupgradedr� r� )rF r8 r8 r9 r
� s
z ClientResponse._response_eofc C s | j S)N)r� )rF r8 r8 r9 r� � s zClientResponse.closedc C su | j r
d Sd | _ | j d k s4 | j j � r8 d S| j d k r] | j j � d | _ | j � | j � d S)NT)r� r� r� r� r� r� �_notify_content)rF r8 r8 r9 r� � s
zClientResponse.closec C sY | j r t � Sd | _ | j d k r> | j j � d | _ | j � | j � t � S)NT)r� r r� r� r� r )rF r8 r8 r9 r� � s
zClientResponse.releasec C sC d | j k r? t | j | j d | j d | j d | j �� d S)Ni� r r� r< )r� r r� r r� r< )rF r8 r8 r9 �raise_for_status� s z ClientResponse.raise_for_statusc C s2 | j d k r | j j � d | _ d | _ d S)N)r� r� rp )rF r8 r8 r9 r� s
z ClientResponse._cleanup_writerc C sE | j } | rA | j � d k rA | j � rA | j t d � � d S)NzConnection closed)r
� exceptionZis_eofr� r )rF r
r8 r8 r9 r s %z ClientResponse._notify_contentc � s: | j d k r, z | j Id HWd d | _ X| j � d S)N)r� r� )rF r8 r8 r9 �wait_for_close s
z ClientResponse.wait_for_closec � sS | j d k rL y | j j � Id H| _ Wn t k
rK | j � � Yn X| j S)zRead response payload.N)r� r
r �
BaseExceptionr� )rF r8 r8 r9 r s
zClientResponse.readc C s� | j j t j d � j � } t j | � } | j j d � } | rq y t j | � Wn t
k
rp d } Yn X| s� | j d k r� | j d k r� d } n t
j | j � d } | s� d } | S)Nr� �charsetZ application�jsonzutf-8�encoding)r< rB r r� r� r Zparse_mimetyper6 �codecs�lookupr� r/ �subtype�chardetZdetectr� )rF �ctypeZmimetyper r8 r8 r9 � get_encoding% s
zClientResponse.get_encoding�strictc � sL | j d k r | j � Id H| d k r6 | j � } | j j | d | �S)z!Read response payload and decode.N�errors)r� r r r� )rF r r r8 r8 r9 �text: s
zClientResponse.textr �loadsr� zapplication/jsonc � s� | j d k r | j � Id H| rv | j j t j d � j � } | | k rv t | j | j d d | d | j �� | j j
� } | s� d S| d k r� | j � } | | j | � � S)z Read and decodes JSON response.Nr� r� z3Attempt to decode JSON with unexpected mimetype: %sr< )
r� r r< rB r r� r� r r� r r� r r� )rF r r! r� r Zstrippedr8 r8 r9 r D s" zClientResponse.jsonc � s | S)Nr8 )rF r8 r8 r9 �
__aenter__] s zClientResponse.__aenter__c � s | j � d S)N)r� )rF �exc_type�exc_val�exc_tbr8 r8 r9 � __aexit__` s zClientResponse.__aexit__)0r0 r1 r2 rb r� r� r
r< r r� r' Zflow_control_class�_readerr} r� r� rp rH rO r: r� rK r� r� r r� r� rR r� r r� r r r
r� r� r� r r� r r r r r r r! r" r& r8 r8 r8 r9 r) sV
7
)Irn r r� r r{ ry rR � collectionsr �hashlibr r r Z http.cookiesr r r �typesr r3 Z multidictr
r r r
Zyarlr r� r r r r r Zclient_exceptionsr r r r r r Zformdatar r r r r r r! r" r# r$ r% �logr& Zstreamsr' rY � ImportErrorZcchardetr �__all__�sr. r* r+ Z
SSLContext�boolr/ rV rZ r[ r( r) r8 r8 r8 r9 �<module> sP "(.."
++" #� �