���ZnY � @ s� d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l Z d d l
Z
d d l m Z d d l m Z d d l m Z d d l m Z d d l m Z d d l m Z d d l Z d d l Z d d l m Z d d
l m Z d d l m! Z! d d
l" m# Z# d d l$ m% Z% dk Z& e j' dl k Z( e j' dm k Z) e) s�d d l* Z* e* j+ � e, � Z- e. e j/ j0 d � � Z1 e2 d d � e3 d d � D� � Z4 e2 d d � e3 d d � D� � e5 d � h BZ6 d d d d d d d d! d" d# d$ d% d& d' d( d) d* d+ e5 d, � h Z7 e4 e6 Ae7 AZ8 e j9 Z9 e9 j: Z; d- e9 _: e j< d. d/ � � Z= e; e9 _: Gd0 d � d e d d1 d2 d3 g � � Z> d4 d5 � Z? d6 d7 � Z@ e jA d8 d9 d: d9 � Gd; d< � d< � � ZB d= d> � ZC d d? d@ � ZD dA dB � ZE e jA d8 d9 d: d9 � GdC dD � dD � � ZF dE dF � ZG d dG dH � ZH d9 dI dJ � ZI GdK dL � dL e# � ZJ GdM dN � dN � ZK dO ZL dP ZM e
jN eL � ZO e
jN eM dQ e
jP �ZQ e
jN eL jR dR � � ZS e
jN eM jR dR � dQ e
jP �ZT dS dT � ZU d aV d aW dU dV � ZX dW dX � ZY d9 dY dZ � ZZ d[ d\ � Z[ Gd] d^ � d^ � Z\ Gd_ d` � d` � Z] Gda db � db � Z^ Gdc dd � dd e j_ � Z` Gde df � df � Za dg dh � Zb di dj � Zc d S)nzVarious helper functions� N)�
namedtuple)�suppress)�ceil)�Path)�quote)�
getproxies)� MultiDict)�URL� )�hdrs)�AbstractAccessLogger)�
client_logger� BasicAuth� � � ZAIOHTTP_NO_EXTENSIONSc c s | ] } t | � Vq d S)N)�chr)�.0�i� r � /helpers.py� <genexpr>. s r � c c s | ] } t | � Vq d S)N)r )r r r r r r / s � � �(�)�<�>�@�,�;�:�\�"�/�[�]�?�=�{�}� � Fc O s d S)Nr )�args�kwargsr r r �noop: s r0 c sg e Z d Z d Z d d � f d d � Z e d d d � � Z e d d d d
� � Z d d � Z � S)
r z!Http basic authentication helper.� �latin1c sa | d k r t d � � | d k r0 t d � � d | k rH t d � � t � j | | | | � S)Nz"None is not allowed as login valuez%None is not allowed as password valuer" z5A ":" is not allowed in login (RFC 1945#section-11.1))�
ValueError�super�__new__)�cls�login�password�encoding)� __class__r r r5 E s zBasicAuth.__new__c
C s� | j � j d � } t | � d k rd | d j � j � d k rW t d | d � � | d } n t d � � y7 t j | j d � � j | � j d � \ } } } Wn! t
j k
r� t d
� � Yn X| | | d | �S) z<Create a BasicAuth object from an Authorization HTTP header.r, � r Zbasicz Unknown authorization method %sr
z%Could not parse authorization header.�asciir" zInvalid base64 encoding.r9 ) �strip�split�len�lowerr3 �base64Z b64decode�encode�decode� partition�binascii�Error)r6 Z auth_headerr9 r> Z to_decodeZusername�_r8 r r r rC R s
zBasicAuth.decoder9 c C sM t | t � s t d � � | j d k r. d S| | j | j pC d d | �S)zCreate BasicAuth from url.z url should be yarl.URL instanceNr1 r9 )�
isinstancer � TypeError�userr8 )r6 �urlr9 r r r �from_urlf s
zBasicAuth.from_urlc C s? d | j | j f j | j � } d t j | � j | j � S)zEncode credentials.z%s:%szBasic %s)r7 r8 rB r9 rA Z b64encoderC )�selfZcredsr r r rB o s "zBasicAuth.encode) �__name__�
__module__� __qualname__�__doc__r5 � classmethodrC rL rB r r )r: r r B s
r7 r8 r9 c C s< t j | � } | d k r% | d f S| j d � | f Sd S)N)r rL Z with_user)rK �authr r r �strip_auth_from_urlu s
rT c C s4 d } t j j d � } y� | d k r6 t | � } n< t j � } t j d k rc | j d � } n | j d � } | r� | j � r� y t j t | � � } Wn> t j
t f k
r� } z t j
d | � WYd d } ~ Xn X| d k r� t j
d � Wn5 t k
r/} z t j
d | � WYd d } ~ Xn X| S)NZNETRC�ntZ_netrcz.netrcz.netrc file parses fail: %szcould't find .netrc filez could't find .netrc file: %s)�os�environ�getr �home�nameZjoinpath�is_file�netrc�strZNetrcParseError�OSErrorr
�warning� RuntimeError)� netrc_objZ
netrc_pathZhome_dir�er r r �netrc_from_env} s& # #rc �frozenT�slotsc @ s4 e Z d Z e j d e � Z e j d e � Z d S)� ProxyInfo�typeN) rN rO rP �attr�ibr] �proxyr Z
proxy_authr r r r rf � s rf c C s d d � t � j � D� } t � } d d � | j � D� } i } x� | j � D]� \ } } | \ } } | j d k r� t j d | � qQ | r� | d k r� | j | j � } | d k r� | ^ } }
t | d r� | d n | d |
� } t | | � | | <qQ W| S) Nc S s1 i | ]' \ } } | d k r t | � | � q S)�http�https)rk rl )r )r �k�vr r r �
<dictcomp>� s z$proxies_from_env.<locals>.<dictcomp>c S s% i | ] \ } } t | � | � q Sr )rT )r rm rn r r r ro � s rl z,HTTPS proxies %s are not supported, ignoringr r
���)
r �itemsrc �schemer
r_ Zauthenticators�hostr rf ) Z
proxy_urlsra ZstrippedZret�proto�valrj rS Zauth_from_netrcZloginsr8 r r r �proxies_from_env� s$
rv c C ss | d k r t j � } t r3 t j d | � } n t j j d | � } | d k ro t | d � ro | j � } | S)N�loop� current_task)�asyncioZget_event_loop�PY_37rx ZTask�hasattr)rw �taskr r r rx � s rx c C s t t d � r t j | � Sd S)N�isasyncgenfunctionF)r{ �inspectr} )�objr r r r} � s
r} c @ sX e Z d Z e j d e � Z e j d e � Z e j d e � Z e j d e � Z
d S)�MimeTyperg N) rN rO rP rh ri r] rg �subtype�suffixr �
parametersr r r r r� � s r� c
C si | s% t d d d d d d d i � S| j d � } g } xy | d d � D]g } | sZ qK d | k rx | j d d � n | d f \ } } | j | j � j � | j d
� f � qK Wt | � } | d j � j � } | d k r� d
} d | k r| j d d � n | d f \ } } d | k r8| j d d � n | d f \ } } t d | d | d | d | � S)a Parses a MIME type into its components.
mimetype is a MIME type string.
Returns a MimeType object.
Example:
>>> parse_mimetype('text/html; charset=utf-8')
MimeType(type='text', subtype='html', suffix='',
parameters={'charset': 'utf-8'})
rg r1 r� r� r� r! r
Nr) z "r �*z*/*r% �+)r� r> �appendr@ r= r )
Zmimetype�parts�params�item�key�valueZfulltypeZmtype�styper� r r r �parse_mimetype� s" 0, 00r� c C sX t | d d � } | rT t | t � rT | d d k rT | d d k rT t | � j S| S)NrZ r r r
r rp )�getattrrH r] r rZ )r �defaultrZ r r r �guess_filename� s 5
r� c K s | s t t | � k r/ t d j | � � � | } | rg } x� | j � D]� \ } } | st t t | � k r� t d j | | � � � | r� t | d � n | } | j | d | f � | d k rN | j d d | f � qN Wd j d d
� | D� � } d j | | f � } | S) z�Sets ``Content-Disposition`` header.
disptype is a disposition type: inline, attachment, form-data.
Should be valid extension token (see RFC 2183)
params is a dict with disposition params.
z!bad content disposition type {!r}z+bad content disposition parameter {!r}={!r}r1 z"%s"�filenamez filename*zutf-8''z; c s s | ] } d j | � Vq d S)r) N)�join)r Zpairr r r r s z-content_disposition_header.<locals>.<genexpr>)�TOKEN�setr3 �formatrq r r� r� ) ZdisptypeZ quote_fieldsr� r� Zlparamsr� ru ZqvalZsparamsr r r �content_disposition_header s" r� c s� e Z d Z d Z d d d d d d d d d
d d d
d d d d d d d d d d i Z d Z e j d � Z e j d � Z i Z
e d d � Z e � f d d � Z
d d � Z e d! d" � � Z e d# d$ � � Z e d% d& � � Z e d' d( � � Z e d) d* � � Z e d+ d, � � Z e d- d. � � Z e d/ d0 � � Z e d1 d2 � � Z e d3 d4 � � Z e d5 d6 � � Z d7 d8 � Z d9 d: � Z � S);� AccessLoggera� Helper object to log access.
Usage:
log = logging.getLogger("spam")
log_format = "%a %{User-Agent}i"
access_logger = AccessLogger(log, log_format)
access_logger.log(request, response, time)
Format:
%% The percent sign
%a Remote IP-address (IP-address of proxy if using reverse proxy)
%t Time when the request was started to process
%P The process ID of the child that serviced the request
%r First line of request
%s Response status code
%b Size of response in bytes, including HTTP headers
%T Time taken to serve the request, in seconds
%Tf Time taken to serve the request, in seconds with floating fraction
in .06f format
%D Time taken to serve the request, in microseconds
%{FOO}i request.headers['FOO']
%{FOO}o response.headers['FOO']
%{FOO}e os.environ['FOO']
�aZremote_address�tZrequest_start_time�PZ
process_id�rZfirst_request_line�sZresponse_status�bZ
response_size�TZ request_timeZTfZrequest_time_frac�DZrequest_time_micror Zrequest_header�oZresponse_headerz/%a %t "%r" %s %b "%{Referer}i" "%{User-Agent}i"z.%(\{([A-Za-z0-9\-_]+)\}([ioe])|[atPrsbOD]|Tf?)z(%[^s])� KeyMethodz
key methodc s` t � j | d | �t j j | � } | sJ | j | � } | t j | <| \ | _ | _ d S)z�Initialise the logger.
logger is a logger object to be used for logging.
log_format is an string with apache compatible log format description.
�
log_formatN)r4 �__init__r� �
_FORMAT_CACHErX �compile_format� _log_format�_methods)rM �loggerr� Z_compiled_format)r: r r r� M s
zAccessLogger.__init__c C s� t � } x� | j j | � D]� } | d d k r] | j | d } t t d | d � } nH | j | d | d f } t t d | d � } t j | | d � } | j | j | | � � q W| j j
d | � } | j j
d | � } | | f S)a� Translate log_format into form usable by modulo formatting
All known atoms will be replaced with %s
Also methods for formatting of those atoms will be added to
_methods in apropriate order
For example we have log_format = "%a %t"
This format will be translated to "%s %s"
Also contents of _methods will be
[self._format_a, self._format_t]
These method will be called and results will be passed
to translated string format.
Each _format_* method receive 'args' which is list of arguments
given to self.log
Exceptions are _format_e, _format_i and _format_o methods which
also receive key name (by functools.partial)
r
r1 r z
_format_%sr; z%sz%\1) �list� FORMAT_RE�findall�LOG_FORMAT_MAPr� r� � functools�partialr� r� �sub�
CLEANUP_RE)rM r� �methodsZatomZ
format_key�mr r r r� ] s zAccessLogger.compile_formatc C s# | d k r d S| j j | d � S)Nz (no headers)�-)�headersrX )r� �request�response�timer r r � _format_i� s zAccessLogger._format_ic C s | j j | d � S)Nr� )r� rX )r� r� r� r� r r r � _format_o� s zAccessLogger._format_oc C s- | d k r d S| j } | d k r) | Sd S)Nr� )Zremote)r� r� r� Zipr r r � _format_a� s zAccessLogger._format_ac C s2 t j j � } | t j d | � } | j d � S)NZsecondsz[%d/%b/%Y:%H:%M:%S +0000])�datetimeZutcnowZ timedelta�strftime)r� r� r� �nowZ
start_timer r r � _format_t� s zAccessLogger._format_tc C s d t j � S)Nz<%s>)rV �getpid)r� r� r� r r r � _format_P� s zAccessLogger._format_Pc C s1 | d k r d Sd t | j | j f | j � S)Nr� z%s %s HTTP/%s.%s)�tuple�methodZpath_qs�version)r� r� r� r r r � _format_r� s zAccessLogger._format_rc C s | j S)N)�status)r� r� r� r r r � _format_s� s zAccessLogger._format_sc C s | j S)N)Z body_length)r� r� r� r r r � _format_b� s zAccessLogger._format_bc C s
t | � S)N)�round)r� r� r� r r r � _format_T� s zAccessLogger._format_Tc C s d | S)Nz%06fr )r� r� r� r r r �
_format_Tf� s zAccessLogger._format_Tfc C s t | d � S)Ni@B )r� )r� r� r� r r r � _format_D� s zAccessLogger._format_Dc s � � � f d d � | j D� S)Nc 3 s- | ]# \ } } | | � � � � f Vq d S)Nr )r r� r� )r� r� r� r r r � s z,AccessLogger._format_line.<locals>.<genexpr>)r� )rM r� r� r� r )r� r� r� r � _format_line� s zAccessLogger._format_linec C s� y� | j | | | � } t � } t � } xo | D]g \ } } | j | � | j t k rf | | | <q1 | \ } }
| j | i � } | | |
<| | | <q1 W| j j | j t
| � d | �Wn" t k
r� | j j d � Yn Xd S)N�extrazError in logging)
r� r� �dictr� r: r] rX r� �infor� r� � Exception� exception) rM r� r� r� Zfmt_info�valuesr� r� r� Zk1Zk2�dctr r r �log� s
'
zAccessLogger.log) rN rO rP rQ r� Z
LOG_FORMAT�re�compiler� r� r� r r� r� r� � staticmethodr� r� r� r� r� r� r� r� r� r� r� r� r� r r )r: r r� s@ ( r� c @ s= e Z d Z d Z d d � Z e d d � Z d d � Z d S) �reifyaR Use as a class method decorator. It operates almost exactly like
the Python `@property` decorator, but it puts the result of the
method it decorates into the instance dict after the first call,
effectively replacing the function it decorates with an instance
variable. It is, in Python parlance, a data descriptor.
c C sG | | _ y | j | _ Wn t k
r6 d | _ Yn X| j | _ d S)Nr1 )�wrappedrQ r� rN rZ )rM r� r r r r� � s
zreify.__init__c C sz yN y | j | j SWn5 t k
rL | j | � } | | j | j <| SYn XWn% t k
ru | d k rn | S� Yn Xd S)N)�_cacherZ �KeyErrorr� �AttributeError)rM �inst�owner� _sentinelru r r r �__get__� s
z
reify.__get__c C s t d � � d S)Nz reified property is read-only)r� )rM r� r� r r r �__set__� s z
reify.__set__N)rN rO rP rQ r� �sentinelr� r� r r r r r� � s
r� z[^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$a� ^(?:(?:(?:[A-F0-9]{1,4}:){6}|(?=(?:[A-F0-9]{0,4}:){0,6}(?:[0-9]{1,3}\.){3}[0-9]{1,3}$)(([0-9A-F]{1,4}:){0,5}|:)((:[0-9A-F]{1,4}){1,5}:|:)|::(?:[A-F0-9]{1,4}:){5})(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])|(?:[A-F0-9]{1,4}:){7}[A-F0-9]{1,4}|(?=(?:[A-F0-9]{0,4}:){0,7}[A-F0-9]{0,4}$)(([0-9A-F]{1,4}:){1,7}|:)((:[0-9A-F]{1,4}){1,7}|:)|(?:[A-F0-9]{1,4}:){7}:|:(:[A-F0-9]{1,4}){7})$�flagsr< c C s� | d k r d St | t � rH t j | � s= t j | � rA d Sd Sn_ t | t t t f � r� t j | � s~ t j | � r� d Sd Sn t
d j | t | � � � � d S)NFTz {} [{}] is not a str or bytes)
rH r] � _ipv4_regex�match� _ipv6_regex�bytes� bytearray�
memoryview� _ipv4_regexb� _ipv6_regexbrI r� rg )rs r r r �
is_ip_address s r� c
C s� t t j � � } | t k r� d } d } t j | � \ } } } } } } } }
} d | | | | | | | | | f a | a t S)N�Mon�Tue�Wed�Thu�Fri�Sat�Sunr1 �Jan�Feb�Mar�Apr�May�Jun�Jul�Aug�Sep�Oct�Nov�Decz#%s, %02d %3s %4d %02d:%02d:%02d GMT)r� r� r� r� r� r� r� )
r1 r� r� r� r� r� r� r� r� r� r� r� r� )�intr� �_cached_current_datetime�gmtime�_cached_formatted_datetime) r� Z _weekdaynameZ
_monthnameZyearZmonthZdayZhhZmmZssZwd�y�zr r r �rfc822_formatted_time! s *$r c
C sI | \ } } | � } | d k rE t t � � t | | � � Wd QRXd S)N)r r� r� )r� �refrZ �obr r r �_weakref_handle7 s
r c C s` | d k r\ | d k r\ | j � | } | r: t | � } | j | t t j | � | f � Sd S)Nr )r� r �call_atr �weakrefr )r rZ �timeoutrw Z ceil_timeout�whenr r r �weakref_handle? s
r c C sB | d k r> | d k r> t | j � | � } | j | | � Sd S)Nr )r r� r
)�cbr rw r
r r r �
call_laterH s r c @ s^ e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d
d � Z d d
� Z d S)�
TimeoutHandlez Timeout handle c C s | | _ | | _ g | _ d S)N)�_timeout�_loop�
_callbacks)rM rw r r r r r� Q s zTimeoutHandle.__init__c O s | j j | | | f � d S)N)r r� )rM �callbackr. r/ r r r �registerV s zTimeoutHandle.registerc C s | j j � d S)N)r �clear)rM r r r �closeY s zTimeoutHandle.closec C sT | j d k rP | j d k rP t | j j � | j � } | j j | | j � Sd S)Nr )r r r r� r
�__call__)rM Zatr r r �start\ s zTimeoutHandle.startc C sM | j d k r@ | j d k r@ t | j � } | j | j � n t � } | S)Nr )r � TimerContextr r r � TimerNoop)rM �timerr r r r a s
zTimeoutHandle.timerc C sO x; | j D]0 \ } } } t t � � | | | � Wd QRXq
W| j j � d S)N)r r r� r )rM r r. r/ r r r r i s
zTimeoutHandle.__call__N)
rN rO rP rQ r� r r r r r r r r r r N s r c @ s( e Z d Z d d � Z d d � Z d S)r c C s | S)Nr )rM r r r � __enter__s s zTimerNoop.__enter__c C s d S)NFr )rM �exc_type�exc_val�exc_tbr r r �__exit__v s zTimerNoop.__exit__N)rN rO rP r r" r r r r r q s r c @ sF e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d
S) r z( Low resolution timeout context manager c C s | | _ g | _ d | _ d S)NF)r �_tasks�
_cancelled)rM rw r r r r� } s zTimerContext.__init__c C s] t d | j � } | d k r* t d � � | j rI | j � t j d � | j j | � | S)Nrw z4Timeout context manager should be used inside a task) rx r r` r$ �cancelry � TimeoutErrorr# r� )rM r| r r r r � s
zTimerContext.__enter__c C s> | j r | j j � | t j k r: | j r: t j d � d S)N)r# �popry �CancelledErrorr$ r& )rM r r r! r r r r" � s
zTimerContext.__exit__c C s: | j s6 x! t | j � D] } | j � q Wd | _ d S)NT)r$ r� r# r% )rM r| r r r r � s zTimerContext.timeoutN)rN rO rP rQ r� r r" r r r r r r z s
r c @ s e Z d Z d d � Z d S)� CeilTimeoutc C st | j d k rp t d | j � | _ | j d k r? t d � � | j j t | j j � | j � | j � | _ | S)Nrw z4Timeout context manager should be used inside a task)
r rx r Z_taskr` r
r r� Z _cancel_taskZ_cancel_handler)rM r r r r � s (zCeilTimeout.__enter__N)rN rO rP r r r r r r) � s r) c @ s� e Z d Z e d d d g � Z d Z d Z e Z d d � Z e
d e j d d � � Z
e
d e j d
d � � Z e
d e j d
d � � Z d S)� HeadersMixin�
_content_type�
_content_dict�_stored_content_typeNc C sI | | _ | d k r* d | _ i | _ n t j | � \ | _ | _ d S)Nzapplication/octet-stream)r- r+ r, �cgiZ parse_header)rM �rawr r r �_parse_content_type� s
z HeadersMixin._parse_content_type�
_CONTENT_TYPEc C s5 | j j | � } | j | k r. | j | � | j S)z7The value of content part for Content-Type HTTP header.)�_headersrX r- r0 r+ )rM r1 r/ r r r � content_type� s
zHeadersMixin.content_typec C s> | j j | � } | j | k r. | j | � | j j d � S)z7The value of charset part for Content-Type HTTP header.�charset)r2 rX r- r0 r, )rM r1 r/ r r r r4 � s
zHeadersMixin.charset�_CONTENT_LENGTHc C s& | j j | � } | r" t | � Sd S)z(The value of Content-Length HTTP header.N)r2 rX r )rM r5 �content_lengthr r r r6 � s zHeadersMixin.content_length)rN rO rP � frozensetZATTRSr+ r, r� r- r0 �propertyr Z CONTENT_TYPEr3 r4 ZCONTENT_LENGTHr6 r r r r r* � s r* c C s | j � s | j | � d S)N)�done�
set_result)�fut�resultr r r r: � s r: c C s | j � s | j | � d S)N)r9 �
set_exception)r; �excr r r r= � s r= )r )r r )r r )drQ ry rA rE r. r� r� r~ r\ rV r� �sysr� r � collectionsr �
contextlibr �mathr Zpathlibr Z urllib.parser Zurllib.requestr Z
async_timeoutrh Z multidictr Zyarlr r1 r �abcr r� r
�__all__� version_infoZPY_36rz Zidna_sslZpatch_match_hostname�objectr� �boolrW rX Z
NO_EXTENSIONSr� �rangeZCHARr ZCTLZ
SEPARATORSr� Z
coroutinesZ_DEBUGZ old_debug� coroutiner0 r rT rc r� rf rv rx r} r� r� r� r� r� r� Z
_ipv4_patternZ
_ipv6_patternr� r� �
IGNORECASEr� rB r� r� r� r r r r r r r r r r r) r* r: r= r r r r �<module> s�
"/' (3 + + & �"! # %
+