���Z�s � @ s� d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z m Z m
Z
d d l m Z m
Z
m Z d d l m Z d d l m Z m Z m Z m Z m Z d d l m Z m Z m Z m Z d d l m Z d d l m Z m Z m! Z! m" Z" m# Z# m$ Z$ d Z% Gd d
� d
e& � Z' Gd d � d e& � Z( d d � Z) d d d � Z* Gd d � d � Z+ Gd d � d � Z, e$ e, � Gd d � d e! � � Z- Gd d
� d
� Z. Gd d � d e! � Z/ Gd d � d � Z0 d S) � N)�Mapping�Sequence�deque)� parse_qsl�unquote� urlencode)� CIMultiDict� )�CONTENT_DISPOSITION�CONTENT_ENCODING�CONTENT_LENGTH�CONTENT_TRANSFER_ENCODING� CONTENT_TYPE)�CHAR�TOKEN�parse_mimetype�reify)�
HttpParser)� JsonPayload� LookupError�Payload�
StringPayload� get_payload� payload_type�MultipartReader�MultipartWriter�BodyPartReader�BadContentDispositionHeader�BadContentDispositionParam�parse_content_disposition� content_disposition_filenamec @ s e Z d Z d S)r N)�__name__�
__module__� __qualname__� r$ r$ �
/multipart.pyr s c @ s e Z d Z d S)r N)r! r" r# r$ r$ r$ r% r s c s\ d d � � d d � } � f d d � } d d � } d d
� } d d j t t j t � � d
d � } | sy d i f S| j d � ^ } } � | � s� t j t | � � d i f Si } x�| rK| j d � } d | k r� t j t | � � d i f S| j d d � \ }
} |
j
� j � }
| j � } |
| k r]t j t | � � d i f S� |
� s�t j t
| � � q� n�| |
� r�| | � r�| | d d � � } q>� | � s>t j t
| � � q� nf| |
� rt| | � r| j d d � \ } }
} | pd } n t j t
| � � q� y t | | d � } Wq>t k
rpt j t
| � � w� Yq>Xn� d } | | � r�d } | | d d � j d � � } nm � | � r�d } nX | rd | | d f } | | � r| j d � | | d d � j d � � } d } | r>t j t | � � d i f S| | |
<q� W| j
� | f S) Nc S s | o t t | � k S)N)r �set)�stringr$ r$ r% �is_token$ s z+parse_content_disposition.<locals>.is_tokenc S s | d | d k o d k SS)Nr r �"���r$ )r' r$ r$ r% � is_quoted' s z,parse_content_disposition.<locals>.is_quotedc s � | � o | j d � d k S)N�'� )�count)r' )r( r$ r% �
is_rfc5987* s z-parse_content_disposition.<locals>.is_rfc5987c S s
| j d � S)N�*)�endswith)r' r$ r$ r% �is_extended_param- s z4parse_content_disposition.<locals>.is_extended_paramc S sV | j d � d } | s d S| j d � r< | | d � n
| | d � } | j � S)Nr0 r Fr* )�findr1 �isdigit)r' �posZ substringr$ r$ r% �is_continuous_param0 s
/z6parse_content_disposition.<locals>.is_continuous_param�chars� c S s t j d j | � d | � S)Nz\\([{}])z\1)�re�sub�format)�textr7 r$ r$ r% �unescape7 s z+parse_content_disposition.<locals>.unescape�;r �=r r, r- zutf-8�strictTFz\/z%s;%sr* r* r* )�join�mapr9 �escaper �split�warnings�warnr �pop�lower�strip�lstripr r �UnicodeDecodeError)�headerr+ r/ r2 r6 r= Zdisptype�parts�params�item�key�value�encoding�_Zfailed�_valuer$ )r( r% r " sz '
"
�filenamec
sH d | � | s d S� | k r( | � S| | k r< | | Sg } t � f d d � | j � D� � } xz t | � D]l \ } \ } } | j d d � \ } } | j d � r� | d d � } | t | � k r� | j | � qt Pqt W| s� d Sd j | � } d | k r@| j d d � \ } } } | p-d } t | | d
� S| Sd S) Nz%s*c 3 s0 | ]& \ } } | j � � r | | f Vq d S)N)�
startswith)�.0rP rQ )�name_sufr$ r% � <genexpr>� s z/content_disposition_filename.<locals>.<genexpr>r0 r r8 r, r- zutf-8r@ r* ) �sorted�items� enumeraterD r1 �str�appendrA r )
rN �namerM Zfnparams�numrP rQ rS �tailrR r$ )rX r% r � s2
c @ s^ e Z d Z d Z d d � Z d d � Z d d � Z d d � Z d
d � Z d d
� Z d S)�MultipartResponseWrapperz~Wrapper around the MultipartBodyReader.
It takes care about
underlying connection and close it when it needs in.
c C s | | _ | | _ d S)N)�resp�stream)�selfrc rd r$ r$ r% �__init__� s z!MultipartResponseWrapper.__init__c C s | S)Nr$ )re r$ r$ r% � __aiter__� s z"MultipartResponseWrapper.__aiter__c � s' | j � Id H} | d k r# t � | S)N)�next�StopAsyncIteration)re �partr$ r$ r% � __anext__� s z"MultipartResponseWrapper.__anext__c C s | j j j � S)z2Returns True when all response data had been read.)rc �content�at_eof)re r$ r$ r% rm � s z MultipartResponseWrapper.at_eofc � s6 | j j � Id H} | j j � r2 | j � Id H| S)z#Emits next multipart reader object.N)rd rh rm �release)re rO r$ r$ r% rh � s z MultipartResponseWrapper.nextc � s | j j � Id Hd S)zPReleases the connection gracefully, reading all the content
to the void.N)rc rn )re r$ r$ r% rn � s z MultipartResponseWrapper.releaseN)
r! r" r# �__doc__rf rg rk rm rh rn r$ r$ r$ r% rb � s rb c @ s6 e Z d Z d Z d Z d d � Z d d � Z d d � Z d d
� Z d d d
d � Z e d d � Z
d d � Z d d � Z d d � Z
d d � Z d d d d � Z d d d d � Z d d d d � Z d! d" � Z d# d$ � Z d% d& � Z d' d( � Z d d) d* � Z e d+ d, � � Z e d- d. � � Z d S)/r z&Multipart reader for single body part.i c C s� | | _ | | _ | | _ d | _ | j j t d � } | d k rQ t | � n d | _ d | _ t � | _
d | _ d | _ i | _
d S)NFr )�headers� _boundary�_content�_at_eof�getr �int�_length� _read_bytesr �_unread� _prev_chunk� _content_eof�_cache)re �boundaryrp rl �lengthr$ r$ r% rf � s ! zBodyPartReader.__init__c C s | S)Nr$ )re r$ r$ r% rg � s zBodyPartReader.__aiter__c � s' | j � Id H} | d k r# t � | S)N)rh ri )re rj r$ r$ r% rk � s zBodyPartReader.__anext__c � s | j � Id H} | s d S| S)N)�read)re rO r$ r$ r% rh � s zBodyPartReader.next�decodeFc � s[ | j r
d St � } x+ | j sC | j | j | j � Id H� q W| rW | j | � S| S)z�Reads body part data.
decode: Decodes data following by encoding
method from Content-Encoding header. If it missed
data remains untouched
� N)rs � bytearray�extend�
read_chunk�
chunk_sizer )re r �datar$ r$ r% r~ � s "
zBodyPartReader.readc � s� | j r
d S| j r- | j | � Id H} n | j | � Id H} | j t | � 7_ | j | j k rq d | _ | j r� | j j � Id H} d | k s� t d � � | S)zWReads body part content chunk of the specified size.
size: chunk size
r� NTs
z3reader did not read all the data or it is malformed) rs rv �_read_chunk_from_length�_read_chunk_from_streamrw �lenrr �readline�AssertionError)re �size�chunkZclrfr$ r$ r% r� � s zBodyPartReader.read_chunkc � sO | j d k s t d � � t | | j | j � } | j j | � Id H} | S)Nz(Content-Length required for chunked read)rv r� �minrw rr r~ )re r� r� r� r$ r$ r% r� s
z&BodyPartReader._read_chunk_from_lengthc � s� | t | j � d k s% t d � � | j d k } | rT | j j | � Id H| _ | j j | � Id H} | j t | j j � � 7_ | j d k s� t d � � | j | } d | j } | r� | j | � } n. | j | t
d t | j � t | � � � } | d k rt| j j | | d � � | | k rL| j d | � | _ | t | j � | � } | std | _ | j } | | _ | S)Nr- z<Chunk size must be greater or equal than boundary length + 2� zReading after EOFs
r T)
r� rq r� ry rr r~ rz ru rm r3 �maxZ unread_datars )re r� Z first_chunkr� Zwindowr: �idx�resultr$ r$ r% r� s.
. z&BodyPartReader._read_chunk_from_streamc � s� | j r
d S| j r( | j j � } n | j j � Id H} | j | j � r� | j d � } | j } | j d } | | k s� | | k r� d | _ | j j | � d SnF | j j � Id H} | j | j � r� | d d � } | j j | � | S)z Reads body part by line by line.r� Ns
s --Tr- ���) rs rx �popleftrr r� rV rq �rstripr^ )re �lineZsliner| Z
last_boundary� next_liner$ r$ r% r� 5 s$
zBodyPartReader.readlinec � s6 | j r
d Sx" | j s1 | j | j � Id Hq Wd S)z0Like read(), but reads all the data to the void.N)rs r� r� )re r$ r$ r% rn R s zBodyPartReader.releaserR Nc � s< | j d d � Id H} | p, | j d d � } | j | � S)z;Like read(), but assumes that body part contains text data.r TN�defaultzutf-8)r~ � get_charsetr )re rR r� r$ r$ r% r< Y s zBodyPartReader.textc � sO | j d d � Id H} | s! d S| p6 | j d d � } t j | j | � � S)z<Like read(), but assumes that body parts contains JSON data.r TNr� zutf-8)r~ r� �json�loadsr )re rR r� r$ r$ r% r� a s
zBodyPartReader.jsonc � s^ | j d d � Id H} | s! d S| p6 | j d d � } t | j � j | � d d d | �S)zXLike read(), but assumes that body parts contains form
urlencoded data.
r TNr� zutf-8Zkeep_blank_valuesrR )r~ r� r r� r )re rR r� r$ r$ r% �formi s zBodyPartReader.formc C s | j S)z<Returns True if the boundary was reached or False otherwise.)rs )re r$ r$ r% rm u s zBodyPartReader.at_eofc C s> t | j k r | j | � } t | j k r: | j | � S| S)zrDecodes data according the specified Content-Encoding
or Content-Transfer-Encoding headers value.
)r
rp �_decode_content_transferr �_decode_content)re r� r$ r$ r% r y s
zBodyPartReader.decodec C s | j t j � } | d k r3 t j | t j � S| d k rV t j | d t j � S| d k rf | St d j | � � � d S)N�deflate�gzip� �identityz unknown content encoding: {})rp r rH �zlib�
decompress� MAX_WBITS� RuntimeErrorr; )re r� rR r$ r$ r% r� � s z BodyPartReader._decode_contentc C sn | j t j � } | d k r, t j | � S| d k rE t j | � S| d k rU | St d j | � � � d S)N�base64zquoted-printable�binary�8bit�7bitz%unknown content transfer encoding: {})r� r� r� ) rp r
rH r� Z b64decode�binasciiZa2b_qpr� r; )re r� rR r$ r$ r% r� � s
z'BodyPartReader._decode_content_transferc C s4 | j j t d � } t | � } | j j d | � S)z>Returns charset parameter from Content-Type header or default.r8 �charset)rp rt r r �
parameters)re r� �ctype�mimetyper$ r$ r% r� � s zBodyPartReader.get_charsetc C s+ t | j j t � � \ } } t | d � S)zoReturns name specified in Content-Disposition header or None
if missed or header is malformed.
r_ )r rp rt r
r )re rS rN r$ r$ r% r_ � s zBodyPartReader.namec C s+ t | j j t � � \ } } t | d � S)zsReturns filename specified in Content-Disposition header or None
if missed or header is malformed.
rU )r rp rt r
r )re rS rN r$ r$ r% rU � s zBodyPartReader.filename)r! r" r# ro r� rf rg rk rh r~ r� r� r� r� rn r< r� r� rm r r� r� r� r r_ rU r$ r$ r$ r% r � s,
c s. e Z d Z � f d d � Z d d � Z � S)�BodyPartReaderPayloadc sn t � j | | | � i } | j d k r8 | j | d <| j d k rT | j | d <| rj | j d | � d S)Nr_ rU Z
attachment)�superrf r_ rU Zset_content_disposition)re rQ �args�kwargsrN )� __class__r$ r% rf � s
z BodyPartReaderPayload.__init__c � s^ | j } | j d d � Id H} x7 | rY | j | j | � � | j d d � Id H} q# Wd S)Nr� r- r� i i )rT r� �writer )re �writerZfieldr� r$ r$ r% r� � s
zBodyPartReaderPayload.write)r! r" r# rf r� r$ r$ )r� r% r� � s r� c @ s� e Z d Z d Z e Z d Z e Z d d � Z d d � Z
d d � Z e d d
� � Z
d d � Z d
d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d S)!r zMultipart body reader.Nc C sS | | _ d | j � j � | _ | | _ d | _ d | _ d | _ g | _ d S)Nz--FT) rp �
_get_boundary�encoderq rr �
_last_partrs �_at_bofrx )re rp rl r$ r$ r% rf � s zMultipartReader.__init__c C s | S)Nr$ )re r$ r$ r% rg � s zMultipartReader.__aiter__c � s' | j � Id H} | d k r# t � | S)N)rh ri )re rj r$ r$ r% rk � s zMultipartReader.__anext__c C s% | j | | | j | j � � } | S)z�Constructs reader instance from HTTP response.
:param response: :class:`~aiohttp.client.ClientResponse` instance
)�response_wrapper_clsrp rl )�cls�response�objr$ r$ r% �
from_response� s z MultipartReader.from_responsec C s | j S)zSReturns True if the final boundary was reached or
False otherwise.
)rs )re r$ r$ r% rm � s zMultipartReader.at_eofc � sw | j r
d S| j � Id H| j r@ | j � Id Hd | _ n | j � Id H| j r\ d S| j � Id H| _ | j S)z#Emits the next multipart body part.NF)rs �_maybe_release_last_partr� �_read_until_first_boundary�_read_boundary�fetch_next_partr� )re r$ r$ r% rh � s zMultipartReader.nextc � sA x: | j s< | j � Id H} | d k r* P| j � Id Hq Wd S)z=Reads all the body parts to the void till the final boundary.N)rs rh rn )re rO r$ r$ r% rn s
zMultipartReader.releasec � s | j � Id H} | j | � S)z"Returns the next body part reader.N)�
_read_headers�_get_part_reader)re rp r$ r$ r% r� s z MultipartReader.fetch_next_partc C s� | j t d � } t | � } | j d k re | j d k rR t | � | | j � S| j | | j � S| j | j | | j � Sd S)z�Dispatches the response by the `Content-Type` header, returning
suitable reader instance.
:param dict headers: Response headers
r8 � multipartN)rt r r �type�multipart_reader_clsrr �part_reader_clsrq )re rp r� r� r$ r$ r% r� s z MultipartReader._get_part_readerc C s� t | j t � } | j d k s. t d � � d | j k rT t d | j t � � | j d } t | � d k r� t d | � � | S)Nr� z!multipart/* content type expectedr| z$boundary missed for Content-Type: %s�F z&boundary %r is too long (70 chars max))r rp r r� r� r� �
ValueErrorr� )re r� r| r$ r$ r% r� & s
z MultipartReader._get_boundaryc � s( | j r | j j � S| j j � Id HS)N)rx rG rr r� )re r$ r$ r% � _readline8 s
zMultipartReader._readlinec � sz xs | j � Id H} | d k r3 t d | j � � | j � } | | j k rR d S| | j d k r d | _ d Sq Wd S)Nr� z#Could not find starting boundary %rs --T)r� r� rq r� rs )re r� r$ r$ r% r� = s
z*MultipartReader._read_until_first_boundaryc � s� | j � Id Hj � } | | j k r) n� | | j d k r� d | _ | j � Id H} | j � Id H} | d d � d k r� | j j | � q� | j j | | g � n t d | | j f � � d S)Ns --Tr- z Invalid boundary %r, expected %r)r� r� rq rs rx r^ r� r� )re r� Zepiloguer� r$ r$ r% r� J s z MultipartReader._read_boundaryc � sf d g } x8 | j j � Id H} | j � } | j | � | s Pq Wt � } | j | � ^ } } | S)Nr� )rr r� rI r^ r Z
parse_headers)re �linesr� �parserrp rS r$ r$ r% r� b s
z MultipartReader._read_headersc � sS | j d k rO | j j � s0 | j j � Id H| j j | j j � d | _ d S)z8Ensures that the last read body part is read completely.N)r� rm rn rx r� )re r$ r$ r% r� n s
z(MultipartReader._maybe_release_last_part)r! r" r# ro rb r� r� r r� rf rg rk � classmethodr� rm rh rn r� r� r� r� r� r� r� r� r$ r$ r$ r% r � s&
c s� e Z d Z d Z d d � f d d � Z d d � Z d d � Z d
d � Z d d
� Z e j
d � Z e j
d � Z e
d d � � Z e
d d � � Z d d d � Z d d � Z d d d � Z d d d � Z e
d d � � Z d d � Z � S) r zMultipart body writer.ZmixedNc s� | d k r | n t j � j } y | j d � | _ Wn! t k
rZ t d � d � Yn Xd j | | j � } t � j
d d | �g | _ t � | _
| j | j
t <d S)N�asciiz(boundary should contain ASCII only charszmultipart/{}; boundary={}� content_type)�uuidZuuid4�hexr� rq �UnicodeEncodeErrorr� r; �_boundary_valuer� rf �_partsr �_headersr� r )re �subtyper| r� )r� r$ r% rf z s !
zMultipartWriter.__init__c C s | S)Nr$ )re r$ r$ r% � __enter__� s zMultipartWriter.__enter__c C s d S)Nr$ )re �exc_type�exc_val�exc_tbr$ r$ r% �__exit__� s zMultipartWriter.__exit__c C s
t | j � S)N)�iterr� )re r$ r$ r% �__iter__� s zMultipartWriter.__iter__c C s
t | j � S)N)r� r� )re r$ r$ r% �__len__� s zMultipartWriter.__len__s \A[!#$%&'*+\-.^_`|~\w]+\Zs [\x00-\x08\x0A-\x1F\x7F]c C s� | j } t j | j | � r+ | j d � St j | j | � rL t d � � | j d d � } | j d d � } d | j d � d S)zyWrap boundary parameter value in quotes, if necessary.
Reads self.boundary and returns a unicode sting.
r� z*boundary value contains invalid characterss \s \\� "s \"r) ) rq r9 �match�_valid_tchar_regexr �search�_invalid_qdtext_char_regexr� �replace)re rQ Zquoted_value_contentr$ r$ r% r� � s
z MultipartWriter._boundary_valuec C s | j j d � S)Nr� )rq r )re r$ r$ r% r| � s zMultipartWriter.boundaryc C s� | d k r t � } t | t � r\ | j d k rF | j j | � n | | _ | j | � Sy | j t | d | �� SWn t k
r� t � Yn Xd S)Nrp )
r �
isinstancer rp �updater� �append_payloadr r � TypeError)re r� rp r$ r$ r% r^ � s
zMultipartWriter.appendc C sH t | j k r | j | j t <| j j t d � j � } | ra | d k ra t d j | � � � | d k rs d } | j j t d � j � } | d k r� t d
j | � � � | d k r� d } | j } | d k r� | p� | r� t
| � | j t <d j d d � | j j
� D� � j d
� d } | j j | | | | f � | S)z)Adds a new body part to multipart writer.r8 r� r� r� z unknown content encoding: {}Nr� �quoted-printabler� z%unknown content transfer encoding: {}c S s( g | ] \ } } | d | d � q S)z: z
r$ )rW �k�vr$ r$ r% �
<listcomp>� s z2MultipartWriter.append_payload.<locals>.<listcomp>zutf-8s
)r� r� r� )r8 r� r� r� )r rp r� rt r rH r� r; r
r� r] r rA r[ r� r� r^ )re �payloadrR � te_encodingr� rp r$ r$ r% r� � s,
z MultipartWriter.append_payloadc C s. | d k r t � } | j t | d | �� S)zHelper to append JSON part.Nrp )r r� r )re r� rp r$ r$ r% � append_json� s zMultipartWriter.append_jsonc C s� t | t t f � s t � | d k r0 t � } t | t � rQ t | j � � } t | d d �} | j t | d | d d �� S)z&Helper to append form urlencoded part.NZdoseqTrp r� z!application/x-www-form-urlencoded)
r� r r r� r �listr[ r r� r )re r� rp r� r$ r$ r% � append_form� s zMultipartWriter.append_formc C s� | j s
d Sd } xl | j D]a \ } } } } | sJ | sJ | j d k rN d S| d t | j � d | j t | � d 7} q W| d t | j � d 7} | S)zSize of the payload.r Nr- � )r� r� r� rq )re �totalrj rp rR r� r$ r$ r% r� s % zMultipartWriter.sizec � s | j s
d Sx� | j D]� \ } } } } | j d | j d � Id H| j | � Id H| sd | r� t | � } | r� | j | � | r� | j | � | j | � Id H| j � Id Hn | j | � Id H| j d � Id Hq W| j d | j d � Id Hd S)z Write body.Ns --s
s --
)r� r� rq �MultipartPayloadWriter�enable_compression�enable_encoding� write_eof)re r� rj rp rR r� �wr$ r$ r% r� s
zMultipartWriter.write)r! r" r# ro rf r� r� r� r� r9 �compiler� r� �propertyr� r| r^ r� r� r� r� r� r$ r$ )r� r% r w s #c @ sO e Z d Z d d � Z d d � Z d d d � Z d d � Z d
d � Z d S)
r� c C s | | _ d | _ d | _ d S)N)�_writer� _encoding� _compress)re r� r$ r$ r% rf : s z MultipartPayloadWriter.__init__c C s= | d k r$ | | _ t � | _ n | d k r9 d | _ d S)Nr� zquoted-printable)r� r� �_encoding_buffer)re rR r$ r$ r% r� ? s
z&MultipartPayloadWriter.enable_encodingr� c C s< | d k r d t j n t j } t j d | � | _ d S)Nr� r� Zwbits)r� r� � compressobjr� )re rR Z zlib_moder$ r$ r% r� F s #z)MultipartPayloadWriter.enable_compressionc � s| | j d k r? | j j � } | r? d | _ | j | � Id H| j d k rx | j rx | j j t j | j � � Id Hd S)Nr� )r� �flushr� r� r� r� r� � b64encode)re r� r$ r$ r% r� K s z MultipartPayloadWriter.write_eofc � s | j d k r1 | r1 | j j | � } | s1 d S| j d k r� | j j | � | j r| j } t t | � d � \ } } | d | d � | | d d � } | _ | rt j | � } | j j
| � Id HnE | j d k r| j j
t j | � � Id Hn | j j
| � Id Hd S)Nr� r� zquoted-printable)
r� �compressr� r� r� �divmodr� r� r r� r� r� Zb2a_qp)re r� �bufferZdiv�modZ enc_chunkr$ r$ r% r� W s" ,!z MultipartPayloadWriter.writeN)r! r" r# rf r� r� r� r� r$ r$ r$ r% r� 8 s
r� )r r r r r r r )1r� r� r� r9 r� rE r� � collectionsr r r Z urllib.parser r r Z multidictr Zhdrsr
r r r
r Zhelpersr r r r Zhttpr r� r r r r r r �__all__�RuntimeWarningr r r r rb r r� r r r� r$ r$ r$ r% �<module> s6 (". b %� ��