a d@sNdZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z gdZ dZdZdZdZd Zd Zeejjd d ejjDZd ZdZdZedjZedj Z!edZ"edZ#hdZ$d@ddZ%Gdddej&j'Z(ddZ)e(fddZ*Gdddej+Z,Gd d!d!Z-z ddl.Z.Wne/yHYn0Gd"d#d#e-Z0e 1d#Gd$d%d%e2Z3Gd&d'd'e3Z4Gd(d)d)e3Z5Gd*d+d+e3Z6Gd,d-d-e3Z7Gd.d/d/e3Z8Gd0d1d1e3Z9Gd2d3d3e3Z:Gd4d5d5e:Z;Gd6d7d7e:ZGdd?d?e@e>ZAe3ZBdS)Aa HTTP/1.1 client library HTTPConnection goes through a number of "states", which define when a client may legally make another request or fetch the response for a particular request. This diagram details these state transitions: (null) | | HTTPConnection() v Idle | | putrequest() v Request-started | | ( putheader() )* endheaders() v Request-sent |\_____________________________ | | getresponse() raises | response = getresponse() | ConnectionError v v Unread-response Idle [Response-headers-read] |\____________________ | | | response.read() | putrequest() v v Idle Req-started-unread-response ______/| / | response.read() | | ( putheader() )* endheaders() v v Request-started Req-sent-unread-response | | response.read() v Request-sent This diagram presents the following rules: -- a second request may not be started until {response-headers-read} -- a response [object] cannot be retrieved until {request-sent} -- there is no differentiation between an unread response body and a partially read response body Note: this enforcement is applied by the HTTPConnection class. The HTTPResponse class does not enforce this state machine, which implies sophisticated clients may accelerate the request/response pipeline. Caution should be taken, though: accelerating the states beyond the above pattern may imply knowledge of the server's connection-close behavior for certain requests. For example, it is impossible to tell whether the server will close the connection UNTIL the response headers have been read; this means that further requests cannot be placed into the pipeline until it is known that the server will NOT be closing the connection. Logical State __state __response ------------- ------- ---------- Idle _CS_IDLE None Request-started _CS_REQ_STARTED None Request-sent _CS_REQ_SENT None Unread-response _CS_IDLE Req-started-unread-response _CS_REQ_STARTED Req-sent-unread-response _CS_REQ_SENT N)urlsplit) HTTPResponseHTTPConnection HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady BadStatusLine LineTooLongRemoteDisconnectederror responsesPiUNKNOWNIdlezRequest-startedz Request-sentcCsi|] }||jqS)phrase).0vrr#/usr/lib64/python3.9/http/client.py kriids[^:\s][^:\r\n]*s\n(?![ \t])|\r(?![ \t\n])z[- ]z[-]>PUTPATCHPOSTdatac Csjz |dWStyd}z@t|j|j|j|jd|||j|j|fdWYd}~n d}~00dS)zd?Z Z!S)FrrNcCsR|d|_||_||_d|_|_t|_t|_t|_ t|_ t|_ t|_ t|_ dS)Nrb)makefilerG debuglevel_methodrHmsg_UNKNOWNversionstatusreasonchunked chunk_leftlength will_close)r7sockrVmethodurlrrr__init__s  zHTTPResponse.__init__c Cst|jtdd}t|tkr*td|jdkrBtdt||sNt dz| dd\}}}WnBt yz| dd\}}d}Wnt yd}Yn0Yn0| d s| t|z$t|}|d ks|d krt|Wnt yt|Yn0|||fS) Nr1rJz status linerzreply:z-Remote end closed connection without responsezHTTP/ri)strrGrDrEr3rrVprintreprrsplit ValueError startswith _close_connrint)r7r;rZr[r\rrr _read_statuss2      zHTTPResponse._read_statusc Cs|jdurdS|\}}}|tkr&qHt|j}|jdkrDtd|~q||_|_| |_ |dvrnd|_ n| drd|_ nt |t|j|_|_|jdkr|jD]\}}td|d|q|jd }|r|d krd |_d|_nd |_||_d|_|jd }|r`|js`zt||_WntyJd|_Yn0|jdkrfd|_nd|_|tks|tksd|krdksn|jdkrd|_|js|js|jdurd |_dS)Nrzheaders:)zHTTP/1.0zHTTP/0.9 zHTTP/1. header:r0transfer-encodingr]TFcontent-lengthrHEAD)rHroCONTINUErIrGrVrhcoder[stripr\rZrlrrSrXitemsgetr2r]r^ _check_closer`r_rnrk NO_CONTENT NOT_MODIFIEDrW) r7rZr[r\skipped_headershdrvaltr_encr_rrrbegin9sd               zHTTPResponse.begincCsv|jd}|jdkr.|r*d|vr*dSdS|jdr>dS|rRd|vrRdS|jd}|rrd|vrrdSdS)N connectionrqcloseTFz keep-alivezproxy-connection)rHr{rZr2)r7connpconnrrrr|s    zHTTPResponse._check_closecCs|j}d|_|dSN)rGr)r7rGrrrrmszHTTPResponse._close_conncs2ztW|jr.|n|jr,|0dSr)superrrGrmr7 __class__rrrs   zHTTPResponse.closecst|jr|jdSr)rflushrGrrrrrs zHTTPResponse.flushcCsdS)zAlways returns TrueTrrrrrreadableszHTTPResponse.readablecCs |jduS)z!True if the connection is closed.N)rGrrrrisclosedszHTTPResponse.isclosedcCs|jdurdS|jdkr$|dS|durRt|}||}t|d|S|jr`|S|j durv|j }n4z| |j }Wnt y|Yn0d|_ ||SdS)Nrrvr) rGrWrm bytearrayreadinto memoryviewtobytesr]_readall_chunkedr_read _safe_readr )r7amtbr8srrrrs*      zHTTPResponse.readcCs|jdurdS|jdkr$|dS|jr4||S|jdur^t||jkr^t|d|j}|j|}|s||r||n&|jdur|j|8_|js||S)z^Read up to len(b) bytes into bytearray b and return the number of bytes read. Nrrv) rGrWrmr]_readinto_chunkedr_r3rr)r7rr8rrrrs$       zHTTPResponse.readintocCsp|jtd}t|tkr$td|d}|dkrB|d|}z t|dWStyj|Yn0dS)Nr1z chunk size;r) rGrDrEr3rfindrnrkrm)r7r;irrr_read_next_chunk_sizes     z"HTTPResponse._read_next_chunk_sizecCs:|jtd}t|tkr$td|s*q6|dvrq6qdS)Nr1z trailer linerA)rGrDrEr3rr7r;rrr_read_and_discard_trailers z&HTTPResponse._read_and_discard_trailercCsj|j}|sf|dur|dz |}WntyBtdYn0|dkr`||d}||_|S)Nrerr)r^rrrkr rrm)r7r^rrr_get_chunk_left$s   zHTTPResponse._get_chunk_leftcCsn|jtksJg}z6|}|dur&q>|||d|_qd|WStyhtd|Yn0dSNrr)r]rYrr6rr^rLr )r7valuer^rrrr<s  zHTTPResponse._readall_chunkedcCs|jtksJd}t|}zv|}|dur2|WSt||kr\||}|||_||WS|d|}||}||d}||7}d|_qWn&tytt|d|Yn0dS)Nr) r]rYrrr3_safe_readintor^r bytes)r7r total_bytesmvbr^r8temp_mvbrrrrJs$         zHTTPResponse._readinto_chunkedcCsTg}|dkrJ|jt|t}|s2td|||||t|8}qd|S)aVRead the number of bytes requested, compensating for partial reads. Normally, we have a blocking socket, but a read() can be interrupted by a signal (resulting in a partial read). Note that we cannot distinguish between EOF and an interrupt when zero bytes have been read. IncompleteRead() will be raised in this situation. This function should be used when bytes "should" be present for reading. If the bytes are truly not available (due to EOF), then the IncompleteRead exception can be used to detect the problem. rr)rGrmin MAXAMOUNTr rLr6r3)r7rrchunkrrrrbs zHTTPResponse._safe_readcCsd}t|}|t|kr~tt|kr>|dt}|j|}n |j|}|shtt|d|t|||d}||7}q |S)z2Same as _safe_read, but for reading into a buffer.rN)rr3rrGrr r)r7rrrrr8rrrrys      zHTTPResponse._safe_readintocCs|jdus|jdkrdS|jr(||S|jdurJ|dksD||jkrJ|j}|j|}|sh|rh|n|jdur|jt|8_|S)zvRead with at most one underlying system call. If at least one byte is buffered, return that instead. Nrvrr)rGrWr]_read1_chunkedr_read1rmr3)r7r8resultrrrrs    zHTTPResponse.read1cCs4|jdus|jdkrdS|jr(||S|j|S)Nrvr)rGrWr] _peek_chunkedpeek)r7r8rrrrs  zHTTPResponse.peekcs|jdus|jdkrdS|jr*t|S|jdurL|dksF||jkrL|j}|j|}|sj|rj|n|jdur|jt|8_|S)Nrvrr)rGrWr]rrDr_rmr3)r7limitrrrrrDs    zHTTPResponse.readlinecCsd|}|dus|dkrdSd|kr0|ks6n|}|j|}|jt|8_|s`td|Sr)rrGrr^r3r )r7r8r^rrrrrs zHTTPResponse._read1_chunkedcCsBz |}Wnty YdS0|dur.dS|j|d|S)Nr)rr rGr)r7r8r^rrrrs  zHTTPResponse._peek_chunkedcCs |jSr)rGfilenorrrrrszHTTPResponse.filenocCsF|jdurt|j|p|}t|ts4t|ds8|Sd|SdS)axReturns the value of the header matching *name*. If there are multiple matching headers, the values are combined into a single string separated by commas and spaces. If no matching header is found, returns *default* or None if the *default* is not specified. If the headers are unknown, raises http.client.ResponseNotReady. N__iter__z, )rHrget_all isinstancerghasattrrL)r7r,defaultrHrrr getheaders zHTTPResponse.getheadercCs|jdurtt|jS)z&Return list of (header, value) tuples.N)rHrlistrzrrrr getheaderss zHTTPResponse.getheaderscCs|SrrrrrrrszHTTPResponse.__iter__cCs|jS)ajReturns an instance of the class mimetools.Message containing meta-information associated with the URL. When the method is HTTP, these headers are those returned by the server at the head of the retrieved HTML page (including Content-Length and Content-Type). When the method is FTP, a Content-Length header will be present if (as is now usual) the server passed back a file length in response to the FTP retrieval request. A Content-Type header will be present if the MIME type can be guessed. When the method is local-file, returned headers will include a Date representing the file's last-modified time, a Content-Length giving file size, and a Content-Type containing a guess at the file's type. See also the description of the mimetools module. )rHrrrrinfoszHTTPResponse.infocCs|jS)aZReturn the real URL of the page. In some cases, the HTTP server redirects a client to another URL. The urlopen() function handles this transparently, but in some cases the caller needs to know which URL the client was redirected to. The geturl() method can be used to get at this redirected URL. )rcrrrrgeturls zHTTPResponse.geturlcCs|jS)zuReturn the HTTP status code that was sent with the response, or None if the URL is not an HTTP URL. )r[rrrrgetcodeszHTTPResponse.getcode)rNN)N)r)r)r)N)"r=r>r?rdrorr|rmrrrrrrrrrrrrrrrrDrrrrrrrrr __classcell__rrrrrs< !H  "     rc@s eZdZdZdZeZeZdZ dZ e ddZ e ddZ d ejd d fd d Zd7d dZddZddZddZddZddZddZddZddZd8d d!Zd9d"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Z d:dd.d/d0Z!d ifdd.d1d2Z"d3d4Z#d5d6Z$d S);rrqzHTTP/1.1r1rcCs t|tjS)zFTest whether a file-like object is a text or a binary stream. )rio TextIOBase)streamrrr _is_textIO%szHTTPConnection._is_textIOcCsd|dur|tvrdSdSt|dr*dSzt|}|jWStyLYn0t|tr`t|SdS)aGet the content-length based on the body. If the body is None, we set Content-Length: 0 for methods that expect a body (RFC 7230, Section 3.3.2). We also set the Content-Length for any method if the body is a str or bytes-like object and not a file. Nrr) upper_METHODS_EXPECTING_BODYrrnbytes TypeErrorrrgr3)bodyrbmvrrr_get_content_length+s    z"HTTPConnection._get_content_lengthN cCsn||_||_||_d|_g|_d|_t|_d|_d|_ d|_ i|_ | ||\|_ |_||j tj|_dSr)timeoutsource_address blocksizera_buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterW _tunnel_host _tunnel_port_tunnel_headers _get_hostporthostport_validate_hostsocketcreate_connection_create_connection)r7rrrrrrrrrdKs zHTTPConnection.__init__cCs<|jrtd|||\|_|_|r.||_n |jdS)aDSet up host and port for HTTP CONNECT tunnelling. In a connection that uses HTTP CONNECT tunneling, the host passed to the constructor is used as a proxy server that relays all communication to the endpoint passed to `set_tunnel`. This done by sending an HTTP CONNECT request to the proxy server when the connection is established. This method must be called before the HTTP connection has been established. The headers argument should be a mapping of extra HTTP headers to send with the CONNECT request. z.Can't set up tunnel for established connectionN)ra RuntimeErrorrrrrclear)r7rrrHrrr set_tunnelas zHTTPConnection.set_tunnelcCs|dur|d}|d}||krzt||dd}WnFty||dddkrf|j}ntd||ddYn0|d|}n|j}|r|ddkr|ddkr|dd}||fS) Nr0]r1rfznonnumeric port: '%s'r[r)rfindrnrk default_portr )r7rrrjrrrrys    zHTTPConnection._get_hostportcCs ||_dSr)rV)r7levelrrrset_debuglevelszHTTPConnection.set_debuglevelc Cs d|jd|jf}|g}|jD]$\}}||d|ddq&|d|d|~|j|j |j d}| \}}}|t j jkr|td |d ||jtd } t| tkrtd | sq| d vrq|jdkrtd| qdS)NsCONNECT %s:%d HTTP/1.0 asciiz:  r$rBrrbzTunnel connection failed:  r1r@rArrr)rr%rrrzr6sendrLresponse_classrarWrohttp HTTPStatusOKrOSErrorryrGrDrEr3rrVrhrM) r7connectrHheaderrresponserZrxmessager;rrr_tunnels.    zHTTPConnection._tunnelc Csz||j|jf|j|j|_z|jtjtj dWn2t yf}z|j t j krRWYd}~n d}~00|j rv|dS)z3Connect to the host and port specified in __init__.r1N)rrrrrra setsockoptr IPPROTO_TCP TCP_NODELAYrerrno ENOPROTOOPTrrr7errrrs zHTTPConnection.connectcCsZt|_z4|j}|r d|_|W|j}|rVd|_|n|j}|rTd|_|0dS)z(Close the connection to the HTTP server.N)rrrarr)r7rarrrrrs  zHTTPConnection.closecCs|jdur |jr|nt|jdkr8tdt|t|dr|jdkrTtd||}|rt|jdkrttd| |j }|sq|r| d}|j |qtdSz|j |WnHt yt|tjjr|D]}|j |qnt dt|Yn0dS) zSend `data' to the server. ``data`` can be a string object, a bytes object, an array object, a file-like object that supports a .read() method, or an iterable object. Nrzsend:rsendIng a read()ableencoding file using iso-8859-1rJz9data should be a bytes-like object or an iterable, got %r)ra auto_openrrrVrhrirrrrr%sendallrr collectionsabcIterabletype)r7r#r% datablockdrrrrs8         zHTTPConnection.sendcCs|j|dS)zuAdd a line of output to the current request buffer. Assumes that the line does *not* end with \r\n. N)rr6)r7rrrr_outputszHTTPConnection._outputccs^|jdkrtd||}|r2|jdkr2td||j}|sDqZ|rR|d}|Vq2dS)NrrrrJ)rVrhrrrr%)r7rr%rrrr_read_readables    zHTTPConnection._read_readableFc Cs|jdd|j}|jdd=|||durt|drN||}nVz t|WnBtyz t|}Wn"tytdt |Yn0Yn0|f}|D]R}|s|j dkrt dq|r|j dkrt |d d d |d}||q|r|j dkr|d dS) zSend the currently buffered request and clear the buffer. Appends an extra \r\n to the buffer. A message_body may be specified, to be appended to the request. )rrrBNrzAmessage_body should be a bytes-like object or an iterable, got %rrzZero length chunk ignoredrqXrrs0 )rextendrLrrr rriterrrVrh _http_vsnr3r%)r7 message_bodyencode_chunkedrXchunksrrrr _send_outputs:             zHTTPConnection._send_outputc Cs|jr|jrd|_|jtkr(t|_n t|j||||_|pHd}||d|||j f}| | ||j dkr|sd}| drt|\}}}}}|rz|d}Wnty|d}Yn0|d |n|jr|j} |j} n |j} |j} z| d} Wnty0| d} Yn0| d d krNd | d } | |jkrh|d | n| d} |d d| | f|s|ddndS)a`Send a request to the server. `method' specifies an HTTP request method, e.g. 'GET'. `url' specifies the object being requested, e.g. '/index.html'. `skip_host' if True does not add automatically a 'Host:' header `skip_accept_encoding' if True does not add automatically an 'Accept-Encoding:' header N/z%s %s %srqrfrridnaHostr0r[]z%s:%szAccept-Encodingidentity)rrrr_CS_REQ_STARTEDr _validate_methodrW_validate_path _http_vsn_strr_encode_requestr rlrr%r& putheaderrrrrrrrM) r7rbrc skip_hostskip_accept_encodingrequestnetlocnil netloc_encrrhost_encrrr putrequest=sP             zHTTPConnection.putrequestcCs |dS)Nr)r%)r7r rrrrszHTTPConnection._encode_requestcCs,t|}|r(td|d|ddS)z&Validate a method name for putrequest.z)method can't contain control characters.  (found at least )N)$_contains_disallowed_method_pchar_researchrkgroup)r7rbmatchrrrrs zHTTPConnection._validate_methodcCs,t|}|r(td|d|ddS)zValidate a url for putrequest.&URL can't contain control characters. r&r'N!_contains_disallowed_url_pchar_rer)r r*)r7rcr+rrrrs   zHTTPConnection._validate_pathcCs,t|}|r(td|d|ddS)z9Validate a host so it doesn't contain control characters.r,r&r'Nr-)r7rr+rrrrs   zHTTPConnection._validate_hostcGs|jtkrtt|dr$|d}t|s:td|ft|}t|D]\\}}t|drl|d||<nt |t rt |d||<t ||rJtd||fqJd |}|d|}||dS) zkSend a request header line to the server. For example: h.putheader('Accept', 'text/html') r%rzInvalid header name %rr$zInvalid header value %rs s: N)rrrrr%_is_legal_header_namerkr enumeraterrnrg_is_illegal_header_valuerLr)r7rvaluesr one_valuerrrrrs"        zHTTPConnection.putheaderrcCs*|jtkrt|_nt|j||ddS)zIndicate that the last header line has been sent to the server. This method sends the request to the server. The optional message_body argument can be used to pass a message body associated with the request. r4N)rr _CS_REQ_SENTrr)r7rrrrr endheaderss zHTTPConnection.endheaderscCs||||||dS)z&Send a complete request to the server.N) _send_request)r7rbrcrrHrrrrr szHTTPConnection.requestc Cstdd|D}i}d|vr&d|d<d|vr6d|d<|j||fi|d|vrd |vrd }|||}|dur|dur|jd krtd |d }|ddq|dt|nd }|D]\} } || | qt|trt |d}|j ||ddS)Ncss|]}|VqdSr)r2)rkrrr rz/HTTPConnection._send_request..rr1rzaccept-encodingrrtrsFrzUnable to determine size of %rTzTransfer-Encodingr]zContent-Lengthrr4) frozensetr%rrVrhrrgrzrr.r6) r7rbrcrrHr header_namesskipscontent_lengthrrrrrr7s0      zHTTPConnection._send_requestcCs|jr|jrd|_|jtks&|jr0t|j|jdkrR|j|j|j|jd}n|j|j|jd}zZz | Wnt y| Yn0|j t ksJt|_|j r| n||_|WS| Yn0dS)a)Get the response from the server. If the HTTPConnection is in the correct state, returns an instance of HTTPResponse or of whatever object is returned by the response_class variable. If a request has not been sent or if a previous response has not be handled, ResponseNotReady is raised. If the HTTP response indicates that the connection should be closed, then it will be closed before the response is returned. When the connection is closed, the underlying socket is closed. Nrr)rrrr5rrVrrarWrConnectionErrorrr`rYr)r7rrrr getresponse5s0      zHTTPConnection.getresponse)NN)NF)FF)N)%r=r>r?r rrr HTTP_PORTrrrV staticmethodrrr_GLOBAL_DEFAULT_TIMEOUTrdrrrrrrrrr rr%rrrrrr6r r7r?rrrrrsJ     & 6    .rcsHeZdZdZeZdddejdfddddfdd ZfddZ Z S) HTTPSConnectionz(This class allows communication via SSL.Nr)contextcheck_hostnamerc stt|j||||| d|dus2|dus2|durHddl} | dtd||_||_|durtt }|j durtd|_ |j tj k} |dur|j }|r| std|s|r||||j durd|_ ||_|dur||j_ dS)N)rrzTkey_file, cert_file and check_hostname are deprecated, use a custom context instead.reTzMcheck_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED)rrCrdwarningswarnDeprecationWarningkey_file cert_filessl_create_default_https_contextpost_handshake_auth verify_mode CERT_NONErErkload_cert_chain_context) r7rrrIrJrrrDrErrF will_verifyrrrrds:    zHTTPSConnection.__init__cs6t|jr|j}n|j}|jj|j|d|_dS)z(Connect to a host on a given (SSL) port.)server_hostnameN)rrrrrQ wrap_socketra)r7rSrrrrs  zHTTPSConnection.connect) r=r>r?__doc__ HTTPS_PORTrrrBrdrrrrrrrCys$rCc@s eZdZdS)rNr=r>r?rrrrrsrc@s eZdZdS)rNrWrrrrrsrc@s eZdZdS)r NrWrrrrr sr c@seZdZddZdS)rcCs|f|_||_dSr)argsrZ)r7rZrrrrdszUnknownProtocol.__init__Nr=r>r?rdrrrrrsrc@s eZdZdS)rNrWrrrrrsrc@s eZdZdS)r NrWrrrrr sr c@s$eZdZdddZddZejZdS)r NcCs|f|_||_||_dSr)rXpartialexpected)r7rZr[rrrrdszIncompleteRead.__init__cCs2|jdurd|j}nd}d|jjt|j|fS)Nz, %i more expectedrfz%s(%i bytes read%s))r[rr=r3rZrrrr__repr__s    zIncompleteRead.__repr__)N)r=r>r?rdr\r(__str__rrrrr s r c@s eZdZdS)r NrWrrrrr sr c@s eZdZdS)r NrWrrrrr sr c@s eZdZdS)rNrWrrrrrsrc@s eZdZdS)rNrWrrrrrsrc@seZdZddZdS)rcCs|s t|}|f|_||_dSr)rirXr;rrrrrdszBadStatusLine.__init__NrYrrrrrsrc@seZdZddZdS)rcCst|dt|fdS)Nz&got more than %d bytes when reading %s)rrdrE)r7 line_typerrrrdszLineTooLong.__init__NrYrrrrrsrc@seZdZddZdS)rcOs(t|dtj|g|Ri|dS)Nrf)rrdConnectionResetError)r7poskwrrrrds zRemoteDisconnected.__init__NrYrrrrrsr)r#)CrU email.parserrN email.messagerrrrercollections.abcr urllib.parser__all__r@rVrYrrr5globalsupdater __members__r2rrrErFcompile fullmatchr/r)r1r.r(rr.rMessager/rIrSBufferedIOBaserrrK ImportErrorrCr6 Exceptionrrr rrr r r r rrrrr_rrrrrrszF       .] 8