o AfJ@sddlZddlZddlZddlZedZde_dggddgdggdd Zd gd gd Z e ee BZ Gd dde Z ddZdddZddZdS)Nzlocation.importerz3https://ftp.afrinic.net/pub/pub/dbase/afrinic.db.gz)z6https://ftp.apnic.net/apnic/whois/apnic.db.inet6num.gzz5https://ftp.apnic.net/apnic/whois/apnic.db.inetnum.gzz5https://ftp.apnic.net/apnic/whois/apnic.db.aut-num.gzz:https://ftp.apnic.net/apnic/whois/apnic.db.organisation.gzz'https://ftp.nic.ad.jp/jpirr/jpirr.db.gzz0https://ftp.lacnic.net/lacnic/dbase/lacnic.db.gz)z9https://ftp.ripe.net/ripe/dbase/split/ripe.db.inet6num.gzz8https://ftp.ripe.net/ripe/dbase/split/ripe.db.inetnum.gzz8https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gzz=https://ftp.ripe.net/ripe/dbase/split/ripe.db.organisation.gz)ZAFRINICZAPNICZJPNICLACNICZRIPEzBhttps://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latestzHhttps://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-extended-latest)ZARINrc@s:eZdZddZddZd ddZd dd Zd d d ZdS) DownloadercCs d|_dSN)proxy)selfr6/usr/lib/python3.10/site-packages/location/importer.py__init__fs zDownloader.__init__cCstd|||_dS)z= Sets a HTTP proxy that is used to perform all requests zUsing proxy %sN)loginfor)rurlrrr set_proxyis zDownloader.set_proxyNc Cs tjdd}tjj||d}|jr||jdtd|j tj |}t d|j D]}t d||j |fq.|} | d }|sIn||qAWd n1sYwY|d d } |j d } | dvrrd} n | ddkr{d} |d | rt dtj|dd}|S)z This method will fetch the content at the given URL and will return a file-object to a temporary file. If the content was compressed, it will be decompressed on the fly. i@)Zmax_sizedataZhttpzRetrieving %s...zResponse Headers:z %s: %sTiNrFz Content-Type)zapplication/x-gzipzapplication/gzipszGzip compression detectedrb)Zfileobjmode)tempfileZSpooledTemporaryFileurllibZrequestZRequestrrr r Zfull_urlZurlopendebugZheadersreadwriteseekgetgzipZGzipFile) rr rtZreqresheaderfZbufZgzip_compressedZ content_typerrr retrieveps<          zDownloader.retrievecC|j||d}t|S)zi This method will fetch the data from the URL and return an iterator for each block in the data. r)r iterate_over_blocksrr rrrrr request_blockszDownloader.request_blockscCr!)zh This method will fetch the data from the URL and return an iterator for each line in the data. r)r iterate_over_linesr#rrr request_linesr%zDownloader.request_linesr)__name__ __module__ __qualname__r rr r$r'rrrr res   @ rccs`t|D](}d}i}t|D]\}}|dd\}}|dkr!|}|||<q||fVqdS)N:rr)r" enumeratesplitstrip)rblocktyperilinekeyvaluerrr read_blockss  r5zutf-8latin1c csg}|D]C}|ds|drq|D]}z||}Wn ty&Yqw|d\}}}|}|r9|s9q|rA||q|rF|Vg}q|rP|VdSdS)N#%#) startswithdecodeUnicodeDecodeError partitionrstripappend)rZcharsetsr/r2charsethashZcommentrrr r"s2   r"ccs"|D] }|}|VqdSr)r<r?)rr2rrr r&s  r&)r6)rZloggingrZurllib.requestrZ getLoggerr Z propagateZ WHOIS_SOURCESZEXTENDED_SOURCESsetZSOURCESobjectrr5r"r&rrrr s2 8 b  ,