Metadata-Version: 2.1
Name: requests_tor
Version: 0.6.2
Summary: Multithreading requests via TOR with automatic TOR new identity
Home-page: https://github.com/deedy5/requests_tor
Author: deedy5
Author-email: deedy-ru@ya.ru
License: MIT
Description: [![Python >= 3.6](https://img.shields.io/badge/python->=3.6-red.svg)](https://www.python.org/downloads/) [![](https://badgen.net/github/release/deedy5/requests_tor)](https://github.com/deedy5/requests_tor/releases) [![](https://badge.fury.io/py/requests-tor.svg)](https://pypi.org/project/requests-tor)
        # requests_tor
        
        Multithreading requests via [TOR](https://www.torproject.org) with automatic TOR new identity.
        
        Wrapper of the [requests](https://docs.python-requests.org) and [stem](https://stem.torproject.org) libraries.
        Returns [requests.Response](https://docs.python-requests.org/en/latest/api/#requests.Response) object.
        
        Masking as Tor Browser by using its default headers:
        ``` "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
            "Accept-Encoding": "gzip, deflate, br",
            "Accept-Language": "en-US,en;q=0.5",
            "Upgrade-Insecure-Requests": "1",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
        ```
        
        ### Install
        
        ```
        pip install -U requests_tor
        ```
        
        ### Dependencies
        Download and start [Tor Browser](https://www.torproject.org/download/) or install [Tor](https://www.torproject.org/docs/installguide.html.en)
        
        ---
        ### Simple usage
        ```python
        from requests_tor import RequestsTor
        
        rt = RequestsTor() #for Tor Browser
        rt = RequestsTor(tor_ports=(9050,), tor_cport=9051) #for Tor
        
        url = 'https://httpbin.org/anything'
        r = rt.get(url)
        print(r.text)
        
        urls = ['https://foxnews.com', 'https://nbcnews.com', 'https://wsj.com/news/world',
                'https://abcnews.go.com', 'https://cbsnews.com',  'https://nytimes.com',
                'https://usatoday.com','https://reuters.com/world', 'http://bbc.com/news',
                'https://theguardian.com/world', 'https://cnn.com', 'https://apnews.com']
        r = rt.get_urls(urls)
        print(r[-1].text)
        ```
        
        ---
        ### Advanced usage
        [Edit torrc file](https://support.torproject.org/tbb/tbb-editing-torrc/):
        
        1. add [socks ports](https://www.torproject.org/docs/tor-manual.html.en#SocksPort),
        ```
        SocksPort 9000 IsolateDestAddr
        SocksPort 9001 IsolateDestAddr
        SocksPort 9002 IsolateDestAddr
        SocksPort 9003 IsolateDestAddr
        SocksPort 9004 IsolateDestAddr
        ```
        2. add password for control port [not necessary]:
        
        generate and add in torrc file [HashedControlPassword](https://www.torproject.org/docs/tor-manual.html.en#HashedControlPassword).
        ```
        HashedControlPassword hashed_password
        ```
        ---
        ```python
        from requests_tor import RequestsTor
        
        rt = RequestsTor(tor_ports=(9000, 9001, 9002, 9003, 9004), tor_cport=9151, password=None,
                         autochange_id=5, threads=8,)
        """
            tor_ports = specify Tor socks ports tuple (default is (9150,), as the default in Tor Browser),
            if more than one port is set, the requests will be sent sequentially through the each port;
            tor_cport = specify Tor control port (default is 9151 for Tor Browser, for Tor use 9051);
            password = specify Tor control port password (default is None);
            autochange_id = number of requests via a one Tor socks port (default=5) to change TOR identity;
            threads = specify threads to download urls list (default=8),
            """
            
        # check your ip
        rt.check_ip()
        
        # new Tor identity. Сalling this function includes time.sleep(3)
        rt.new_id()
        
        # test automatic TOR new identity
        rt.test()
        
        # rt.get(url). TOR new identity is executed after (autochange_id * len(tor_ports)) requests.
        rt.get(url, params=None, **kwargs)
        """
            url – URL for the new Request object.
            params – dictionary, list of tuples or bytes to send in the query string.
            **kwargs – optional arguments that request takes:
                data – dictionary, list of tuples, bytes, or file-like object.
                json – a JSON serializable Python object.
                headers – dictionary of HTTP Headers.
                cookies – dict or CookieJar object.
                files – dictionary of 'name': file-like-objects (or {'name': file-tuple}) for multipart encoding upload. 
                        file-tuple can be a 2-tuple ('filename', fileobj), 3-tuple ('filename', fileobj, 'content_type') 
                        or a 4-tuple ('filename', fileobj, 'content_type', custom_headers), where 'content-type' is 
                        a string defining the content type of the given file and custom_headers a dict-like object 
                        containing additional headers to add for the file.
                auth – auth tuple to enable Basic/Digest/Custom HTTP Auth.
                timeout (float or tuple) – how many seconds to wait for the server to send data before giving up, 
                                           as a float, or a (connect timeout, read timeout) tuple.
                allow_redirects (bool) – boolean. Enable/disable redirection. Defaults to True.
                verify – either a boolean, in which case it controls whether we verify the server’s TLS certificate, 
                         or a string, in which case it must be a path to a CA bundle to use. Defaults to True.
                stream – if False, the response content will be immediately downloaded.
                cert – if String, path to ssl client cert file (.pem). If Tuple, (‘cert’, ‘key’) pair.
                """
        
        url = 'https://httpbin.org/anything'
        params={
            "id": 12345,
            "status": 'passed'
            }
        headers={
            "Origin": "https://www.foxnews.com",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"
            }
        r = rt.get(url, params=params, headers=headers)
        print(r.text)  
        
        # get urls list concurrently. TOR new identity is executed depending on the number of socksports and 
        # autochange_id parameter. In case of 5 socksports and autochange_id=5, after downloading 5*5=25 urls
        # TOR identity will be changed. It does matter, because calling TOR new identity includes time.sleep(3).
        # get_urls(urls) can accept params, headers and other arguments from requests library.
        urls = (f'https://checkip.amazonaws.com' for _ in range(10))
        results = rt.get_urls(urls)
        for r in results:
            print(r.text) 
        
        
         ```
        ### Example: downloading list of urls concurrently with unique ip for each url
        Urls:  https://habr.com/ru/post/1 - https://habr.com/ru/post/50
        
        ```python
        from requests_tor import RequestsTor
        
        rt = RequestsTor(tor_ports=(9000, 9001, 9002, 9003, 9004), autochange_id=1)
        
        urls = (f'https://habr.com/ru/post/{x}' for x in range(1, 50))
        r = rt.get_urls(urls)
        print(r[-1].text)
        ```
        
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.6
Description-Content-Type: text/markdown
