Fork me on GitHub

起因

要用http请求探测服务的有效性，多进程，多线程，感觉似乎没有必要，看看有没有协程的方案

1. 简单用法

grequests 利用 requests和gevent库，做了一个简单封装，使用起来非常方便

import grequests
import time
import requests
urls = [
    'https://docs.python.org/2.7/library/index.html',
    'https://docs.python.org/2.7/library/dl.html',
    'http://www.iciba.com/partial',
    'http://2489843.blog.51cto.com/2479843/1407808',
    'http://blog.csdn.net/woshiaotian/article/details/61027814',
    'https://docs.python.org/2.7/library/unix.html',
    'http://2489843.blog.51cto.com/2479843/1386820',
    'http://www.bazhuayu.com/tutorial/extract_loop_url.aspx?t=0',
]

def method1():
    t1 = time.time()
    for url in urls:
        res = requests.get(url)
        #print res.status_code

    t2 = time.time()
    print 'method1', t2 - t1

def method2():
    tasks = [grequests.get(u) for u in urls]
    t1 = time.time()
    res = grequests.map(tasks, size=3)
#    print res
    t2 = time.time()
    print 'method2', t2 - t1

def method3():
    tasks = [grequests.get(u) for u in urls]
    t1 = time.time()
    res = grequests.map(tasks, size=6)
#    print res
    t2 = time.time()

if __name__ == '__main__':
    method1()
    method2()
    method3()

运行结果如下：

method1 8.51106119156
method2 5.77834510803
method3 2.55373907089

可以看到使用协程以后，整个程序的完成时间有了大大缩短，并且每个协程的并发粒度也会影响整体时间

2. 重要参数

这里需要补充的是几个
grequests

def grequests.map(requests, stream=False, size=None, exception_handler=None, gtimeout=None)
参数 说明 备注
size 协程的并发度 当一个协程进行IO等待的时候，就会将CPU交给其他协程序，一般设置为50 ~ 100足矣
exception_handler 协程的并发度 捕获单个请求的异常
gtimeout 整体请求的超时设置

另外，由于grequests底层使用的是requests，因此它支持
GET，OPTIONS， HEAD， POST， PUT， DELETE 等各种http method
所以以下的任务请求都是支持的

grequests.post(url, json={"name":"zhangsan"}) 
grequests.delete(url)

3. 事件钩子

grequests的底层库，是requests，因此它也支持事件钩子

def print_url(r, *args, **kwargs):
        print(r.url)

url = "http://www.baidu.com"
# 1.
res = requests.get(url, hooks={"response":print_url})

# 2.
tasks = []
req = grequests.get(url, callback=print_url)
tasks.append(req)
res = grequests.map(tasks)

参考资料:

  1. Requests 事件钩子

15 对 “用grequests实现并发http请求”的想法；

  2. On this platform, you can find lots of slot machines from top providers.
    Visitors can try out traditional machines as well as feature-packed games with stunning graphics and interactive gameplay.
    Whether you’re a beginner or a casino enthusiast, there’s something for everyone.
    slot casino
    Each title are available anytime and compatible with PCs and smartphones alike.
    No download is required, so you can get started without hassle.
    Site navigation is easy to use, making it simple to explore new games.
    Sign up today, and enjoy the world of online slots!

    回复

  3. On this platform, you can discover a wide selection of online slots from leading developers.
    Users can enjoy traditional machines as well as feature-packed games with high-quality visuals and interactive gameplay.
    If you’re just starting out or a seasoned gamer, there’s a game that fits your style.
    money casino
    The games are instantly accessible 24/7 and designed for PCs and mobile devices alike.
    No download is required, so you can get started without hassle.
    The interface is user-friendly, making it simple to explore new games.
    Sign up today, and dive into the excitement of spinning reels!

    回复

  4. Оформление страхового полиса перед поездкой за рубеж — это разумное решение для финансовой защиты путешественника.
    Сертификат гарантирует медицинские услуги в случае несчастного случая за границей.
    Также, полис может включать оплату на репатриацию.
    страховка осаго
    Многие страны требуют оформление полиса для въезда.
    При отсутствии полиса медицинские расходы могут быть финансово обременительными.
    Приобретение документа до поездки

    回复

  5. Questo sito consente la selezione di lavoratori per incarichi rischiosi.
    I clienti possono selezionare esperti affidabili per incarichi occasionali.
    Le persone disponibili sono valutati secondo criteri di sicurezza.
    sonsofanarchy-italia.com
    Sul sito è possibile consultare disponibilità prima di assumere.
    La qualità rimane un nostro valore fondamentale.
    Iniziate la ricerca oggi stesso per ottenere aiuto specializzato!

    回复

  6. Данный ресурс — подтверждённый онлайн-площадка Боттега Вэнета с доставлением по стране.
    У нас вы можете заказать эксклюзивные вещи Боттега Венета официально.
    Каждый заказ подтверждены сертификатами от производителя.
    парфюм bottega veneta
    Доставление осуществляется быстро в любое место России.
    Наш сайт предлагает удобную оплату и лёгкий возврат.
    Доверьтесь официальном сайте Bottega Veneta, чтобы получить безупречный сервис!

    回复

  7. 在此页面，您可以雇佣专门从事单次的高危工作的专业人士。
    我们汇集大量经验丰富的从业人员供您选择。
    无论需要何种挑战，您都可以轻松找到胜任的人选。
    chinese-hitman-assassin.com
    所有执行者均经过筛选，保障您的安全。
    任务平台注重专业性，让您的任务委托更加顺利。
    如果您需要服务详情，请与我们取得联系！

    回复

  8. Questo sito offre la selezione di operatori per compiti delicati.
    Gli utenti possono trovare operatori competenti per lavori una tantum.
    Tutti i lavoratori vengono verificati con cura.
    assumi assassino
    Sul sito è possibile visualizzare profili prima di assumere.
    La qualità continua a essere la nostra priorità.
    Contattateci oggi stesso per ottenere aiuto specializzato!

    回复

  9. La nostra piattaforma rende possibile l’assunzione di persone per lavori pericolosi.
    I clienti possono selezionare candidati qualificati per missioni singole.
    Ogni candidato vengono scelti con cura.
    sonsofanarchy-italia.com
    Utilizzando il servizio è possibile consultare disponibilità prima della selezione.
    La qualità continua a essere un nostro valore fondamentale.
    Sfogliate i profili oggi stesso per ottenere aiuto specializzato!

    回复

  10. On this platform, you can find lots of online slots from top providers.
    Visitors can enjoy classic slots as well as feature-packed games with stunning graphics and bonus rounds.
    If you’re just starting out or a seasoned gamer, there’s something for everyone.
    slot casino
    Each title are instantly accessible 24/7 and optimized for PCs and mobile devices alike.
    All games run in your browser, so you can get started without hassle.
    Platform layout is user-friendly, making it convenient to explore new games.
    Register now, and dive into the excitement of spinning reels!

    回复

  11. Looking for someone to take on a one-time risky assignment?
    This platform focuses on linking customers with freelancers who are willing to perform high-stakes jobs.
    Whether you’re dealing with emergency repairs, unsafe cleanups, or risky installations, you’ve come to the right place.
    Every available professional is pre-screened and qualified to guarantee your security.
    hire a killer
    This service provide transparent pricing, comprehensive profiles, and safe payment methods.
    No matter how difficult the scenario, our network has the skills to get it done.
    Begin your quest today and find the ideal candidate for your needs.

    回复

  12. Here you can discover practical guidance about ways of becoming a hacker.
    Content is delivered in a precise and comprehensible manner.
    It explains diverse strategies for accessing restricted areas.
    Furthermore, there are hands-on demonstrations that manifest how to implement these abilities.
    how to learn hacking
    Comprehensive info is periodically modified to remain relevant to the up-to-date progress in information security.
    Distinct concentration is focused on practical application of the developed competencies.
    Take into account that each maneuver should be applied lawfully and with moral considerations only.

    回复

  13. This site you can come across unique promo codes for the popular betting platform.
    The set of discount deals is periodically revised to guarantee that you always have access to the fresh proposals.
    By applying these discounts, you can save substantially on your wagers and improve your probability of success.
    Every discount code are precisely tested for authenticity and working condition before being published.
    https://slidesgeek.com/pages/trudnoe_obschenie_kak_nayti_obschiy_yazyk_s_chelovekom_vulkanom.html
    Furthermore, we give detailed instructions on how to activate each enticing proposal to improve your gains.
    Keep in mind that some offers may have distinct clauses or predetermined timeframes, so it’s critical to scrutinize carefully all the details before applying them.

    回复

  14. This resource can be found valuable promocodes for 1xBet.
    Such codes make it possible to acquire supplementary bonuses when placing bets on the website.
    All available discount vouchers are constantly refreshed to maintain their usability.
    Using these promocodes it is possible to enhance your opportunities on the gaming site.
    https://swastamita-bcaf.telkomuniversity.ac.id/pages/ob_opyte_bankovskoy_reformy_v_vengrii.html
    Furthermore, full explanations on how to apply discounts are given for better understanding.
    Remember that particular bonuses may have limited validity, so check them before redeeming.

    回复

  15. This online service offers various medical products for easy access.
    Users can conveniently access needed prescriptions from anywhere.
    Our range includes both common drugs and specialty items.
    The full range is acquired via licensed distributors.
    how long does cenforce 100 last
    We maintain quality and care, with secure payments and timely service.
    Whether you’re managing a chronic condition, you’ll find affordable choices here.
    Begin shopping today and get stress-free support.

    回复

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注