用grequests实现并发http请求
起因 要用http请求探测服务的有效性，多进程，多线程，感觉似乎没有必要，看看有没有协程的方案 1. 简单用法 grequests 利用 requests和gevent库，做了一个简单封装，使用起来非常方便 import grequests import time import requests urls = [ 'https://docs.python.org/2.7/library/index.html', 'https://docs.python.org/2.7/library/dl.html', 'http://www.iciba.com/partial', 'http://2489843.blog.51cto.com/2479843/1407808', 'http://blog.csdn.net/woshiaotian/article/details/61027814', 'https://docs.python.org/2.7/library/unix.html', 'http://2489843.blog.51cto.com/2479843/1386820', 'http://www.bazhuayu.com/tutorial/extract_loop_url.aspx?t=0', ] def method1(): t1 = time.time() for url in urls: res = requests.get(url) #print res.status_code t2 = time.time() print 'method1', t2 - t1 def method2(): tasks = [grequests.get(u) for u in urls] t1 = time.time() res = grequests.map(tasks, size=3) # print res t2 = time.time() print 'method2', t2 - t1 def method3(): tasks = [grequests.get(u) for u in urls] t1 = time.time() res = grequests.map(tasks, size=6) # print res t2 = time.time() if __name__ == '__main__': method1() method2() method3() 运行结果如下： ...