admin 管理员组

文章数量: 887007

python 并发爬虫的快感

import time
from tomorrow import threads
from requests_html import HTMLSession
session=HTMLSession()
@threads(50)  # 使用装饰器，这个函数异步执行
def download(url):return session.get(url)def main():start = time.time()urls = ['.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879','.2.0/','.html','','','.html?yyue=a21bo.50862.201879']req_list=[]for i in urls:req_list.append(download(i))print(req_list)responses = [i.html.xpath("//title/text()") for i in req_list]print(responses)end = time.time()print("Time: %f seconds" % (end - start))if __name__ == "__main__":main()

转载于:.html

本文标签： python 并发爬虫的快感

版权声明：本文标题：python 并发爬虫的快感 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.freenas.com.cn/jishu/1732355900h1534428.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。