Just a snippet with aiohttp.
import aiohttp
import asyncio
async def get_status(url, id):
r = await aiohttp.get(url)
print(r.status, id)
r.close()
tasks = []
for i in range(100):
tasks.append(asyncio.ensure_future(get_status('https://api.github.com/events', id=i)))
if...I was tired of writing the similar bit of code every time I wanted to craw something from the web. So, I wrote this helper to save the boring part of the work.
I call it a helper because it's definitely not a framework and it's so simple that I don't want to call it a library. It's also the firs...
So I'm using flask-mongoengine as my ORM in many projects.
The pagination comes with it is very useful and easy to use, especially the iter_pages() function.
However, there's a problem with flask-mongoengine's pagination that can cause a huge performance lag. The count() function used to calcula...