前言
前段时间想着把一个python服务的接口逐渐改成异步的，其中用到requests的地方就要改成httpx或者aiohttp，有点好奇异步请求相较于同步请求有哪些提升，遂做了点小实验。
首先有个服务A提供接口，这个接口会停顿1秒，模拟数据库操作。服务B去请求服务A的这个接口，并把响应返回给客户端C。服务B提供4个接口，这4个接口分别用requests、httpx同步、httpx异步和aiohttp去请求服务A。
客户端使用wrk做请求测试。
实现服务A
服务A使用Go编写，用标准库即可完成
package main import ( "net/http" "time" ) func main() { mux := http.NewServeMux() mux.HandleFunc("GET /a", func(w http.ResponseWriter, r *http.Request) { time.Sleep(1 * time.Second) w.Write([]byte("ok")) }) if err := http.ListenAndServe("127.0.0.1:8000", mux); err != nil { panic(err) } } 先用wrk直接请求试试，以此作为基准
$ wrk -t8 -c1000 -d30s http://127.0.0.1:8000/a Running 30s test @ http://127.0.0.1:8000/a 8 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.00s 7.62ms 1.17s 97.34% Req/Sec 194.17 266.07 1.24k 86.82% 29491 requests in 30.10s, 3.32MB read Requests/sec: 979.85 Transfer/sec: 112.91KB 服务B: FastAPI
先用FastAPI做服务B试试
from fastapi import FastAPI import httpx import aiohttp import uvicorn import requests app = FastAPI() url = "http://127.0.0.1:8000/a" @app.get("/sync1") def sync1(): try: resp = requests.get(url, timeout=2) resp.raise_for_status() except Exception as e: print(f"sync request failed, {e}") else: return resp.text @app.get("/sync2") def sync2(): try: with httpx.Client() as client: resp = client.get(url, timeout=2) resp.raise_for_status() except Exception as e: print(f"sync2 request failed, {e}") else: return resp.text @app.get("/async1") async def async1(): try: async with httpx.AsyncClient(timeout=2) as client: resp = await client.get(url) resp.raise_for_status() except Exception as e: print(f"async1 request failed, {e}") else: return resp.text @app.get("/async2") async def async2(): try: async with aiohttp.ClientSession() as session: async with session.get(url, timeout=2) as resp: resp.raise_for_status() content = await resp.text() except Exception as e: print(f"async2 request failed, {e}") else: return content if __name__ == "__main__": uvicorn.run("server1:app", host="127.0.0.1", port=8001, workers=4, access_log=False) wrk请求结果。httpx不仅同步请求性能不如requests，没想到连异步请求性能也不如requests。而aiohttp以五倍多第二名的性能冠绝群雄。
| API | Total request | QPS | timeout | comment |
|---|---|---|---|---|
| /sync1 | 4640 | 154.15 | 4480 | requests 同步请求 |
| /sync2 | 3631 | 120.87 | 3570 | httpx 同步请求 |
| /async1 | 4313 | 143.40 | 4254 | httpx 异步请求 |
| /async2 | 25379 | 843.35 | 0 | aiohttp 异步请求 |
异步比同步性能还差，着实有点费解，遂找大模型问了下，大模型回复说httpx默认配置参数不高，可以额外指定参数，还需要避免反复创建http client。似乎有点道理，但是同步性能不如开箱即用的requests，异步性能不如开箱即用的aiohttp，我为什么还要折腾httpx呢？
服务B: Flask
Flask 2.0 也支持异步接口，但是之前测试性能并不是很好，拉出来一并测试瞧瞧实力。
Flask 版本：3.1.1。因为gunicorn运行异步接口会报错，所以用的flask内置webserver。
from flask import Flask import requests import httpx import logging import aiohttp app = Flask(__name__) url = "http://127.0.0.1:8000/a" @app.get("/sync1") def sync1(): try: resp = requests.get(url, timeout=2) resp.raise_for_status() except Exception as e: print("request failed") return "request failed" else: return resp.text @app.get("/sync2") def sync2(): try: with httpx.Client() as client: resp = client.get(url, timeout=2) resp.raise_for_status() except Exception as e: print("request failed") return "request failed" else: return resp.text @app.get("/async1") async def async1(): try: async with httpx.AsyncClient(timeout=2) as client: resp = await client.get(url, timeout=2) resp.raise_for_status() except Exception as e: print("request failed") return "request failed" else: return resp.text @app.get("/async2") async def async2(): try: async with aiohttp.ClientSession() as session: async with session.get(url, timeout=2) as response: response.raise_for_status() resp = await response.text() except Exception as e: print("request failed") return "request failed" else: return resp if __name__ == "__main__": werkzeug_logger = logging.getLogger("werkzeug") werkzeug_logger.disabled = True app.run(host="127.0.0.1", port=8001) 测试结果。看来flask还是跟requests更搭，异步还不如同步。
| API | Total request | QPS | timeout | comment |
|---|---|---|---|---|
| /sync1 | 13279 | 441.27 | 248 | requests 同步请求 |
| /sync2 | 2324 | 77.26 | 2323 | httpx 同步请求 |
| /async1 | 2330 | 77.46 | 2330 | httpx 异步请求 |
| /async2 | 8277 | 275.03 | 6887 | aiohttp 异步请求 |
服务B: Sanic
再用Sanic测试一遍
from sanic import Sanic from sanic.response import text import requests import httpx import aiohttp app = Sanic(__name__) url = "http://127.0.0.1:8000/a" @app.get("/sync1") def sync1(request): try: resp = requests.get(url, timeout=2) resp.raise_for_status() except Exception as e: print(f"sync1 request failed, {e}") else: return text(resp.text) @app.get("/sync2") def sync2(request): try: with httpx.Client() as client: response = client.get(url, timeout=2) response.raise_for_status() except Exception as e: print(f"sync2 request failed, {e}") else: return text(response.text) @app.get("/async1") async def async1(request): try: async with httpx.AsyncClient() as client: response = await client.get(url, timeout=2) response.raise_for_status() except Exception as e: print(f"async1 request failed, {e}") else: return text(response.text) @app.get("/async2") async def async2(request): try: async with aiohttp.ClientSession() as session: async with session.get(url, timeout=2) as response: response.raise_for_status() content = await response.text() except Exception as e: print(f"async2 request failed, {e}") else: return text(content) if __name__ == "__main__": app.run(host="127.0.0.1", port=8001, debug=False, access_log=False, workers=4) 可能是我对Sanic了解不多，单就这个测试结果来看，Sanic根本不适合编写同步API。而且使用httpx异步请求的时候有大量报错，wrk结果显示 Non-2xx or 3xx responses: 1244
| API | Total request | QPS | timeout | comment |
|---|---|---|---|---|
| /sync1 | 37 | 1.23 | 35 | requests 同步请求 |
| /sync2 | 16 | 0.53 | 16 | httpx 同步请求 |
| /async1 | 5481 | 182.09 | 5339 | httpx 异步请求 |
| /async2 | 28116 | 934.67 | 0 | aiohttp 异步请求 |
服务B: Go
最后再用Go实现下请求
func GetA(w http.ResponseWriter, r *http.Request) { resp, err := http.Get("http://127.0.0.1:8000/a") if err != nil { log.Println(err) } defer resp.Body.Close() body, err := io.ReadAll(resp.Body) if err != nil { log.Println(err) } w.Write(body) } func main() { mux := http.NewServeMux() mux.HandleFunc("GET /b", GetA) if err := http.ListenAndServe("127.0.0.1:8001", mux); err != nil { panic(err) } } 测试结果，和直接请求服务A差别不大。
$ wrk -t8 -c1000 -d30s http://127.0.0.1:8001/b Running 30s test @ http://127.0.0.1:8001/b 8 threads and 1000 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.02s 85.69ms 1.66s 96.55% Req/Sec 210.49 175.05 740.00 63.65% 29000 requests in 30.08s, 3.26MB read Requests/sec: 963.97 Transfer/sec: 111.08KB 小结
用python编写同步请求还是老老实实用requests，异步接口应该用aiohttp，httpx的性能只能说能用。
