database - python mysql.connector fetchall() run extremely slow -
i have simple sql query using python , mysql.connector
query = 'select age, score patient' cursor.execute(query) dr = cursor.fetchall() or cursor.fetchone() print(dr)
in phpmyadmin, query fast (0.003s). runs fast in python if use fecthone(). however, becomes extremely slow if use fetchall(). take few minutes , prints result. failed , halt.
my data 440,000 lines. want plot score against age, have read data out. can me?
ps: if rid of fetchall(), fetchone() , print, runs fast. how data , plot it?
fetching all 440k records can legitimately slow.
if want draw plot, don't need 440k points; screen 2000 4000 pixels wide, 100-200 times smaller. can show parts of plot fetching relevant part of data, or can pre-compute scaled-down version (iirc mysql not have native table sampling support).
as side note: if care database performance @ all, routinely inspect query plans. plan fetching 1 record can drastically different plan fetching of them, , have different priorities if database serves many concurrent requests. affects data latency significantly.
Comments
Post a Comment