We've mentioned in a couple of earlier posts that JFK gets an always-fresh daily feed while the other 12 origins run on a 24-hour-delayed one. Worth explaining why two separate pipelines exist at all, rather than just one.
A dedicated daily job fetches JFK alone, once a day — simple, cheap, and it's been the primary pipeline since before Sparkfare tracked any other origin. A separate hourly job fetches all 13 origins, including JFK again, genuinely more often. They're not redundant: the daily job is the one guaranteed, always-on source of truth for JFK; the hourly job is what makes real per-origin freshness possible for everyone else, and is being extended to serve fresher data for paying users as the tiered model comes online.
Right now, honestly: mostly nobody directly, yet. The hourly pipeline's data already flows into the 24-hour-delayed feed every non-JFK visitor sees, and into the serving layer that will eventually distinguish free from paid access — but there's no live paid tier yet, so today it mostly just makes the free 24-hour-delayed feed for other origins possible in the first place. That's a real, useful job even before any tier distinction goes live.
It would be easy to describe "hourly data" in a way that implies real-time freshness for everyone. It doesn't, today. See why a JFK deal and an ORD deal never share a baseline for the fuller picture of how origins are actually served differently right now.