Why the hourly pipeline exists, and who actually gets it
We've mentioned in a couple of earlier posts that JFK gets a dedicated daily feed while the other 22 origins are compiled from an hourly one. Worth explaining why two separate pipelines exist at all, rather than just one.
Two pipelines, two jobs
A dedicated daily job fetches JFK alone, once a day — simple, cheap, and it's been the primary pipeline since before Sparkfare tracked any other origin. A separate hourly job fetches all 23 origins, including JFK again, genuinely more often. They're not redundant: the daily job is the one dependable, always-on source of truth for JFK; the hourly job is what gives every other origin its data.
Who actually benefits today
Everyone, since October 10, 2026. Before that, the free feed for non-JFK airports was held back 24 hours on purpose. That hold-back is off, so a visitor looking at any airport sees data from the latest hourly run, not yesterday's. There is no paid tier, so nobody gets a faster feed than anybody else.
Why we're saying so plainly
It would be easy to describe "hourly data" as real-time. It isn't. The job is scheduled hourly, but the scheduler we use often runs it every few hours, every price shows when we retrieved it, and the fare itself comes from a search cache and can be older. See why a JFK deal and an ORD deal never share a baseline for the fuller picture of how origins are actually served differently right now.