data analysis: flying from san francisco to new york

May 10, 2013: Updates!

CheapAir.com has performed a similar analysis and created a graph that I think looks similar.  The peaks and valleys have been smoothed out because they’ve got a lot more data to average out.  For domestic flights, the average cheapest flight is 49 days prior to departure.  This is earlier than my graph, but I didn’t start my analysis nearly as early as they did.  Clicking the graph links to their informative post.

I also found a study by ARC (Airlines Reporting Corporation) via marketplace.org that created a very similar looking graph to that of CheapAir.com’s, but from a different set of data.

—

Data Analysis: Flying from San Francisco to New York – when is the cheapest time to buy tickets?

Kayak.com has this nice feature where you can subscribe to price alerts for certain itineraries.  This is helpful as fares change fairly frequently and it’s hard to know when to purchase tickets.  Microsoft purchased a company called farecast.com back in 2008, which originally grew by using data to predict when prices would rise, fall, or hold steady.  Microsoft has since integrated into bing travel.  They claim about a 75% accuracy.

I visited New York a few weeks ago, and when searching for a ticket, I decided that I didn’t really trust bing travel’s technology.  I decided that I’d monitor fares on my own using Kayak’s emailed price alerts, and then make a purchase when prices seemed to be reasonable.  I identified travel dates for a round trip where I’d depart on June 17th at any time of the day and return June 21st, at any time of the day.   San Francisco International Airport (SFO) and Oakland International Airport (OAK) are both just about as easy to get to for me.  It also didn’t matter whether I arrived at John F. Kennedy International (JFK) or LaGuardia (LGA) in New York.

I took the prices from all of the emails, put them together in a data set, and plotted them.  One of the big assumptions here is that the travel dates are fixed – if you’re able to fly on different days, you’ll of course most likely be able to find cheaper tickets.

but first, key findings:

* Prices go up at the last minute.  In this case, they almost doubled.
* In the 6-week monitoring period, the cheapest flights were found about 3 weeks prior to departure
* There seems to be some truth to prices being lower mid-week
* There doesn’t seem to be a big price difference for OAK vs SFO or JFK vs LGA
* When one airline dropped fares, others seemed to follow

onto the graphs:

when should I buy tickets?

One interesting finding is that buying early (I am speaking relatively here as I didn’t start my search until about 6 weeks before departure) isn’t always the cheapest.  In this case, the cheapest fares were found about 3 weeks prior to departure.  Tickets may have, of course, been cheaper prior to 6 weeks before departure.

An ABC News article states that “Airfare sales tend to occur early in the week … And increases tend to occur at the end of the week.”  My data set isn’t very large, but here’s a histogram of prices, grouped by day of the week:

What does the histogram show?  For my set of data, the cheapest prices occurred on Wednesday and Thursday.  You can see the little bumps of lower fares on the left side of the graph for Wednesday and Thursday.  I’m not sure if much can be made of the rest of it – there aren’t too many data points to draw any strong conclusions.

Prices were probably also the highest Tuesday-Thursday because those were the last 3 days before the flight and as can be expected, last-minute tickets were much more expensive.

where should I fly from/to?

I had two theories about the relationship between airfare and the size of the airport.  I was thinking that flights might be cheaper out of SFO since it’s a much more popular airport (Based on what I could find here and here, they handled about 45 million passengers in 2010 compared to about 9.5 million for OAK).  Conversely, I also thought that flights may be cheaper out of OAK since I know that a strategy of low-cost carriers like Southwest, JetBlue, and AirTran is to use secondary airports in larger markets (think Midway for Chicago, BWI for DC, Providence for Boston, and Love Field for Dallas) to keep costs down and thus offer lower fares.

There doesn’t look to be a big price difference, on average.  There’s a piddly $4 to $6 difference between flying out of SFO vs OAK and landing in JFK vs LGA.  Maybe the two theories are both correct.  Or incorrect.  Also, the Kayak data doesn’t include Southwest, since Southwest doesn’t make its data available to third parties.

why did prices drop?

The lowest price I encountered was on May 25th, when United/Continental dropped their prices for a nonstop flight from SFO to JFK to $319 from $549 a day earlier.  American and Delta also lowered their prices for nonstop flights that day to $439 and $359.  Some of the airlines also lowered their prices from SFO to LGA (note: no nonstop flights).  This may have been because the price of connecting flights was reduced and the SFO to LGA and SFO to JFK trips share similar legs.  Interestingly, flights out of OAK didn’t change by much when prices of flights from SFO dropped by over $200.  These prices didn’t last long – the $319 fare was available for only two days.  $319 seems like a pretty good deal.  I don’t have historical flight price data, but from what I can recall, this appears to be near the bottom of the fare range.

There was another temporary price drop from SFO to LGA offered by Delta on May 31st to $341.  Prices were back up by the next day.

parting words

If you’ve made it this far, thanks for reading.  I’ve been wanting to get more into data analysis on topics that we all can relate to and this is part of my foray into the field.  There’s a lot more to learn and study out there, so if you have any suggestions of things I should look into regarding airfares or anything else, let me know.

When my schedule freed up, I ended up changing my travel dates in order to find flight times that worked better for me and found two nonstop flights from SFO to JFK on Virgin America.