Are you an aspiring housing nerd or home buying? Most stats are garbage

Housing data isn’t as reliable as you might think

You don’t have to be an economist to be an aspiring housing data nerd, and you don’t even have to be in the market for a home. Many people are just intrigued by housing and its broader impact on the national economy. The problem with this data is that a lot of it is garbage, and opinions are formed on faulty information and disseminated across cable news outlets, either giving people false hope or needlessly panicking the nation. Let me explain.

Let’s say you’re fascinated by music, but you’re not in the industry, you’re just a fan. You want to know the hottest songs right now. You’ve dug around the top lists on Spotify, you’ve read all of the music bloggers and journalists’ take on new sounds, and while driving, you hear a local DJ proclaim they have the list of the nation’s hottest songs. Bingo – that is what you’re looking for. But when you go to his blog, there’s nothing fresh, nothing quantified, no science, just a hodge podge of songs he likes. That has merit, but it wasn’t what you were looking for.

This happens in housing all the time. I’m not anywhere near an economist, I’ve just been an observant housing writer for years (and secretly a data nerd, shh!). That said, here are just three examples of faulty stats:

1. ILikeHousesAndPizzaAndCats.com (a site I just made up) reports today that their data shows housing in a downward spiral, with foreclosures up 12 percent, sales down 80 percent and housing starts remaining stagnant. Scary, right? The problem is that it’s flat out wrong, but some talking heads don’t vet their information, they just see the word “report,” and opine that the sky is falling.
2. SomeNationalHousingSite.com (another site I just made up) reports that housing is improving, as home prices are up 35 percent from 2012. Unlike the first example, the information isn’t wrong, it’s just extremely limited, as most real estate search sites are reporting the home prices of houses for sale on their site. The data is not wrong, it’s just incomplete, and let’s face it, most people don’t read the methodology portion of reports to understand just how limited this data is (especially true for the very small search sites that offer data reports as economic indicators).
3. SomeUniversityThatLikesHousing.com (obviously, another made up name) reports that consumer sentiment is up 12 percent over last month, and home buyers are feeling super happy about the market and are ready to jump in. It’s not false data, but if you read the methodology, it is often a web survey placed on the University (or search portal’s) site and is based on 250 responses. The data is not wrong, but the data set is too limited to consider scientific.

How to find legitimate data

There’s no need to be cynical, just critical – not all housing indicators are bad (hence, why we report on only a handful and discard many others). How can you tell which indicators are legitimate? Let’s talk about that.

Often, the federal government puts out numbers, such as the housing starts data, and that’s reliable, because where do you think builders apply for permits? The government. Fannie Mae and Freddie Mac offer stats, and while it’s only based on borrowers’ whose homes are backed by Fannie/Freddie, that accounts for the majority of active loans, so it’s a large enough number to extrapolate from.

What you may not know is that realtor.com is widely accepted as the most complete data set in the industry, as they have the most direct feeds from MLSs (where Realtors originally enter your house for sale from their computer, before it is magically disseminated across the world wide web (a discussion for another day)). With this much reliable data, it is ironic that they were the last of the major search sites to finally hire a Chief Economist – no other site has this amount of historical raw data to dig into.

Zillow has acquired Trulia, and combining their data will put them pretty high up there with realtor.com, but they’re not quite there yet, despite having highly respected economists on staff (that personally admire and respect). They’re growing their data set though, so give it a few years and we suspect they’ll have nearly the accuracy of realtor.com (unless every broker in America revolts and refuses to let their listings appear on Z or T). Like any other search site, their information is based on their users and listings, which for now remains incomplete.

Another reliable source is the National Association of Realtors (NAR), as they not only study all of the data as a whole, but constantly research Realtor practitioner trends. You may think they have an agenda because they’re a trade group representing their members (so you think “yeah, yeah, it’s always a good time to buy or sell or whatever”), but we’ve found their data in recent years to be amazingly accurate… even their predictions and forecasts are pretty close to right on. And want in on a secret? If you read NAR data, watch for other economists to mimic their wording. That’s how you know a source is legit.

So how can you tell if data you’re getting is complete, scientific, and reliable? Read the methodology. The red flags are outlined for you in the aforementioned sites I made up as examples. Regarding national stats, anything Harvard puts out is going to be golden, CoreLogic has more data and reliable analysis than almost any other information provider in the world, and for local stats, your city or state’s Board of Realtors (or “Realtor Association”) will have the most accurate information (which your Realtor has access to).

Why this nerdy geeky dorky stuff matters

Data accuracy isn’t just for the hobbyists – if you have a home on the market or are hunting, you want to make damn sure that the price is fair. You want to know the accurate average days on market for your subdivision, not just some projected number some stranger pulled out of their you-know-what.

Listing data accuracy has been a huge tangled mess in the real estate industry, as listing syndication took off without the proper groundwork being laid (no one could have predicted this Jetsons-like future). Organizations like NAR have been devoted to untangling the mess through data standards and policies, but when any Joe Jack Bob can start a website, invite brokers post their listings there for free, call it a national search site, and tell CNN that housing is falling apart based on their data (and for whatever reason CNN runs with it), consumer sentiment is improperly swayed.

It’s such a touchy subject that realtor.com’s latest marketing campaign is entirely about data accuracy (as seen in this video below (which we enjoy and chuckle at every time)):

Any economist will tell you that what they do is scientific, and not having large amounts of raw data can lead to improper analysis. When reading a study’s Methodology, consider the source (is it Joe Jack Bob in a basement or a reputable national organization with a real economist on staff?) and consider the science (a 200 person web survey is not a scientific study that yields national data results). Read the methodology folks, because most housing indicators are garbage.