Did Google Scrape My Website? You Be The Judge. Rules Threaten Realtors & IDX Providers.

cease and desist

[Update] “NAR has supported this ruling! That is the scary thing. My local board threw it on NAR for intrepretation and they indeed said Google is a scraper site and no we must “protect the data” from scaper sites. I think the fear is if it starts with our board, it could easily be implemented as a nationwide policy like Benn said.”  See Mike Taylors comment below.

Warning – Long, but worth it if you want to have or keep a dynamic IDX solution for your real estate website or blog.

The Back Story

Recently, my website was reported for allowing the MLS data on my website to be indexed, which led the Indianapolis Metropolitan Board of REALTORS®, (MIBOR), to issue a cease and desist letter to my broker for both our sites, or they would cut our feed.

Since April 10th, Google can not index the detail pages of the IDX on our websites. Our Title tags can not contain any details about the property (Example #1). The decision affected about 15 other brokers and agents in Indianapolis. While I firmly believe I have not violated any MLS rules, we complied! It’s hard to fight an 800-pound gorilla

hometoindy_mls_listings1



Advertise at AG

On April 30th, another demand! The description can not contain ANY MLS data, no address, the MLS number, neighborhood, subdivision, price, bedrooms, baths, property type or public remarks. (Example listing 2)

But Realtor.com can:

results_for_carmel_listing1

The New Rule: Google is a Scraper

MIBOR classifies Google as a scraper site and therefore, I am allowing Google to scrape or reproduce the MLS data from my site.  This interpretation is supported by the National Association of REALTORS®, based on the MIBOR’s listing service rules and regulations.

Section 15.2.2  – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database

The entangled web of rules we abide by to be a participating member of our local board and MLS are subject to the interpretation of the local authorities and agents who police the web for offenders. They say scraping and indexing is the exact same thing.

I guess then we would have to define scraping vs. indexing. While there are detractors who say Google or any search engine are indeed scraper sites, the generally accepted charachteristics of a scraper site should be explored:

  • Scraper sites copy and blend information to make it look less like plagiarism
  • Scraper sites directly copy another’s site without attribution or copy for the purpose of generating revenue through paid advertising.
  • Scraper sites use  automated means to harvest data.
  • Scraper sites steal content without adding utility.
  • The biggest difference is people go to Google looking for information – and invite Google  to visit their site.

You can’t have it both ways; call Google a scraper, then invite them to crawl your site.

A Step Back in Time

Start your newspaper ad with your clients home featured  in black and white and gather up your open house signs, we’re heading backwards. Let’s hide the information the client wants, put our MLS listings in a book and wait a week for them to be published. Let’s force the hand of the client to only have access to the agent who listed the home.

Why stop there – we should just go back to the days of sub-agency, when everyone represented the seller.

Talk about going backwards:

Look what happened to the information for 2 separate listings before and after April 10th. This is the basic search for the MLS number and city MIBOR used to issue their last demand.

Home listed before April 10th and still indexed

Home Listed after April 10th

I’ll bet the second home owners are really happy with their online marketing!

That one little listing with a Diverse Solutions RSS feed will soon be gone too, if the interpretation of the rules is enforced. Until then, at least this home has some marketing exposure.

I have been told this will be a subject discussed at the mid-year NAR meeting next week. Can’t confirm it, though!

My Client Doesn’t Care

It doesn’t bother me if my clients listing ends up on other agents websites or indexed by another REALTOR® and I’m willing to bet my clients don’t care either.  I’m thrilled when there are multiple returns for my clients address. I know many of the entries will be by my efforts and more will be from the collaborative effort of REALTORS® who understand the dynamics of the Internet Data Exchange and those who have harnessed the power of online marketing.

So far, the rules have not impacted my website. I’ve actuallly moved from #13 to #6 since April 10th for Indianapolis Real Estate and traffic to my site has been unaffected. This is bigger than my one site; it affects REALTORS®, our CLIENTS, IDX providers, and the future of our ability to market for our clients and our client’s access to data.

Who Will Control the MLS Data?

My predictions: Control of indexed IDX will be held by our local boards through their public facing site, along with third party aggregators who will gladly take all the listings we give them, and charge us to submit them or have a premium account. They will continue to charge us for leads generated by our listings.

Oh and let’s not forget REALTOR.com, who can use the data because they have a special deal with our board. We know what they charge to have an enhanced listing package. [Insert Russell’s Graphic]

I’m certain there are many unsuspecting REALTORS® and IDX Companies who have no idea their marketing and websites are about to be axed. No longer will we be able to have RSS feeds, dynamic IDX listings or indexed MLS data.

I know others in the industry are working on new ways to deliver the data through advanced feeds and integrated IDX. Unless REALTORS® are allowed to provide this information to the public – why should someone else be allowed to?

It’s a new world of change – where nothing really changes – two steps forward, three steps back. March on…………..

Note from AG staff: If you would like to voice your opinion regarding this ruling to NAR, please email Cliff Niersbach. You can also email Steve Sullivan, MIBOR CEO or call him directly (317) 956.5000 ext. 237. Please keep your words professional if you choose to take action and be sure to reference this article.

Paula is team leader for The "Home to Indy" Team in Indianapolis . She is passionate about education and client care and believes an empowered client is better prepared to make good decisions for themselves. You'll find her online at Agent Genius,Twitter and sharing her insights about her local real estate market at Home To Indy.


445 Comments

  1. This is a case of the policy lagging behind technology it seems. Clearly, this needs to be addressed if we are to continue to serve the consumer. The days of white knuckling information are long gone and as you stated, seeing our listing showing up all over the web sounds good to me. It’s still my listing and part of my job is to make sure we get exposure.

    The double standard with Realtor.com only adds to the ongoing list of frustrations I have with them (love Russell’s image BTW).

  2. Wow, I think that the IDX concept is an excellent example of the new transparency in the business. I know that my broker has a signed IDX agreement with MRIS to make/allow the listing information to go public.

    Next week, I will be at the mid-year NAR conference in DC and will sit in to hear this juicy discussion.
    -doug

  3. It’s pathetic and yet, fascinating to watch a dinosaur stuck in a tar pit struggle for life.

  4. I confess that I hadn’t paid any attention to the NAR-DOJ settlement, and I was warned that MLS rules were changing. Even so, I was shocked to find my gallery of sold listings suddenly get stripped of all photos but the main one and all descriptions I’d added to the mls with property details eliminated. My web provider explained that I could not have many details on my site and no properties more than 3 years old displayed in my gallery of solds.

    The fallout of that decision is still being discovered by all of us. I have to wonder how all of this is supposed to benefit the public and make the playing field more level.

    So far, I’m not seeing it. And you raise a lot of good points in this post that I hadn’t even considered.

    • I am not an agent but might as well be from the knowledge I have acquired over the last 5 years. I just can’t believe how close-minded some NOT all agents could be to think technology would not get a hand in the control of the NAR MLS monopoly. There are good agents and there are bad agents but during our times 2010 people are smarter and the internet can educate buyers to a point of using Zillow.com and Redfin. Those two should sharks will be eating agents commissions left and right.

  5. Someone needs to wake up, smell the coffee and get a clue what century we’re living in…

    • Hey Jay,

      That seems to be the problem. The root of the question is who owns the listing information and is that information part of the public domain. Your clients would probably argue that they own the information that describes their home and want Google to display the listing…it’s a sticky situation.

  6. This seems like the doing of some juicy paw getting a cut for their sole indexing – I always wonder who’s behind all this? Could it be the big dogs that don’t want individual Realtors to get the juice? could it be IDX companies that don’t want to share the wealth? Could it be third parties that sell leads to Realtors and would lose $$$$? Or is it a matter of “lethargic technology” – I choose not to be naive.

  7. I cant believe in this world of instant results based on technology that an antiquated idea like this would even have enough merit to warrant a discussion. Quite possibly the dumbest thing I have heard….ranks up there with a solar powered flashlight.

  8. How anyone who has ever searched on Google (everyone) could confuse indexing with scraping is just beyond ridiculous. It is getting tougher for us to serve the consumer and market their home/product as widely as possible. Free market economy anyone?

  9. The 70ks called. They want their MLS rules back…

  10. These policies clearly needed to be changed. And can someone please explain to me how REALTOR.com is able to post MLS data without violating anything?

  11. Sad. Another example of folks leading when they have no clue who their leading. NAR is supposed to support our best interests. MLS boards are supposedly run by Realtors. How do we keep getting shot in the foot by those who are supposed to represent us?

  12. Perhaps some fail to realize that the horse left the barn long ago….

  13. Do any of these people realize that a majority of buyers find their property via IDX or their agent searching the MLS? Stats don’t lie and there’s reason more and more agents are ditching the books and print ads.

    BTW< when is the meeting at Mid-Years about this. This sounds like something many of us need to witness.

  14. Sad and pathetic.

    Let’s hope that this gets reversed and NAR comes out in support of their dues-paying members – the Realtors.

    Is this on any part of the agenda for Mid Year? I’ll be there and will gladly speak up. @tcar?

  15. Welcome to my world in Lynchburg, VA. We have to put our listings in the Roanoke, VA MLS to get them onto REMAX.com because the Lynchburg Association of REaltors will not allow to go on national sites except for Realtor.com of course. The LAR MLS listings do go on individual IDX sites but the rules are very confusing. We may be coming into the 21st Century this summer we’ll see. Yes the clients want their properties everywhere you can get them on the internet

  16. Utter crap. They couldn’t stop media companies, so they’ll lean on their own membership?

    Paula, I know how detailed this situation is, so I have to give you kudos for boiling it all down in simple terms. I gave you a 5 star rating, thanks for sharing this information with all of us, I know you didn’t go public lightly.

    Benn

  17. by the way, clicking those two Google before and after results is staggering, what an awesome illustration of the devastating impact controlling the data can have on the visibility of our listings.

  18. Forgot to subscribe to comments. This is a conversation I don’t want to miss.

  19. WTF? This is absolutely ludicrous. Am I paying dues to be ushered back in time? NAR, if you see this you need to know that your REALTORS think you have lost your ever-loving mind.

  20. Wow, it is amazing to me that Realtors would put up obstacles to advertising listings. Our SELLERS want the widest possible exposure to their listings, and what better way than on Google! Boy, am I glad that our local MLS is owned by brokers, not the local Realtor board!

  21. But Kevin, isn’t that the crux here, if this rule is carried, then it could pave way for a landslide backwards across the board- what works with one board, may work with all boards, right? Is this ultimately in the best interest of even broker owned mls?

  22. It begs the question of why would the local board even look at this issue. Was it brought to their attention by an agent, was a committe put in place that wants to excercise some power, is the DOJ agreement part of this…? I’m curious at the motivation behind the ruling.

    Also, if the NAR regulations support this ruling, is a mandate to all local boards far behind?

  23. Hi all. I am the broker in the middle of this and wanted to say thanks to Paula for such a great write up and explanation of the events and thanks for all the support.

    @Bob – Yes a local agent did raise the issue. My guess is he was mad other sites(Paula’s) were showing up for his own listings and he chose to file a complaint to serve his best interest not his clients.

    NAR has supported this ruling! That is the scary thing. My local board threw it on NAR for intrepretation and they indeed said Google is a scraper site and no we must “protect the data” from scaper sites. I think the fear is if it starts with our board, it could easily be implemented as a nationwide policy like Benn said.

    @Jim – This is supposed to be addressed next week and we would LOVE for you to speak up.

  24. Linsey – It appears the only people who don’t mind are those who don’t use technology or understand the ramifications of the policy.When we were first notified, I contacted other brokers/agents to garner support. No one wanted to fight – one told me they would watch from the sidelines.

    My seller does have the right to have their listing promoted online! This is my biggest complaint.

    Russel’s graphic is a classic – I laugh everytime I see it.

  25. Mike –

    When/where/which committee/who’s the chair?

    NAR Mid-Year is a busy time and a chance to meet those who can directly impact our business. if we don’t speak up there, we’re mostly silent.

    It’s all good and well to talk about it here, but it’s there that we really have to express ourselves.

  26. Sorry, but my comment comes late due to uncontrollable laughter. Been watching local & state boards, and NAR step on their own whatsits since Dad explained to me which way was north on the ‘organized’ real estate industry map.

    Every time I’ve made my thinking public, Realtor supporters have come after me big time. Most have been civil, but their logic has generally been so tortured, it would be outlawed in Gitmo.

    I wonder if this might be the last straw for my friend, Benn? Working within NAR to improve things has never produced anything but millions of man hours of wasted time and effort. What most agents fail to see is the obvious. NAR has no excuse whatsoever for being so utterly clueless OR if you really wanna be cynical, doing pretty much all that they do with full knowledge — the maintenance of their power the only goal.

    For 40 years now I’ve witnessed them eat their young, then complain because they weren’t cooked just right. Either rise up, destroy them and start over, or realize you’re just another pebble in the road in need of crushing.

    NAR couldn’t organize a one man picnic if they had two consultants, their own park, free KFC, and a week’s lead time. Until the majority of their membership sees this as reality, nothing will change. In fact, Dad told me that back in 1969, and nothing has changed.

    There, I feel much better.

  27. @bawldguy :) 3 hours later the NAR is still deciding whether the answer is yes or yes. Either way, yes is still a defining answer.

  28. Jim – Can’t say I know for sure, but this committee has been copied on some of the emails about this topic: http://www.realtor.org/educsess.nsf/allpages/Mgov09AWIR-5UPUAQ

  29. Doug – Yes it is until it is taken away. We are trying to find out IF this is a confirmed meeting at the conference. We were told by MIBOR that it is.

    John – :) It certainly is!

    Mary – I wasn’t aware of the rules which are affecting you. I guess if they quietly change the rules, a few at a time, they assume it will not be noticed. Somehow, I don’t think it is meant to benefit the public.

    Jay – Yep – a triple shot of espresso!

    Ines – Definitely big paws – everyone has their hand out and their pockets lined with the fruit of our labor. You are wise to not be naive. I personally believe it is a power issue!

  30. Forgot to check subscribe box. Maybe I shouldn’t be organizing one man picnics either. :)

  31. Benn — Welcome to the dark side.

  32. I wonder if we could just start printing books with our listings in it. You know, with addresses, a nice black and white photo and the address of the listing agent so we can pick up a key prior to showing the home. Heck, a weekly book per area would probably suffice.

  33. Clint – You mean there is no use for a solar powered flashlight:) Someone should tell the good ol boys at MIBOR and NAR.

    Elaine – I’m all for free market economy and bet my clients are also.

    Todd – Paused to laugh out loud :)

    D Patrick – We have been asking that for years – the answer is, they have a deal with our board.

    Matt – Oh yes, we have leaders and they have decided what is best for them, not the Realtor or the public.

  34. Interesting, I did a search a bunch of active listings, by address, here in Long Beach and found that only a few sites were still being indexed “by address”:


    Trulia

    Yahoo
    Redfin
    Ziprealty
    Movoto

    I have to wonder how Google disseminates the difference when it deindexes? If no listings are allowed to be advertised by address, per MLS rules, why are these still indexed? Are they picking and choosing WHO can be indexed? Is money changing hands to make this happen?

    No matter what they do, there will always be a way to get your listings to the top of the engines and get exposure for your clients. It will just require a bit of creativity.

    While I agree that this is horrible, its not the end of the world and may actually have a silver lining – of sorts.

    It should eliminate many of the aggregator sites paving the way for you to market your listings with less competition on the engines. This just means that you cannot market them with the “address” in the title according to what you wrote. The listings can still be marketed on the internet.

    Just looking for a solution.

  35. Update:

    from @tcar to @toddwaller via twitter “I’m walking into a three hour, unrelated meeting, but someone will respond. We want to make sure we have a clear response.”

    [PUNT]

  36. On another note, I have wondered how long Google itself would tolerate the flood of duplicate content in its results. With so many flooding the internet with IDX solutions, and more coming daily, and the information being pretty much the same, I figured that Google itself would begin deindexing at some point.

  37. Jim Rake – Not in Indiana – our horses are safely locked in their stalls :)

    Matt Wilkins – Oh, I think they DO realize it and want to control the data. Like I said, I can see an indexed IDX solution in the future of our Board’s public site.

    Jim – Sad indeed! I know at NAR is where the change will be made and Realtors will be heard. We were told by our board this is going to be a discussion at NAR next week. We have not been able to confirm, but will post here if/when we are.

    Kathy – It’s hard to keep up when the rules become the flavor of the month.

  38. Laurie,

    I’ve also been curious when Google would de-index RE listings. Putting the almighty Google aside for a moment, don’t our selling clients deserve as much online marketing exposure as possible?

    As long as the content coming off those aggregators is the content I fed into the MLS, I want as many sites as possible (yes, even my competitors’ IDX fed site) showing my listing and being indexed. With over 80% of home buyers using the ‘net at some point in their buying process, I want as many potential buyer eyeballs on my clients’ homes.

    And yes, you are bang on correct that there are other ways to market properties online. A big concern I have is consumer frustration:

    Sellers: “You mean we can’t just search my address to see your marketing effort?”

    Me: “No, you need to search the term ‘Gorgeous West Side Colonial’.”

    Sellers: “But our home is a contemporary ranch on the NorthEast side….”

    Me: “Well, that term is in the MLS description, and NAR says I can’t use that term because Google is a ‘Scraper’.”

    Sellers: “Huh?”

    Me: [long pause, and a reflection on what the term ‘common sense’ means]

    I don’t look forward to those conversations….

  39. Todd,

    So true……..

    What NAR/Google has done is reprehensible. My mind just always looks for the immediate solution to solving the problem.

    The level of ignorance at NAR is astounding. The people who make these decisions have no business being in the positions they are in and not having educated themselves on what is going on in the here and now. In the meantime, we still have to run our businesses and represent our clients.

    NAR working against us just make it that much more of a challenge. I’ve always been good at puzzles.

    Laurie

  40. So the NAR supports this ridiculous ruling? And now they “want to make sure they have a clear response”.

    While they are working on that, they may want to go ahead and revise the Mission & Vision statement of the NAR:

    Mission & Vision
    The core purpose of the NATIONAL ASSOCIATION OF REALTORS® is to help its members become more profitable and successful.

    Yeah, whatever.

  41. Laurie & Jay,

    yeah, it appears as though the association’s leadership is taking cues from congress on how to deal with “the constituents.”

    Laurie – I hear you on looking to solve this puzzle… a real game changer, actually…

  42. Question … my broker uploads OUR listings only to other search engines, like Trulia, Propsmart, Google Base, etc. The broker owns the listings, not the MLS. It would seem that this could still be done since the feed isn’t coming from the local MLS nor from an IDX and only includes listings owned by my broker.

    It seems that this issue is with IDX systems – which includes ALL broker’s listings – allowing “others” to use the info without having the appropriate approval of the reciprocating brokers or the required listing agent name/broker/phone number on the listing. Or am I not understanding the issue?

  43. Benn — Punt indeed…good grief.

    Jay — I dont think I could agree with you more. The fact that NAR supported this decision flies in the face of just about everything they “say” they support with regards to its membership.

    This whole thing is retarded. There, I said it.

  44. It would seem to me that your best bet in fighting this is going to be on the local level first. Since you feel you haven’t violated any of the rules, you should fight it there, with your local MLS. Waiting for the behemoth that is NAR to move on this issue could take forever. Not to mention, your local MLS could always ignore NAR’s interpretation. NAR doesn’t make the rules for your local MLS, only suggestions.

    I agree that what your MLS is doing is ridiculous. My MLS has the exact same rule, but as of yet, it hasn’t interpreted in the way your MLS has interpreted it. The only way to change it is to actually go through the process of changing it.

    I say, fight it. Fight it hard. Get those 15 other brokers in your MLS to fight it, too. Appeal it, go through the channels, force the person who filed the violation to stand up and defend their point of view. Put it all out in the open, and make people think about it. Tell your clients about it, tell them how it hurts their marketing/advertising exposure. Have them contact your MLS. Do whatever it takes.

    All of us here are going to agree that it is ridiculous, but we’re not the ones who need to be convinced of that, nor are most of the readers here. The folks that need to see how detrimental an interpretation like this is to our profession are sitting on the boards of our local associations.

  45. Mike –

    Thanks so much for coming on and commenting. That NAR supports this shows that they clearly lack understanding of this “internet” of which we hear so much.

    Re: @tcar – meh. I expected as much. Give him a few more hours. At least he came on and said that they were working on it. My greater concern is that the NAR thought this was a good idea in the first place.

    Stupid stuff like this that has the potential to so greatly impact our collective businesses make it harder and harder not to agree 100% with @bawldguy. I mean, seriously.

    What I want, as a member of the NAR is this – the minutes of the meeting where they supported this change. @tcar, will you please guide me in the right direction to make that request, as well as confirm that this will be on the agenda for the meeting? Thanks.

  46. These MLS boards are idiots. I have been in a dispute with my local board over posting information on my site [I won’t get into it, but it actually is information the consumer should know]. After 15+ email exchanges, I have finally given up. They know nothing and after skip tracing this guys history, his linkedin profile shows previous work experience in the medical field and has only been with the MLS board for a couple of years.

    Like Laurie says, just have to stay two steps ahead.

  47. Elaine wrote: “It seems that this issue is with IDX systems – which includes ALL broker’s listings – allowing “others” to use the info without having the appropriate approval of the reciprocating brokers…”

    At least in Arizona (and I suspect everywhere else) when a broker gives approval to have their listings in the IDX feed, they are also giving their approval for those listings to be displayed on other web sites. That is pretty much the point of IDX — to get listings displayed on other sites.

    You can opt out of having listings in the IDX. Why you’d want to, is beyond me. But some still do it.

    The agent that complained about his listing appearing on Paula’s site needs to get a grasp of what it means when his broker opts in to the IDX.

  48. RETARDED!!! That is the word I was looking for… that about sums it up!

  49. Thanks for the post Paula. Although I’m dumbfounded, I can’t say that I’m surprised.

    ::Wonders if the new, retro printed version of the listings book will be available on the Kindle.::

  50. This just proves how F’d up the real estate industry is! Boards and big brokers would love to have us back in the days when they “held the only key” to property information. Forcing the public to go through them. They need to wake the F up and smell the transparency! It’s 2009 right???

    And R.com – don’t get me started on that – I pay sick amounts of money to enhance… but they have us by the balls!

  51. I think a few things need to be cleared up here. First of all, you can advertise your own listings however you see fit and allow Google to index them or have them scraped all day long. The issue is what you do with other board participant’s listings on your site that you get through your IDX.

    The problem is this ruling gives/ will give an unbelievable boost to lead aggregators like Trulia, Zillow, and our good friend Realtor.com. R.com aside, the lead aggregators are not members of NAR or the local boards and are not subject to the rules they are imposing on its members. So while NAR chooses to cuts its members legs out from underneath them with their archaic IDX blue laws, the national lead aggregators are the real beneficiaries here.

    @Jay – I think you are wrong here: “when a broker gives approval to have their listings in the IDX feed, they are also giving their approval for those listings to be displayed on other web sites. That is pretty much the point of IDX — to get listings displayed on other sites.” Or at least the semantics of this needs to be cleared up. The IDX approval gives permission to be displayed on other’s sites, but NAR is saying you must not let Google or any other search engine get a hold of that data. So I guess it means you can display the data, but not have it be found so as to be useful to anyone.

  52. @Mike – Sorry about the semantics. It’s tricky. And I can only speak for Arizona and our MLS. But here, when a broker opts in to the IDX (to be precise I should say that a broker has to opt OUT of the IDX, and by default, all listings are in the IDX feed) they are giving permission for their listings to be displayed on other broker/agent sites. I don’t see anything in our IDX agreement (YET!) that says these listing must be unindexed and “hidden” from search engines.

    “Yet” being the key (and scary) term. My IDX solution is framed, so it isn’t indexed anyway, but the simple fact that a board, with the support of the NAR, can make such a ridiculous ruling when it’s painfully obvious they are clueless as to how search engines (and consumers) work is frightening.

    What’s next?

  53. Once again, NAR has made a decision that craps on their dues-paying members, their clients and consumers in general. NAR continues to think about their own (aka the big brokers who control NAR) self-interets at the expense of everyone else.

    Todd said it well with his comment regarding the conversation he’ll now have with his sellers. NAR should ammend its “Right Tools, Right Now” materials to include the scripted response Realtors should give their seller clients when that conversation comes up. Oh wait…that’s right..there is no good respone except, “The organization I and other Realtors belong to doesn’t care about your best interests Mr Seller”.

  54. Jay- Are you sure about this “I don’t see anything in our IDX agreement (YET!) that says these listing must be unindexed and “hidden” from search engines. ” Our IDX agreement comes directly from NAR’s policy and I would be shocked if AZ was different. Look at section 15.2.2 that Paula mentioned in the post.

  55. This ruling in my opinion is highly problematic. It not only ties ones hands in being able to compete with the agregators sites but also opens the doors to more action by the DOJ. When it comes to the best interests of Realtors we want our fellow agents to be indexed rather than the big boys. This is true for several reasons first of all the whole point of sites like Zillow or Trulia is to shave off a protion of the commission dollars if not replace the broker all together. That in no way can be considered a positive for agents. Next the more times that information is found by a modern search engine the less value it has towards building a sites authority. Those large agregators cannot compete with our ability to add content to our sites that shows us as the authority for our local markets, but if they are the only ones with listings indexed they can trump us with access to unique content. This being the case because if those sites are the only ones with the listing data protrayed in a crawlable manner it becomes unique in the eyes of the search engines. I am really dissappionted by the lack of vision by those who are suppose to be our leaders. It is truly sad!

  56. Jay – dont you use an rss feed for listings also? say goodbye to that if you do!

  57. It’s really hard for me to receive updates on the comments here if you delete the comment I used to subscribe to the comments. Can you leave this one please?

  58. Are you kidding me! This is ridiculous. Holy Crap! I need to check with our vendor and find out if/when this might hit our site. I don’t even know what to say at this point.

  59. Jay and Mike – you are both saying the same thing :) YET, being key!

  60. Gnaws off the hands that feeds, then pukes them back up…all in the vain of trying to stay relevant.

  61. Jay, the main point I meant to make is that the listing broker/agent/phone number must be included – at least that’s how our Board works. In the pre-Apr 10 example above, only the broker’s name showed up. There’s no phone number or agent name. I think the IDX is responsible for making sure they adhere to their Board agreements.

    By requiring the listing agent/broker/phone, it prevents some “questionable” people from presenting the listing as their own – which we know would happen. Consider what is happening now on Craigslist with people “renting” vacant listings.

    On a similar experience, I found an old (3 yrs ago) listing of mine on a scraper site. It so happens that this home is for sale again with a different agent. That puts me in an uncomfortable position since it appears that I’m “advertising” a listing that’s not mine. Since there’s no enforcement for these scraper sites to update, it can become a problem.

  62. Mike –

    Yes, there is a very similar “anti-scraping” section in our IDX policy:

    “IDX Brokers must protect the IDX Database from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction or use of the IDX Database.”

    But the argument that Google is a “Scraper” is ludicrous (that of course, won’t prevent a board or the NAR from proclaiming that they are). Google is not accessing, reproducing or using the IDX database.

    And yes, I use an RSS feed to display a small number of listings — all of which meet all the required terms of the IDX agreement.

    And yes, I’m very concerned that a clueless board or national association will slam down the gavel and make that practice “illegal”.

    Sadly, it’s the members of the boards and NAR (and our clients) that are suffering from this sort of reckless rule making. How something like this protects the consumer and helps us, the dues paying members, “become more profitable and successful” escapes me. It is simply beyond my comprehension.

    And don’t even get me started on Realtor.com, the “Official site of the NAR”. I’ve yet to hear a reasonable explanation of why they can violate rules that members must adhere to.

    Prime example. Various boards across the country prevent agents (with NARs backing) from using the term “MLS” and variations on their websites. Yet at this very moment, Realtor.com has “mls, mls listings, multiple listing service” in their meta keywords.

    The only thing I’ve ever heard from NAR on this is “we don’t have any control over realtor.com” (I’m paraphrasing).

    Really. I find it … stupid beyond belief … that the NAR has no control over the “Official site of the NAR”.

  63. I wonder why the boards think it’s a great business model to compete with the members who make it exist. I really believe there needs to be an alternative board created.

  64. Ok folks – I was going to keep quiet about this (after discussing with many of my clients including Mike Taylor – the broker who was involved in the initial issue and Paula’s Broker) mainly because we didn’t want to raise awareness of this issue if it could in fact be resolved (why plant the idea in other MLS boards pea brains right?) – would not bode well for those of us who make a living (agents and vendors) from having listings successfully syndicated on the web.

    But now that the cat is out of the bag with Paula’s post – and after speaking with Mike about it, I suppose the best thing we can do now is raise awareness in an attempt to get NAR to overturn this ridiculous decision and realize that search engines are NOT the scraper sites that certain clauses of the IDX policy were created to protect.

    BTW For those of you who don’t know me, I am the CEO and director of technology at http://www.realestatewebmasters.com and provide the IDX’s in question (to both Paula and Mike) that have been the target of this witch hunt.

    I am also the person that was handling the communication between NAR, REW, MIBOR and Mike (on the REW side anyways, Mike did a lot of his own lobbying as well)

    To get started and to provide some behind the scenes, I have decided to document the communication that went on between NAR and REW / MIBOR /REW etc

    Here is my first installment – you can expect much more to follow.

    Ironically (and I will post more on this later) – the broker that filed the original complaint against Mike (in a move of absolute hypocrisy) was in the midst of negotiating a search engine spiderable IDX of his own. What was he thinking!

  65. hey @respres my apologies, it appeared you were trying not to interrupt the flow, wasn’t aware it killed your subscription- I’m working on a post comment email subscriber and will have it up shortly- thank you for your patience. In the meantime, there is a comment RSS feed if that would better suit you.

    You have my email, phone number, or can reach me via twitter if there is anything else I can do.

    Sincerely,

  66. No wonder you’re angry! I can’t believe that policy is so antiquated and naive.

  67. Benn – Thank you for your support and the opportunity to post here on AG. I did put a lot of thought into this and when the second sanction was laid down, I felt it was time to give a voice to my fellow REALTORS(r) and the unsuspecting who may not know what is ahead for them.

    The only way to stop such ludicrous interpretation is to bring to light what happens behind closed doors, as evidence by the two listing comparisons.

    My one little site is a miniscule piece of the whole! The implication this represents for our clients, agents and IDX providers is scary.

  68. Elaine –

    I don’t know what the rules for Paula’s MLS are with regard to what must be visible on an IDX display. Here, it’s the brokerage name, an ARMLS logo, and a “Information Deemed Reliable But Not Guaranteed” disclaimer are required. Every MLS is different.

    None of these things appear in on a Google search result page, but they DO appear if you click through to the actual IDX page on a compliant broker/agent site.

    And I see similar things on clicking through in the pre-April 10 results Paula linked to.

    There is no question that scraper sites need to be dealt with. The problem as I see it is the NAR or a local association has no jurisdiction over these sites. And to say Google is a scraper site is utterly ridiculous.

    If the local/NAR “solution” to scraper sites is to prevent listing information from being indexed in search engines, then what will happen is only the true scrapers will have listing information. They will still scrape sites, and Google will index THEIR information on the listings!

    It’s utterly asinine.

  69. This should be interesting to follow.

  70. Hey, Jay, thanks for clearing that up for me.

  71. I just want to clarify for some folks here – the issue is not that users can access data on the sites nor is it with any specific disclaimers etc – everything is compliant – the issue is that because of the way the REW technology works (it allows you to query compliant previews of MLS data into your content pages to improve conversion) the MLS board takes issue with it because now Google sees things like the MLS #, address or whatever is in the preview – for example this page – http://www.reddoorindy.com/avon-real-estate.php is a spiderable (indexable) page of content and it has listings data (previews of listings or search results) on it. That is the main thing they are taking issue with at this point.

  72. And here is a google search (an example) of what MIBOR takes issue with http://www.google.ca/search?q=+MLS+ID%3A+2865482&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a

    See how Paula’s http://www.hometoindy.com shows up in the SERPS for that query because of the snippet of MLS data – that is what they are telling her (and her broker) that they must prevent. It’s BS if you ask me.

  73. Hi everyone,

    Here is the response from Cliff Niersbach, NAR’s Vice President of Board Policy & Programs:

    The purpose of the NAR IDX policy is to give listing brokers a method to consent to display of their listings by other participants on those participants’ websites. Absent the listing broker’s consent, advertising (whether on websites or otherwise) of her listings by other brokers is prohibited by the REALTORS® Code of Ethics and by the license laws of many states.

    When the IDX policy was revised in May 2005, the following provision was added: “Participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and to prevent ‘scraping’ or other unauthorized accessing, reproduction, or use of the MLS database.” For nearly four years no questions were raised about this new requirement.

    Recently, though, questions have arisen about the scope of the requirement that IDX site operators protect the listings of other participants displayed on their IDX sites from “scraping”. Specifically. whether the policy distinguishes between “malicious” scraping and what might be considered “good” or “benign” scraping. Also, whether “indexing” is a type of scraping. The Center for REALTOR® Technology (“CRT”) advised that while the intent of “scrapers” may be malicious, and the intent of “indexers” good, the two practices from the Web server’s view appear to be the same. Consequently, NAR staff responded to questioners that the requirement to prevent scraping includes indexing.

    Obviously that conclusion hasn’t sat well with some IDX site operators and some IDX vendors. What should be clear is that staff’s interpretation isn’t the last word. Rather, if REALTORS® believe the IDX policy should be changed to permit indexing while continuing to prohibit scraping, or if they believe the policy should be revised in some other way, the appropriate step to bring about clarification or change to propose changes/enhancements to the NAR Multiple Listing Issues and Policies Committee. The IDX policy is currently being looked at with an eye toward incorporating certain aspects of the recently revised VOW policy, and distinguishing between scraping and indexing could be part of the effort.

    If you would like to propose a change to the policy, send your suggestion to the MLS Issues & Policies Committee in care of Cliff — cniersbach@realtors.org.

    Thanks much,

    Hilary Marsh
    Managing Director, REALTOR.org

  74. I have an important question: Why is it that Realtors are held up to a code of ethics, yet the organization that sets forth the code of ethics is allowed to get away with being so stupid?

    Kinda makes you wonder about the process of getting on these boards that lead us. Just look at your associations and who is leading. Do they understand technology? I hope everyone sends this post to those that are in leadership in their boards; like I did.

    Get the conversation started before it is too late.

  75. “Obviously that conclusion hasn’t sat well with some IDX site operators and some IDX vendors.”

    Nor has it sat well with a certain segment of the NAR membership.

    I hope those opposed to this short-sighted lumping of malicious scrapers and internet search engines into the same category will send Mr. Niersbach their thoughts, as he requested.

  76. I’ll leave my suggestion out here in the open:

    I would suggest that if an agent or broker does not want to cooperate with another broker online, then he/she should not opt IN to web promotion in any way shape or form- level playing field for all.

    The consumer can decide if the broker or agent that has opt’d out is of value or not.

    Simple.

  77. Well slap my ass and call me Becky. Just signed up with Diverse Solutions, oh … Yesterday(!) and look forward to being stomped on for providing listing feeds in the coming months. I feel like the 16 year old who just walked into a party with the keg, only to find the cops in the living room.

    Toga! Toga!

  78. I didn’t have time to post this morning because I really did have a long meeting to go to (not punting Benn), but wanted to give you my PERSONAL reaction to this story.

    Who Google searches for “2920292 carmel”?

    Because, if I was the listing agent for 3741 Carwinion Way, and found that most of my clients, or ANY of my clients found their way to my site by Google searching “2920292 carmel”, I guess I would be astounded.

    If I was a listing agent, I wouldn’t count on my IDX feed for my listing’s Google presence. Not for “2920292 carmel” or for “3741 Carwinion Way”. I would build a single property site. Or a featured listings page. Or I would write a post on my blog with the address in the title, announcing my cool new listing. The last thing I would do is count on my IDX, the same IDX that all my competitors can buy, as my primary source for Google SERPs.

    I might even be happy that my MLS/Local Board/National Association makes it harder for the agent down the street to outrank me in Google for my own listing.

    Again, PERSONALLY I would rather that listing be propagated all over the web, in as many places as possible. But I know not all real estate agents agree. Which is probably why this rule was crafted (again, by members, other REALTORS) in the first place. If I was an agent, and didn’t like this rule, I guess I would work to change it.

  79. Benn –

    I’d encourage you to also send your suggestion directly to Cliff as he requested. I’m not confident he’ll come here to read any suggestions.

  80. Im with you, Benn. Isnt that the purpose of having the freedom to choose??

  81. lol okay Becky! Toga!

  82. When technology changes rules have to change to keep up with the times. They need to be reviewed and revised. Nothing new about that. I will send in my opinion on this via email. :)

  83. I don’t care how a buyer finds the listing, it’s why we offer cooperating brokers a commission- just get the buyer to the house and process the transaction in the most professional way possible is all we ask.

    however, if the broker isn’t cooperating, nor allowing me to market products using the most powerful modern tools possible to move inventory, then that broker is not actually cooperating, so that broker shouldnt opt into web promo.

    It’s pretty simple really, I pay for IDX, I blog about what I see on that IDX if I so choose, but now I should mark it nofollow? seriously?

    This rule has serious consequences and this decision has been floating around the halls of NAR since at least the 10th of April-

    simple, tell all the realtors where the meeting is in the best way possible, and let the members decide.

  84. @Todd – You wrote, “I might even be happy that my MLS/Local Board/National Association makes it harder for the agent down the street to outrank me in Google for my own listing.”

    I’d be happy if the MLS/Local Board/National Association stayed out of managing my business, stifling my creativity, and attempting a socialistic move like leveling the playing field for everyone.

    If the agent down the street outranks me in Google, that’s MY problem, not the MLS/Local Board/National Associations.

    If I have the minimal technical skills required to put up an RSS feed of listings — listings that meet every single requirement of my MLS’s IDX policy — and the agent down the street does not, why is it the MLS/Local/National Association’s role to stifle MY technical capability? So it’s “fair” for everyone? Puhlease.

    I don’t count on my IDX feed for my internet presence. But I also don’t expect the NAR or anyone else to step in with a bunch of ridiculous rules that penalize me because I’ve put in the hours required to learn something about Internet marketing.

  85. I think the 80/20 rule applies here. 80% of you think of IDX as pure advertising… it is specialized advertising in that its authorization comes from local IDX MLS Rs & Rs which are allowed some variance (subject to state law) from the NAR model. The broker is presumed to have opted in unless he/she has specifically opted out of IDX (all or nothing.)

    It is supported through the MLS vendor who is paid by the dues of ALL local MLS members whether they have oodles, one, or no listings (100% buyer agency.) The idea is to pull Internet traffic to the IDX member websites to benefit ALL MLS IDX Participants. Everyone has a shot at selling any of the listings (MLS spirit of cooperation.)

    Brokers are free to deliver “their” listings to any third party aggregator (3PA), including Realtor.com, the mother of all 3PAs, that will best serve them and their sellers. And, yes, the data will be used by 3PAs to generate leads that often are “sold” back to the brokers. Broker’s choice. But, IDX is a special opportunity to get leads directly to MLS IDX Participants.

    The Internet is a universe of its own and IDX should have been the portal opportunity to benefit MLS members exclusively… then we started giving away MLS data and created our own Internet competition. Thus, we pay for leads that would have come directly to IDX websites.

    If you are thinking only as a “listing agent”, perhaps IDX should be changed to a charge per listing. Then IDX will be no different than any 3PA… expect most, if not all, of the IDX Participants to opt out of any new “open” IDX as their direct benefit will be greatly diminished as listing agents will insist their full contact information be on each listing.

    Sometimes, you don’t know you have a good thing until you wake up and find you have given it away.

  86. Wow, spent 27 years working for ATT and sbus before coming to real estate. Deva Vu – its our turf you can’t play on it unless we approve. thought that mindset was dead. Face it, consumers want the info, consumers will get the info (one way or another). I’d rather be a source for the info and want my listings spread as far and wide as possible! Exposure is a good thing.
    Please lose the monopoly mindset before you become extinct. It has killed other giants.

  87. The DOJ settlement with NAR will have an effect here, too, as “solds” will soon be available for most IDXs.

    The act of trying to prevent data from being “out there” isn’t sustainable and it gets these boards into trouble. The agents who provide the best information get the leads. That’s the way it always has been.

    We provide IDX in Charlotte, Dallas, and Northern New England, and we always recommend to our clients to put as much “out there” as possible.

  88. @Jay, you said,”I’d be happy if the MLS/Local Board/National Association stayed out of managing my business, stifling my creativity, and attempting a socialistic move like leveling the playing field for everyone.”

    It does not level the playing field. IMO, It benefits the listing agent. Right? Probably the reason the rule was crafted in the first place. To protect listing agents.

    Long before I came to work at NAR, I wrote a posts on why I thought MLS data should be more open. I was crucified by agents who still wanted it protected. This policy was written by your fellow members Jay. Many of which, still want this data placed behind an iron wall.

  89. So … why do we pay NAR dues every year? Hm. Just sayin’

    If the organization does not support the members then why do the members support the organization?

  90. Fiery discussion going on here. :)

    A dumbass question for y’all….

    The rule is horrible since indexing isn’t the same as scraping. Okay, got that.

    Why is scraping so horrible and bad?

    Say I setup a “malicious scraper” company that goes out and scrapes every single piece of IDX content I can find and puts it all on the web. Some consumer finds my scraper site and calls the agent.

    Why is this bad again?

    Scraping is horrible for media sites (e.g., NYTimes) since it means that readers don’t go to the copyright owner’s site to be counted as an eyeball to be monetized. So Trulia should care greatly about scrapers. But agents? But for listings that are meant to generate leads… why is scraping so bad?

    -rsh

  91. This is quite as silly as the REALTOR vs Realtor (R) silliness that we usually have to put up with, but it certainly makes me embarrassed to be a member of NAR.

    Honestly, if I didn’t have to be a member to join the MLS I probably would leave NAR on the side of the road.

    @tcar I’m sorry you have to clean up this mess and that you have to explain over and over what a mess it is.

  92. Todd, I have to respectfully disagree with you. Maybe the intent isn’t to level the playing field. Hell, it’s beyond me what the intent really is. But the reality is it does (at least somewhat) level the playing field.

    It removes one technical advantage I may have (an advantage, by the way, that I taught myself via Googling ‘converting RSS to HTML”. If I can do it, anyone can).

    The rule was crafted to “protect listing agents”? Why do they need protecting? So it’s the NARs job to protect certain agents? Funny, I thought their mission statement was “to help its members become more profitable and successful.”

    I took that to mean ALL its members, not just listing agents, or lose with less technical skills than others. But that’s just my interpretation.

    I guess that’s my fundamental issue with all this oversight, policy, and rule making. It interferes with my ability to run my business.

    I agree that there needs to be some rules. This isn’t, after all, the wild west where anarchy rules. But good grief, lets at least use some freaking COMMON SENSE. Equating scrapers with search engines because some guy at the CRT says that on the server level they do the same thing? How ridiculous. So instead of saying, “Huh, the intent of the policy is to stop malicious use of the data, how do we do that?” we now have “Nothing can be indexed!”.

    You said yourself Todd that you “might even be happy [if NAR] makes it harder for the agent down the street to outrank me in Google for my own listing.”

    Well, I *am* that agent down the street that’s outranking a lot of people in Google. Not because of IDX data, mine isn’t even indexed. But that’s not the point. The point is, suddenly today we’re finding out we can’t do this. What is tomorrow going to bring?

    And please, the whole “this policy was written by your fellow members” thing is beyond tiresome. This “no indexing, Google is a scraper” was NOT written “by my fellow members”.

    As for keeping the MLS data behind an iron wall, I hear you, and I fight it every day. It’s arcane thinking that I’m doing what I can to fix. Having boards/locals/nationals step in with decisions like indexing and scraping are equal sure doesn’t help fix that mentality. In fact, I think it just got set back 20 years.

    OK, enough of me whining. I’m going to sleep on the email I have for Cliff (and hope when I wake up tomorrow some other rule hasn’t changed.)

  93. Rob –

    A reasonable question.

    My opinion is, if some site (feed, whatever) is scraped, and left intact, then no harm is done. Heck, I could look at is like, “there’s another site out there with my listing on it”. And I like that.

    The problem manifests itself in the scraping and alteration of the data.

    If your scraper company goes and removes the listing brokers info, or the contact info from the agents site you scraped it from, and replaces it with your own information, then that may be problematic. You’re now representing yourself as the seller. Or, you just basically copied my IDX — that I pay for — to generate your own business. Why should I support your business as well as my own?

    Personally, I don’t care what agent brings a buyer to my listing. After all, I’m pretty sure my seller’s main goal is to … brace yourself … get their home sold. That’s why if you want to “scrape” my listing and plaster it all over the web, please knock yourself out.

    I also don’t practice dual agency, so I’m not worried about some agent getting the buy side instead of me. But let’s face it, there is a certain percentage of brokers/agents out there who do everything they can to double-side deals. Those types aren’t usually quite so keen on getting their listings exposed to as many as possible.

  94. Jay — Not being an agent, I want to thank you for your comments on this discussion! Very insightful and useful toward this argument/discussion/whatever you wanna call it.

    Im in your corner on this.

  95. One of the problems with some of the thinking out there is that they still believe that the only value they hold for the consumer is that they have *the listings*. They believe that if they can keep control of *the listings* they don’t have to prove their worth otherwise in order to obtain business. The reality is that *the listings* are out there and that, as most real estate bloggers and other ‘new marketing’ types know, consumers are pretty smart and choosing us for lots of different reasons, and not just because we have *the listings*.

  96. Jay –

    I guess that’s what I was wondering. Because this:

    If your scraper company goes and removes the listing brokers info, or the contact info from the agents site you scraped it from, and replaces it with your own information, then that may be problematic. You’re now representing yourself as the seller. Or, you just basically copied my IDX — that I pay for — to generate your own business. Why should I support your business as well as my own?

    is not just scraping, but copyright violation and potentially criminal fraud depending on what was altered and to what purpose.

    Straight up copying of IDX without paying for it I suppose can be a big issue, but that’s pretty straightforward theft of intellectual property — no different than pirating DVD’s. And the realtor shouldn’t much care, whereas the local MLS and the IDX vendor likely would care very much indeed.

    -rsh

  97. My understanding of scraping is when somebody copies all or part of my site, or runs it as an rss feed without my permission, and puts it on theirs to promote their own business or advertising sleeze site.

    My understanding of IDX feeds is that the companies that we purchase this from are licensed by the individual MLS’s/Boards and have given permission for these feeds to flow onto the engines and to be indexed.

    So what the heck is the problem here? Was the offending feed not licensed? If it was, the question is why is NAR interfering with decisions made at the board level?

    I’m not liking any of this at all and happen to agree. Perhaps its time to toss the captain overboard.

  98. Todd –

    “Who Google searches for “2920292 carmel”? ”

    MIBOR does; my examples were based exactly on the criteria they used to create the screen shot they sent along with their last demand that we further restrict our sites.

    I do market my listings, but not for MY benefit, for my clients benefit. The mentality which says you can not have my listing indexed on your website comes from the nearsighted “limited agency” or “dual agency” proponent who does not want another agent to bring the buyer – or from an agent who longs for the “good ole days”.

    Like you said – it benefits the listing agent only – not the client, not all agents; only the listing agent. I’ll go ask my clients if it’s okay with them if I don’t market their listing everywhere I can, so I can protect my listings.

    I thought my clients expected me to protect their interest throughout the entire real estate transaction.

    Now, you have me thinking; I must have been wrong all along! Back behind the iron curtain!

  99. Lisa…

    “They believe that if they can keep control of *the listings* they don’t have to prove their worth otherwise in order to obtain business.”

    There may be some that think that, but it is not an “either or” situation. I believe exercising your right to your “exclusive” listings does not diminish the need to prove your professional value. You can and should pursue both.

  100. I wonder if the meta=”noarchive” tag might be a nice compromise. This tells Google not to cache a copy of the page (what they’re claiming is scraping) but allows Google to index the page so that users can navigate to it.

  101. Oh, look at that! I’m still censored here! How wonderful!

  102. Laurie – Your understanding of scraping is a generally accepted definition, except in this case.

    My feed is licensed and I am a licensed REALTOR(r). The problem is not all feeds or IDX solutions are indexed. Some are framed and therefore, search engines do not see the data

    The reason NAR is involved is my local board conferred with them about the definition of a scraper site and their view was the same as my boards – that Google and search engines are scraper sites.

  103. Oh my…Google is NOT a scraper site.

    Now if your board doesn’t get that much how in the heck are you going to work on the local level.

    And…why would NAR agree with such foolishness.

  104. @tcar

    “…I would rather that listing be propagated all over the web, in as many places as possible. But I know not all real estate agents agree.”

    Who cares if the other real estate agents agree?

    The agents HERE get two things very well: fiduciary responsibility and the power of data accessibility.

    The OTHER agents may not know one or the other at all.

    THIS group of agents is trailblazing a new way to meet the needs of the consumer.

    THOSE agents are simply yearning for the “old ways.”

    If I KNOW of a better way to market my sellers’ properties, and don’t use it, I am failing in my fiduciary responsibility. Simply because some real estate agents don’t agree with data accessibility, doesn’t mean my clients should suffer. It means the OTHER agents need to come up to speed and accept that since AlGore invented the internet, housing information was going to be as free flowing as moolah in politics!

    This faux pas reeks of the Rose Bowl parade float: an insensitivity to the consumer and their desire for straight housing information.

  105. @tcar – “Which is probably why this rule was crafted (again, by members, other REALTORS) in the first place. If I was an agent, and didn’t like this rule, …”

    Who?
    Which Committee?
    Who is the Chair?
    I would really like to see the minutes. Will you please email them to me if posting here (or elsewhere) isn’t kosher?

    “I guess I would work to change it.”

    How? Seriously, how does a member voice an opinion to the members who crafted this rule? Respectfully, Cliff is just one person (despite the fact that I know him and know him to be remarkable, intelligent, capable … tall, good looking … ) he’s just one person. The members who crafted this need to hear from the members who are affected/offended/disaffected.

    So … who and how?

  106. Jim,

    Here is the list of IDX policies that were passed back in 2005: http://budurl.com/lf9k (the link requires you to log in to REALTOR.org). It was the MLS committee, I believe.

    Best,

    Hilary

  107. @Tcar- You said, “Who Google searches for ‘2920292 carmel’?”

    That’s a pretty weak argument, especially for you.

    If no one searches Google based on MLS number and/or city, or address, then why are we even having this discussion? If no one searches on it, then there is no need to regulate it.

    I do agree with you, however, in that if someone disagrees with the rule interpretation, something should be done about it. That is why I still advocate fighting it on the local level. As much as I sympathize with Paula, (in fact, I can empathize with her, as we have been the target of bizarre interpretations of MLS rules and the CoE) the only way that this would be resolved is if Paula’s broker chooses to fight it. Capitulation is still capitulation, even if you believe you did nothing wrong.

    I don’t know the details, but I’m willing to bet that MIBOR sent one heck of a nastygram to Paula’s broker. Heavy-handedness and legalese tend to be the strategy and tactics of choice for large organizations. I’m pretty sure that Paula’s broker could have appealed the rules violation. To my knowledge, that is possible in every association. Paula’s broker chose not to fight the “800-lb. gorilla.”

    If there is a rabid gorilla out there, shoot the thing before it hurts someone, don’t avoid it and hope it goes away.

    That’s why I’ll reiterate my initial comment– fight it. Fight it hard. Tell your clients how you feel it handcuffs your ability to market their home. Whatever it takes to get the issue heard openly in front of people who have the ability to change the rule, or at least, its interpretation.

    Going through NAR leadership on this one is gonna be tough. First of all, the committees can only make “suggestions” for policies and rules. Let’s say that you convince NAR leadership, there is still the possibility that your local association will ignore it. They have every right to do so.

    I say, do what you can, where you can. We all have much more influence with our local associations than we do with national leadership.

    On a side-note: the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE. It is completely indefensible. The fact that our National Association controls the use of its data by its paying members, but not by a private company, is a disgrace.

  108. I think Jay said it best with…

    “If the local/NAR “solution” to scraper sites is to prevent listing information from being indexed in search engines, then what will happen is only the true scrapers will have listing information. They will still scrape sites, and Google will index THEIR information on the listings!”

    It all comes down to NAR’s hair brain interpretation of the term “scraping”. Mr. Bramlett probably has the right idea to add a “noarchive” tag to listing pages if NAR is so stressed about this. However, if that is the case than they need to require R.com and others to do the same.

  109. Sigh… I am literally sitting in the hospital, admitted for cardiac issues – reading this isn’t helping.

    I am going to be uncharacteristically brief:

    1. Go Trulia! (just saying…)

    2. Agents should be cautious of harvesting sites, but not to the exclusion of marketing to the best of their ability and to promote their client’s best interest.

    3. There are issues with harvesting / scraping, such as: Agents putting remarks such as “not a foreclosure or shortsale” and then the algorithm classifies it as a foreclosure or shortsale. This has happened to several local agents on Oodle and others.

    4. MLS should be doing everything to help the practitioner and not themselves.

    5. To NAR: If you want people like me, to keep trying to promote the Association and involvement, you must stop making such dumb effing rulings. Leave it to the locals to screw up all on their own!

    Now, my morphine and nitro is wearing off – I’m going to sleep.

  110. Laurie Manny said “If it was, the question is why is NAR interfering with decisions made at the board level?”

    Laurie, NAR is not interfering. This decision was made at the board level. When asked for feedback, NAR determined that the local board’s decision fits within the policy included in this original post. (Section 15.2.2 etc…)

    Again, if I were an agent, I would work to clarify the policy. It’s four years old. Technology moves fast.

    Daniel, my argument over who searches for “2920292 carmel” is every bit as as relevant as the links to that search included in this post were. There’s a dramatic difference in those two results. But it’s worth pointing out that it’s a dramatic difference over an obscure search. Google the addresses instead of the mls number and the results are different. Benn later commented,

    “by the way, clicking those two Google before and after results is staggering, what an awesome illustration of the devastating impact controlling the data can have on the visibility of our listings.”

    So I think it’s relevant to take a step back and look at what this illustration represents. This is a search term that nobody would use, and that nobody really tries to own. I wonder how hard it would be to own “2920292 carmel” on Google. I’m guessing not very.

  111. As a Broker/Owner, practicing REALTOR, and Vice President of a local REALTOR Association…

    [ASIDE: I am a voluntary member of NAR because I believe change comes from within and not because I am required to be a member to access the local MLS. In fact, in Metro Atlanta our MLS’ are not owned by the local REALTOR Associations)

    …this policy needs amending immediately. Jay is right, so is Rob, Matt is spot on as well, and especially Daniel when he says, “On a side-note: the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE. It is completely indefensible. The fact that our National Association controls the use of its data by its paying members, but not by a private company, is a disgrace.”

  112. This policy is an outrage and even nefarious. I cannot entertain that adults with any heart for consumers would try to state with a straight face that indexing by search engines is an unethical scraping of data. How can MIBOR intentionally try to prevent search engines from indexing the IDX data? How is that serving the marketplace and consumers? Intentionally making it harder for consumers to find the homes for sale for which they are searching is anti-consumer. It is this 20th century mentality of trying to control the market and data that has damaged the reputation of realtors and brokerages in the past.

    Are you blind, MIBOR? Are you anti-consumer? Have you actually thought this out?
    And what about the sellers? You better believe they want their listing indexed by google and easily found from anywhere. How is MIBOR’s position allowing brokers to better serve their sellers? It is consciously making the broker less effective either in exposing the listings or in connecting with and forming relationships with buyers/consumers.

    What is also aggravating is that they are intentionally making it harder for agents/brokers who are modernizing their business and business models to be harmonious with the behavior and preferences of today’s consumers to connect with and serve more clients. MIBOR is intentionally penalizing those who are on the cutting edge of reaching out to serve consumers most effeciently. What is gained by this control MIBOR is trying to maintain other than a sense of controlling others’ businesses and thwarting buyers from finding what they are looking for quickly? How is this serving the marketplace MIBOR? It most decidedly is NOT.

    Mike said it well:
    “The problem is this ruling gives/ will give an unbelievable boost to lead aggregators like Trulia, Zillow, and our good friend Realtor.com. R.com aside, the lead aggregators are not members of NAR or the local boards and are not subject to the rules they are imposing on its members. So while NAR chooses to cuts its members legs out from underneath them with their archaic IDX blue laws, the national lead aggregators are the real beneficiaries here.”

    That any MLS board would endeavor to either be this backwards, controlling of their members or anti consumer is alarming and not what our country and capitalism is about. Handcuffing the most creative and entreprenurial is disgusting and depressing and is not what being a realtor or small businessman is about. Why would NAR stand for this abuse? I pay so much money so they can do what for me–handcuff my business through which I feed my 5 children?

    MIBOR supposedly is open to reviewing their policy with more input from its members and other interested parties. I hope they are sincere in saying that….Everybody is watching you MIBOR.
    And perhaps the obvious solution is that all listings should default into an IDX open to indexing by search engines when they are put into the MLS. I CANNOT IMAGINE 1 SELLER NOT WANTING THEIR LISTING TO BE INDEXED BY GOOGLE. Do you care about that MIBOR? Let’s be honest about this anti-consumer policy. If a seller does not want their home exposed on the IDX feed that can be indexed by search engines, they can opt-out when they sign the exclusive right to list contract and the listing agent can opt-out when they list the home in MLS.

  113. Daniel –

    My broker did fight – he did not cave to the first email sent out or the second for that matter. We talked to other brokers who were affected by the rule and no one wanted to go the distance. There were several email exchanges and phone calls, to no avail. They sent the cease and desist and told Mike he could present his case to the local Board in a month. Mike was invited to this months meeting, but unfortunately, did not make it. He will continue to try to pursue his options through the board.

    Honestly – I believe it would not have made a difference! They sent the second email stating we could no longer index any MLS data, knowing there was a meeting coming up.

    Why is NAR involved? Because that was our boards “way out”. They deferred to NAR and NAR said :

    “You are correct that under the IDX policy and rules, Participants have an obligation to protect IDX information from “scraping” which includes “indexing”. I’m told that employment of a robots.txt file can prevent scraping of information Participants are required to protect.”

    We will continue to fight!

  114. @Todd

    “his decision was made at the board level. When asked for feedback, NAR determined that the local board’s decision fits within the policy included in this original post. (Section 15.2.2 etc…)

    Again, if I were an agent, I would work to clarify the policy. It’s four years old. ”

    Then why did NAR agree w/local board issue, rather than advise the local board to wait and allow for a FULL review of the outdated policy?

    just sayin…

  115. Matthew – Pease take care! I’m praying for you!

  116. In the words of NOFX —> “Dinosaurs will Die!”

  117. I think that Hillary Marsh hit the nail on the head. The rules were written back in 2005. In the tech world that was the 18th Century. Unless the association we are all required to belong to comes into the 21st Century with technology we will be fighting this until we are all dead.

    NAR is meeting in San Diego this year. Perhaps a revolution should be planned to take place during the convention. If enough are willing to do this, they will have to listen. Otherwise, we are SOL.

  118. @tcar – As stated earlier – I only used what MIBOR used in their screenshot.

    “Google the addresses instead of the mls number and the results are different.”

    Oh fun – let’s do!

    Okay for 290292 – that is 13872 Ninebark Ct for those of you who want to play along. Results: 2 real estate returns and one of those will soon be gone because it is from an RSS feed – the other is real estate.yahoo

    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=13872+ninebark&btnG=Search

    The second one – 3741 Carwinion Way – about the same returns, except the first 8 are not realtor sites


    Trulia

    Trulia

    Realestate.yahoo
    homes.com
    realestate.com

    zillow

    realestate.aol
    realestate.aol

    Then we have in #9 spot another Realtor site which will be getting the axe.

    http://www.google.com/search?q=3741+Carwinion+Way&hl=en&rlz=1T4GGLL_enUS318US319&start=0&sa=N

    Now, that makes me feel much better.

  119. @Benn,

    Let me just relate my own personal experience for a moment. Every week, at least half a dozen local associations email or call me about social media. Every week, I get at least one invitation to go speak somewhere for a local board. Most associations don’t know I exist yet at NAR. Yet working with local associations is a part of my job every day.

    Less than 400 staff members serve over 1,000 local boards and over 1,000,000 members.

    Earlier today, you noted hadn’t responded after three hours. That’s a common expectation. We don’t have the luxury of waiting to respond to our members. Every policy on technology will be outdated almost the instant it is written. NAR staff works with what the elected committees and leadership has given them today.

  120. @tcar – one more note – although the probability is low anyone would use the mls # and city for a search, the results I posted ARE indicative of the information available using the exact same phrases before and after I was told to stop aloowing Google to index my site.

    Even if I did want to own the mls # and city, now I can’t!

  121. First off, let me just say that I was always a fan of Jay Thompson, but now I’m a huge fan. Almost every comment he made I had something to say about, but by the time I got down to this box, I realized I could write for about five hours in agreement with some of the things he had to say. What I really enjoyed, was that he said:

    “Obviously that conclusion hasn’t sat well with some IDX site operators and some IDX vendors.”

    Nor has it sat well with a certain segment of the NAR membership.

    I was afraid I was the only one that caught that. Thanks NAR for completely marginalizing all that you just read on this post. Nice work – it should quell any furor over this and make your membership feel pretty calm.

    I’m the new guy around here (and real estate in general), I make no bones about it. Sometimes I don’t understand the ins and outs of topics that some of you have dealt with years. So if I say something completely ludicrous, feel free to call me out on it – I’d love to learn more.

    What shocks me the most (and this is a general disgust of mine) is that here the board is going after a dues paying member for opening themselves up to be scraped (I’m not even touching the idea of whether Google is a scraper). Ok, so let’s compare scraping to theft, which I think is a logical comparison for the purposes of my argument.

    If I leave my wallet on a table in a restaurant and it gets stolen – who’s the criminal? Me or the thief? Now I realize no one has been arrested and this isn’t criminal in nature, but give it a second to sink in. Who’s the problem here? Paula (her broker and IDX vendor)? Or the scrapers? If we are truly worried about scrapers, let’s take steps to stop them, not stop agents from working hard.

    I’m sure the argument then becomes that it’s too hard to stop them or would cost too much. Poor excuses if you ask me. It’s too hard to stop the illegal flow of drugs from Mexico these days, but that doesn’t mean we don’t keep trying (and again there’s arguments about our attempts at that too).

    I hate to harp on my experiences within the music industry, but I’ve seen how technology can confuse the hell out of a giant machine that works on “steady as she goes” mentality. I watched as Napster took over the world and worried that my own money was disappearing everyday. The arguments flew in different directions and whether one side was more right than the other didn’t matter one bit. The world changed and the music industry was left holding the bag. They failed to see the change and adapt to it. You can argue for days whether it was theft or not, it didn’t matter – it was too late. Change happens, whether we want it or not. Ignoring it doesn’t not make it go away, laughing at it and saying it won’t affect us won’t make it go away…but what does it matter – I’m preaching to the choir here.

    I read this early this morning and reserved my comments until now, now its late and I still have to write my post for AgentGenius, how in the world can I compete with this one?

    P.S. Extra credit goes to Jason Sandquist for dropping NOFX into an AgentGenius comment thread.

  122. @Paula, after linking the google search to that MLS number, you remarked,

    “I’ll bet the second home owners are really happy with their online marketing!”

    The MLS# based results are full of jibberish. Google the address and the home owner’s property is all over the place. On Trulia, Zillow, Homes.com etc. That’s a world of difference worth noting. I can see why it’s still far from ideal for you, but there’s a huge difference.

    The homeowner would have a different opinion as to how well their house is being marketed online after looking at that address search.

    The first time I clicked on both links I thought, “wow, that’s huge”. But then I looked at what was being searched for. The difference is huge because no one would take the time to own these results. Because even though it would be extreemly easy to own them, they are not worth owning.

    BTW, I now own number one placement for “2920292 carmel” on Google. I’ve yet to see any inbound leads from it. :p

  123. @matt, “What shocks me the most (and this is a general disgust of mine) is that here the board is going after a dues paying member”

    Last one, and I’m out for the night. That board took action because another dues paying member asked them to. As I noted before, not every REALTOR agrees that this data should be open. NAR policy is written by it’s members. If you want to change the policy, get involved.

  124. @tcar – You said, ” Every policy on technology will be outdated almost the instant it is written. NAR staff works with what the elected committees and leadership has given them today.”

    Absolutely correct, but in this case, from what we can discern, the NAR affirmed an obsolete policy. Shouldn’t NAR give guidance such as, “while this does fit in with the existing policy, we feel that it would be prudent to withhold judgment as this policy was written four years ago and a lot has changed since then.”

    And I almost did a separate post with nothing but this quote from Daniel –

    On a side-note: the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE. It is completely indefensible. The fact that our National Association controls the use of its data by its paying members, but not by a private company, is a disgrace.

  125. Last one for this evening myself –

    “If you want to change the policy, get involved.”

    How? Who should members contact to get involved? For many members, this is the problem – they don’t know where to go or whom to contact.

  126. @tar – I think you cut that quote a little short (the next few words change the meaning a bit), but I’m not going to argue about that. I know what you’re saying in regards to a dues paying member brought the action in the first place, but my point was that Paula is not the person who did the “scraping” – it was Google (!?!?!?!?! – I can’t believe that’s even being said). I just thing the finger is being pointed in the wrong direction (and I know the response will involve Section 15.2.2, but my point is that I think that might be where the disconnect is). I can understand if Paula was selling the data to scrapers, but the scrapers are doing the act, not Paula – I guess that’s what bugs me about it all.

    Its kind of like the giant trucks I always seem to get behind on the highway. The one that just came from the construction site and doesn’t have his load properly covered. The one that hits a bump and sends rocks flying and bouncing down the highway as the driver breaks the speed limit. The same truck that says “not responsible for broken windshields” on the back.

    The logic never makes sense to me.

  127. Un momento por favor

    We keep referring to Realtor.com being allowed but we may not- should we not be lumping that into ANY media company?

    Seriousy, I won’t list them all again, but Zulia comes to mind…

  128. @ Todd, I am not 100% sure of your role at r.com or completely familiar with your past prior to your employment with the NAR so forgive me if I assume something I should not. I can certainly understand you are dealing with the hand you are dealt in relation to technology, committees, et al. However; it seems you “have your ear to the ground” more than your average NAR aristocrat and can actually relate to and understand the conversation we are having here in addition to having an audience at the NAR.

    I guess as the broker in question here; I would implore of you, as the ostensible middle man between the powers that be and the tech savvy brokers and agents that are (like it or not) ushering in a new era of real estate, you cogently explain to the NAR committees that this is a HUGE issue with major ramifications for thousands and thousands and thousands of websites and needs updating. Judging from your own comments as well as the other the 100+ comments in support of this, it seems the NAR’s policy is clearly outdated and needs addressing/updating immediately.

    If there is anything else that we can constructively do to proactively effect change in NAR’s seemingly archaic policies, please let me know and I would happy to participate. You tell us to “get involved” but at this point is there anything we can do besides email Cliff Niersbach?

  129. @tcar – Let me try one more time! The results for the listing you are searching does have results, but it is a listing which was listed before April 10th and had already been indexed. Here’s some results for homes listed after April 10th.

    The results ARE astounding. It’s almost like the feed was cut!

    3810 Brigade Circle
    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=3810+brigade+circle

    5803 Coralberry
    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=5803+coralberry+carmel&btnG=Search

    13187 N Camillo
    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=13187+N+Camillo&btnG=Search

    10412 Breckenridge
    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=10412+Breckenridge

    3817 Earhart
    http://www.google.com/search?hl=en&rlz=1T4GGLL_enUS318US319&q=3817+earhart

    I was going to post some of these, but didn’t want the post to be any longer than it already is.

  130. *sigh*

    Yet more “NAR policy is written by its members” bullshit. Todd, this “policy” WAS NOT WRITTEN BY THE MEMBERS. It was *the NAR* that affirmed a boards interpretation of a policy that was written by a few members several years ago. A policy that by your own admission is outdated.

    The argument that no one would search for a MLS number is weak, at best. That’s not the point. The NAR has chimed in that Google is the equivalent of a scraper site. Surely someone with your technical acumen Todd can see how completely asinine that comparison is.

    Paula and her broker didn’t do anything wrong. The MIBOR *and the NAR* are accusing Google of “scraping sites”. This complete and utter cluelessness is what I find shocking. And frightening.

    And if a local board, and the NAR, are THAT clueless, what is next?

    “If you want to change the policy, get involved.”

    I think the commenters here ARE getting involved. Maybe not in the formal manner (whatever that is) that the NAR bureaucracy requires, but they are getting involved.

    What exactly is the formal process to get involved in changing this ridiculous policy that the NAR just affirmed and “clarified”? Sending Cliff Niersbach an email? Surely there is more to it than that. I’ll freely admit it, I don’t know. It’d be nice if my national association would clue me in rather than side-step the issue by making statements like “Who Google searches for ‘2920292 carmel’?” and “I might even be happy that my MLS/Local Board/National Association makes it harder for the agent down the street to outrank me in Google for my own listing.”

    (And yes Todd, I realize you said those were your personal thoughts. But you also must surely understand that as an NAR staffer, your personal thoughts are now associated with your employer. Like it or not, they are.)

    I’m done with this argument. All I seem to get back is “the policy was created by the membership” and “not everyone wants open data”. With that attitude, it’s pointless to try to affect any change on a forum such as this. I’ll get “involved” in the way the NAR prefers as soon as I can figure out how.

  131. @Todd Carpenter You said “this is a search term that nobody would use” – I am going to have to disagree with you here, – the fact of the matter is, I process literally millions of unique visitors through my servers every month for obscure terms with no traffic and these terms are some of the best converting terms in many cases.

    While you are right that most of these terms only get searched 1-2 times per month (if you are lucky) what happens when you rank for tens of thousands of them? Thousands of targetted visitors that’s what happens – and these are real buyer prospects.

    Now don’t get me wrong – not everyone with a spiderable IDX gets thousands (or tens of thousands) of unique visitors per month due to the fact that in order to truly make a search engine spiderable IDX (or any site with great content) work in your favor from a search engine perspective you need to bring a lot more to the table – but for those that can pull it off – those that learn their craft (ethical SEO in this case) – why should they be punished (Jay already makes a great point above) simply because they put the time in to learn how to take advantage AND invested significantly in the technology (great IDX products like Paula’s certainly don’t come cheap)

    Now here is the most disgusting part about this particular situation – I have it on very good authority that the member that reported Paula’s broker Mike Taylor was another member named Mike Woods. The very same Mike Woods that contacted Real Estate Webmasters with the following inquiry –

    “IIwould like a site that automatically displays listings and can be indexed by the search engines: 1. Display listings by Subdivision 2. Display listings by Zip Code separated into price range categories 3. Display listings by City separated into price range categories 4. Display listings by Township separated into price range categories 5. Display listings by Bank Owned 6. Display listings by Pre-Foreclosure 7. Display listings by Probate 8. Display listings by fixer-upper Each page would need to be automatically generated with the correct title tag, H1, tag, meta description, meta-keywords and the keywords bolded within the body copy of the result pages. We also need to use friendly URLs. Let’s talk about price and time. Mike Woods President and Owner Broker msWoods Real Estate, LLC 317-578-3220 msWoods.com” [link redacted per commenter]

    This was not some “old school” broker who did not understand the value of the product to listings clients or the value to his competitor – he knew all to well – in fact if you read his request for estimate you will find that he actually knows better than most exactly what a solution like this can do for him – so why did he report our cusotmer? You would have to ask him that, but I have a few ideas

    #1 – He found out how much something like that would cost and didn’t want to pay, so instead he figure he would just report the competition

    #2 – Perhaps he thought that he could have his competition removed while building his own and seeing as he was the only one smart enough to report, he would have the competitive advantage alone when the dust settled.

    I have some more theories (and at this point #1 and #2 are just that theories) but the rest (including the quote request) is factual and verbatim.

    So what now? Sure NAR didn’t pass the judgement (MIBOR did) but NAR backed them on their interpretation of the rule which basically gave MIBOR their out – NAR should have stayed neutral, but they didn’t, they got involved and spoke before they practiced responsible due dilligence – I think it more than reasonable to expect NAR retract their support of MIBOR ruling until such time as the rules can be reviewed and adjusted (because technically there is no verbiage in the IDX policies doc that states that search engines are scraper sites) in which case MIBOR would have to reverse their demands due to the fact that they weighted so much of their justification on deferring to NAR.

  132. Gack is there any way you can unlink or nofollow that link to MS Woods – certainly did not mean to send any link juice in that direction :) – thanks

  133. Boy, where have I been?????

    The first thing I would like to see happen is Google Slap Realtor.com and de-index their site for agreeing with MIBOR in their assessment of Google being a scraper site (wishful thinking). That’s just plain incompetence and to think these people are running NAR. I know that’s pretty harsh but come on, Google a scraper site.

    This reminds me of what took place with the Sarasota Association of Realtors taking Marc Rasmussen’s site and NAR doing nothing. NAR’s Mission is “to help its members become more profitable and successful.” I might be a little cynical, but I just don’t see NAR holding true to it’s mission statement.

    @Todd – I was really hoping to see more from you in support of this issue. You know how important Technology is and by enforcing out dated rules like this, it’s only hurting the progression of our industry on the Internet. NAR should not be my competitor and unfortunately it is and rulings like this will only strengthen Realtor.com and hurt the little guy.

    @Morgan – That is pretty compelling information that you provided on the Broker. If I were one of his agents working for him, I would definitely reconsider hanging my license with him after seeing how he responds when he doesn’t get what he wants.

    This is just a matter of time before other local associations begin to follow suit especially with NAR supporting the ruling. Hopefully with a strong enough voice, we will be heard and we can get these rulings changed for the betterment of all. Having a listing displayed on thousands of websites and indexed in the search engines should be left up to the homeowner, not NAR or the local associations.

  134. Since the time the NAR sold off realtor.com to a private company, a downhill slide of agent member service has grown into a fiasco. NAR has evolved into a nitpicking and meddling bunch Akin to a Home Owners Association committee with way to much time. I agree with many of the remarks on this feed, Were it not for the mls I would not be a realtor (copyright logo here).
    Sure, I’m one of those simple minded old time agents who enjoyed the art of selling real estate without having to stay abreast of technology and terns like indexing and scraping. I also remember the time when the IDX was introduced knowing immediately it was going to get exploited by those with superior tech knowledge and or deeper pockets than me.
    I guess what Irks me the most, we have a housing crisis, and the folks at NAR are fiddle faddling around with hair splitting accuracy. Surely the time would be better spent away from the nit picking ideas
    Reducing member costs ( yep we are hurting as well) and putting the focus on cleaning up this mess with. Something a little better than the TV advertising of Sally the Real-Tor who is selling a house to a Doc-Tor.
    And whilst I’m enjoying the rant, let’s put an end to the lavish self appreciation events. Take the money saved. Fire the tech committee for a 12 month period, and apply the funds to local workshops where homeowners would receive assistance and advice (providing of course we are still allowed to give it)
    This whole scrapping indexing debate is a terrible allocation of time. Put it on hold, get back to it next year. Phew what a rant

  135. The post from Hilary Marsh of REALTOR.org seems to offer a possible solution to this issue or at least a way to suggest one.

    It says:

    When the IDX policy was revised in May 2005, the following provision was added: “Participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and to prevent ’scraping’ or other unauthorized accessing, reproduction, or use of the MLS database.”

    It also says:

    If you would like to propose a change to the policy, send your suggestion to the MLS Issues & Policies Committee in care of Cliff — cniersbach@realtors.org

    How about proposing a change in policy and sending it to Cliff Niersbach? What about changing the previous paragraph with something like this:

    “Participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and to prevent unauthorized accessing, reproduction, or malicious use of the MLS database. ‘Scraping’ done for the sole purpose of search engine indexing or where the data will be displayed in accordance with the terms of this policy shall be allowed.”

  136. I got here a little late and I think everything has been covered except for…

    Doesn’t MIBOR have anything better to do?

  137. Pingback: FBS Blog » Blog Archive » Calling @crtweet for clarification, please!

  138. Great point, Tony. Until NAR speaks out on this in a pro-consumer and pro-realtor fashion Google should consider deindexing them for equating through their silence that indexing = scraping. It’s ridiculous and insulting to everybody’s intelligence….

  139. WHO IS THE CORRECT POINT OF CONTACT AT GOOGLE TO ALERT THEM TO NAR’S POSITION ON THIS???? I’m sure google would like to know about this discussion.

  140. Benn,

    The reason that I didn’t mention Zillow in my comment is because it has always been my understanding that Zillow operates with data that it gets directly from brokers, not from MLS’s. This makes it different than REALTOR.com. REALTOR.com is getting the data from the same place as Paula’s broker, the MLS. Therefore, it should be subject to the same rules. Zillow, on the other hand, is getting the data directly from the brokers, so would not be subject to those rules.

    Now, if my understanding is incorrect, and Zillow does get its data from MLS’s in certain areas, then Zillow, too, should be subject to the MLS rules regarding usage and display of the data that affect participants.

  141. It is our JOB to get the sellers information OUT THERE, in the most efficient way possible, with NAR stating over 80% of buyers are online looking, why is NAR trying to hobble us and our efforts. This is totally doublespeaking by NAR and shortsighted by the Indianapolis board to prevent their agents in aiding the sellers. Isn’t that our job? Get the house sold, not hide or make difficult to get the facts and encourage DUAL AGENCY? JMHO.

  142. “2920292 carmel” – would I search for something like that? Why duh! yes!
    I would also try to rank for it if I found value with it, MLS Data or no MLS Data.
    So just how hard would it be to rank for the term “2920292 carmel”? There are 14 results there this morning, most are in a foreign font and illegible. Perhaps everyone that is in this discussion should post a blog entry on the subject titled “2920292” and then the NAR and MLS could just search for that term to find experts in the field and have a source of those that “get it” to put in an advisory capacity to the NAR and MLS on technology. No technology skills – no technology advice!

    Here is my post on the subject – http://www.mygulfcoastbeachteam.com/2920292-carmel/

  143. @tcar – Oooops -I spelled Corralberry incorrectly. There are returns for this home. I could give you hundreds more examples of homes which have no information -it’s just wasted space and time, though. It does not change the facts which originated this post.

    http://www.google.com/search?hl=en&q=5803+corralberry

  144. Again, wow, and to think that I thought all we had to do is sell houses?

    I wrote up an offer this morning for $405,000… but maybe I should have been SEO-ing.

    It is time to get our priorities in order.
    -doug

  145. Mike Taylor said>> “I am not 100% sure of your role at r.com or completely familiar with your past prior to your employment with the NAR so forgive me if I assume something I should not”

    I’m the social media manager here at NAR. I do not work for REALTOR.com. Again, ALL of my comments here are my personal opinion despite what Jay says. I’ve been expressing my personal opinion on Agent Genius LONG before I ever worked for NAR.

    I don’t make policy at NAR. To be frank, I won’t be doing anything to get this policy changed. That’s on you guys. The members. An official statement for NAR was published in the comment stream. For everyone wondering what to do now, read that statement. Or read Teresa Boardman’s comment as I believe she understands this better than anyone.

  146. @Matt

    P.S. Extra credit goes to Jason Sandquist for dropping NOFX into an AgentGenius comment thread.

    hahaha, I knew somebody would get it!

  147. That NAR or MIBOR or perhaps Todd would be okay with 3rd party sites ranking higher for home searches than the listing and selling agents says everything….IT’S WRONG AND ANTI-REALTOR! If NAR is okay with this they are not on REALTOR’s side.

    They do not have the agents’ best interests in mind. They are like elected politicians who may have had ideals and principles on the front end but quickly focus on maintaining their power, control and reelection after in office.

    Pathetic.

  148. “Again, ALL of my comments here are my personal opinion despite what Jay says.”

    I understand you are giving your personal opinion Todd, I said that in my previous comment.

    I also understand that when the NAR Social Media Manager speaks, people listen. And the simple fact of human nature is, when the NAR Social Media Manager says anything, personal opinion or not, it will be construed as the opinion of the NAR itself. That may not be right, we may not like it, but that’s the way it is. I spent years inside corporate America in Human Resources dealing with “but it was just my personal opinion” from management and staff. Believe me, what you say ANYWHERE Todd will be taken by many as coming from the NAR. That fact can’t be ignored. Yes, of course you are allowed to express your personal opinion, just understand that it can, and will, be perceived as your employers position as well. That’s one of the shitty things about working for The Man, which you now do.

    So I guess an email to Cliff is our only option to “get involved” as we are so often told to do. That seems odd, as I thought there was a committee or some sort of governing body we could address our concerns to. I don’t know Cliff, but if Jim Duncan says he’s a good man, that’s all I need to know to trust that Cliff will take my email and express my concerns to whoever it is that listens to these things. I wish I could tell them directly, but I guess I’ll use Cliff as the “middle man”.

    I never expected you to fight this policy Todd, that’s not your job as I understand it. I am disappointed because in the social media space there is obviously a lot of passion surrounding this policy clarification and your response seems to be simply “I didn’t do it. You guys made the policy, deal with it”. I still like ya, and I certainly respect you. But I’m disappointed in the response to this first big “blow up” in the social media space since you took this role.

    Granted the numbers here are small considering the overall membership, but I believe one could successfully argue that these passionate few are the ones that truly understand how ridiculous, and potentially damaging, this whole thing of equating indexing to scraping might become.

  149. What should I have done differently Jay? I’m not NAR’s PR person. Nor do I set, advise, or interpret NAR’s policies. Pretty much every comment you’ll ever see me make online will be my own. My opinion in this thread would be the same if I didn’t work for NAR.

    My role in this “blow up” was to identify it, gather the facts about the post, determine if the post was based on credible information, find the appropriate people (or people in this case, half a dozen people discussed it here yesterday) to comment on the issue, and ask them to respond. This all happened in less than one business day.

    It’s understandable that you don’t agree with the response. It’s unfortunate that you are unsatisfied with the remedy. But there is a remedy. In your opinion, what else should I be expected to accomplish for you?

  150. What I wonder is why it’s taking so long for some smart tech company to put together a national site to replace all the local MLS systems that agents could use without being members of the NAR. The site could be free, supported by Google ads, with no control by the NAR since the agents would no longer need to be members.

    Seems simple, so why isn’t it happening?

  151. @tcar
    So if we get involved (in our way which is to raise awareness that in less than a week’s time, NAR will discuss this exact policy and no one knows about it), it’s a blow up? I’m not sure that’s a fair choice of words.

    I think the folks here have voiced sincere concern and displeasure to a “4 year old rule” being endorsed by one person at NAR that has greatly affected an entire marketplace in the middle of a recession- that’s not a blow up, that’s outrage.

    I don’t think anyone here is hanging this around your neck Todd, or ever wanted to, but you are looked at as a truth bringer, and the truth you brought us here today is that you’re not here for us, you’re here for you.

    So, what you might do differently is find out where and when the exact meeting will be on exactly this subject, and extend an invitation so that those that can afford to spend money on behalf of the membership can go and do what you’ve so brilliantly expressed here- be members of an organization that didn’t come here for us, but for themselves.

    Also, you may want to change your link to Realtor.org.

    “To be frank, I won’t be doing anything to get this policy changed. That’s on you guys. The members.”

    Of note, nearly 5,000 uniques read this post yesterday, and another 450 read it between the hours of 12a and 1a.

  152. Benn – in all fairness to Todd, I first used the term “blow up”.

    Todd – What could you have done differently? What Benn said. It’s been repeatedly requested on this thread for someone at NAR to let us know what we can do to “get involved”. Is sending Cliff Niersbach an email really all we can do?

    Is there a meeting scheduled at mid-year to discuss this?
    What committee handles this policy?
    Who is that committee chair?
    If I send an email to Cliff, what is done with it? Is it “presented” to someone? If so, who? Can I contact them directly?

    Maybe I’m an idiot and I’m the only one who doesn’t know how to “get involved”. As I previously mentioned, it’s hard for me to believe that sending a NAR VP is the only way I can get involved. After all, as you’ve stated a couple of times, the NAR didn’t make the policy, Realtor members did. Why then is my only method of getting involved to email a NAR VP? Why can’t I directly address the members responsible for this flawed and outdated policy?

    I suspect I can address them beyond an email to Cliff. I just don’t know how. If you can’t answer the “how questions” can you forward them to someone that can?

    I sincerely hope that as the NAR SMM, that you’ve forwarded this post and encouraged the staff within the NAR that might have some concern to read the comments. You may not need to care what we’re thinking here, but someone at NAR should. I don’t know who that person is, but I would think that as the SMM when a discussion of this magnitude that is occurring within the social media space occurs, with it’s obvious emotion and (hopefully) obvious potential ramifications, that it would at least be pointed out to the appropriate people within the organization. Maybe that has happened, maybe not. Only you know and you certainly don’t have to tell me or anyone else every step you take.

    Now I’m going to be blunt and honest. You know me, that’s how I roll.

    You are well aware that before you were even named to this position, there were questions out there that asked regarding how long it would be before the SMM was “assimilated” by the NAR. Well, some of your personal opinions in this thread — to me — have answered that question. It didn’t take very long.

    I hope I’m wrong about that. And maybe that’s just my interpretation and my feelings/emotions getting the best of me, but that’s how I felt when I read many of your responses.

    I can’t tell you how to do your job Todd. But this whole thing is of very deep concern to me. It clearly demonstrates how poorly some policies are written and interpreted. The potential ramifications of NARs “clarification” or “support” for MIBORs decision is profound. It seems to have been done in a relative vacuum, and some of us are obviously struggling with how we can get involved.

    I just wish someone, anyone, at NAR could answer those questions. If an email to Cliff is really the only action I can take, then please explain it to me like I’m 3 years old — “Yes Jay, all you can do to get involved in this matter is email Cliff”.

  153. Well, a little unfair to Todd. He is, after all, in charge of social media services, and that’s why he’s in on this discussion. But Member Policy geek he isn’t. I suspect he’s making sure that the MP staff knows about this discussion…but member policy does belong to the members. And NAR is still encumbered with a governance structure that moves slower than technology, and slower than its members themselves. That’s not blame, it’s the way things are.

    What that means, I think, is that members (and local MLSs and associations) need to weigh the risk of moving ahead to meet member needs, and hoping NAR catches up rather quickly. Many of the existing policies once had a reason for existing, but in this case the reason has been rendered ludicrous.

    I think the issue needs to be decided on the operations level–what is going to work best for you as members? Don’t settle for less….

  154. Going to the whole email “cliff” thing. Will he actually read it or are we going to get some bull s**t PR written letter back just like I get every time I seem to write my congress representative. For some reason that’s how I think it will all go down.

    Putting effort into an email just doesn’t do justice if it goes unread or he happens to hit the delete key.

  155. @Jay

    Duly noted :) But the larger point remains :)

  156. Hey, wait a minute! I’ve got an idea. Why don’t we just require consumers to be blindfolded while trolling the Internet for real estate listings? It would have about the same effect as this ruling without inconveniencing all the hard-working Realtors in Indianapolis. In fact it would be better than the ruling because then consumers wouldn’t run the risk of landing on an actual scraper site. Moreover, the national sites like Trulia, Zillow, Craigslist, Realtor.com etc wouldn’t have an unfair advantage either.

    Sheesh.

  157. I am going to just jump in here quickly. I’ve been following on my blackberry while sitting around…

    This comment stream is getting personal, slowly but surely and life is too short to create offenses.

    NAR has a sucky rule – it’s one of many. This is a good point of origination to try and find a resolution.

    MLS isn’t proactive – it rarely has been. This is one more nail in the coffin.

    The question is, how are you going to respond? Are you still doing single sites for each listing and adding to Google under “add URL?” Are you using systems like postlets.com and others effectively? Sometimes agents are going to just have to work around stupid obstacles.

    Someone in one of the emails commented that they just wrote an offer, instead of “SEO’ing” that’s very agent-centric thinking, unless you’re solely a buyers agent. Listing agents should be very knowledgeable about SEO and online marketing techniques. Just my two cents.

  158. I emailed Cliff, cniersbach@realtors.org, and he responded very quickly. If you have not done so, please do so now and let him know how you feel about the issue.

  159. Note from AG staff: If you would like to voice your opinion regarding this ruling to NAR, please email Cliff Niersbach. You can also email Steve Sullivan, MIBOR CEO or call him directly (317) 956.5000 ext. 237. Please keep your words professional if you choose to take action and be sure to reference this article.

  160. @Braxton Beyer Do you care to post his response? I would be curious to hear what he has to say before contacting him myself.

  161. Hi Everyone.

    I’ve been reading this thread in between many appts yesterday and today, and very much want to (and will) go back and read the entire thread.

    WHAT CAN BE DONE REGARDING “GETTING INVOLVED”: Here’s my 2 cents….on this. I have written a concise email to Cliff asking if a meeting can be held at #midyear in DC next week to discuss this. I just hit the send button, and make the same request to Cliff herein this post. The best “first step” would be a meeting at #midyear organized by NAR showing their interest in working with and for their members. In the alternative, if NAR indicates that this is not doable for them, those of us who will be at midyear should gather ourselves and discuss what we….as members want to collectively communicate to NAR.

    Deborah Madey – Broker
    Peninsula Realty Group – New Jersey
    732 530-6350

  162. Follow up to my post…..
    Who will be at midyear? please note here or email me Deborah [@] PeninsulaFirst dot com

    Tx
    Deb
    732 530-6350

  163. My recommendation is to send anyone you know who will be in DC Thursday of next week to the MLS Policy meetings at 9 am to 12 pm. I expect this issue will be discussed there. I’ve commented over at the FBS Blog that I believe a key issue here is that the interpretation provided be changed and that this not require a change in the actual policy. As others have commented, changing the policy could take a long, long time to get through the process, but the interpretation was issued and can be clarified relatively quickly. This isn’t a policy change, but rather an interpretation. I suggest the language used (“scraping”) in the current policy does not need to be changed at all. Rather, the interpretation of that term is what needs to be changed and that doesn’t require the entire process of a policy change.

  164. Perhaps in lieu of (or in addition to) emailing Cliff, a petition could be created and circulated asking for NAR to review the pertinent portions of NAR policy dealing with this issue. A few hundred thousand signatures couldn’t hurt. Just a thought…

    I’ve spent the last few hours, (yes, HOURS. I have no life.) reading this post and every comment. I’ve gone to Morgan Carey’s blog and read his exchange with Cliff Niersbach. I’ve read Cliff’s “official” response here, which is pretty much summed up in his response in these comments. Here’s what I know:

    My head hurts. Bad. I need medication, or an intervention, before my brain melts into a pool of yucky gray matter and runs all over my laptop. The stupidity of this decision confounds me. The logic escapes me. How can a large, national organization that is supposed to be doing everything it can to make it’s members more successful suddenly doing something that is so Neanderthal in scope and consequence? Why can they not see the harm this will do to the members it so boldly claims to serve, but those members certainly can? Their entire argument is based on the premise that:

    “The Center for REALTOR® Technology (”CRT”) advised that while the intent of “scrapers” may be malicious, and the intent of “indexers” good, the two practices from the Web server’s view appear to be the same. Consequently, NAR staff responded to questioners that the requirement to prevent scraping includes indexing.”

    From the web server’s view??? Are we now making policy based on what an inanimate object decides is good and bad? Whatever happened to applying good old human reasoning and common sense to a situation to resolve it? Those CRT people should have said, “Well, the web server sees scraping and indexing to be the same, but it’s not capable of making an ethical or moral judgment in this case , but we are. We know that indexing is a good thing, exactly the opposite from scraping, so we, in our infinite wisdom, hereby decree that indexing shall be allowed so our hard-working, innovative members can do their utmost to uphold their fiduciary duties to their clients and expose their listings in as many places as possible. So say we all.”

    THAT would have been the logical, smart, HUMAN way to interpret the rule, thereby ensuring that the people they are supposed to be serving and supporting have all the tools at their disposal they need to effectively market and sell their client’s property. But NO, they let a machine do their thinking for them. If the server says scraping and indexing are the same then, by God, they’re the same, and who are we to argue with a Dad-blamed machine?? Ahh, my head…

    Well, here are some other things I know (or at least BELIEVE):

    1. Left unchallenged, this decision will stand and become a precedent for other MLS’s and Boards to emulate;

    2. Debating the point endlessly here and on other forums will not, by itself, reverse the antiquated and narrow-minded thought processes of the NAR (although it does get the word out about the issue);

    3. An email to Cliff Niersbach will not, in and of itself, convince NAR to reverse this ridiculous position. It must be supplemented with stronger, more focused action;

    4. This decision by the NAR has finally and irrefutably shown that they do NOT have the best interests of their members at heart. They are so wrapped up in politicking, regulating and pompous chest-thumping that they have lost sight of the lofty purpose they so glibly espouse, that of making us more successful. As Jay so eloquently said, “Yeah, whatever”.

    5. It’s time that a concerted, grass roots effort to change the direction that the NAR is headed in be formulated and initiated before we as Realtors(r) are regulated out of existence. I know this sounds drastic and perhaps even a little melodramatic, but I for one am not happy with paying dues to an organization that has become so blind to the needs of its constituents. The time to take back “our” organization is now, before the shackles become too tight.

    OK, I’m done now. I didn’t realize I was hijacking this thread with this rant, but as others have said, this is an emotionally charged issue that affects the livelihood of every one of us, or will sooner or later. Benn, if you choose to delete this comment, I understand, but the sentiment will remain long after the words are gone. Time for me to write to Cliff Niersbach now.

    And my head still hurts…

  165. Cliff responded to my email……….

    The MLS Forum is held immediately before the Multiple Listing Issues and Policies Committee. Attendees at the Forum have the opportunity to comment prior to the Committee’s meeting so members of the Committee can benefit from those comments and insights.

    The MLS Forum will be next Thursday, May 14, 2009 beginning at 9:00 a.m. in the Regency Ballroom, Lower Level, Omni Shoreham Hotel.

    I will arrive in DC on Wed afternoon.

    Deb
    732 530-6350

  166. I’m “in” for the policy-updating effort, too! Now that we’re arming ourselves with appropriate e-mail addresses — for Cliff, local MLS and AoR, etc. — here’s to collaborating at NAR Mid-Year in DC (preferably Wednesday or Thursday 5/13-14/09, since I’m nearby in Northern Virginia) and soon!

  167. @Madison real estate I have posted the thread here. Hopefully they will be discussing this at midyear as Deborah mentioned.
    http://braxtonbeyer.com/2009/05/email-exchange-regarding-nars-recent-decision-to-back-mibor/

  168. @tcar writes – “not every REALTOR agrees that this data should be open. NAR policy is written by it’s members”.

    With the technology available today why does NAR still use a committee approach to major policy and procedure guidelines? Why not after committee recommendation an every member vote with majority rule, then at least we would all have an actual voice, instead of being (in my opinion) under-represented.

    And before someone preaches at me to “get involved”, I serve as a director at my local association, but feel that we have very limited voices on a national level.

  169. [Update]

    This issue will be discussed at NAR Midyear in DC per Cliff

    “The MLS Forum is held immediately before the Multiple Listing Issues and Policies Committee. REALTORS® attending the Forum have the opportunity to comment prior to the Committee’s meeting so members of the Committee can benefit from those comments and insights.

    The MLS Forum is next Thursday, May 14, 2009 at 9:00 a.m. in the Regency Ballroom, Lower Level, Omni Shoreham Hotel. The Committee meeting follows immediately in the same room.”

    Please continue to voice concerns here and to parties involved- contact information is at the bottom of this post.

  170. Of note of the update on the event that your comments are PRIOR to the actual committee meeting.

  171. I forwarded this post and thread to my contact at Google and he is forwarding it his real estate contact (that I met at NAR in Orlando).

    So I am confident they will read it.

    As a 7 Year Director on our Ann Arbor Area Board of Realtors here, we have often had to contact NAR for an opinion.

    My guess is they responded, and decided to stand with the local board, not thinking through the implications.

    This should be a call to action to get involved with your local associations, serve on committees. That is the way up to get involved at the State level and National level.

    Of course this is a bad policy and should be changed asap. Not fair at all to us and how we market the homes for our sellers.

    Not sure how many folks involved in committees at NAR are that technological like here on AG. Forward thinking people see things first and the implications.

    Regardless, I don’t think the policy is meant to hurt us just not well thought out with long term implications.

    I can tell you if you depend on a committtee to change this it will take forever, we made decisions yesterday at our BOR meeting that had to be made quickly and did not send it back to the MLS committee. So it can be done by the Directors.

  172. This is what happens when you have a bunch of real estate agents that turned into policy makers on an issue they don’t know enough about. While the rest of the world moves forward in the sharing of information, they are moving backwards. If they would look at how the recording industry failed at keeping music from being shared, they might learn a lesson.

  173. As already noted, The MLS Forum is held immediately before the Multiple Listing Issues and Policies Committee on Thursday, May 14th, 9:00 a.m. in the Regency Ballroom, Lower Level, Omni Shoreham Hotel. The Committee meeting follows immediately in the same room.

    After a considerable amount of behind the scenes discussions here at NAR, we decided that the most effective way to assure the MLS Committee sees how this verbiage is effecting agents would be to have them engage two members who really understand it’s consequences.

    I have invited both Paula Henry and Jay Thompson to speak before the committee. NAR will be covering their travel expenses as well. Lots of items are on the agenda, this will not be the dominant topic discussed, but Jay and Paula will be able to bring this issue to the committee’s attention.

    If you are interested in getting more involved, try serving on a committee. Here’s a couple links to help you along.

    http://budurl.com/serveonacommittee
    http://budurl.com/mbrhome

  174. I would also appreciate if you would consider adding Michael Wurzer to that list- I believe he has an outstanding grasp of NAR, IDX, and this issue.

  175. Thanks for the update, and I hope the meeting is open to the REALTOR’s as I plan on listening in. Promise, I’m not an agitator.

    Omni Shoreham, not the Marriott Wardman Park. Got it!
    -doug

  176. Ummm…why don’t one of you call a lawyer and file suit…that would probably tie things up for a year two

  177. @TCAR thanks for getting face time for Jay and Paula at Mid-Year…

  178. This is a good first step – I’m happy to hear that Paula and Jay have been invited to represent our voices.

  179. I’ll be there on Thursday. Thanks, @tcar and NAR for addressing this so quickly.

  180. I, too, am happy that Jay and Paula will be speaking on our behalf. What concerns me is that this issue will not be the dominant issue being discussed. Makes me wonder what else could be more important at this particular moment in time, and how how much time Jay and Paula will be accorded to voice their concerns. Let’s hope it’s enough…

  181. Todd – THANK YOU. It’s greatly appreciated.

    All – I’ll do my best to represent everyone’s interest. Feel free to email me at jay [@] thompsonsrealty.com if there is something specific you’d like me to know and don’t feel comfortable posting it here.

    Deb (and others attending Midyear) – I don’t get into DC until late Wednesday afternoon but would love to hook up either Wednesday evening or early Thursday morning (those that have seen me early in the mornings should know I have a strong preference for Wednesday evening….)

  182. @The Harriman Team – It has something to do with how short sales are treated. I don’t know any of the details other than that it is an issue with lots of attention right now. I just want to set the proper expectations here. You eat an elephant one bite at a time.

  183. @Todd. You are correct that it’s not your job as Social Media Director, to make rules at NAR, and no is expects that of you, but you can act as a conduit for the group as a whole, and explain why everyone is so outraged over this. Moore’s Law still stands, as it did when Gordon Moore first voiced it. Tech moves at warp speed and although it’s nearly impossible to remain parallel to tech changes, smart companies and organizations do their very best to be kept aware of those changes and act accordingly.

    You CAN tell the powers that be at NAR that the times have changed, a lot, and NAR must work with the new rules of the day. After all, this is 2009 and things will be different even by year’s end, so NAR if they care about what’s best for us — the people paying their salaries — should be listening to this dialog and taking it for what it’s worth and I certainly don’t want it to be people screaming, but people thoughtfully telling them they need to move into this year with tech.

    The people who really understand what this means is really only a small percent of all of the NAR membership, but this is a group who is very powerful online and if gathered together could possibly make real change. We need to do that thoughtfully, quickly and if possible by the time the mid-year meeting takes place.

    Remember, we must do this with a well thought out plan. Jay you have been the most thoughtful of anyone writing this stream. Maybe you could put something together that we could use?

  184. Ack, I stuck an email address in my last comment which sent it to moderation Neverland.

    Todd – THANK YOU. It’s greatly appreciated.

    All – I’ll do my best to represent everyone’s interest. Feel free to email me at jay AT thompsonsrealty DOT com if there is something specific you’d like me to know and don’t feel comfortable posting it here.

    Deb (and others attending Midyear) – I don’t get into DC until late Wednesday afternoon but would love to hook up either Wednesday evening or early Thursday morning (those that have seen me early in the mornings should know I have a strong preference for Wednesday evening….)

    I have to say this — given my “somewhat vocal” expression of opinion here and elsewhere, the fact that the NAR is willing to send me to DC to share that opinion directly is a pretty decent thing to do, and I think it signals a very positive step.

    Unless of course, the real plan is to meet at the airport and dump my body in the Potomac River. ;)

  185. Thanks for that info, Todd! At this point, we’re just thankful that the elephant is even on the menu…

  186. Jay / Paula – if you guys need any analytics data or information of other kind let me know if I can help – I think you both know my email and you can call me at 250-753-9893 as well.

  187. I also emailed Cliff Niersbach (before I was invited to Midyear) and got a swift response. I’m posting it below, with the permission of Cliff . Of note, Cliff did copy the Committee leadership, and included my original email. So (assuming they read it and I don’t doubt they will) they did get my input:

    On Thu, May 7, 2009 at 12:07 PM, wrote:

    Good afternoon REALTOR® Thompson:

    I very much appreciate your comments and your suggestions related to the IDX policy as it deals with scraping.

    I will, by copy of this email, share your thoughts and suggestions with the REALTOR® leaders of the NAR Multiple Listing Issues and Policies Committee for their careful consideration. I anticipate this issue will be discussed in the MLS Forum and the Multiple Listing Issues and Policies Committee next week.

    It should be understood that the focus of the rule in question has never been on blocking indexing by search engines. That potential effect was not contemplated when the rule was adopted by the NAR Board of Directors in 2005 and only came to light in the past few weeks. Simply put, the issue is whether – and how – indexing by search engines can be accommodated while at the same time clearly and objectively distinguishing that functionality from the scraping the IDX policy prohibits to protect MLS databases from misuse and misappropriation. Hopefully that can be accomplished with input and insight from thoughtful REALTORS like yourselves.

    Cliff Niersbach

  188. Jay, that was the exact same response he emailed to me. Word for word. I’m sure he couldn’t craft a personal response to everyone, though. But it was, as you pointed out, received quickly and copied to NAR leadership. Good luck to you and Paula in the District of Corruption!!

  189. I’d say this whole discussion is a textbook case of why every association needs to utilize social media. Knowledgeable members were able to discuss and identify a concern, and Hillary and Todd were able to participate as NAR representatives. Other NAR departments got involved, and the response was timely and fair. A good step forward in understanding the issues and–we can hope–resolving the problem.

  190. Hi,

    The MLS policy forum on Thursday morning will have many attendees with a multitude of concerns. In support of grass roots efforts to make a stronger presence and unified message during the forum and committee meeting on Thursday, those interested in a an informal meet up on Wed evening, Lee Ellis will help us organize a time and place. Anyone interested in meeting like-minded people who support “involvement” “good policy making” and “social media” along with a few drinks, laughs and maybe a bite to eat…….please come! Jay indicated a preference for Wed eve…… Surely, we hope Paula will join in…..as in will everyone who has a passion for good MLS policy on IDX. Stay tuned for info from Lee Ellis for details. Anyone with any comments or suggestions?
    Deb Madey

  191. Deborah/Jay/Paula,

    any chance someone can send updates on Twitter to keep the folks who can’t attend updated somewhat? If not during the proceedings, then maybe afterwards while you’re celebrating your victory (and before the party gets too involved! LOL)??

  192. Am I over simplifying this by looking at the needs of the actual home sellers?

    I honestly believe that the sellers really don’t care how a buyer finds their home on the internet as long as they see the home and have a way to find more information about their home.

    The listing agent is always happy to recieve an offer from an interested and qualified buyer regardless of where they saw the home advertised. Print media was for along time the main source of information for buyers and they would scour through the magazines making notes of homes they liked, listed by numerous realtors, and then present that list to their realtor and then ask him/her to find out more about the homes and set up showings if the homes were still available.

    The downside with the print media was that as soon as the publication was printed the information was out of date which is why the internet has become so popular for buyers researching information on homes for sale.

    I still believe that buyers utilize the internet, browsing from one realtors site to another, much like they did when picking up 4 or 5 of the Homes for Sale magazines and when they have the information they want they still present this to the agent they have chosen to work with.

    It would seem that restricting the the IDX sites from displaying the available homes for sale, in terms of the way the search engines pick up the data, would really hurt the actual sellers of the home and the listing agent as now the homes exposure on the internet has been reduced thereby reducing the number of potential homebuyers who can see the home.

    Hopefully I haven’t missed the point too much as the whole concept of scraping etc is a bit above me but if I were selling a home I would want it to be seen as much as possible and as long as I was dealing with an offer from a qualified buyer I’d be happy to sell my home to the best offer I got.

    Gary :)

  193. @tcar No one is mad at you, they are however infuriated about the issue at hand and while you are the NAR social networking czar / communicator what you say will be interpreted as being the gospel. Don’t forget the Michael Russer, aka “Mr. Internet” article published in the official REALTOR magazine. He was seemingly a spokesperson for NAR and many followed his advice only to be trampled on by the very organization that publishes the magazine.

    Additionally, you distanced yourself from R.com in a comment above and stated that your were involved with NAR not R.com. Many REALTOR’s associate R.com as being a part of the NAR, is it, or is it not? I never have understood how Trulia, Zillow, et al can provide great display of content fed from the broker at no cost and linked back to the listing agent, but R.com charges a huge premium for a lesser good fed directly from the MLS. Why isn’t R.com run for the benefit of all REALTOR members? Anybody here ever get a lead from R.com that wasn’t paying the premium? I never have.

    @Jay, can you get the MLS Committee to touch on R.com too? I don’t think R.com should get MLS feeds unless they treat all REALTOR members equally and inclusive of their present NAR fees. Otherwise take their feeds away. I may be wrong, but I don’t think they are us! You could certainly get clarification on this.

  194. Go Jay & Paula! Thanks in advance for representing our justifiable outrage and utmost concern about this issue. And please send many Tweets before, during and after the meeting – if possible!!

  195. Take a step back and look at this… within the past 24+/- hours:
    – Paula provided all NAR members with an issue of importance and immediate relevance
    – Benn & Lani host a great forum for information and public debate
    – NAR SMM (Todd Carpenter) does his job by paying attention to blogs and NAR members’ comments
    – Jay, Daniel, Paula, Morgan, Matt and a host of others share great concerns
    – NAR makes a formal announcement within this forum and provides a contact email
    – More great debate amongst thought leaders
    – NAR steps up and gives a nice gesture by flying Jay and Paula to the meeting for change
    (now we know where our dues dollars go)

    Isn’t this how social media is suppose to work for real networks (like a realtor association)? There is NO CHANCE this would have happened even 1 year ago. I feel this proves @tcar’s new position is an invaluable resource for NAR to engage it’s members more effectively. Just think of how much better this will go now that one is under our collective belts.

    Kudos to everyone who participated in the new web of change. (even Jason, who doesn’t wear regular shoes, got to enjoy jokes with a real rockstar – very cool)

    I like to call this process: CHAOTIC GOOD

  196. If you dig this thread, you’re invited! Meet colleagues keen on good policy-making at the “REALTOR Mixer Before the May 14 MLS Forum” at Petits Plats in Woodley Park, DC, Wednesday evening. Check Face book for event details or contact Deb Madey, Deborah [@] PeninsulaFirst dot com, or Lee Ellis, Lee [@] HomeInNOVA dot com, for more info.

  197. @Cal I wasn’t hired to be a spokesperson for NAR. When I tell you I’m stating my personal opinion, you should trust that I am. For people to accept my personal statements as NAR gospel is a mistake. If you look at my first comment, you will see that I do not fully agree with this policy.

    I don’t work for REALTOR.com. More specifically, I don’t work for Move, who operates REALTOR.com. I’m not at all qualified to speak for their product. If you want REALTOR.com to be involved in this conversation, I recommend pinging them on Twitter @realtordotcom.

  198. I heat @tcar, just saying.

  199. opps heart… lol

  200. Sorry, but with the comparison to the music industry, this laughable attempt immediately made me think of NAR and this situation.

  201. NAR can pick on us because we pay them to that. I wonder at what point will they do the right thing and look after our best interest.

    They will not even lean in favor for us on such a simple issue.

    I have to wonder what is in it for people who make decisions against their supporters. That is what concerns me.

    Do we not have enough working against us already with national sites that do not pay dues but use our listings to convince the consumer that they actually have better data than us?

    Removing our rights with the core idx will just tie our hands behind our back and make it more difficult for us to communicate to home buyers and sellers. This will make it harder for us to communicate with our prospective clients, and let companies who not even sell real estate or support our cause have better technology to market to our clients.

    The real fight here is who has the rights to our prospective clients?

    It looks as is if some people think that sites belonging to companies outside of NAR (non Member). If we let them hobble us now.

    More agents will have to rely on buying leads from companies who have better technology than us. That will only be if we give up on this. This is one fight we have no choice but to win.

  202. Deuce.

  203. This is so old hat, I am surprised the board actually thinks it can put the genie back in the bottle. I do recall the old books of our listings we eagerly awaited to get. Heck, I remember our first 300 baud modem, my my weren’t we advanced back in 1989 ?

    To have one’s information indexable makes it so much more easily found. Isn’t that why we use google – to find stuff. The information is public, so is it then someone wants a royalty or link back?

    Maybe electing a more modern forward thinking board is the solution? I am surprised about NAR, but we as a collective business have always been behind the technology curve. I remember we had supposedly our first all electronic( paperless) transaction in 1989,Florida if I recall.

    Heck now we have our documents and contracts on line, but we still have to print them out to get them signed. Anyone sign on the pads at any department store……. we have an industry that is slow to change and these aren’t earth shattering things that need to be updated. I am glad at least my office has all of our transactions online. That way I and my clients can access our data from anywhere whenever we want. Isn’t the point to make things easier and more efficient?

  204. A good majority of our team’s business revolves around representing sellers. We met with over 150 sellers last year and not one of them asked us:

    1. What techniques we would use to hide their property from as many buyers as possible.
    2. How we would not cooperate with other brokers so we could keep both sides of the commission for ourselves.

    I can assure you that as an agent of the seller, I want the following:

    Article 1
    When representing a buyer, seller, landlord, tenant, or other client as an
    agent, REALTORS® pledge themselves to protect and promote the
    interests of their client.
    © 2009, NATIONAL ASSOCIATION OF REALTORS®, All Rights Reserved

    And when promoting the interests of my seller, I want their listing on as many websites as possible and in front of as many buyers as possible(as long as the information is accurate).

    Do I take issue with a nationwide company displaying inaccurate scraped information……Absolutely

    But complaining about an individual/broker displaying accurate information about my listing through a data feed given to them by my own board of realtors…..give me a break.

    MIBOR, you should be ashamed of yourselves. Maybe you should be spending more time creating public awareness about why it is such a great time to buy real estate instead of listening to a bunch of cry babies whine about why they can’t compete in this age of technology.

  205. Well said Matt…

    How the decision by MIBOR has anything to do with protecting the consumer or protecting the best interests of it’s members is beyond me.

  206. This post is a perfect example of why I read AG and why it is important.

    Special thanks to Jay Thompson, Benn Rosales for their insight and the willingness to communicate them.

    I have to add that the following brilliant comment from Daniel Rothamel belongs in the history books:

    “On a side-note: the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE. It is completely indefensible. The fact that our National Association controls the use of its data by its paying members, but not by a private company, is a disgrace.”

    And I want to add that I totally appreciate NAR’s response to this thread! :-)

  207. Russell:

    I completely agree with you about Daniel’s side note. I gave him props for the statement in my first comment, but it certainly warrants more attention.

    On a side-note: the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE. It is completely indefensible. The fact that our National Association controls the use of its data by its paying members, but not by a private company, is a disgrace.
    – Daniel Rothamel

  208. Reflecting on Brad’s comment

    Now the real work begins. It seems that we have convinced each other and enough NAR staff to have the conversation – In Real Life – with the 112* members of the Committee.

    Here’s my suggestion –

    If you’re a Realtor, go to this page for the Multiple Listing Issues and Policy Committee.

    Find members from your state – EVERYTHING IS ABOUT POLITICS – and contact them. Fast.

    Respctfully find out their position and awareness of this policy and educate them. Equally fast. Send them resources to educate themselves. Heck, does anybody want to put together – quickly – a simple fact sheet of talking points that we could send them? I will be contacting the three members from Virginia and encourage others to do the same in your respective states. Personally, I know the three from Virginia and know them to be open minded and informed on current matters … and hope that they will listen and act on Thursday.

    If you want an idea of the vastness that is the NAR bureaucracy – check out the Committee page on Realtor.org (which I found after nearly 10 minutes of digging and clicking)

    If this is about members helping members, now is the time for us to take our actions from the blogs to the members.

    * I may have lost count.

  209. Jim, I’d love to see that list to see who I know but do not have access to Realtor.org.

  210. GREAT idea Jim! I looked up GA and I know our 1 committee member and will be glad to reach to him. It would be very beneficial to have a One-Sheet to help us all stay on the same page and deliver a consistent message to committee members. Any takers on assembling that doc?

  211. Michael, here is that list you requested: http://maxsell.net/nar2009mlscommittee.xls

    Jim, let me know if you find some help to get the talking points together.

  212. Here would be my suggested talking points:

    1. Recently, a broker in Indiana was forced by their local Association to prevent Google from linking to (indexing) the IDX listings on their site.

    2. This means consumers can no longer find the IDX listings through Google.

    3. The local Association said Google couldn’t index the site because they believe indexing is the same as “scraping”, which is against their IDX policy.

    4. Indexing by Google is NOT scraping. Google links back to the source site and drives consumers back to the site. In contrast, the anti-scraping IDX policy is to prevent someone from stealing the content to present it as their own. These are very different results.

    5. The local Association reached their conclusion based on advice from NAR and so NAR’s MLS Policy Committee should clarify this advice immediately for other local Associations facing similar issues.

    6. The Committee should clarify that Google is not a scraper.

    7. If that clarification cannot be issued immediately, the MLS Policy Committee should state that local Associations are free to interpret the term scraping as they see best and does not, at this time, have a final position on the issue.

  213. Michael…Well stated. A common sense approach. Let’s hope that NAR simply will respond with “my bad” without any overreach.

  214. Go for the fight! This issue is a huge topic that will affect all of our business! I hope the real reason Paula and Jay are invited to talk is because there is REAl concern for US the REALTOR and not because they recognize there is actual concern and talk ON LINE that will not go away.

    I completely believe because of their relationship with MIBOR and lets not forget Realtor.Com fits in there somewhere, they have closed their eyes to who they are representing, US !

    This could be a main topic for a long time….. how many consumers will be denied true access?

  215. Michael, are you going to midyear? I think you bring up some excellent talking points to be taken into consideration. Make sure to bring up that R.com scam as well, big props to Daniel on that one.

    I wonder why some of those aggregator sites haven’t stopped by yet? Unless they know something that we don’t.

    Whew, I thought a riot was going to erupt… I guess what it takes to be heard.

  216. Perhaps while you are in front of them:
    http://activerain.com/blogsview/1065941/homes-on-realtor-com-without-being-on-local-mls-

    Where is the policy on this? I would have emailed but don’t have everyone’s email address. So feel free to delete if I am being bad.

    Jim, I have been on realtor.org for an hour and can’t find all the state members to call. Very cumbersome.

  217. Jason, I will be at mid-year. Good point about R.com. I’d phrase it like this:

    4a. Treating Google as a scraper under the IDX policy puts broker and agent sites at a serious disadvantage on the web compared to sites like Zillow, Trulia and Realtor.com not subject to the IDX rules.

    The aggregator sites haven’t said anything yet because it’s like a train wreck — too frightening to get close but impossible not to watch.

  218. Well since this is apparently going to get in front of the MLS Committee, I propose that R.com be included in the conversation and taken back for the benefit of all REALTORS.

    If the MLS is going to feed that monster, then it should feed all listings with all photos and all virtual tours for all listing agents with links back to the agent and no additional fees for the agent members.

    If they want to monetize the thing, let them stick some Google Adwords in the thing and non agent real estate service providers and Trulia and Zillow can pay for their ads to show there!

    The monster has to be a huge profit center for somebody, who is it? I feel as if I am a member of a giant co-operative, but I haven’t received my rebate check on those profits yet, have you?

  219. @Jason – the agregators are watching – check out my “MyBlogLog” widget here – Trulia personnel would not have found their way here otherwise -mygulfcoastbeachteam.com

  220. My advice: when in front of the MLS Policy Committee, stick to MLS Policy. The Realtor.com issue is a whole different can of worms and will probably distract everyone from the point you want to accomplish.

  221. Better late than never….DOH.

    I think Brad Nix summed it up nicely. NAR drops one in the punch bowl. Members respond, passion runs hot [a good thing], Tcar does his thing [grace under pressure], NAR listens, invites in-person feedback [Paula/Todd}. It all happens fast…it’s not over…Lani provides contact information, more pressure, conversation, explanation, clarification and education is/should be applied….the cogs turn and we’ll see how it shakes out. I predict they’ll get a clue and amend.

    Moreover, this post/thread/will be used as a supreme case study example of how Social Media & Community effect change, educate and impact consumers, practitioners and what ever you’d call NAR.

    Fascinating.

    Thank you all for you insight, I’ve learned some things and my world view on this issue has expanded.

  222. You are right Judith, it has just bugged me for a long time that agregators can and will display all you want for free and our own namesake URL charges huge premiums for a lesser product. Should be the other way around.

  223. Forgot to subscribe.

  224. Judith said

    when in front of the MLS Policy Committee, stick to MLS Policy

    true, but what should be consulted is what Daniel said about the fact that REALTOR.com uses the same data that brokers use, from the same source (the MLS of the local association), but are not required to adhere to the same rules, is UNCONSCIONABLE <— this needs some massive attention

    Who knows… maybe we can kill 10 birds with 1 stone. I agree with Missy in R.org being such a cluster bomb as well.

  225. @Cal you said: If they want to monetize the thing, let them stick some Google Adwords in the thing and non agent real estate service providers and Trulia and Zillow can pay for their ads to show there!

    I agree and the recent deal with R.com and C21 doesn’t make them look any better.

  226. “If the MLS is going to feed that monster, then it should feed all listings with all photos and all virtual tours for all listing agents with links back to the agent and no additional fees for the agent members. The monster has to be a huge profit center for somebody, who is it? I feel as if I am a member of a giant co-operative, but I haven’t received my rebate check on those profits yet, have you?”

    @Cal Carter Well put. VERY well put.

  227. Judith Lindenau said “My advice: when in front of the MLS Policy Committee, stick to MLS Policy. The Realtor.com issue is a whole different can of worms and will probably distract everyone from the point you want to accomplish.”

    I can’t agree more. I’m not even sure this committee governs the realtor.com agreement. My pure guess is that it doesn’t. Even if it does, Judith’s advice is spot on.

  228. Regarding the Realtor.com/NAR relationship/debacle/fiasco/almostcriminalactivityagainstNARmembers:

    I humbly suggest we take this one step at a time. Realtor.com absolutely is a festering sore on the relationship between Realtor members and their/our trade organization, NAR, but let’s fight this IDX/scraping battle first, learn from our experience and contacts and then attack the problem in an organized, cohesive fashion.

    We need to present a solution to the NAR such as, “let’s buy Realtor.com – Their stock is two bucks a share and then fold them into the NAR as a not-for-profit entity that benefits all members who are contributing their content and listing data to Realtor.com”

    Something like that.

    But with the behemoth that is the NAR, I humbly suggest patience, planning and a concerted, directed attack to solve the problem.

    I see getting on the agenda as a huge victory for us – the members who feel disconnected from our organization, and see this as a winnable battle.

    Honestly, I think that the Realtor.com battle is one for another committee and/or Leadership Team … or we could act locally to pull our respective MLS feeds from Realtor.com and seek to bankrupt them that way. Just a thought.

    (and @laniar I think that this may be the foundation for my post next week :) )

  229. hmm… tweet on twitter by Hilary Marsh about dues doubling if NAR bought back r.com

    for another day, something to look at I guess going to back to what Jim said.

  230. I feel like by this time next week I’m going to be a complete failure in a lot of people’s eyes.

    I think it’s fabulous that NAR has agreed to send Paula and I to Midyear to discuss the scraping/indexing issue at the MLS committee. And I am more than willing to take the time away from my business and family to do whatever I can there.

    But “killing 10 birds with one stone” and fixing the Realtor.com fiasco are WAY beyond the scope of what Paula and I could even remotely hope to accomplish.

    I won’t say the MLS committee doesn’t care about Realtor.com, but I don’t believe they have any jurisdiction over it. Even if they do, this particular battle is about the “clarification” that search engines are the equivalent of scrapers.

    I don’t really want to stand in front of this committee with a litany of complaints. I think we need laser focused attention on this one specific problem. As Todd said earlier, the way to eat an elephant is one bite at a time.

    We have to be realistic here folks. None of this is going to be fixed overnight. It’s entirely possible (probable in fact) that this specific indexing/scraping issue won’t be resolved next week. I can see some “focus group” being formed to examine the issue further. And I’ll happily join such a group.

    Those that know me know I’m passionate and I hate to lose. I will take those traits to DC next week and do my best to fix this mess. But please, let’s have reasonable expectations. The fact that NAR is sending two people there on such short notice is remarkable in and of itself. I hope I don’t let anyone down, but I’ll be honest — reading some of these comments makes me feel like I will.

    Jim Duncan’s suggestion to contact your state’s committee members is outstanding. Emailing Cliff Niersbach will get your own thoughts into the hands of this committees leadership.

    How many reading here have done BOTH of these actions? This is how grass-roots, membership driven change works folks. It’s not me and Paula standing in front of the committee. It is ALL of us letting our state committee members be aware. It is ALL of us sending an email to the NAR VP that asked for emails. (My suggestion – be polite, stay on topic — ONE topic — search engines are not scrapers.)

    I’ll do my best next week, but I’ll tell you right now — Realtor.com will not be fixed next Thursday.

  231. I feel like by this time next week I’m going to be a complete failure in a lot of people’s eyes.

    Yep. If you you don’t fix the MLS/IDX/Realtor.com/Health Insurance/Eminent Domain/Global Warming – you’re fired. :) Or at least we’ll dock your pay.

  232. I am sure you will do an excellent job as always Jay! Just because someone is given an opportunity to speak doesn’t mean they will be heard, but I do think they are listening. Regardless of outcome, you will not be seen as a failure, but as a well informed and well spoken advocate and spokesperson for REALTORS on the web. I don’t think they could have picked a better person to go on short notice and I am glad that instead of festering even longer within forums, the message will be delivered to the policy makers.

  233. Jay –

    I have a lot to respond to here and quite frankly, feel a bit overwhelmed. To be asked to fly to DC and represent my fellow REALTORS is quite an honor; one which I am committed to. Like you, I am busy rearranging my schedule, my business, closings and life to get to DC and present our collective point of view on this topic.

    It is important we stay on topic and accomplish what we were asked to do. Taking everything to the committee wll only confuse the importance of the task at hand.

    I have privately thanked you and now publicly want to say, thank you for your wisdom, passion and thoughts you have shared here. I can’t think of anyone I would rather have beside me in DC next Thursday.

  234. Jay/Paula, Your efforts are appreciated what ever the outcome. I believe you’re right, progress or instant change isn’t likely, you two/we are wrestling with a Goliath sized tar-baby. This whole drama is an exercise in action, reaction, chain reaction…. I’ll sent my email and stay tuned. Speed, persuasion and all the best.

  235. Jay — Despite the fact that you might “be a complete failure” in this attemptl…the mere fact that you and Paula are going to be presenting this automatically makes you heros in our book. All of us in this industry appreciate what you are attempting to do for us. And I, for one, have your back.

  236. @Jay

    Look, you’re right, but you’ll not be a failure if this ruling is at least given serious thought- and with you there, it surely will. Passion… lol anyway

    Here’s what’s in your favor- if the committee refuses to really educate themselves on this issue and take into account what the reaction of a small percentage of their membership is already, I assure you, the day that 1.4 million realtors lose their IDX freedom the committee will not have a choice but to get it- pitchforks & torches come to mind…. ponder.

    Thanks @mwurzer with flexmls.com for your input, I hope you will forward your thoughts to Cliff in advance of the meeting at the very least

    Thanks @jimduncan for your outstanding suggestion, I’ve forwarded the link to this post to several boards I’m close to do just what you recommended.

    Thanks @energizerthread :)

  237. Hey folks….just chiming in to offer support and agreement. There are lots of NAR issues…but the task at hand and focus for this MLS Forum (morning) and Committee Meeting (Jay and Paula in the afternoon) is the scraping/indexing issue. The support and faith in Paula and Jay to speak on our behalf is overwhelming. A huge thanks to Paula for setting the first ball in motion. I look forward to meeting you! As always, Jay is so spot on and willing to step up. Share thoughts, drinks and smiles Wed eve (before the Forum) http://www.facebook.com/event.php?eid=74635886294&ref=mf

  238. Since I’m not a REALTOR, and therefore voiceless in this issue, I’ve had great learning from reading this thread.

    I did want to make one small suggestion, however, for the One Sheet and for the MLS Forum. I’m sure it would have been done without, but just in case…

    Make sure you do not make this a Google vs. scraping, but Search Engines vs. Something Bad. There are other search engines than Google, although it may not feel that way sometimes. And there will be other search engines in the future that will one day take the place of Google, as Google took the place of Yahoo. (Wolfram anyone?)

    The larger question then if you guys aren’t going to have this whole discussion again a few years from now about some Expert Systems Search Engine that takes data, mashes it up with local blogposts, local data, financial analytics, then presents the Answer to the user rather than a longass list of URL’s, is to distinguish between whatever is Good Behavior (using the data to send traffic to the agent) vs. Bad Behavior (fraud, theft of IP, etc.).

    Also, I find the little sideline about @tcar’s role rather interesting. That will be addressed separately, and elsewhere so as not to derail this thread.

    -rsh

  239. One disadvantage of unplugging is you miss something like this every now and again …

    @tcar – I read you before you took the job. No way you would have made an argument like “I wonder how hard it would be to own “2920292 carmel” on Google” once upon a time, knowing full well that that’s not the issue being debated. Just sayin’.

    In general … and maybe this is why I unplugged … it’s great that Jay and Paula are heading to this meeting to discuss the issue. It’s great that 100 or so like-minded people (read: those who likely understand IDX versus the agents who continue calling me asking for status on the IDX listings they find on my site because they can’t work the MLS) who feel strongly about this issue.

    But there also are hundreds of agents and brokers who feel strongly that the data should be protected. And there are thousands who don’t know the difference.

    Two weeks ago I wrote on open plea for the local MLS to adjust and/or enforce their rules so that “active” listings actually mean “active.” Apparently, there’s a groundswell of support from listing agents who prefer having “active” be ambiguous for their own benefit. And, as I was told, the board has to balance both sides.

    The argument that members write these policies is valid, at least once you accept that the vast majorities of these (and all) members don’t have any idea what they are talking about. Or they’re so concerned for protecting their own little niche that they don’t care about the large-scale ramifications of their decisions.

    That a 2005 policy hasn’t been revised is laughable in its own right, given the leaps of technology seen over the past two years in the real estate world. One of the best things the Arizona Regional MLS did here was finally open up the IDX to multiple vendors. The notion that they’d then cut the legs out from under those who used these other vendors saddens me, but wouldn’t surprise me in the slightest.

    Jay and Paula, I wish you the best of luck as you tilt at the windmill. I hope you prove more successful but absolutely will understand if you’re not.

    And with that – and with a quick mi shebeirach to Matthew, who I hope is feeling better – I’m going back to my shack in the woods.

  240. Certainly, Realtor.com should NOT be exempt from the rules that any IDX site has to abide by, but what happens when a small broker like myself is advertising their listings to Google and a large competitor with higher placement is advertising the same listing? I own the listing and agreed to let my competitor advertise under their name on their site, not on google or other sites linked to it. Yet this is what’s happening. My listing shows up with their name on it, and leads I would otherwise be able to get are directed to my competitors site. Since not all brokers can afford to put up an IDX site advertising everyone’s listing (PDX RMLS provides no frame or anything else), I’m at a big disadvantage to the big name brokers here.

  241. @tcar – I read you before you took the job. No way you would have made an argument like “I wonder how hard it would be to own “2920292 carmel” on Google” once upon a time, knowing full well that that’s not the issue being debated. Just sayin’.

    Johnathan, I don’t think you “read me” well enough back in the day. Because I make these sort of arguments all the time. Just sayin’ as well.

    It was part of the argument in the original post. I think it’s worth qualifying what’s a stake here. This is an important issue. But quantifying exactly what is at issue was worth noting.

    Early comments like Matt’s caught my attention, “Do any of these people realize that a majority of buyers find their property via IDX or their agent searching the MLS? Stats don’t lie and there’s reason more and more agents are ditching the books and print ads.”

    If you already know the MLS number, or the address, you are not finding the home online. You found it in real life. I understand the reason’s why it’s worth owning SERP’s for addresses in your market, the issue is important. But my argument was designed to focus attention to that single issue.

  242. @Mary, the thing you are forgetting is if you have the listing your job is not to generate leads, its to sell that listing for the client. Dont you think having that listing come up on any realtor website a potential buyer visits is a good thing for your seller. What would your seller think if they read your statement?

  243. Doug, that is not at issue here. The issue is if those listing results that appear on all those Realtor websites can be accessed by spiders that can replicate/index/scrape, etc… Google included.

  244. I totally agree on leaving the Realtor.com issue OUT of this fight. This should be a single issue and we are there to WIN (not to “be right”).

    I will never give up on the Realtor.com issue. Never. But to matter, ABOVE ALL ELSE BE EFFECTIVE.

    The mere fact that Jay and Paula are going to NAR is – in itself a victory, even if a small one. And I believe that Todd is owed a big thanks, as well.

  245. @tcar, scraper sites should be shut down as they are found, but to call Google a scraper site is completely laughable. It is one of the more outrageous statements I have ever heard about Google.

  246. If you already know the MLS number, or the address, you are not finding the home online. You found it in real life. I understand the reason’s why it’s worth owning SERP’s for addresses in your market, the issue is important. But my argument was designed to focus attention to that single issue.

    Assuming that customers are not going to google/search for any and every piece of information about a particular property is assuming that customers are not doing their due diligence. Why try to limit the amount of information that they can find?

    There is a clear difference between scrapers’ intent and indexing sites’ intent.

  247. @tcar I guess I didn’t read you then and neither did some others who are equally perplexed.

    I’ll concede that someone googling an address found a home in real life (not so the MLS number, which rarely is found.) But if we put the consumer at the forefront – their ability to access the information – then it’s damn hard to argue there’s a problem with IDX sites.

    The central issue really remains the notion of Google as a scraper site, which is two steps beyond laughable.

  248. Hey Jim Duncan,

    You just coined a key summary statement, IMO…..

    “There is a clear difference between scrapers’ intent and indexing sites’ intent.”

    Lots of good points on the thread.

  249. This is typical of our main trade association. What a bunch of Neanderthals!

  250. I just picked up a buyer that was searching for Purple Pink Dinosaurs in Las Vegas… ;)

    Better enjoy it while I can before somebody makes a claim that it is a deceptive practice… LOL!!

    Anyways… I think this whole ruckus is just another example of policy makers not busy enough selling real estate so they become policy makers…

    And the technology minded agents too busy actually selling real estate (or dealing with all of the far more important issues taking place)…

    It certainly is refreshing though that some savvy agents will have a say in the upcoming meeting and the NAR has reached out for their input.

    Also nice to know that the short sale mess is the main topic…. A little bit late but it’s better to be late then never.

    Of course… it’s always better to be Proactive then Reactive but that’s for another discussion I suppose.

  251. There’s not a clear difference between scrapers and indexers. Not from the perspective that matters in this conversation. Because what matters here is what has to be done from a technology perspective to allow Google in and other scrapers/indexers out. Look again at NAR’s comment here.

    The Center for REALTOR® Technology (”CRT”) advised that while the intent of “scrapers” may be malicious, and the intent of “indexers” good, the two practices from the Web server’s view appear to be the same.

    Not from my view, your view, or the consumer’s view. The web server’s view. Web server’s can’t measure intent.

    When you open the gate, anyone can come in. Our IT department looked at this long and hard. It’s tricky. Alpha geeks with linux laptops were tasked to look at what the rule said, and apply it to how the web works.

    On top of that, what if you just wrote a clause that says “Google is okay”, and had the technological means to do that? What about the other search engines? Who decides? Plus, what happens down the road if Google, or some other search engine makes a serious real estate play? Will it still be okay to let Google index your IDX? Maybe it will be. But this is what we were grappling with.

    To really move forward, that entire clause might need to be removed instead of amended. What happens then?

  252. Dean Ouellette said, “@tcar, scraper sites should be shut down as they are found”

    Again, go back to the rule.

    Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database

    The rule says steps have to be made to PREVENT scraper sites from accessing the data in the first place. All our staff can do is work within the rules.

  253. @todd if all that jazz held water, then wouldn’t we have beaten back trulia, zillow, et al? The bottom line is the cat’s out of the bag, and tieing our hands behind our backs so that big brokers/MLS can own the marketshare at least above the agent is what you’re actually communicating to us lowly agents.

    I get how you are trying to frame this debate, but you’re not just talking to brokers here in this thread, you’re mostly talking to agents who need to put dinner on the table tonight.

    Please take my tone as sincere, I get where you’re coming from, but until you put yourself in our shoes, you’re never going to understand why we could care less about how a server reads data- everyone knows that if a scraper steals content, all it takes is a drop of a url to google and it’s gone, so anyone to inferring that someone can’t monitor that sort of thing locally is not going to fly with an online audience.

    It just isn’t that complicated out here in the real world man…

  254. Pingback: Online Home Search Technology Heading Back to the Ice Age if NAR / MIBOR Ruling Stands

  255. Todd – I hear exactly what you’re saying, but why would NAR make the ruling while knowing it’s not that easy of a decision? NAR should have let the local association make the decision and not make a call that could set a precedent for associations across the country until NAR had truly vetted this out.

    Also, I don’t think we need to solve the technical issues with Policy (in fact, I think that is impossible). Instead, we should have goals the membership hopes to achieve and best practices for achieving them. Writing a policy that dictates how a web server should perform is not where NAR should be concerned. NAR should be concerned with the goals of it members. It’s obvious that a number (at least those here) of the members want ‘indexing’ to be allowed and not ‘scraping’. I think it’s possible to write a policy that allows indexing (or whatever you want to call ‘good’ stuff – and this probably means having an open policy as servers are concerned) and then include provisions to penalize, pull the feed, seek legal damages, etc… for those who practice scraping or intentionally misuse the data.

    I think of it like this… it’s illegal to steal another persons car, but the law doesn’t say we have to lock our doors.

    Isn’t this sort of like our data policy with MLS’?

  256. OK, so if we are going to make policy based on what happens at the web server level, then we need to ban web browsers, you know, Internet Explorer, Firefox, Safari, Chrome, etc. Because from a purely technical perspective, they do pretty much exactly what search engines and scrapers do.

    That’s why this “web server level” argument is absurd.

    And let’s face it, there is NO WAY to block malicious scrapers short of making a rule that NO information can be displayed on a web site.

    Unfortunately, you can’t “legislate” common sense, and you can’t write enough lines of policy to prevent misuse and abuse.

    Personally, I don’t care who or what indexes my listings. I list a home for one express purpose — to sell it. The more sites it’s exposed on, the better. You want to advertise my listing on the web, in the Sunday paper, on a billboard? Please, knock yourself out. The more my listing is out there, the better chance it has of being sold.

    If I EVER go to one of my sellers and say “I’m only going to put your listing on my site. That way I get all the calls and can either 1) double-side the transaction; or 2) generate leads from your home, then please take my license.

    I don’t list a house to generate leads, I list a house to sell it. So what if Trulia, Zillow, realtor.com or “Joe’s Real Estate Brokerage” ranks higher for my listing than I do? It’s exposure for the listing, and that gets it sold.

    But wait, what if I want to double-side the deal? Then maybe I do want to be the only site people can find the listing on…. Hmmm… perhaps that’s a reasonable argument.

    And it’s also why dual agency should be eliminated.

    But that’s a whole ‘nuther can or worms.

  257. I’ll probably get railed on for this, but here goes …

    It seems to me that NAR is not calling Google a scraper site but considering what Google does as “scraping.” This may, in fact, be the truth depending on how you define the term. Google does scrape the text from our web sites to index it. Does this make them a “scraper site?” Absolutely not. Are they scraping? Yes. Obviously the intent of Google is not to steal content as the malicious “scraper sites” are BUT Google DOES sell ad space on their search pages just the evil sites do, so just how different are they?

    A lot of people here are concerned that if this ruling stands that our individual sites will have to compete with Truilia, Zillow, et al. Well, who’s fault is that? We GIVE them out listing data! They are not IDX sites – they get their listings from US! We have created a monster that many of you are afraid of! R.com isn’t an IDX site either. They’re the same beast as the others except that they get their feed directly from the MLS. Even if you don’t pay for enhanced listings your brokerage name is on the listing and they can find you that way.

    If you are concerned about your web site ranking at the top of Google why would you rely on an IDX feed to do that? If you do a single-property site OR write a keyword-rich blog about your listing you’ll get there. Personally, I am of the belief that it is my job as a listing agent to get the property sold for my client. I don’t care if a buyer finds the listing on another site as long as it gets found.

    I believe that IDX is a tool to be used by buyers to find a house while they’re on a particular site. I really don’t see the point of having an IDX that gets indexed in the first place. Maybe I’m missing something on that point, but my goal would be for my web site to rank high for the particular set of keywords I am aiming for and not necessarily for a particular property.

    Is the concern of the masses here the fact that your listing is found on another agent’s site before yours or that your listing won’t be found at all? Believe me, your listings will be found … So what if a co-op broker brings the buyer? If you are looking for buyers rework your site so it ranks higher in your area – post local info, blog about your listings … Sorry, after reading all the comments here and on other related blogs, I just don’t understand the issue.

    NAR and MIBOR isn’t saying IDX is not allowed – they’re saying that the IDX feed should not be plastered all over the internet. The way Google works, it scrapes the text from your site on a listing that may be owned by another broker, links back to your site as if you were the owner of the listing, and the true owner of the listing isn’t revealed until they are on your site. You have effectively drawn in a potential buyer by advertising a listing that is not your own – which is illegal/unethical – and do not disclose the true owner of the listing until the buyer is on your web page with the full IDX info. It may not be your intent to misrepresent but that is what happens.

    The more I think about it, the more I’m leaning towards the NAR side here.

  258. Pingback: Friday Afternoon Theme Music - May 8 | Phoenix Real Estate | Phoenix Homes for Sale | All Phoenix Real Estate

  259. “Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database”

    Misappropriation is the key word here. If a search engine indexes a page from somebody that has authorization to display the limited (IDX info is limited for our MLS) information on their own website….is this really “misappropriation”??

    The page being indexed is owned by somebody who has authorization for the feed. Is a search engine indexing the page misdirecting the seeker of the information?

    http://en.wikipedia.org/wiki/Misappropriation

    “In law, misappropriation is the intentional, illegal use of the property or funds of another person for one’s own use or other unauthorized purpose.”

    So… is Google or any other search engine illegally using the information for their own use/benefit?

    Now… letting non MLS members having the same feeds or allowing them to scrape our IDX feeds for their benefit…. I think would be classified as “Misappropriation”. (Realtor.com and all of the advertising they sell certainly comes to mind here…. since they are technically an advertising site and their revenue is from the advertising they sell/display.)

    Hmmm… something tells me that selling advertising on all of the individual home detail pages from our IDX feeds would be a huge NO.. NO… but of course… it’s Ok for R.com to do so…

    Sorry… just procrastinating on some real work issues I need to get to….

  260. A few points stuck with me in reading (ok, I skimmed a few) these last 250 comments.

    First, I agree this is an issue of defining Indexing vs. Scraping

    Two ways that scraping concerns me:
    1. Inability to track back to the original source of information can mean lack of control in how information is indexed ie. misreading entry of “near water access” to mean “Waterfront”, which is something I’ve encountered, and had trouble correcting- would have been much harder if it hadn’t been on a site using a bona fide IDX feed.
    2. Scraping sites using our information to create leads and then sell them back to us.

    How to get involved? Start local. Sit on a committee. Sacrifice your time for the greater good. Be willing to take the next step in leadership. Go to State and National meetings. If you’re lucky, you’ll find a great mentor with a big brain like I did in Judith Lindenau, who was our Association Exec. when I jumped in.

    I’ll be in DC again this year, and at the 9:00 meeting. IMO the Midyear Meetings are the best NAR offering. But I’m also interested in the governance of our association, and I’m one of those sick bastards that enjoys going to Capitol Hill and telling his legislators how what they do impacts our business, our customers. Real work gets done at Midyear by people who are passionate about making NAR work best for us, the people on the front lines.

    Want to hear something really smart? NAR does not charge to attend Midyear.

    Oh, and I’ve refused to pay for an upgraded profile on REALTOR.com since they switched to their current pricing model. Maybe that goes back to my distaste for paying someone to use my data to drive consumers to their site.

  261. Jay Thompson >>”And let’s face it, there is NO WAY to block malicious scrapers short of making a rule that NO information can be displayed on a web site.

    Unfortunately, you can’t “legislate” common sense, and you can’t write enough lines of policy to prevent misuse and abuse.”

    Exactly why (IMO) I think the rule itself might need to be abolished. But it’s still a rule. We still have to act within the rule.

    Benn Rosales – “I get how you are trying to frame this debate, but you’re not just talking to brokers here in this thread, you’re mostly talking to agents who need to put dinner on the table tonight.”

    I originated my first mortgage in 1992 and have personally managed a career pipeline of over one billion dollars in real estate transactions. Please don’t suggest that I don’t understand who I’m talking with.

    I’m framing this debate in the only way that matters if you all hope to change it in a meaningful way. If Jay stands in front of the committee and says something to the effect that any idiot can tell the difference between Google and a scraper, this issue will probably die next Thursday.

    I’m relating to you why we are where we are. It might not be all that complicated in your world Benn, but if you want to change this, you’re going to need to step into mine.

  262. Let me ask this? Is the data not better protected by REALTORS who are ethically obligated to protect that data and where the rules can be enforced?

    I am thoroughly amazed by those who seem intent on preferring a 3rd party to have the data over a REALTOR who can provide accurate data to the public. When I submit a listing through my local board, I am obligated by my the COE and Board rules to provide accurate information.

    How many listings have you seen on a non realtor site which is no longer
    active or have inaccurate detailed onformation? Have you ever had someone call you about a listing they found on a 3rd party site, which was no longer available. Heck, some are so old, they’ve been sold twice since.

    How does that possibly help promote and protect the interest of our clients?

    Do you know how we lose control of the data? Have you read the terms of service on some of these sites?

    Here’s one for you:

    you grant [name redacted] an irrevocable, perpetual, worldwide license to (1) use, copy, distribute, transmit, publicly display, publicly perform, reproduce, edit, modify, and translate your Submission, in connection with the Services or in any other media, and (2) sublicense these rights, to the maximum extent permitted by applicable law.

    So tell me which is better – a indexed idx on a REALTORS site or on a site where they basically say – sorry, we can’t be responsible for the data and we have an irrevocable right to said information.

    Can you say SCRAPE? Becasue they have NO vested interest in whether our data remains protected.

    Please, someone tell me they understand this!!!

  263. Properly framing the issue requires acknowledging that this is an issue of interpretation of the existing policy, not making a new policy at this time. To that end, I’m glad Todd quoted the policy above, because I had not focused enough on the specific language previously:

    Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database

    Because I had not focused on the specific language, I fell into the same trap others have, namely interpreting the term scraping. Importantly, however, the central term in the provision above is not scraping but rather misappropriation, and that’s where the disconnect in interpretation occurred. The NAR technical team was asked whether scraping was occurring, and they stated that it is not possible to distinguish between scraping and indexing at the web server level. But that was the wrong question.

    The question is whether the broker has employed “reasonable efforts” to prevent misappropriation. Focusing on the specific issue before MIBOR, the question becomes whether Google was misappropriating the data by indexing it. There is no legal precedent in the United States of which I’m aware (though I am not a practicing lawyer) that indexing is misappropriation. Accordingly, the broker need not prevent Google or any other search engine from indexing their site.

    There’s little doubt that the NAR and local Associations will need to revisit this policy in the near future. Importantly, however, that’s not going to happen next week. Accordingly, a focused goal for next week would be to ask the MLS Policy Committee to issue a statement that this policy provision needs to be interpreted on a case by case basis by the local Association to ensure that misappropriation is not occurring and that reasonable efforts are being taken to prevent it. That’s what the policy requires, not some difficult assessment of intent at the web server level.

  264. Funny, I see now that as I was writing my comment, J. Paul Francis had posted the exact same thing. Sorry for the duplication, but I’ll reiterate again that the proper interpretation here is of the term misappropriation and not scraping.

  265. Pingback: NAR and MIBOR at odds with reality of what Google is. | Property in Santa Cruz Real Estate Blog

  266. In regard to Google – the Cache – it is a carbon copy (redundant) of pages that are kept handy for indexing purposes – and is not scraped content (it is called the Cache, not the Scrape( A cache is a temporary storage area where frequently accessed data can be stored for rapid access – http://en.wikipedia.org/wiki/Cache ) ).

    All of the original content is there, all the original links are there, all of the original disclosures and required contact information is there, and all links resolve to the original URL of the responsible party that was approved for IDX by the MLS board. In essence, it is a redundant copy of the approved website that is cached only for the purpose of indexing, which is good for buyers, sellers, and agents at large.

    The only persons that will benefit from choking this off will be the listing agents that don’t want the buyer side represented in any capacity other than dual agency and the lead agregators. Those that would loose would be the consumer public at large and many agents that choose to work with buyers instead of listings (I am sure from an agent count standing, there is a very large percentage of the NAR that does not list property).

    The cache is not scraped content, it is “The” content. It is “The” content stored redundantly on a different server using all original linking structures, graphics, photos, contact info, et. al. There are no new links generated! Searches return snippets of information from the redundant server and all links returned in a search go to that host site – all done in compliance with IDX rules and approval of local board via IDX agreement.

    The only reason the pages are held on the redundant space is to accomodate efficient indexing of the site. It would make absolutely no sense for the index spiders to have to go out and reanalyze the entire internet every time a search was made (billions of times a day). In fact it would probably by physically impossible to implement in that manner and would lock up Al Gores communication network.

    Long ago when I thought of knowledge, I thought of Libraries, when I thought of Libraries I thought of the dewey decimal system. Now when I think of Knowledge and Libraries I think of the Internet and the dewey decimal indexing system has been replaced by search engines.

    When I think of website content being choked from being indexed I think of lost knowledge, which makes me think of book burnings:

    “Where they burn books, so too will they in the end burn human beings.” – I hope you all will spend 5 minutes viewing this wikipedia entry – http://en.wikipedia.org/wiki/Book_burning

  267. Michael and J Paul – Thank you both for taking the time to disect both the rule and the term we need to be focusing on; misappropriation.

    The beauty of blogging and social media on a post as controversial as this, is the great thoughts of many and possibly, resolution. At the very least, each of us has a better understanding of the issues at hand and the goals we are working toward.

  268. @michael :) home run

  269. Web caching is the caching of web documents (e.g., HTML pages, images) in order to reduce bandwidth usage, server load, and perceived lag. A web cache stores copies of documents passing through it; subsequent requests may be satisfied from the cache if certain conditions are met.

    It is not to be confused with a web archive, a site that keeps old versions of web pages.

    http://en.wikipedia.org/wiki/Web_cache

  270. Please forgive me if this is a silly question, but I believe it goes to the heart of the rules interpretation at a different level than ‘Google vs. Scrapers’. Three points to lead up to the question:

    (1) The rule states that “participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database.” To go back to the car theft example earlier, this IS a law that says “you’re required to lock your car doors to prevent theft.”

    (2) The rule doesn’t say you’re subject to penalty if your IDX info ends up Google (or on a scraper site for that matter). Rather, it says you’re subject to penalty if you don’t take reasonable steps to prevent the activity. If you implement appropriate security measures and somebody figures out how to steal your IDX data, you (theoretically, at least) haven’t violated the rule.

    (3) There have been very knowledgeable comments that the Google indexing feature and “real” scraping are viewed by a web server as essentially the same activity, although they have different intents.

    So here’s the question…If Google indexing and malicious scraping look and feel the same to a web server, couldn’t it follow that those sites being so thoroughly indexed are also NOT being protected from actual malicious scraping activities? Or to put it another way: if Google and a true scraper would come in through the same “unlocked door” to your site, are you really protecting against scrapers if your information is indexed by Google? I’m not taking sides in the discussion, I’m just pointing out that there’s maybe a different way of approaching the issue.

    Now a disclaimer, with a few suggestions on how to address the issue within NAR:
    I am a senior staffperson with a state real estate association and am staff liaison to several statewide committees. Last year I also began serving on a large NAR committee (not this one!). My observation from having been on both sides of the table is that a coherent, well-reasoned, narrowly targeted argument (preferably with graphics and/or demonstrations of your points) is always the most effective way of trying to win the day — especially when you’re dealing with a 100+ person committee with widely varying knowledge and virtually no time to get educated on a technical issue such as this one.

    Further, if you’re looking for a change in rule or policy, it is always better to go it with a very specific written recommendation (preferably made available to staff in advance) for consideration rather than simply saying “this is wrong and you need to change it.” If you just say your piece and leave it to the Committee there’s always a chance that they’ll fix it, but not in the way you want it fixed.

    So to focus on the rule at hand — if there’s a way to set up a site to allow search engine indexing while denying scraping, it seems to me that such an explanation would be pretty persuasive to the Committee. That would provide clear guidance either for a reinterpretation of the existing rule or for a rules change that narrows the scope of the rule. And if you can provide a rewritten rule that clearly distinguishes between the two activities, even better. Can’t guarantee that all this will get you what you want, of course, but it will certainly increase the odds.

  271. Hank, your analysis is the same one that led to the interpretation by MIBOR, and it points out the flaw in focusing on scraping versus misappropriation. The only reasonable way to prevent scraping is not to post the content to the open web. Interpreting a policy that way would defeat the entire purpose of the policy, which is to allow brokers and agents to display the IDX information on the web.

    I agree with you wholeheartedly that what should be asked of the MLS Policy Committee should be as narrow as possible. To that end, I suggest they be asked to issue a statement that:

    “The IDX policy requiring Participants to use reasonable efforts to prevent misappropriation does not require Participants to remove any of the IDX information from their web site or to instruct search engines not to index the site.”

    Or, as a fall back:

    “Interpretation of the requirement in the IDX policy that Participants use reasonable efforts to prevent misappropriation is to be made by the local Association that issued the policy. The NAR Policy Committee will be researching this issue in the coming months to provide guidance on the issue.”

    What is unstated in the above is that no guidance is issued at this time. Perhaps that could be stated, but I don’t believe it is necessary.

  272. @tcar – Plus, what happens down the road if Google, or some other search engine makes a serious real estate play?

    It’s a non issue, at least as far as this debate is concerned. It’s already been proven that NAR rules have no impact on non-NAR members which is why the Trulia and Zillows of the world can do what they choose while NAR selectively enforces archane on its own paying membership.

    I was going to save this for a post of my own but I’m trying really hard not to go down this road (having said that, my gloves are off and I’m skating in circles trying to let the aggravation dissipate) … but the idea of allowing IDX so long as we don’t let anyone really see the IDX reminds me of Al Pacino’s line in The Devil’s Advocate:

    “He’s a prankster. Think about it. He gives man instincts. He gives you this extraordinary gift, and then what does He do, I swear for His own amusement, his own private, cosmic gag reel, He sets the rules in opposition. It’s the goof of all time. Look but don’t touch. Touch, but don’t taste. Taste, don’t swallow. Ahaha. And while you’re jumpin’ from one foot to the next, what is he doing? He’s laughin’.”

    The boards are pranksters, my friends. They give you the IDX and then they set the rules in opposition to the very methods by which the IDX can be used most effectively.

  273. Michael you have the best grasp here and I’m so glad you are involved in the post.

    Here’s the problem though. Our IT department’s ongoing conclusion is unchanged. What the IDX provider is doing to allow Google to index this data. The technical way in witch that data data is presented on the web so that Google can read it and index it, violates the policy because it allows the real bad guys to then come in and easily misappropriate it.

    From my conversations with our IT department, I don’t think anyone there is philosophically opposed to seeing listings on Google. (I could be wrong, it’s Friday night). What they are saying is that their isn’t a reasonable way to open the door to Google indexing without opening the door to real scrapers as well.

    Point of disclosure, we are moving out of my technical grasp here. I probably can’t clarify this until Monday.

  274. Todd/Michael, You are both making great points about the technical details, but I think it’s really about the big picture.

    Todd hit on it earlier. Maybe we should just delete the rule altogether? Rules/laws are in place to protect the innocent and this one seems to punish those who are doing things right. As REALTORS, we have codes of ethics and the ability to file complaints & grievances if other members behave badly. Why bog us down with minutia of technical detail about web servers? NAR should just empower the members to sell real estate with as little difficulty adhering to rules as possible. It just doesn’t make sense to constrict our members when 3rd parties do not have to follow the same silly rulebook. We should broaden our scope when writing policies.

  275. @tcar “What they are saying is that their isn’t a reasonable way to open the door to Google indexing without opening the door to real scrapers as well.
    I really am trying to stay”

    I am really trying to stay on task here, but have to ask – how is it that they can open the data to Google through R.com?

    My guess is the thought of gioving REALTORS access to their data is going to cost someone money.

  276. Okay – so that was a hurriedly written statement – hope you can make sense of it. :)

  277. Please understand that I’m not defending the wording of the current rule, nor the interpretation at question. What I’m trying to do is help focus the discussion on an approach that may get the Committee from Point A to Point B effectively.

    The way things stand right now, there’s a rule that says “you have to take reasonable steps to stop the Bad Guys from doing Very Bad Things by preventing scraping,” and there’s an opinion from CRT saying that the computers can’t tell the difference between the Good Guys and the Bad Guys, so you may have to keep everyone out just to be sure. Seems like CRT won’t be changing their opinion, and it is probably reasonable to assume that as a group the MLS Committee is not likely to just discount the CRT. So that leaves a focus on either attacking the interpretation or attacking the rule.

    Michael said, “The only reasonable way to prevent scraping is not to post the content to the open web.” Assuming that to be true, the rule as written means you can’t post anything to the open web since there is no reasonable way to prevent against scraping. You can split hairs on interpretation all you want, but if this is the reality of the universe, it will be tough to simply interpret the rule to mean something other than what it says.

    That leaves an attack on the rule as the last option. I like Michael’s focus on the term “misappropriation.” Seems to me that a major problem you’re running into with the rule as it stands is that it has very specific wording regarding scraping. Members must “protect IDX information from misappropriation BY EMPLOYING REASONABLE EFFORTS TO MONITOR AND PREVENT ‘SCRAPING’…” So no matter what else you do to protect the data, you ALSO have to monitor and prevent scraping. Rather than try to finesse highly technical descriptions of scraping, or try distinguish between ‘good scraping’ and ‘bad scraping,’ what if you removed the reference to scraping altogether and rewrote that part of the rule to be more reflective of the reality of the Web?

    For example, what about “Participants must use reasonable efforts to protect against the misappropriation of IDX information and the unauthorized access, reproduction or use of the MLS database. For the purposes of this rule, search engine indexing is considered to be an authorized use of IDX data.” No more fights over the word ‘scraping,” search engine indexing is allowed, and everyone can sleep at night knowing that Realtors still have to protect against Very Bad Things as best as they can. Just a thought.

  278. @tcar said: “What they are saying is that their isn’t a reasonable way to open the door to Google indexing without opening the door to real scrapers as well.”

    You’re absolutely right. someone alluded to books earlier so take this example. What if you owned a book store. You wouldn’t know whether every person that walked through the door was there to do business with you or steal your books. But you still let them all in. If someone steals, you do your best to catch and punish them. But if you follow NAR’s rationale, you would just have to close the shop altogether. Since you can’t tell who is criminal and who isn’t, you just shut them all out.

    In fact, NAR’s rationale is worse because it keeps the ones you know are good out while doing nothing to keep the criminals out.

  279. Well, this has been a saga… Many well-intended folks who have missed what has been excellent definitions of indexing and scraping. And, they are clearly not identical. So, NAR should revisit and revise its language to allow indexing and prohibit scraping. (If you don’t grasp the difference, kindly read Michael Wurzer’s posts.).

    Once that is resolved, the issue of whom has the authority to submit listing data to third party aggregators (3PAs) should be reviewed. IDX is a cooperative (authorized) effort between brokers in a specific MLS. Brokers, with approval of their sellers, can additionally authorize their MLS to provide the broker’s data to 3PAs, including REALTOR.com (a 3PA authorized to use the “Realtor” name.). MLSs should not be directly authorizing data delivery to 3PAs. It is a broker/seller business decision.

    3PAs compete with broker/agent web sites for clients/customers surfing the Internet. As an example, if you have a MLS with 2,000 MLS Participants/Subscribers with web sites, it is not likely that an additional 1,000 web sites operated by 3PA’s will “create” more clients/customers. The 3PA’s will simply reduce the number of client/customer visits to Participant/Subscriber web sites. And, having provided the 3PAs with the MLS data necessary to be competitive, brokers can then expect to pay them a fee for any referred clients. Of course, web sites with the best search engine optimization will rank higher.

    Unlike other types of media where distribution of advertising in physical locations may catch clients/customers not otherwise reached, the Internet is dimensionless and any IDX web site can produce full search results without the need for clients/customers to go to a 3PA web site.

  280. Hank, I suspect most, if not all (especially NAR’s IT team), would be happy with a revision of the policy to state as you suggest: “Participants must use reasonable efforts to protect against the misappropriation of IDX information and the unauthorized access, reproduction or use of the MLS database. For the purposes of this rule, search engine indexing is considered to be an authorized use of IDX data.”

    Instead of suggesting revisions to the policy, I’ve been trying to stick to interpretation of the existing language, because of the concern that many Associations have adopted the same language and it will take significant time to work through the process of correcting it. In the meantime, Paula and others in MIBOR are impacted in their ability to compete on the web and there is the possibility (hopefully remote now) that other Associations will follow suit. Accordingly, I think a good result here would be for the MLS Policy Committee to say to MIBOR and all other Associations that they should hold off on any interpretations until the policy can be reviewed.

    Todd, I know you’re right that NAR’s technical team is not anti-Google or trying to frustrate broker or agent competition on the web. They’re simply providing a technical interpretation of the term scraping, which, on its own, does not deal with misappropriation. In other words, NAR’s technical team was never asked the right question.

    What I’m suggesting is that a proper interpretation of the current language must start with the term misappropriation, which precedes and controls the term scraping in the context of this policy. I do not believe NAR’s technical team interpreted the term scraping in this context, but instead focused only on the technical interpretation of scraping. The context of the language, however, is critical.

    For example, the importance of the term misappropriation is further strengthened by the subsequent reference to “other unauthorized access”. In other words, the only scraping that must be prevented is unauthorized misappropriation. If it were just any type of scraping that was to be prevented, then the term “other” would not have been used. It is “unauthorized” scraping and scraping that is “misappropriation” that is to be prevented. To conclude: indexing by search engines is neither misappropriation or unauthorized.

    I believe this interpretation is both internally consistent with the language and the overall intent of the IDX policy to provide MLS Participants with a tool to compete on the web. Nonetheless, if the Committee is not comfortable with agreeing with this interpretation of the existing language, then I would advocate they advise Associations that the current language needs further study before NAR can provide a formal recommendation or revision to the language.

    Unless the current language can be interpreted as I suggest above, then the only logical conclusion is that the language is inconsistent with the purpose of the policy itself and so needs to be revised as Hank, Brad and others have suggested. I merely suggest it is possible to reach the same result with the current language and, if not, the Committee should make a statement next week that advises other Associations that the issue needs further study and that the language may be revised in the near future.

  281. The statement I made above that “the only scraping that must be prevented is unauthorized misappropriation” should have said: “the only scraping that must be prevented is scraping that is unauthorized and misappropriation.”

  282. Spot on, Michael Wurzer! It’s about “misappropriation” and “unauthorized access”, and we already have means in place to protect the data from both.
    MW wrote above: “For example, the importance of the term misappropriation is further strengthened by the subsequent reference to “other unauthorized access”…. It is “unauthorized” scraping and scraping that is “misappropriation” that is to be prevented. To conclude: indexing by search engines is neither misappropriation or unauthorized.” Yes!
    From what I know of the technical side, locally, IDX providers agree to terms with the MLS for specific access — through an account, passwords and other specifications. Those are “gates” and they protect those data. The broker/listing agent and seller have some control and can opt out of IDX. Another gate. The agent selects an IDX provider actively incorporates data. Still another gate.
    How the system works now with these and other “gates” does support an interpretation of policy to allow “indexing” and what Paula and many NAR members desire — to use plenty of listings on their sites and stay in business.
    Hope to see you all at the Wednesday “REALTOR Mixer” event with Paula and Jay and supporters! – L

  283. Question:

    1) Where do the 3rd Party Vendors get their feeds?

    2) Where is Louis, David and Rudy?

    3) Are they getting them from Realtor.com?

    When I was with RE/MAX I got the feed for them from our local MLS, they could not get it being a 3rd party. When Remax closed in A2 in 07, they lost their feed and the data for A2 Listings.

    (They just re-opened here so I am sure they will again have it.)

    4) If the 3rd party vendors can get the data and display it and beat us in SERP’s, I am confused as to why we the paying, and in my case an active Realtor on my board can’t display the data as I see fit. Doesn’t seem fair that members of NAR can’t display their own data however they want to (like Paula) with MLS numbers.

    I don’t particularly care for the data displayed the way the post started but I respect any Realtors right to display and reach the consumer the way that they see fit.

    Ok, there are more 3rd party vendors out there I only named the big 3. I’m not saying they shouldn’t have the data, I have my own 3rd party that I use, which is responsible for 44% of my closing in 08.

    But, where are they gettting the data?

    Perhaps all of them would like to share with us.

    And why the silence here on this debate?

  284. Missy –

    I think that is the crux of this debate, and this is part of what I’ve been thinking for a while. With this interpretation, the NAR is seeking to limit how its members display data while allowing (b/c they don’t have ‘jurisdiction’) Third Party Aggregators to do whatever they please with – OUR data/content.

    1 – I think DavidG or Rudy or Glenn Kelman were likely on the committee guiding the conversation, as the Z/T/H/C of the world are the ones who benefit.
    2 – Big brokers will rule if this is allowed to stand. (more on this in a forthcoming post) The internet has “leveled the playing field” for those who choose to (try) to leverage it to it fullest extent.
    3 – And this is my main point – if you take away the law-abiding citizens’ guns, only the criminals will have ‘em. In this case (and I’m not calling anyone criminals, except the true scrapers, but it serves to make my point) – the Big Brokers and the Aggregators will be permitted and encouraged to have an undue competitive advantage … and the lowly agent – you know, the ones who are the “boots on the ground grunts” making the money – will suffer and die a slow death in committee meetings.

  285. Another point –

    I’d *LOVE* to see someone from the NAR MLS committee come here to debate/participate/discuss/let us know they’re listening to this matter.

    It shouldn’t fall on Todd to make or interpret an argument or policy that 1) he didn’t make and 2) doesn’t affect his business in any way shape or form the way it does ours – “ours” being fellow Realtors on this board and on the Committee.

  286. Jim, they are not going to come here. (NAR MLS Committee Members) I serve on too many boards and when they speak they speak as one voice.

    We don’t discuss what is behind closed doors. Oh if those walls could talk….

  287. Missy –

    Your third party vendor is a framed IDX solution. Google can not read the data to display the information. Although I use the same IDX as you, it would not bring you business without PPC.

    Additionally, I have an IDX on my website, which is embedded (not framed), which allows Google to read the pages. Granted, I could never get the whole thing indexed; we have 28,000 properties available here.

    Both companies offer a service to REALTORS(r) and receive their feed from our local board. The 3PA’s (except R.com)do not get their data from an MLS feed, they get it from agents who put their listings on the sites.

    Of important note: the companies you and I use have signed a contract with our boards to use the IDX data: 3PA’s have not.

    I understand you and other agents may not like the way it is displayed in my first example, but go look at T,Z,H,C and R.com – this is exactly the way theirs is displayed, which gives them the advantage you speak of.

    Many agents who do not understand the difference; they “kick and scream” about their listing being displayed on another agents site. In their infinite wisdom, they would rather have their listing displayed on T/Z/H/C, where we actually lose any control of the data. I Pointed this out earlier, but has anyone read the terms of service for some of these 3PA’s; here’s one short exerpt:

    you grant [name redacted] an irrevocable, perpetual, worldwide license to (1) use, copy, distribute, transmit, publicly display, publicly perform, reproduce, edit, modify, and translate your Submission, in connection with the Services or in any other media, and (2) sublicense these rights, to the maximum extent permitted by applicable law.

    Our data is not protected from misuse, scraping, editing, retransmitting or anything else deemed acceptable by the company. So are all agents who advertise their listings ther in violation of the MLS rules and regulations.

    Think about it though, someone finds your listing on T, for example and I have a TPro account, with my picture on the side – is that any different than someone finding your listing on my site. Isn’t the goal to get the home sold and provide as much exposure to your clients home as possible?

    As a REALTOR, I have a vested interest in ensuring the data is correct and protected. It’s in the interest of all REALTORS that we have the same right to display as the 3PA’s and REALTOR.com do.

    In doing so, we can ensure the data is protected in its integrity.

  288. Paula, I know Tiger is framed. I don’t get indexed but I am in support of the policy being changed for all Realtors that have unframed sites and want to get indexed that way.

    Most of my leads are NOT from the PPC side of Tiger only about 80 per month most come from my own organic and viral marketing of the site. (200 plus)

    I am in total agreement with you the goal is to get the home sold and I don’t care who displays my listings, I just want an even playing field.

    People search so many ways it is just shocking so I want to be there for HOWEVER they search. If they search my MLS number or address or any other weird term so be it.

    Good luck, I’m trying to get a cheap flight in to NAR.

  289. Missy – I hope you do make it – you have been a mentor to me and I would love to meet you!

  290. Missy said

    We don’t discuss what is behind closed doors. Oh if those walls could talk…

    anyway to get a live stream or recording on this meeting so ALL realtors can see this who are unable to attend?

    oh and btw big thanks goes out to Jay & Paula for taking time out of their schedules to go to DC. Thank You

  291. Yes, yes!! I would love to see that meeting live-streamed for all Realtors to watch (monitor?) who cannot attend. This issue is so critical, I can’t think of anything else that is more important or has the potential to impact my business more than this. 290 responses to date on this post – WOW! That’s a lot considering most Realtors would not have a clue what is being discussed here.

    Kudos to Jay & Paula for all they are doing. We are all so very appreciative.

  292. I think it is obvious that the real reason for this is the big dogs do not like a local Realtor showing up higher in the search results. Plus the potential to charge us for yet something else. NAR needs to realize this is not the best for it’s members, or more importantly for the clients of it’s members.

  293. I don’t think it’s quite fair to separate Google from the rest, many search engines index (spider) sites in similar ways, but I guess Google being #1 puts it square in the spotlight.

    “Scraping” or “Spidering” or whatever the term is, search engines page through sites using an almost text-based browser. Engines like Google rate certain HTML characteristics and content which (using a KFC-style secret recipe) give sites their placement. It is possible to restrict certain pages – but not all search engines follow these restrictions – if it’s out there, it can potentially be indexed. Added to this is that just because a site isn’t added to Google doesn’t mean it won’t be found (and indexed or scraped) by Google or another engine. Thus is the viral online world we live in. The only way to secure data is by an old-fashioned authentication system whereby you have to know who the person is before they can see the data (in theory, making electronic browsing more difficult). This may be partly what’s going on between the lines.

    If NAR decides to stick to and enforce this policy, it’s going to negatively impact agents across the board. People pay for IDX feeds, they’re not getting stuff for free and, as long as the IDX feed is adhering to NAR rules, if it boosts an agent’s Google rating, then it can only (in theory) benefit their clients who are paying for the best marketing they can get. If agent Joe’s MLS listing shows up on agent Mary’s site, Joe will still be compensated…

    Not only is the interpretation archaic, it’s ethically questionable and, to my mind, an attempt at control. As Barry says, look what happened to Realtor.com. If agents are going to be corralled into ever-tighter regulations, then any “personality” or “creativity” will go by the wayside and they might as well look for a “desk job”. Add in all the companies that provide supporting services and you’ve got a pretty big issue on your hands.

    I’m sure the original thought behind this “scraping” interpretation was somewhat noble, to revent data theft and resale, but the cure in this case might be worse than the disease.

    My prediction, for what it’s worth: If this ruling does start being enforced across the baord, it won’t stop with IDX. The next thing will be listing-specific websites (of which I do several) because it will be deemed “unfair”. Then they’ll add any form of e-flyer, Craigslist or other advertising medium, emailing, twitter, blogging and so on, until everyone is forced to go through “approved” and mandated systems.

    Eventually, enough agents will drop out of the system and a new, less restrictive body will form.

    Apologies for the novella. I’m not even an agent and it infuriates me that these enforcing bodies go after the benign when they should be focusing on bigger fish.

    Great post!
    Dave

  294. Jay and Paula – please read this post – I think you will find it valuable for your meeting – mygulfcoastbeachteam.com/mibor-nar-grasp-internet-technology-works/

  295. Pingback: Real Estate’s Deep Dark Secret: NAR (and Many Brokers) Don’t Want You the Public to See the Data | Phoenix Real Estate | Phoenix Homes for Sale | All Phoenix Real Estate

  296. Personally, I don’t care who or what indexes my listings. I list a home for one express purpose — to sell it. The more sites it’s exposed on, the better. You want to advertise my listing on the web, in the Sunday paper, on a billboard? Please, knock yourself out. The more my listing is out there, the better chance it has of being sold.

    If I EVER go to one of my sellers and say “I’m only going to put your listing on my site. That way I get all the calls and can either 1) double-side the transaction; or 2) generate leads from your home, then please take my license.

    Jay – I couldn’t agree more. I want my listings to be found by everyone. I’m not worried about where they find them or who put them there. I want to serve my clients and sell their home. That’s what I’m paid to do. Dual-agency or not, there is no reason (in my opinion) to not want every agent and home buyer to see your listing everywhere they can.

    I think of it like this… it’s illegal to steal another persons car, but the law doesn’t say we have to lock our doors.

    Isn’t this sort of like our data policy with MLS’?

    Brad – I made a similar point earlier. Why are we responsible for “theft” by others? I know that the argument is that “it’s the policy” – so I question that policy. Regulated and giving us rules that deter our abilities to sell homes is not the answer. Giving us the tools and “having our backs” should be. PS I hear Jason wears his Birkenstocks to work.

    You’re absolutely right. someone alluded to books earlier so take this example. What if you owned a book store. You wouldn’t know whether every person that walked through the door was there to do business with you or steal your books. But you still let them all in. If someone steals, you do your best to catch and punish them. But if you follow NAR’s rationale, you would just have to close the shop altogether. Since you can’t tell who is criminal and who isn’t, you just shut them all out.

    Braxton – Much like Brad’s point, I think you’re on the same track as I am.

    I know these are not the talking points for Paula and Jay, but I do see the need to point them out anyway.

    Jay said he’s worried to leave us disappointed, but I know for me, I won’t be. Why? Because Jay and Paula took the opportunity presented to them and are going to do their best. If NAR chooses not to listen, Jay and Paula have not failed. NAR has. Oh wait, no… the members failed, since we wrote the rules. Sorry, I’m tired of that argument. Turning it around and putting the blame on us isn’t going to help any. Let’s work to change the rules (or the interpretation) written by people who didn’t see the future four years ago.

    I wasn’t here in 2005 to even think about joining a committee. I’m ok with the idea that I have to deal with the issues of those that came before me, so I’m not whining about it. And I do plan on working to effect change in the industry, but I’m a long way off from joining a NAR committee. NAR committees require some level of qualification and experience (would you really want a guy with a year under his belt guiding your business, when so many of you are much more qualified?). For instance, the MLS Committee requires 1 – 3 years experience on the Multiple Listing Policy Committee at the local or state level/1 – 2 years experience on NAR’s Multiple Listing Policy Forum (from NAR’s website). Since I have neither, I wouldn’t even begin to suggest that I could serve on this committee. It also takes a certain level of financial commitment which I am not ready to make – yet.

    Jay and Paula – Just do what you do. Be honest, outspoken, and focused. I am proud to have you two representing us and I am glad that there are those out there willing to shoulder the burden of something like this for the good of the rest of us. I hope someday to repay you in kind. I appreciate your willingness to drop what you’re doing and be, to borrow a (probably trademarked) phrase from NAR, “The Voice Of Real Estate.”

    This is a great test for the real power of social media and no matter what the outcome, I think something bigger than this rule is going on here (not to downplay the importance of that whatsoever) and I hope this is not the last time I see this sort of outpouring of concern. I think the importance of Todd’s position is truly shown here. Although we all may not agree with him throughout this thread of comments, the importance of what is taking place by having him exist is undeniable. Before his position was created, do you think we would have seen a direct effect of this post as we are seeing now? I don’t. I don’t envy Todd either, for as I mentioned in my AG post, I think Todd will always take the brunt of harsh reactions from those of us that are connected to the online world. It’s not fair, but I think it will come with the job. That doesn’t mean we should use him as an outlet for our anger, but I do think it will occur naturally, especially since he will more than likely always be “first on the scene.”

  297. Hello Matt…

    How about this… Upon application (authorization and hold harmless release) a MLS will release listing data to any 3PA of the broker’s choice. Brokers can pick the 3PAs, one or all. However, the MLS does not make a blanket release (all brokers’ listings) to 3PAs as it should be an individual broker/seller decision where they want the listings to appear.

  298. That’s it. I just moved across state and am in search of a progressive broker to hang my license with. What you’ve pointed out here in this post leaves me more convinced than ever that I have no business being a “Realtor.”

    Finding a broker should be easy now. If you’re a central pa broker and not a member of NAR, give me a call!

    (Awesome comment count btw!)

  299. Here is a good related post – notorious-rob.com/2009/05/10/message-control-and-social-media/

  300. How would consumers be harmed because IDX listings wouldn’t be indexed in Google? The IDX listings are still available on the agent’s websites.

  301. Does anyone find it odd that MIBOR may be allowing “scraping” of listing details in violation of their own policy?

    Judge for yourself – http://www.google.com/search?hl=en&rlz=1B3GGGL_enUS305US305&q=+site:listings.listhub.net+listings.listhub.net/pages/MIBORIN/

  302. And I am sorry, but please note the “channels” in the URL’s of the Listhub indexed pages from MIBOR – “Zillow, Secondspace, Foreclosureclicked, etc.”

  303. So…Is it still ok to scrape Realtor.Com?

    bloodhoundrealty.com/BloodhoundBlog/?p=8352

  304. This is totally crazy. I suggest that all of the web savvy forward thinking Realtors do what they can to get on their local MLS board or committees so that when things like this come up someone with a brain can actually vote on the decision.

  305. Wow, what a tangled web. (Thanks for the tip, Ryan.) Get rid of the co-broke and every bit of this — plus all the rest of the NAR’s insane stunts — goes away. No matter which side of this debate you seem to be on, there is really only one side: “How do we make an inherently unworkable policy of secrecy and deception continue to work in a world of unlimited transparency, plummeting infomation overhead costs and rapidly balancing information symmetry?” The answer is that you can’t. The reasonable thing to do is to stop trying.

  306. Can R.com prevent scraping of the MIBOR listings data? Can you say misappropriation?

    http://www.bloodhoundrealty.com/BloodhoundBlog/?p=8352

    http://www.retechulous.com/category/nar-blows/

  307. @Confused Consumer – if you were a seller, wouldn’t you like it if someone googled a street name after seeing your house was for sale and they actually found it! Suppose they had to search for “Bob the Broker”, and then end up on Bob the Brokers site with no street level search, so they look through all the listings in the 300 to 400K range because surely that cute home was worth every bit of that, they don’t see your home because it is listed at only 299K, and the people that might have bought your house ended up buying the same floor plan in an adjacent subdivision built by the same builder and paid 319K for it! The buyer paid more for the same thing and the seller with a comparable home priced at a lower price and which originally piqued the buyers interest ends up with a home that did not sell! So, the seller is a consumer of services and the buyer is a consumer of services and both appear to be harmed from lack of transparency.

  308. Confused Consumer,

    In answer to your question about how consumers benefit from search engine indexing….

    IDX rules state that Realtors are limited in what data may be displayed. This discussion does not challenge what data can be displayed. This discussion challenges “how” it is displayed. Indexed data delivers better results for both the seller and the buyer. “How” it is displayed can alter the frequency and accuracy of the property information presented to buyers.

    A database search on an IDX site requires the person searching enter the exact criteria to cause a “hit” or “return” of the listing within the database. Google (and other search engines) are able to produce results based upon whatever criteria is entered into the search box on the browser. A quick look at Google Analytics shows how many times keyword terms are searched. Buyers use Google (and other search engines) to find properties, and they do it a lot. As a real estate broker, I can tell you that buyers often overlook properties that do match their criteria when using an IDX database, simply because the entered search terms did not exactly match the IDX search fields.

    Buyers benefit from indexed properties because they find properties that match their criteria easier and faster….and they use Google. Many buyers prefer to begin their searches without Realtors, and just gather information on their own. And, these buyers want valid , accurate info with no strings. Indexed data delivers better search results under these terms.

    Listing agents are charged with the responsibility of getting the product in front of the buyer. Sellers are entitled to benefit from the marketing expertise of the Realtor they hire to represent them. Sellers benefit from increased exposure of their properties to the actual buyers who are searching for them. Sellers have a right to have their properties be where their best potential buyer is more likely to find them. Sellers have a right to hire a Realtor based upon savvy skill sets and commitment to get the job done.

    Accurate info is more likely to come directly from a Realtor. And, we have many rules and regulations demanding it. Media sites are far less concerned with the accuracy of data, and far more concerned with building hit count and selling advertising space….their source of revenue and mission. The mission of a Realtor is to bring sellers and buyers together. Their livelihood depends upon it. How many times I have heard complaints about outdated or inaccurate data on third party media sites? Yikes! Too many. It makes more sense to promote, not deny, the Realtors to index property data from MLS.

  309. @ Cal Carter – As long as my agent’s personal listings, which would include my house, were indexed in Google any potential buyer who googled a street name after seeing my house was for sale could still find it.

    @ Deborah Madey – My home will still show up on any and all websites that are displaying IDX listings for my area. The difference it seems is that the IDX listings won’t be indexed in Google with the exception that my agent’s personal listings, which would include my house, will be indexed in Google.

    SUMMARY: Preventing IDX listings from being indexed in Google will not harm consumers. An agent’s personal listings can still be indexed in Google for consumers to find. IDX listings can still be displayed on cooperating broker’s websites for consumers to find.

  310. NAR, an organizations that you must pay to get the title of “Realtor.”

    The ethics training that differentiates you from the primordial ooze that has no ethics is not extrapolated from the real estate agent gene pool after taking the “mandatory” ethics training classes. You either have ethics or you don’t!

    I am not any more proud of the title…just feel like the “mafia” is protecting me for a fee. Protecting me from what…I have yet to understand?

    NAR is a slow moving dinosaur the will go the way of the “dodo” bird should they not evolve with the times.

    Our local board is made up of a number of “school marms” that had to find a job after they failed as “Realtors.” Their job is to implement fines for incorrect MLS listings and hold lots of meeting that “Realtors” have to pay for to get information that is freely available in a Google search…coincidentally. The thought of any of these employees making a decision affecting search engine IDX and its ability to find my MLS listings is a threat to my future as a “Realtor.”

    I applaud any efforts to challenge NAR rules of “scraping.”

    I think we need to get NAR along with Al Gore (inventor of the internet) in the same room and have Mr. Gore redesign the internet…that would solve this matter.

    If the above sounds absurd…I have archived my objective. NAR = absurd!

    Revolt now…stand up for your clients fiduciary concerns and fight these feeble minded decisions that you are paying NAR to make!

  311. @Confused Consumer – I think there is a possibility that the name you chose may be an attempt to be deceptive and that you have an ulterior motive.

    Your comment really smacks of an agent that does not want to identify himself and is pretending to be a consumer. I can’t imagine why a consumer would have had any interest in this blog, this post, or reading through the 300 comments that follow the post.

    However, the fallacy in your comment – “As long as my agent’s personal listings, which would include my house, were indexed in Google any potential buyer who googled a street name after seeing my house was for sale could still find it.” is:

    1) that there are large numbers of agent and broker websites that are not indexed by Google and never will be because the agents never gained the knowledge nor took the effort to get it indexed. However, they go to listing appointments waving their web address URL all the time knowing that no one has ever visited their website and that they have never fielded an inquiry from their website.

    2) many agents never manually enter their listings on their website. Why should they, they have a cool IDX that they can even set up to display their listings first and then their company listings. Yet, since their IDX is being IFramed, the content never gets indexed thus negating your point completely!

    The agents commenting on this blog are tech savvy and I would bet that 99.9% would concur with my statement and do so in an instant in that they know it is correct without having to do further research.

    So, dump the “Confused Consumer” name, drop the “My Agent” charade, and come back to the playground with your real identity! All you will need to do is introduce yourself and state you were “formerly known as Confused Consumer” and we will be glad to engage you in civil discussions about the way the cow eats the cabbage.

  312. I am not “Confused Consumer” but he echoed my comment prior. The point is not that an agent does not do but that an agent is able to do. If an agent is not updating their own website, blogging about their listings, never gained the knowledge, etc then that is their own fault and quite possibly a good question for consumers to ask when interviewing an agent.

    Instead of railing on an anonymous poster, acknowledge that their point is valid as was mine before that no one decided to respond to.

  313. @Confused Consumer and @James M –

    Why should anyone, be they consumer or agent, be concerned about this ruling?

    Because it could easily change the way real estate is done on the Internet.

    Any organization that is oppressive enough (and dare I say clueless enough) to equate search engine indexing with malicious scraping is oppressive and clueless enough to do anything.

    When they figure out their little policy can not stop misappropriation of MLS data, that no words (or technology) can stop a scraper, what is next?

    This is the ONLY policy that can prevent scraping and misappropriation of MLS data:

    No real estate agent or broker can display listings on their web site.

    That would include YOUR HOME Concerned Consumer.

    Right now I bet someone is saying, “Oh good grief, they aren’t stupid enough to make a rule like that”.

    Yeah, and I bet last week you would have said, “No one could possibly claim Google is a scraper site”….

  314. Jay: Google scrapes but is not a “scraper site. Here is a quote from my original comment here …

    The way Google works, it scrapes the text from your site on a listing that may be owned by another broker, links back to your site as if you were the owner of the listing, and the true owner of the listing isn’t revealed until they are on your site. You have effectively drawn in a potential buyer by advertising a listing that is not your own – which is illegal/unethical – and do not disclose the true owner of the listing until the buyer is on your web page with the full IDX info. It may not be your intent to misrepresent but that is what happens.

    Now even though an agent isn’t necessarily trying to be malicious, is this not considered misappropriation? Now, if we want to redefine or more clearly define misappropriation, than I say that’s a conversation worth having.

  315. Being an Association of Realtors member I am appalled by the statement that Google is “scraping.”
    In my opinion scrapers are people intentionally pulling listings to put on their site. I.E. if a Trulia / Zillow site were to start up and then went to all the broker websites and got listing info to put on their site for it to have listings to advertise, then that would be a scraper.

    Google is not in the business of hosting the property data, pictures, etc on it’s site, it’s simply directing people back to the broker site. According to this ruling Single Property Sites would have to be completely shut down, because Google would be scraping then, individual broker sites would have to set noindex tags on all idx / vow pages etc. This would be a travesty if it holds up. The technological Realtors that will dominate the next phase of the business, in my opinion are being discriminated against.

  316. James, that would be a great conversation…but not here in this thread of a different topic. of course in that discussion one would point out that the owner of the listing was consenting to putting their listing in the IDX in the 1st place and if they didn’t like its ramifications–such as less exposure for their Seller’s home on the web–they could opt out of placing the listing in the IDX. However some MLSs do not have that option. And if that it bothersome to agents in their markets it’s a rule change worth pursuing.

    I also do not know of any MLS where the listing broker of record is not required both in the detailed version of the listing and the summary/snippet version of the IDX listing. So I do not know that would be an issue.

    Write a column on it and start the conversation, Malanowski.

    @ Concerned Consumer:

    Just having your listing for example placed on your agent’s website and not in the IDX does not mean it would get indexed by google. Most agents have very stagnant websites without a regular flow of new content and therefore are crawled rather infrequently by google. And this is especially often the case with big time listing agents who often operate in the 20th century with their newspaper ad campaigns to get new listings. You almost never find their websites when you search for homes for sale or city state real estate….

  317. James –

    Therein lies the absurdity of this argument.

    If we are just going to look at what happens on a web server level, then the NAR and local associations need to modify the policy and add web browsers to the list of what is forbidden. IE, Firefox and other browsers effectively “scrape” sites too.

    So it looks like now the policy needs to state:

    No real estate agent can have a web site

    Extreme, yes. But that’s the path we are headed down.

    Here’s another example. I frequently write blog posts or build entire “blogsites”about local neighborhoods/subdivisions. They rank *very* well in Google for localized search terms (like “Agritopia real estate” or “Power ranch homes”). In fact, between AgritopiaRealEstate.com and my blog, I currently have 3 of the first 10 results in Google for the search term Agritopia real estate

    I don’t currently have a listing in Agritopia, but I do have an IDX search on the site (that is not indexed by the way). People call me ALL the time, wanting to see “my listing” they found on my site. **AGENTS** call me wanting more info on “my listing” in Agritopia (why? I can only assume they either don’t understand how IDX works, or they can’t use the MLS).

    I don’t “advertise other brokerages listings” on the Agritopia site. Yet it ranks very well for Agritopia real estate related searches.

    Rulings like this one, made 4+ years ago and with what appears to be a very minimal effort to “clarify” the ruling, are going to lead to yet more ridiculous rulings. People can chose not to believe that, but that won’t change the fact. I’ve already got pissed off brokers because my Agritopia site brings me buyers. Soon, one of them is going to file a complaint, and the board or the NAR, in all their cluelessness, is going to shut down my business.

    When (not if, but when) that day comes, there will be a lawsuit filed. If nothing else I can tie the mess up in court for long enough to figure out what to do.

    Until that day comes, it would behoove every agent on the planet to do everything they can to educate our boards and associations to help them leave 1975 behind and join the 21st century.

  318. Here is the email I sent to both Cliff & Steve:

    Dear Sirs:

    I am extremely concerned that MIBOR & NAR support of the interpretation of Google Indexing as something contrary to the rules & regulations for members is a poor, shortsighted policy.

    Seeing that NAR has been supporting and encouraging its members to participate in today’s cyber marketplace (ePRO® designation?), endorsing this ruling is contradictory to the actions and teachings of the national industry body. Our industry is behind the curve – especially in the eyes of the consumer, in competing within the electronic market. To classify someone who utilizes Google Indexing as a rogue agent, subject to sanctions is akin to teaching a person to eat with a fork then applying handcuffs and denying food because they might be more acceptable in public situations.

    NAR constantly touts the fact that over 80% of all buyers find their homes on the internet, is constantly improving REALTOR.com and providing more detailed information to the consumer—because the consumer demands it—buyer AND seller. To deny its members the right to compete on a level playing field with non-member agents is irresponsible and backwards. Today’s marketplace IS on the internet and tomorrow’s too. If we don’t evolve with the marketplace to satisfy the demands of the consumer, then NAR will lose its viability as the voice of the industry.

    I implore you to revisit this interpretation of the rules, as another look at it may provide the opportunity to correct a bad policy.

  319. Nice points Kent! Especially in regard to the E-Pro designation. Ironically this e-mail invitation just came in from my MLS:

    SEI VOW Seminar
    The seminar will cover the common questions about Virtual Office websites
    (VOW) & Listing Syndication (NLS), in conjunction with Internet Marketing.
    This seminar emphasizes how to get MAXIMUM Exposure for your listings and
    collecting more credible leads along the way.

    LOL

  320. @ James Malanowski – “Instead of railing on an anonymous poster, acknowledge that their point is valid as was mine before that no one decided to respond to.” — Your response demonstrates a level of professionalism that can be lacking in these types of discussions.

    @ Arlington real estate guy – “Just having your listing for example placed on your agent’s website and not in the IDX does not mean it would get indexed by google. Most agents have very stagnant websites without a regular flow of new content and therefore are crawled rather infrequently by google. And this is especially often the case with big time listing agents who often operate in the 20th century with their newspaper ad campaigns to get new listings. You almost never find their websites when you search for homes for sale or city state real estate….”

    — I said, “An agent’s personal listings CAN still be indexed in Google for consumers to find. IDX listings CAN still be displayed on cooperating broker’s websites for consumers to find.”

    — IF and HOW an agent or broker chooses to market their listings is their decision to make.

    — WHICH agent or broker a consumer chooses to use is their decision to make. If the consumer wants to take into account agent and/or broker advertising policies when making their decision they can.

  321. @All…

    A broker, with consent of the sellers, can place his/her listings in any media they choose, including 3PAs. As an IDX Participant, with the consent of the sellers, listings can be accessed through IDX websites. Most often there is a copyright notice to protect listings from “misappropriation” wherever the listing appears.

    I think NAR is wading into dangerous waters if it attempts to restrict (hello DoJ) indexing of listings by search engines.

  322. Jay: I agree, this argument has gotten absurd and the main issue has been lost and people are arguing about 3 or 4 different things here. As I stated in my original comment, NAR and MIBOR isn’t saying that IDX isn’t allowed, they’re saying we need to do everything in our power to prevent it from being “scraped.”

    I do believe the term “scrape” needs to be more clearly defined (I think we all agree on this point) because in the general public’s view scraping is related to a malicious act which search engines are obviously not doing. However, using the loose definition, indexing = scraping BUT scraping when related to “scraper sites” is a completely different animal. Would everyone feel better if NAR came back and said they agree with you and made the rule read “A REALTOR must do everything in their power to prevent scraping and indexing of IDX feeds?” :) That would get people REALLY pissed off, I’m sure, but I could see them doing it.

    As I stated in my prior comments, the way Google indexes can make an IDX snippet appear to be a listing owned by the site it has indexed which may or may not be the site of the broker that owns the listing. This may be considered to be against IDX policy and a form of “misappropriation” of IDX data.

    Now, the intent as I understand it, of NAR and the MLS in general, is to protect the proprietary IDX feed where brokers have signed on to allow advertising of their listings by other brokers in the manner deemed appropriate by the IDX rules. The NAR/MLS is not saying anything about brokers/agents advertising their OWN listings and allowing them to be indexed. How you )and others here) translate that to mean NAR is against listings being indexed totally or “No real estate agent can have a web site” is completely confusing to me.

    Your own example of your SEO building for Agritopia proves my point. Agents that properly use the tools available to everyone can rank highly for whatever terms they wish and often times blow the big guys out of the water – IDX or no IDX. I also receive a lot of calls from buyers and agents about other listings that are not mine solely because my name is all over the internet for my market area and keywords. I do not have an indexed IDX but I do blog my listings and submit them to the “enemy” (Zillow, Truilia, R.com, etc).

    3rd party sites do not have to deal with IDX rules because they aren’t using an IDX feed. WE, the listing agents/brokers, are GIVING them the information and then we have the nerve to sit back and bitch about how much higher they rank than the typical RE agent site?? This I don’t understand.

    I see nothing, nor do I get the feeling that NAR is trying to restrict the advertising we do on our listings they are only protecting the IDX feeds as we know them. If it’s the IDX rules we don’t like we should be trying to change those.

    Arlington RE Guy: This is exactly the conversation that should have been happening – somewhere along the line it went off in 15 different directions.

    As far as I know, EVERY MLS has the option for a broker to opt out of IDX … I believe it’s a part of the IDX rules. And yes, IDX rules require that the listing broker info be in every feed. I believe this is the whole problem NAR/MLS has with this topic … The search engine’s indexing of the listing data probably will not include the broker info as the index snippet usually scrapes (there’s that word again!) the first few lines of the data whereas the broker info is typically towards the end and therefore would not get indexed as part of the snippet shown in the search results.

    I’d be willing to be that if there was a way the the broker info was included in the index snippet EVERY TIME this would be a non-issue and we could go on and start bitching about how R.com treats us like crap instead!

  323. James wrote, “Agents that properly use the tools available to everyone can rank highly for whatever terms they wish and often times blow the big guys out of the water”

    Agreed. For today. Let’s just hope we don’t wake up tomorrow to yet another policy “clarification” where the ability to do that hasn’t been taken away. Trust me, I know many brokers that would love to see sites like my Agritopia site shut down.

    And that’s my concern that I can’t seem to make clear.

    When people that don’t understand the technology make rules, anything can happen.

    Declaring Google a “Scraper site” sets a very dangerous precedent.

    “Would everyone feel better if NAR came back and said they agree with you and made the rule read “A REALTOR must do everything in their power to prevent scraping and indexing of IDX feeds?” :) That would get people REALLY pissed off, I’m sure, but I could see them doing it.”

    Uhm, that’s exactly what they just did — “Consequently, NAR staff responded to questioners that the requirement to prevent scraping includes indexing.”

  324. Whoops, “Let’s just hope we don’t wake up tomorrow to yet another policy “clarification” where the ability to do that hasn’t been taken away.” should read, “Let’s just hope we don’t wake up tomorrow to yet another policy “clarification” where the ability to do that HAS been taken away.”

    Sure wish we could edit comments here….

  325. Malanowski, As you can see from this search of “mclean high school district homes for sale” http://www.justnewlistings.com/mclean-high-school.html that each snippet of mine has the listing broker clarified even before opening up the details. I think this is propert IDX feed and it is the MRIS policy. Here the defaul is for brokers to opt in the IDX, but they have the freedom to opt out now which I think solves everything. If I was listing agent I cannot imagine opting out of the IDX feed–it only makes it less convenient for buyers to find my listing. Why limit myself to MLS? I’d rather have my listings on 100s of broker sites/feeds/pages indexed by google.

    And of course here the address search for the 670 chain bridge road mclean virginia: http://www.google.com/search?q=670+chain+bridge+road+mclean+virginia&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a It’s great to beat out realtor and move.com. I wish individual agents’ sites dominated all these 3rd party sites and hopefully they will soon more and more with excellent websites. Mine is from real estate webmasters and if you’re interested contact me so I can be the referral.

    jay

  326. >>I’d be willing to be that if there was a way the the broker info was included in the index snippet EVERY TIME this would be a non-issue<<

    James, from my understanding of the analysis made (indexing = scraping, and intent and results do not matter), including the listing broker name wouldn’t make a difference to the conclusion reached by NAR and MIBOR. Moreover, many (if not most) IDX policies do not require display of the listing broker on the summary or short display display page but rather only on the detail page. When users click through from Google, the required disclosures about ownership of the listing information are present. Accordingly, I don’t see how indexing goes against IDX policy in terms of disclosure of the listing broker information.

  327. Jay: WooHoo, we agreed on something! I’m sure many brokers would like to see your site go away. As long as your site is ranked high because of your own craftsmanship and not built on the “misappropriation” of another broker’s data then you’ll be going strong until you decide to pull the plug yourself. Your concern is very clear, I just personally don’t believe the issue at hand is a catalyst for that fear to become a reality.

    When people that don’t understand the technology make rules, anything can happen.

    Declaring Google a “Scraper site” sets a very dangerous precedent.

    Hey, we agree again! I’ve sat on the Tech committee for our MLS and we had members on the committee that were there solely so us “geeks” didn’t “go too far!” I totally share your feelings on that one. However, on point #2, I say that NAR is NOT declaring a “scraper site” but they are considering what Google does as “scraping” in the widest use of the word … Again, we need to have a clear definition of terms.

    We all agree what Google does is not malicious, but the end result may appear to be a misappropriation of IDX data. This is what NAR is saying NO to. This is what I believe to be the core of the issue and the reasoning behind NARs response. Of course, I wasn’t there, so it is possible that the NAR folks involved with the initial statement are complete idiots, actually DO consider Google to be a “scraper site” as you and I and everyone else here defines one, and should have nothing to do with tech whatsoever.

    As I previously stated, if we could format the IDX feed so Google picks up the info of the broker that owns the listing then an index of IDX data would conform to IDX rules. Since it doesn’t (usually), then I would have to agree that an indexed IDX feed is against the rules and should be stopped. Or change the IDX rules.

  328. Jay: No worries … I understood your intent. And I do take intent into consideration unlike NAR! :)

    Michael: Okay, fair enough. Now we’re back to changes in technology. When IDX first came around, I’d venture to say that the designers of the original IDX solutions never envisioned the IDX system coming out of a framed solution. Now that we have the capability the rules need to be revisited as many have pointed out already. If the change needed is as simple as having separate definitions for “scraping” and “indexing” and allow one and not the other, GREAT. Something tells me that there’s something more behind the reasoning, though.

    All: Please understand, I’m playing the Devil’s Advocate here and trying to understand why the NAR did what they did. I am not necessarily opposed to indexing IDX data. I don’t really see the need for it other than for the agents that have no listings of their own and are trying to build their web presence on the backs of others instead of their own content. If you want to take a RSS feed of my listings to advertise them on your site, more power to you … My purpose in life is to get my listings sold. The more places they’re out there the better off I am.

  329. James,

    I don’t have a problem with displaying the broker name as a condition of the feed,nor do I think anyone else here does. There weren’t C&D orders issued because the listing broker name was omitted.

    Even when the broker name is included (my IDX is not indexed), many consumers are not able to determine who owns the listing….nor do they care.

    Are you saying that Google would not index the listing broker? I wouldn’t see that Google would omit that…it’s simply another term to them.

    I think the concern is that a malicious scraper will pick up the feed from “Paula’s” site. But, as we can all see from this example…….it’s very simple to have a bot scrape Realtor.com and put up a site. Since properties from the country are currently scrapable, how does denying “Paula” keep data out of the hand’s of scrapers?

    retechulous.com/category/nar-blows/

    Getting ready to head to DC on Wed. Be at Petits Plats earlier in the eve…and probably head to Perry’s later in the night.

  330. I could not get through 392 comments so I apologize if someone brought up this point (if someone did, let me know so I can shake their hand if we meet).

    This decision does not belong to Realtors as a group and perhaps not to listings agents (unless authorized by their clients) but it belongs, in right, to THE HOMEOWNER. Has anyone bothered to ask them?

    The recent NAR-DOJ settlement gives the home seller the right to decide the extent of internet marketing, including none at all. ANY MLS that usurps this owners right runs the risk of a class action for interference with prospective economic advantage or similar type tort that says ‘Don’t get in the way of my sale”.

    Hi Todd. Is the kitchen getting hot?

  331. I am not an attorney, but have given much thought to how the DOJ would view this.

  332. James – Even in the snippet view of many listings, my IDX does state the listing brokers name. It is also on the detailed view. No violations here!

  333. Deborah: You’re right, the buyers call the agent whose site they’re on at the time (if you’re lucky) – the broker info is rarely paid attention to. What I was saying is that the broker info is usually towards the bottom of the feed so Google wouldn’t index it because it is too far down the line to make it into the snippet – not that Google ignores it on purpose.

    I believe that is exactly their concern – they are trying to protect the IDX feed from malicious offenders. If Paula’s site had a framed solution it would not be scraped/indexed therefore eliminating the problem.

    R.com is not populated by an IDX feed so the comparison is invalid. It is the IDX rules that need to be changed if the majority wishes to loosen the current restrictions on how the feeds are handled. I’m sure R.com uses RSS feeds so scrapers can take advantage of them.

  334. James, what do you mean R dot com is not an idx feed, if not it is a raw data feed from our MLS’s and they put it in the fields like they want to.

    They download from A2 every day several times, in smaller MLS’s it is not everyday.

    It is a feed, whether it is IDX or raw data, I don’t know but they get their data from us. Just sayin’.

  335. On a lighter note – there are currently about 20 returns for “2920292 carmel”; unfortunately not one will help sell the home.

    Thanks all!

  336. “I am really trying to stay on task here, but have to ask – how is it that they can open the data to Google through R.com?”

    It’s a great question for r.com. It’s my understanding that they give Google special access that is not open. If you could figure out how to replicate what they do for a reasonable amount of money, and without getting banned by Google, you wouldn’t have a problem.

  337. Pingback: Google Helps Buyers Find Homes

  338. “It’s a great question for r.com. It’s my understanding that they give Google special access that is not open. If you could figure out how to replicate what they do for a reasonable amount of money, and without getting banned by Google, you wouldn’t have a problem.”

    I think that Paula’s IDX does exactly that! But isn’t that being defined as “scrapeing” by NAR?

  339. Todd – A misunderstanding here – let me rephrase; how is it they (the local boards) can provide the data to R.com for Google to index?

    Special access that is not open – what exactly does that mean? Not open to what?

    Why should I have to try to figure out what they are doing without getting banned from Google? I should be allowed access to the feed because I am a dues paying member not because I have a special deal with the NAR or the local boards.

    IF Google is a scraper site, then the NAR and local boards should be equally concerned about the data being fed to R.com as they are about the data displayed on my site.

  340. @tcar

    Hats off to you for getting Jay and Paula to the event to speak. In order for members to make informed decisions, they must understand the issues. BY hearing opposing viewpoints, members can decide, knowing what effect their decisions will have on their CLIENTS’ interests. In the case of listing agents, it is the home owner.

    The debate on scraping and indexing reminds me of the famous Sony Betamax case. Those who opposed the copying technology argued it could be (and would be) used for bad, i.e. copyright infringement, and thus should be outlawed.

    The US Supreme Court ruled in favor of Sony 5-4, because the technology in question had significant non-infringing uses.

    Scraping is NOT evil in and of itself and has significant non-infringing uses.

  341. jfsellsius – I like the comparison with the Betamax issue. Even if you prevent “scraping” (as it works now), if someone really wants the data, they’ll get it. Whether they (being people who want the data for their own nefarious purposes) scrape a site or employ people to cut and paste text, images, and code – they will find a way to do it (DRM anyone?)

  342. We need to see more agents and brokers speaking up.

    Here is how we are now taking action.

    http://skyrealtyaustin.com/blog/why-we-no-longer-syndicate-our-listings.html

  343. When we submit listings to our MLS there is a ‘yes’ or ‘no’ box for adding that listing to the IDX searches. If you do not want your listing to show up under other brokers IDX’s then you can do it here.

    To take away the ability of the REALTORS to put these listings in front of their consumers makes no sense! Giving that ability to other sites that are just advertising machines and not REALTORS makes even less sense.

    P.S. Is it just coincidence that Trulia has a banner ad at the bottom of this page?!

  344. I have enjoyed a real estate career for almost 20 years now and have been proud to be a Broker for about 18 years. Being a Realtor has definitely been a highlight of my life and I have always been so proud to perform the duties of a Realtor and a Broker.

    Today, as I read your comments in this latest battle between Realtors and local boards and NAR, I am even more proud that Realtors are willing to fight anyone who gets between them and their idea of performing their duties and enhancing their methods of service to the public. Realtors are the “real deal” and we cannot lose sight of that.

    Go Realtors!

  345. I am all for syndicating to broker/agent sites, but not other sites

  346. What good can Trulia or Zillow do without the information that we provide them? None! Why should consumers go to them, and not us, to find homes for sale and home values? Go to a local Realtor and his/her IDX-equipped website! God help us if we ever get to the point of being unable to display listings of other brokers on our websites.

  347. Unfortunately, I am no longer surprised at these types of decisions by NAR.

    Ever since the MLS decision fiasco and their so called independent researchers, I’ve lost complete faith in them doing anything that actually helps their members.

    Their methods are out dated and tend to side on hurting their members rather than helping.

    I wouldn’t be surprised if this was yet another ploy for them to push more agents to only advertise on r.com and force agents websites into oblivion.

  348. When I hear of these decisions, it bothers me immensely that there continues to be one-sided decision making running NAR.

    When will it end?

    While not a highly technical person myself, even I know that big G is NOT a scraper site.

    It’s ludicrous to even consider it as such.

    Let’s all hope for the best and hold on tight. Though I’m not holding my breath on yet another decision that may negatively impact my business.

  349. I had a few phone calls with MLS Committee members. Thanks Jim Duncan for bringing up this as a suggestion! I took your suggestion to heart. It’s great that we discuss the issue here, and there is do much value in the viral nature of this thread, and the power of AG. Additionally, this message needs to reach others who might not come here. For some MLS Committee members, this is a verbal discussion that contained many foreign terms and concepts. I wrote a summary sheet for MLS Committee members that lightly explains indexing, scraping, framing, along with the challenges and opportunities presented by the current rule. I wrote this summary to provide as a follow up to the conversations.

    Although it really is a summary vs. a typical blog post. I did throw this up on a WordPress blog on a URL that has just been parked and collecting dust. OK…the domain name is RiverPurple….so go figure that one out. Because I just went live with the URL today, it won’t come up on a SERP. But, you can get it via link or in the address bar. It’s not really a blog…but Worldpress is such a great vehicle and served the purpose of giving me a place to park the summary.

    I also emailed the document as a PDF in follow up to telephone conversations.

    If anyone finds my summary useful for any discussions they might have, feel free to use it.

    Online via WordPress via the link on my name for this post. There’s also a link for a PDF there.

    Daniel’s quote is in the summary…..and I am DMing him to ask him if I can attach his name in place of Realtor DR.

    If anyone thinks I need to tweak this or make any corrections…please let me know. Tks much! My litmus test was to give this to someone who didn’t even know what an IDX was to see if they could understand it. After reading it, they told me that it was like not letting anyone come in the library and read the books there, because a few people might break copyright laws by making copies.

    I am driving to DC from NJ tomorrow. Hope to see some of you Wed night at Petits Plats at 7pm.

    Please……if any tweaks or changes are needed on this, do let me know.

    Deb

  350. Pingback: Social Media Tips from NAR’s Todd Carpenter : Speaking of Real Estate

  351. Deborah, a couple of things to consider. In your introductory paragraph, you state: “When the above rule was adopted in 2005, IDX data was delivered to Participants (REALTORS®) only in “framed” solutions. In these “framed” solutions, Google is unable to “crawl” and “index” property information. Technology has evolved and IDX Solution Providers can now deliver “crawable” and “indexable” compliant previews of MLS data.”

    I don’t believe this is technically accurate. In 2005, one of the most common ways of delivering IDX data was by FTP and the broker could put the data on a web site that would be indexed by Google. Also, Google can crawl and index framed solutions, but they don’t (or, if they do, the juice goes to the URL of the framed solution). So, there really hasn’t been a change or evolution in technology, but the success of the SEO on Paula’s site finally got someone’s attention.

    In your discussion of 3PAs, you include ListHub but I think they’re more accurately categorized as a syndicator, which sends data to 3PAs. ListHub does have landing pages to help track click-rates from the syndication for those that don’t have landing pages, so perhaps they could be considered a 3PA. I think the more common examples of 3PAs, though, are Realtor.com, Google, Yahoo!, Zillow, and Trulia.

    Last, and most important, I think your conclusion could benefit from specifying exactly what you want to happen. How would you modify the language? Further, I suggest that you provide direction for what the Committee should do in the interim before the language can be modified. Any change to this language will take months or years to change at all the MLSs in the country, and so I think it’s important for the Committee to clarify interpretation of the existing language or at least tell Associations that NAR is studying the issue and so they should hold off on interpreting that provision.

  352. Deborah,

    The River Purple site works well…and I appreciate the link to “Retechulous.Com,” but it’s not working :)

    [Seeing it in lights, I’m regretting the “NAR-Blow’s” category nomenclature slam a bit. Probably a little crude, I know…but at least I didn’t make fun of anyone’s tie :) ]

  353. Pingback: Is the National Association of Realtors Calling Google An Evil Scraper?

  354. Hi Michael,

    Thank you so much for taking the time to review and respond. I am anxious to meet you in person! I am still in NJ, going to start my drive in the next few hours.

    I do understand, completely agree and acknowledge every point you made. I was trying to keep the doc shorter…..and minimize the levels. Example: If I say Google can crawl the framed solutions, I have to explain why and how it’s a different URL. If I say that Google could always crawl, the question immediately rises to why this now becomes an issue when it previously was not. I am certainly aware that more IDX solution providers are responding to the demand of those of us who want an SEO friendly solution at our domain. I was trying to cut through layers with a goal of simplicity of defining and understanding the issues. Your points are very well taken.

    You are on point again, with your accurate classification of ListHub as a syndicator. I lumped them into the 3PA category because I am aware of MLSs who provide feeds to ListHub, and sought to establish the role the MLSs are playing in making the data accessible or at risk as they might opine.

    My follow up, after #Midyear will include more specific suggestions for the next steps. I was curious to know what conversations would transpire in the next few days at #Midyear on this.

    It was a difficult summary for me to write, knowing the reader may be unfamiliar with some of the jargon. Myself, I don’t know much….just enough to take another step and not enough to know when I shouldn’t. Full disclaimer here……I am NOT a techie.

    Maybe I should modify the doc. I have to wrap up here in NJ and hit the road and will give it thought along the drive. Thanks again for taking the time to read and comment. I look forward to meeting you soon.

  355. Hi Ryan and also Daniel,

    I fixed the link…it should work now. Sorry :-(

    I was unphased by the subdomain name. I really thought it was very telling and educational to illustrate that the barn door is already open. Demanding that Paula refrain or modify her website does not protect data and guarantee data integrity.

    What you did was great……as a demonstration.

    I did not identify Daniel by name, but rather by his initials. So, Daniel…..I am more than happy to edit the intials to full name and link. I fully acknowledged that the quote was not from me, but another Realtor. Daniel’s quote was blunt…..and so spot on.

    It’s great that Ryan made this feed, Michael imparts expertise and we have great representatives in Paula and Jay. So thanks to all…including the many who were not mentioned by name.

  356. Ugh. Any possibility you can get Trulia to stop advertising on the bottom of your blog? It makes it look like you endorse them which I’m quite sure you don’t.

  357. We have links to Deborah’s write-up and Paula’s original article (this) on our Facebook event page for tonight’s REALTOR Mixer at PetitsPlats (.com) across from Woodley Park Metro in DC, 7 p.m. 2653 Connecticut Ave. 20008 — so please join the discussion, online, by phone or in person! Twitter updates likely – RealtyLee

  358. We are in the age of the consumer!!! We are no longer an agent focused or brokerage focused business. The consumer is screaming at us that it is not about us it is about them.

    They want information freely and the majority of consumers will go back to a trusted agent to assist them through a transaction.

    I have had the opportunity to speak personally with Richard Barton from Zillow and he has 3 basic rules about age of the consumer.

    1. If it can be known, it will be known
    2. If it can be free it will be free
    3. If it can be rated it will be rated

    We all now that our business is complex and this is the reason that consumers turn to us. What they are asking from us is to be competent, relevant, and professional.

    Consumers use the internet to search for property and they should be able to find property in as many places as possibly.

  359. Pingback: Online Marketing Blog » Blog Archive » Jumping into the Real Estate Fray, Part 1

  360. This is bizarrely strange, the impact of which has still not fully realized itself. I have to wonder if there will come a time when sellers will get wind of these rules, realize the handcuffs they place on their agents ability to market their own homes and begin refusing to sign over the copyrights of their homes, images and details to be turned into listings owned by the MLS.

    That day is coming.

  361. As an emerging technology company and provider of an advanced home search solution we continue to be amazed that MLS organizations (and NAR, I might add) seem to believe that their job is to “protect the data” at all costs, even if it means severely restricting the ability of their MEMBERS to do their jobs. We all look forward to the day when consumers are given what they need to thoughtfully search and find their dream homes without distrusting the sources. Consumers are getting more and more savvy and they won’t put-up with MLS organizations, NAR or local brokers restricting their access to information. Google’s case is interesting, and they likely are breaking the traditional MLS rules, but millions upon millions of consumers go to Google because they trust them. I wish I could say the same thing about NAR and Realtors in general. My advice, give access to all the data, make it easy for smart agents and brokers to use technology to SERVE consumers. Maybe someday, consumers will actually trust the industry. Twitter @JeffPref

  362. The idea is to sell homes. The sooner the better. With >70% of homebuyers going to the internet first, we are all better off with more access not less. As Jeff Johnson wrote, “give access to all the data, make it easy for smart agents and brokers to use technology to SERVE consumers.” Amen Brother.

  363. Mike Russo May 13th, 2009 at 10:53 pm

    “We are in the age of the consumer!!! We are no longer an agent focused or brokerage focused business. The consumer is screaming at us that it is not about us it is about them.”

    Mike Russo is absolutely right!

    When will the real estate INDUSTRY realize this. Many agents and brokers are getting it, but are being threatened by MLS and associations to conform to the old ways.

    I have set-up a hashtag at #GoogleIDXScraping

    Great to see an ongoing discussion on this topic on Twitter

    Jeff Johnston, CEO
    besthomepro.com

    Twitter
    @JeffPref

  364. “If Paula’s site had a framed solution it would not be scraped/indexed therefore eliminating the problem.”

    Google can and does index content within frames depending on how the frame is set up. Page rank can flow as well.

    @jsellsius has it right with regard to the DOJ. Unfortunately, I still think the only thing NAR will understand is a lawsuit.

  365. Well-vetted, reasonable, consistent policies and interpretations will benefit everyone — real estate professionals, technology and IDX providers, clients and consumers!

    We as REALTORS have got to give the consumes what they want – we can no longer hide behind a MLS… If we don’t give the consumer what they want – they WILL find it without us…

    In the ever chaning field of technology a set of 2005 policies are out dated and need to be revised…

  366. [Update]

    Just in: Read the proposed committee action here.

    Great job.

  367. Tulia posted and open letter to Sky Realty

  368. Tulia posted and open letter to Sky Realty
    trulia.com/blog/pierre_calzadilla/2009/05/an_open_letter_to_mr_red

  369. Pingback: MLS Committee Approves Update to IDX Policy : Speaking of Real Estate

  370. I like the revised language… Good job!

    On another issue, I replied on Trulia’s Voices, to
    Curtis Reddehase as follows:

    As you know from my earlier post on your blog, I think your idea is good, but with sellers clinging to the “everywhere is better” approach, it will be difficult to get many brokers on board early on. I think the question to the “everywhere is better” group is “if all third party aggregators disappeared tomorrow, where would the Internet buyers go to find properties for sale?” The question is rhetorical, of course, as the answer is obvious.

  371. I don’t believe we’ve seen the final round with NAR/DOJ; as independent contractors, we have little choice but to join NAR, even when the majority disagrees with it’s stance on issues like this one. This and at the local level, the information being owned by controlling powers in the market or among boards is just as troubling. Is is possible that a national real time database by a third party entity to replace the local boards’ and NAR’s control of the information will be the game changer that technology has brought to other industries?

  372. Pingback: NAR’s IDX Rule Changes Need More Study : Speaking of Real Estate

  373. Ok, so I just paid my yearly board dues and elected not to pay the suggested RPAC contribution. :)

    Finally, a good decision.

    The should just unplug this whole thing and then we can send smoke signals into the air.

  374. Kevin- actually your RPAC contributions help more than anything else NAR does.

  375. yeah, maybe …but I’m cutting my losses.

    It’s ludicrous that Zillow and Trulia can use our content to make money…but WE can’t.

    There needs to be cleaning out starting from the top down.

    I think that the MLS boards and NAR are old-fashioned and, in general, douchebags.

    Yup, I said it.

    Douchebags.

    It shows exactly how little they know about the profession that has all but passed them by.

  376. i don’t disagree with your sentiment Kevin. in fact, i hope these desires can effect some real change. what would an improved NAR look like? what goals would you have to making things ‘better’?

  377. Kevin :) – Thanks for the support! I’ll take all I can get – now, I’ll have to practice starting a fire with kindling.

  378. Brad – I believe you are right about a tipping point. We can’t let this opportunity to make a change be lost. I also wonder about a future of new leaders in the industry – first, take back or get rid of R.com. Then bring in the best and brightest leaders at the local level, who are commited to agents success.

  379. Paula: the third step is to streamline governance at local and national level so that there is at least a prayer of a chance of creating change in a timely manner. If organizations can’t do that, all those best and brightest leaders you recruit will get discouraged and go away.

  380. Finally, the beginning of the END of the MLS monopoly! Third parties will rule very soon.

  381. Simply amazing how NAR and REALTOR(r) organizations push electronic media, social networking and 21st century type technology, however when it comes to doing our diligence as a listing agent to expose our sellers property in the best means possible we get lambasted. YIKES!

    I had heard about this earlier in the month and it left me shaking my head, still shaking my head.

    On a ligher note, check out the video “Loan Mod Fail Think Short Sale” song http://www.youtube.com/watch?v=ADS-LE-J3s4

  382. Quoted from Four People Who Don’t Get The Internet – http://www.webpronews.com/topnews/2009/05/21/four-people-who-dont-get-the-internet

    “Entrenched power and money structures need predictability and control if they are to continue to succeed. And that makes the Internet a problem for them, not for the rest of us. “The Internet” carries the only current populous hope of the people and it’s driving the powers that be absolutely crazy, save for Nader, who just thinks it’s pointless.”

    “It isn’t lack of control that’s the problem, nor populism, nor cultists, nor fragmentation. It’s the ever-increasing desire of the few to control what has become the masses’ medium of choice. Governments, ISPs, Entertainment, Newspapers, and Others want this new Wild West reformed into something orderly, something controlled, something (immensely) profitable. The desire to lump “the Internet” under one easily understood label is only very nearly as strong.”

  383. Loan mod fail… think Short Sale…

    Thanks Frederic — Now I have that jingle stuck in my head.

    @ Paula,

    You hit the nail right on the head with your last comment. Here in Las Vegas… our MLS is much more in tune with technology in comparison to another MLS I belonged to. (Hint… second largest in the country owned by Big Brokers with personal interests and as archaic as the dinosaurs.)

    Let’s face it… the majority of us are busy and sleeping at the wheel when it comes to local elections which have an impact on national leadership.

    The moral of the story is to get more involved such as Brad Nix has been suggesting since day 1 of this topic.

    If we really are technologically superior to the people currently in leadership positions… then what does that really tell us about our marketing skills?

  384. The Justice Department may need another call. Here inlays the problem, everybody is entitled to the same benefit as Realtors….Anyone…..not just certain ones….any Realtor. Some however, got innovative and learned how to get their IDX solution search optimized and now do a better job of advertising listings than Those folks who actually take the listings. So, maybe its time for the folks who b & moan to update their business model or become irrelevant to real estate. No, but not doing nothing for clients and expecting a commission still seems to be best preserved.

    This has nothing to do with rules, this has to do with the blatant and perverse anti-competitive business promotion by a bunch of old stalwarts running boards around the country. The exact same anti – competitive issue that was Addressed by The Department of Justice These folks had better get with the program before Trulia, or Zillow become the new MLS system. Pissing off Realtors who are innovative is the worst thing NAR and local boards. This rulling just further confirms my belif that cronyism and hipocrcacy has a love affair with brokers whom still belive clients come through the office front door. or whose only souce of advertising comes through Realtor.com…the NARs cash cow. Yes.. its nice to know Realors dues go protcect that golden goose. Sorry, I coulden’t publish my actual name for fear 2 old brokers would beat me down with an old MLS book.

    This has gotten to the point of infuriating, i’m pissed.

  385. I’m sure many have quit following this post, but this is the best way to post this current Google Alert I received in my email today. On the Weichert national site but not their listings. A local, traditional companies listings. Broker: Edward Surovell Realtors But, links to Weichert

    Google Web Alert for: houses for sale in ann arbor

    1207 W LIBERTY Ann Arbor MI 48103 MLS2905280 Weichertcom

    1207 W LIBERTY Ann Arbor MI 48103 Home for Sale for $429900 with 3 bedrooms … MI real estate listings virtual tours and open houses in Ann Arbor Michigan …

    423 BURR OAK Ann Arbor MI 48103 MLS2905272 Weichertcom

    423 BURR OAK Ann Arbor MI 48103 Home for Sale for $284900 with 4 bedrooms … MI real estate listings virtual tours and open houses in Ann Arbor Michigan …

  386. Missy,
    I received the alert in my email on a new comment. Thanks for the update!
    Deb

  387. I have found some other IDX solutions that are getting indexed if you get your website and the IDX solution from the same vendor. Ironically the vendor is the provider of the MLS platform for the board also. Anybody using SEI?

  388. Nuts.

  389. Pingback: ARMLS Comes Out In Favor Of Realtors! | Real Estate Blog Magazine - Opinion Column - AgentGenius

  390. Fat Tuesday – In light of this weeks news from MIBOR that they will not change their position or allow the rule to remain as it was until a decision is made, your idea is finding merit in the real estate community. The fear is, it will take another 5 years like it did with the VOW policy.

  391. Pingback: Real Estate Search – Google to Punk Your Local MLS? Is It Possible? | Real Estate Opinion MAG - AgentGenius

  392. Personally, I don’t think listings should get indexed although I will do it to remain competitive. I think it’s against our “exclusive right to sell” contract. Yeah, it gets more exposure, but we work hard and spend a lot of money to get the listing in the 1st place. Allowing someone to take advantage of my efforts is double you are oh in gee. I’m all about IDX and using technology to marketing yourself. I’ve been doing real estate Internet marketing for almost a decade now, but having one of my listings post as if it’s “Featured” on somebody else site is quite in fact plagiarism. Plagiarism is also wrong, and have always been an advocate of writing custom content. This is no longer taking advantage of technology. This is called cheating and lying since simply modifying an IDX data feed really doesn’t make anybody a good agent.

    May I laugh before any of you who disagree with me laugh first? I’ve generated a shit load of business off the Internet by taking advantage of technology, and I will in fact do whatever is possible to remain competitive. However, I will trust my karma over trying to be sly or thinking some generic marketing strategy will make me rich before it makes me poor.

    I say do business the right way instead of looking for ways to cheat those who are. Besides, simply getting at the top of serps doesn’t mean you’ll either be there for long, or convert any of those to sales. If you’re a cheater, pretty sure it will show when meeting the client in person.

    Supporting tricky ways to prevent listings for better search results may be a good selling point when doing a listing presentation, but still comes down to being a cheater–and that people won’t trust. Funny that I wonder sometimes why people feel they can’t trust agents. Can’t beat ‘em…join ‘em I guess?

  393. Justin –

    I hear you saying two things here:

    1) Indexing listings is wrong, unethical, cheating, lying, plagiarism, against our exclusive right to sell, doing business the wrong way

    2) You’ll do it anyway to remain competitive

    For the record:

    * my site ranked before I indexed listings and still does, because I do have original content.
    * I do NOT feature anyone’s listing as my own, they show up in a general search based on the criteria entered.

    I work as hard as anyone else to get listings and if another agent indexes my listings on their site – great – my client gets more exposure. If they bring the buyer through their online efforts, better yet!

  394. Pingback: Benn/AgentGenius

  395. Great article. Overwhelming number of comments. At the end of the day I want to represent my seller and make sure a real live buyer can find their home when the buyer is ready willing and able to purchase it. And I want my buyers to be able to have simple easy efficient way to find the houses they want to look at. Thats really all I want.
    Ken

  396. Pingback: Welcome to the Property Indexer Blog! – Property Indexer

  397. Taking away the ability for Realtors to have IDX listing data indexed by Google will benefit the new media and lead aggregation companies and will hurt the Realtors that have built consumer friendly websites that help people find information about the homes that they are looking for.

  398. Perhaps another (late, I realize) answer is in order to this question – we have a website with an IDX map-based search. Recently, we received an email from our provider, informing us that they can no longer provide IDX in our area. The reason, as I explored further, was that our MLS had decided to levy charges of $50 for each website, or $300 for an unlimited number of websites served with IDX by each provider. Our IDX provider had been providing IDX for the amazing price of $199/yr, which we thought was a wonderful buy for a great product. Problem is, it isn’t cost effective for them to continue in a market until they have large enough numbers to justify the monthly fees from our MLS.

    It is my impression that our MLS board decided to structure these charges as they did to remove the lower priced IDX providers from the marketplace. Those lower priced IDX providers appeal to the new small brokerage on a budget. We were able to provide a search map reasonably at the prices our provider was able to offer. The large brokerages can easily afford $50, or even $100s per month for IDX maps for their websites, so now, they are in the catbird seat, if they are able to limit the number of small brokerages with their own websites providing the same or similar map based searches. I strongly suspect that this is the real reason they don’t want Google to index their data.

    Perhaps Google is not attempting to become a “giant MLS” as some have theorized. An MLS really needs to be somewhat local in order to best serve the needs of its community of agents and buyers and sellers.

    Our IDX provider now has a new map-based search, using Google data. It doesn’t sound as if Google is attempting to run IDX providers out of business, does it?

    As an occasional listing agent, I’m delighted when anyone wants to feature my listings on their websites. (Wellllllllllll, perhaps not just ANYONE. I had quite a lot of difficulty a while back when the “Zestimate” on Zillow was tens of thousands of $ off base – they were using townhomes and homes in other, lesser neighborhoods as comps for a single family home in a very desirable neighborhood). But, most of the other sites which indexed our listing were respectful of the data and photos, and I believe they helped to sell the house. I deliberately posted our listing on as many other sites as I could find. So, I don’t think Google’s intent is evil. I think they are going for internet market share by providing as much information on any topic possible to the searching public. If we are able to provide map based search from Google data, when our own MLS has seemed so intent on overcharging in order to discourage us for purchasing IDX data from an MLS we already pay large monthly fees to support, we will be happy.

    It surprises me that some folks seem to consider the use of IDX- or Google-data map based searches to be “cheating” or “stealing”. Do they not understand the benefits to them and to their clients which are garnered from multiple listing? Real estate is such a funny marketplace. I’m disappointed that we don’t all consider the needs of clients to get their homes sold or to find homes to buy are paramount, and that some agents’ selfish wishes to occasionally double end a transaction are of little importance in an ethical marketplace.

  399. Vicki – I find your response very interesting and most timely. As we near the NAR national convention next month, there seems to be more interest about the issue.I know many people have been diligently working to bring a satisfactory resolution to the question of Google’s (or any search engine) ability to index the MLS data.

    Google doesn’t really have an intent as it relates to the data, except to index relevant information and return that information to the end user. Google does not take data from our websites to build their own database of properties; they didn’t have to. Brokers and agents have willingly given Google their data through Google Base. The theory was, “at least I will get the lead, if a buyer inquires”.

    Now, another company is using the data to bring a mappable IDX to anyone who wants one and I applaud them.

    I am not surprised by the number of IDX companies who are working on indexable IDX solutions or the number of brokerages who opposed it, now scrambling to provide it. I believe we will see changes in November; but am not sure to what extent.

    My stance on this has always been, “Realtors and agents are the people who have the most to gain and the most to lose by allowing the data to be indexed”. Our livelihood depends on maintaining the accuracy and integrity of the data. Our clients expect accurate, complete information, but once it goes to a site who has the authority to disseminate the data, we lost it.

    The truth is, we already lost control of the data and I believe there is no going back. In our efforts to control the data, we gave it all away. Now, we have many third party sites with the data, who are competing with us in the search engines using the data we gave them. Having the ability to index will at least provide us the opportunity to compete and since it comes from our local MLS, it will be the most accurate.

    I hope your boards decision to now charge IDX vendors a monthly fee doesn’t become the option for other boards looking to eliminate IDX companies from providing options. It’s already been suggested there is some anti-competitive behavior attributed to the our National and local boards decision, and the last thing we need as a profession is more infighting and rules settled by the DOJ.

  400. For those still following this saga (and honestly, you should be) just moments ago the NAR’s Board of Directors just passed, without debate, changes in the MLS policy that allows indexing of IDX listings:

    Old Language:

    2. Participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction or use of the MLS database.

    New Language:

    MLS participants may not use IDX-provided listings for any purpose other than display on their websites. This does not require participants to prevent indexing of IDX listings by recognized search engines.

  401. I’ve got you bet! Our MLS actually has an opt-out / opt-in option when listing your property. You can choose to not share your listing with any other on-line IDX provider( except big brother Realtor.com). When I first saw this option earlier this year I was just floored. Why would any Realtor hold back his listing from the web??? I recently heard some Brokers telling their agents to opt-out of Internet marketing??? Like I said crazy stuff. I think its just fear. Those in charge know that knowledge is power and only want transparency on their terms. Great blog!!
    .

  402. Chuck,

    You asked
    “Why would any Realtor hold back his listing from the web???”
    Some MLS’s send the information out with the Broker contact information rather than the agent contact information, you would have to check your MLS out and see how the information is being presented on the sites they share with. In the event your MLS is placing the broker contact information you would not receive any leads, they would go to your broker and be distributed. If that is the case then you would want to opt out and place the listing on each site yourself with YOUR contact information.

    Laurie

  403. Laurie –

    With that argument, it sounds like the primary reason for putting a listing on the internet is to generate leads for the listing agent.

    I though it was to sell the home.

  404. Jay,

    First, it is not an argument, it is a fact. Second the primary purpose is to sell the home. As the listing agent, nobody is in a better position than you to sell that home to either another agent or to a buyer, nobody other than the seller should know more about that home than the listing agent. It is in the sellers best interest that the phone calls (READ: LEADS) are directed at the listing agent.

    Further. It is expensive and time consuming to generate both listings and leads. It is despicable that anyone, yourself included, would make disparaging remarks about agents generating leads in any form.

    Without leads there would be no buyers for those listings.

  405. Laurie, that approach puts you and your seller at a distinct disadvantage. I have a list of all agents who have done that in San Diego. I’ll pull it out at a listing presentation if the seller is considering listing with other agents and let the seller know that the agents on this list have taken deliberate steps to PREVENT the widest possible online exposure of their listings because of the argument you just made.

    Do you disclose to the seller up front that you will prevent their listing from being included in IDX?.

  406. Jay,

    Further to that conversation. It is in the Brokers best interest to have the leads directed at them. Directing leads at agents with smaller commission cuts leave a nice meaty pot for the broker.

    You are a broker owner now aren’t you Jay?

  407. Bob,

    Scroll up. Nowhere did you see me say that I do not include my clients in that option. I was bringing to light that fact that agents should check how their MLS handles it and I went on further to suggest that if they left it out they should post the listings themselves individually.

  408. Relax Laurie.

    Yes, I am a broker owner and I direct all inquires on listings back to the listing agent. Even though in Arizona the listing belongs to the broker, I consider them my agent’s listing, and as you point out, they are in the best position to sell the home.

    Sorry you found my comment “despicable”, it wasn’t intended that way. We both know there are agents out there that DO think the primary reason for taking a listing is to generate leads. (Of note, I am NOT saying you are such an agent.) Your comment, to me, sounded like the primary reason for putting a listing on the internet was to generate leads.

    “Directing leads at agents with smaller commission cuts leave a nice meaty pot for the broker” is a despicable practice as well. Yes, that happens too, but not in my brokerage.

    “It is despicable that anyone, yourself included, would make disparaging remarks about agents generating leads in any form.”

    “Any form” of lead generation is acceptable? Not in my opinion. There are unethical forms of lead generation. No agent anywhere should have the ability to generate leads in ANY form.

  409. Laurie, you said the following:

    “If that is the case then you would want to opt out and place the listing on each site yourself with YOUR contact information.”

    Having built an IDX and a VOW, I know that I (as the IDX) can choose to list either the agent and broker name or just the broker to fulfill the IDX requirement. The IDX opt out Chuck referred to, which is similar to what agents can do in San Diego, doesn’t discern between those IDX systems that include the agent name and those that dont. Its an all or nothing proposition. Since you stated that “you would want to opt out and place the listing on each site yourself with YOUR contact information.”, I assumed (and I assume Jay assumed as well) that was also your stance.

    If an agent opts a listing out in San Diego, then they lose the huge exposure that IDX provides. I still dont see how you can defend that.

    • You assumed correctly Bob. “Opting out” *removes* the listing from the IDX feed. “Placing the listing on each site yourself” (ie: Trulia, Zillow, and all the other listing aggregator sites) doesn’t get a listing anywhere close to the exposure it gets on local agents IDX displays.

      Personally, I want my sellers listings on as many sites as possible. Who picks up the phone when someone calls on it is the least of my concerns.

      Anyone on the planet is welcome to “Feature” one of my listings in any legal manner they like and they are more than welcome to plaster their name and contact info all over it.

  410. Very happy to hear that NAR came to its senses on this issue!

  411. I agree with Laurie’s instincts on checking what your broker is up to. Most brokers’ strategies have been about building their brand to recruit and plunder agent earnings rather than helping their agents’ sell more homes – more easily.

    Here’s the Dirty Secret
    IDX Home Search sites are a great way to “capture leads” and generate business for yourself off “other agents’ listings.” You have a good website that shows a lot of listings, you probably end up getting the lead.

    But, does it really help you to have your own listings shown on other people’s websites?

    We know it helps other tech savvy smart agents with good websites. People will go on their websites and call the other agent, not you.

    How about on sites like Trulia? That’s a little better since they link to you.

    However, my hunch is that if your listings are NOT syndicated through IDX, it doesn’t matter in terms of getting the house sold. If it’s in the MLS, most agents – with ready, willing, and able buyers – i.e.the kind you want, as opposed to internet browsers…are going to pull it up and bring their client through it.

    I’d love to see some research on whether or not any buyer ever finds a home online that his or her agent wouldn’t have shown anyway.

    If you really want maximum Agent Traffic to your home, try offering 4 or even 5% to the buyers’ agent. Not to be too cynical, but a lot of agents will drag their client through your home, just praying that the client will like it. Then when it comes time to negotiations, play hardball and let the agent sell the deal to their client. They’ll have more incentive to do so.

    • Sam – Your last statement about offering a high co-broke is what give Realtors a bad name. It would be truly disgusting if ANY agent would do what you are saying. Only greedy agents, who don’t give a crap about their client, would do this.

  412. Thanks, Sam, for making one of the best arguments for

    1 – Divorcing the commission
    2 – Exclusive Buyer Agency.

    If you really want maximum Agent Traffic to your home, try offering 4 or even 5% to the buyers’ agent. Not to be too cynical, but a lot of agents will drag their client through your home, just praying that the client will like it. Then when it comes time to negotiations, play hardball and let the agent sell the deal to their client. They’ll have more incentive to do so.

  413. Yep Jim, it is.

    Buyers should pay for their agent. Sellers should pay their agent. Buyers who are uneducated and need a lot of therapeutic hand-holding at all hours of the day and night – should be responsible for paying for it. Buyers who know what they want and are involved and smart should only pay for the services provided to themselves – not the aforementioned.

    The commission structure rips off consumers – and most agents.

    Fred, you know what really gives agents a bad name?

    Agents saying, “The Seller pays for it,” when in reality the buyer WILL pay for it when they sell their home and in consideration that price is scaled to take into account the buyer agent commission.

    Here’s another one…. not telling buyers that in a normally appreciating market they will LOSE money if they sell in under 5-7 years and of course omitting the fact that the buyer is very likely to do so.

    : ) “Oh of course owning a home is a great idea! It’s the best investment you can make! Let me introduce you to my lender who will also assure you that buying a home during the bubble is a good idea just like the NAR says.” yow!

    I just think we should be honest about what sells a home i.e. other agents bringing qualified buyers, and to pretend that agents aren’t fundamentally motivated by their own financial pressures and needs (which are huge if you believe any of the data on how many transactions most agents do per month) is the real problem.

  414. Sam wrote:

    “But, does it really help you to have your own listings shown on other people’s websites?

    We know it helps other tech savvy smart agents with good websites. People will go on their websites and call the other agent, not you.”

    Sigh. You’re kidding me, right?

    If someone calls another agent about one of my listings, GOOD. After all, THAT’S THE POINT! I’m trying to get the home sold, not pick up a f’ing lead or double-side the transaction.

    This focus on getting people to call you — and God forbid another agent — on your listing is ridiculous.

    This industry, and real estate consumers, would be a lot better off if single-agent dual agency was illegal.

  415. Jay,

    Naw, I’m not kidding.

    Seriously…Laurie’s problem was a HUGE problem out here in PA. 2 huge brokers having good IDX sites, not letting agents have their own sites, and then scooping up even agents’ past clients and demanding large referral fees.

    I’m an IDX junkie, but am speculating that “exposure” via the internet to non-serious consumers – defined as those who don’t have an agent yet – might be over-rated.

    Showing homes online drives consumers early in their real estate search to tech savvy agents. That’s great! but 99% of the time when they were serious about buying would have found an agent that would have shown them the property even if it or they had never been on an IDX website.

    Homes get shown to people who will actually buy them when an agent prints out a listings sheet from the MLS – which includes properties that are not available via IDX – and presents those to the client as fitting their needs. Even the worst agents that I’ve met don’t skip this “review the MLS listings” step – especially now as they need some way to come up with something the client has not seen.

    This is why a dual strategy of having a great IDX website to capture leads and something targeting other agents (like a 5% commission….see “multiple offers within a week”…ha ha…might be a good idea.)

    And, back to Laurie’s point….Just because someone calls about your listing doesn’t mean you’ll end up representing both parties. They are more likely to want to see a bunch of homes with whatever agent they end up with…who could be a schmuck.

    So, if your participation in IDX ends in people calling your broker who then gives the lead to someone else or, God forbid, sells the lead to you for a higher commission split for him….don’t participate.

    Instead, create a single property website with RealBird and run some Google Ads on real estate related keywords in that town. Or, best of all, do a good job of pricing and staging it – and take nice photos for the MLS.

    • Sam, you seem obsessed with the “lead” for the listing to go back to the listing agent. Who cares who sells the home as long as it gets sold! Your warped (pre internet) thought process is that of a greedy agent wanting to double-deal all your listings, so who’s best interest are you looking out for?

      As for whacking the seller (or as you say the buyer, since they pay the commission???) with a higher fee to give the co-broke a 5% commission, that’s completely ridiculous! Is that really looking out for the seller’s best interest?

      If you work for a broker that keeps your leads, leave and find another one that will give them back to you without any fees. If I had agents that’s what I would do.

  416. Wow. I cant believe agents really believe that is how it works. Of course I am not in PA so I dont know how agency and fiduciary work in that state or any other state outside California, but i would argue that a deliberate approach like that to squelch widespread (and free) advertising here falls short of putting the seller’s interest before the agent’s.

  417. Apparently I have stepped into the Twilight Zone…

    “2 huge brokers having good IDX sites, not letting agents have their own sites, and then scooping up even agents’ past clients and demanding large referral fees.”

    Sorry, but I can’t feel any sympathy for any agent that puts up with that. Find another broker. That shouldn’t take more than 15 minutes.

    I get calls and emails, all the time, from prospects that see other broker’s listings on my site. I’ve sold many of those exact homes (of course I show and discuss more). I’d love to see the data that supports your statement, “but 99% of the time when they were serious about buying would have found an agent that would have shown them the property even if it or they had never been on an IDX website.” Or is that 99% just more “speculation” on your part?

    If I’m recalling a previous comment conversation correctly Sam, you no longer an agent. Is that correct?

    I would love to see a transcript of a conversation with a seller when an agent says “I’m not putting your listing into the IDX because someone other than me might get a phone call”.

    Any seller with an IQ above 75 would show that agent to the door.

  418. Awe Fred, I’ve been making a number of points. I’m actually a big fan of your business model and think the real estate industry’s commission structure is a huge problem. Jay appears to be a terrific broker, running his business ethically (in a corrupt industry), and provides good advice for a lot of people.

    I actually spend my entire day working with internet marketing, social media, lead capture, etc…technology…and love it. But sometimes I try to break out of my box – especially when i’m in a forum like this where most everyone agrees and “gets technology” – to look past the conventional wisdom, or to just speculate and watch what happens.

    There are a lot of brokerages – small and large – where ridiculous stuff happens, and that’s what I was also responding to.

  419. Sorry to take so long to respond here – I have been traveling for three days from Phoenix to Indianapolis.

    Thanks to all who supported me and commented here.

    A special thanks to Jay, who went way beyond the call of duty to join the national MLS committee and of course, Lani and Benn for providing the AG platform from which to express my opinion.

    As great as I feel about the outcome; it is still mind boggling the amount of agents who have no idea why this issue is important enough to care about.

  420. Pingback: NAR Reverses it’s Decision on Property Indexing by Search Engines | Property Indexer

  421. Hey Paula,
    I hope you feel vindicated… at least just a little. :-)

    Do you have any plans to get your site back up, or do you think MIBOR will try to block it again?

    This is such an important decision. It was vital to the success of our new small business that brings the technology to the table that allows proper indexing of listings. We were definitely “hanging on the edge of our seats” for this one…

  422. Justin –

    I do feel a bit vindicated. :) because I know it is the right result. Right now my plans to get my site back up is dependent on the MIBOR BLC (MLS) Committee’s recommendation to change the policy. They are meeting on December 1st. Apparently, any changes to the current policy must be recommended by them, then we will be notified of any changes. I’ll keep you posted!

  423. Pingback: Top Ten Changes in Technology That Defined MLS This Decade « FBS Blog

  424. Pingback: Benn/AgentGenius

  425. Pingback: MIBOR not understanding true scope of Google indexing MLS data | Denver Knox | Denver SEO Trainer and Consultant | iPhone Templates

  426. Pingback: Benn/AgentGenius

  427. The Realtor bodies and MLS will just have to get used to the idea that the internet is an information free zone. Search is there to help consumers find what they are looking for.

    The belief held by listing agents and brokers that they own the listing is an indictment against the agent and or broker. They have been hired to help their client sell a property and not as they choose to believe make a commission. How many clients have been lost by the selfish actions of agents trying to milk a deal for both sides of the sale?

    The reason most realtors are useless is because they believe the reason they get out of bed in the morning is to make money from their clients in the form of commission. The reality is that their purpose as agent is to ensure that their client gets the best possible price for their property in the least possible time, and if this is best attained by co brokerage and sharing a commission so be it.

    The act of hiding property information in the hope of getting both sides of the deal is counter to the spirit of agency and is also counter to MLS and Realtor principles.

  428. Pingback: Benn/AgentGenius

  429. Pingback: Benn/AgentGenius

  430. Pingback: Benn/AgentGenius

  431. Pingback: Maadi Real Estate TG

  432. Why are they so behind on things.Not letting us do things but given others the right to is not fair to any one. Why don’t they just ask us to shut down our sites and have us all going door to door to see if we can find a seller or a buyer.

  433. Pingback: Benn/AgentGenius

  434. I’m not sure if it’s just me, but NAR always seems to make an exception for all of Realtor.com’s practices. I think that the whole IDX indexing is much more a function that is out of our control and if NAR wants to put a stop to it, they’ll have to work it out with Google.

  435. Pingback: Benn/AgentGenius

  436. Not really going to say toom uch more than the 400 other comments, except to agree that Realtor.com seems to be able to do whatever they want. It annoys me to pay them so my listings don't look crappy, and then compete with them for google position. Something is not right there.

  437. Pingback: IDX, syndication battle in Austin may lead to sweeping national changes - AGBeat

  438. Pingback: SellYourHomeServices

Leave A Comment