For a guy who doesn’t get terribly excited about hamburgers or, for that matter, fast food, how is it that I deliberately went out of my way to find one of the worst fast food hamburgers I have ever eaten? I blame crowdsourcing. More specifically, I blame Yelp.
In many ways user review websites like Yelp and TripAdvisor are brilliant. Their millions of members review far more locations than a professional opinion writer ever could. In fact, it’s hard to find something in the world these days that hasn’t been assigned a specific number of stars. And unlike the magazine or guidebook reviews of old, these never go stale because they’re updated continuously.
They’re also supposed to be better. Instead of turning to a crusty old food critic, Yelp relies on “the wisdom of crowds” for its rankings. In theory, the collective view of a large population is better than that of a single individual, even a very knowledgeable one. This idea is the basis for everything from the efficiency of capital markets to Wikipedia. A million internet users, after all, can’t be wrong.
Why, then, are they so often wrong?
Garbage In. Garbage Out.
Yelp, in particular, has directed us to more mediocre meals than I care to recall. One particularly awful restaurant garnered “4 Stars” and a review describing it as “the best Mexican food in Livingston, TX.” Sadly, it might have been. But that’s part of the problem with such reviews. Judging something to be the best in Livingston is not the same thing as judging it to be the best anywhere or, for that matter, even judging it to be good. The best turd pie is still a pretty terrible meal.
For a long time I thought our problems with Yelp stemmed from this phenomenon; that its members use too narrow a sample set in their rankings. Whereas a professional food critic would judge a Mexican restaurant against the broad category called “All Mexican Food,” the typical Yelper seems to judge a place mostly against other local establishments. If all the local restaurants are bad, a good food critic will tell you to skip them all. Yelp, meanwhile, might rate most of them average and a couple of them good or even great.
Our personal experience partly bears that out. Where we tend to have the most luck with Yelp is in large cities that feature plenty of local dining options. The more robust a city’s food competition is, the more likely its residents will have experimented with different restaurants. Big cities also have larger transient populations who bring regional and international food experience to the equation. A better informed crowd must result in better crowdsourced ratings. And so it seems.
Fast and fabulous Shake Shack burgers
Of course none of this explains how I ended up with that regretful burger. That story doesn’t begin with Yelp or even a bad burger, but with a recommendation from a friend – still the best kind.
On a recent trip to New York City, this friend directed me to a relatively new burger joint called Shake Shack. There I enjoyed what is probably the best burger I didn’t have to cook myself. The “SmokeShack” is a cheeseburger, with applewood smoked bacon, chopped cherry peppers and, of course, a secret burger sauce. It was spicy, smoky, savory, greasy goodness on a bun. Everything a fast food experience should be.
Afterwards I made the mistake of consulting Yelp to see what other people were saying about Shake Shack. There I discovered a surprisingly heated East Coast / West Coast burger rivalry. Who knew?
“Shake Shack sux balz, ya’all! In-N-Out Burger’s da BOMB!” according to one reviewer (or something like that, I paraphrase). More serious and common were sentiments similar to those expressed by Seth V.
There really is just no comparison in my mind. In N Out’s burgers are fresher, tastier, and cheaper. New York wins in terms of pizza and bagels, but they can’t touch LA’s burgers. It’s just a fact.”
Salad on a Bun Does not a Great Burger Make
A fact I decided to experience for myself. After completing my quick trip to New York I found myself back in California searching for this West Coast burger Mecca. I didn’t have to look far because they’re basically everywhere. What I got for my effort was a wafer thin patty of beef, a large wedge of iceberg lettuce, and quarter-inch slab of grilled onion, all packed on a roll.
Thinking back I now understand the incredulous tone the cashier used when asking me if I really wanted my burger with onion. It was by far the dominant flavor. Unfortunately, removing the onion only marginally improved things. Mostly lettuce and bun remained. What was supposed to be a great burger experience tasted an awful lot like salad, and a bad one at that.
It’s something I’m at a loss to explain. How could hundreds of reviewers steer us so wrong? It’s possible we just hit In-N-Out Burger on a bad night. But if you can’t count on consistency in fast food, what hope is there for mankind? No, the reason lies elsewhere.
I’m left to wonder whether our individual tastes are just at odds with those of the crowd. It certainly wouldn’t be the first time we and the pack parted ways. These differences of opinion may actually explain a lot. After all, we have a similar, yet completely opposite, problem with hotel ranking sites like TripAdvisor and Hotels.com. Here the difficulty isn’t that people narrowly judge hotels against others in the neighborhood, but that they seem to narrowly judge them against some ridiculous standard; perhaps their bedroom at home. In comparison, all but the most expensive hotels are deemed disasters.
One Person’s Crack Den . . .
Concerning our New York City hotel, one TripAdvisor reviewer warned us to
avoid this crack den. This hotel is a disgrace!!! On arrival we were sent to our room and to be honest i would not have sent my dog there to sleep . . .”
Apparently we have lower standards than his dog because we spent three comfortable and uneventful nights there. Alas, we found no crack.
But then our standards aren’t those of most folks. For our hotels we really only ever want a safe, comfortable bed in a location convenient to where we want to go. We have no need, and would not choose to pay extra, for marble lobbies, extra large rooms, daily maid service, mints on our pillows, or anything else, really. We’ve never once traveled anywhere to enjoy the hotel room. Many reviewers apparently feel otherwise and rate rooms accordingly.
Those differences in preferences and expectations make navigating rating aggregation sites an exercise in advanced psychology. How else can we hope to understand what it means for the same establishment to receive a “5 Star” rating from one user and only “1 Star” from someone else other than by delving deep into each reviewer’s psyche? What is it, for example, that leads a person to describe a place as a crack den? Is the lobby really filled with drug addled addicts or is the reviewer just a bombastic blow hard? And how do we decipher the fairly common all-cap admonishment “ABSOLUTELY DO NOT STAY HERE!!!!” Should we take the emphasis seriously or assume the reviewer is posting on TripAdvisor while taking a break from his manifesto writing?
It’s hard to know in advance which warnings or accolades to take seriously. According to the wisdom of crowds theory we shouldn’t have to care. With enough reviews all of the lunatics balance each other out, resulting in an accurate assessment of a place. Of course that still leaves the mystery of the bad burger, not to mention all the other bad meals, unexplained. Maybe, for us at least, there is no wisdom to be found in crowds; only a kind of madness that defies explanation.
Everywhere Once 
I don’t what the latin equivalent of caveat emptor (buyer beware) would be if you wanted to say “review reader beware”. You can’t take the reviews as necessary valid or to your taste but it’s better than just seeing an ad in the Yellow Pages-which seem to be following in the footsteps of the dodo bird. It is reader beware as you say. If you ever are planning travel to Ottawa, Ontario, Canada I can however give you a first hand review on a great burger joint.
LikeLike
The reviews are certainly better than yellow pages ads, so we have that going for us.
And we’ll take you up on your offer when we make our way to Ottawa – thanks.
LikeLike
I completely agree with your assessment of “the wisdom of crowds” Give ten people an identical meal and you will get ten differnet responses as to its flavor and eatability. Same goes for rooms. Like you, I only want a base to travel from, a place to spend the night and contemplate my day (although a good bar is a plus)
LikeLike
Reblogged this on digger666 and commented:
Spot on…
LikeLike
You said a mouthful here, but I’d also like to add that the Food Network, specifically the show Diners, Drive-In and Dives has done the same thing for us. Twice. Once in St. Louis where we sought out a soul food joint that turned out to be less than soulful, serving cafeteria style fare on styrofoam plates, and most recently, Chop House Burgers in Arlington, TX. Featured as a top notch burger joint with gourmet choices, it was a BIG let down. Talk about salad on a bun. The “burger” was hidden under a giant wad of shredded lettuce so that it presented with the bun top slightly eschew, giving it a big burger appearance. I actually said, where’s the beef? I also ordered onion rings. They were huge, but tasted like fried biscuit dough, the onion part was tough and flavorless. A big greasy fried dough ring. If I were grading the triple D, I give it a big fat F!
LikeLike
Interesting. I often regret that we never include places that have appeared on Diners, etc in our travel itinerary. They seem like fun, and yet also seem like places that might be overrated and over crowded almost by definition.
LikeLike
Well, I use yelp.com and review places on it. The site is helpful compared to knowing nothing at all about a place. The more reviews, the better feel one will get for a place. Paying attention to how current the reviews helps too. The more reviews, the better. In general, take reviews with a grain of salt or a slice of onion. More of my feelings on Yelp, etc. –> http://irvent.wordpress.com/2012/09/08/critic-reviewer-or-complainer/
LikeLike
Sometimes knowing nothing is better than being misinformed.
In the past we’ve used Yelp for two different kinds of inquiries: 1) We know we’re going to be out all day and need to find someplace to eat 2) We feel like dining out, even though we don’t have to, and want a great meal.
In the first case, knowing anything is better than knowing nothing and services like Yelp are super helpful. In the second case, being directed to a bad place is worse for us than just dining in. Through hard experience we’ve mostly decided to only use sites like Yelp in the first situation and almost never for the second.
LikeLike
You might try chowhound.com for the second case
LikeLike
Looks promising. Thanks!
LikeLike
When I started my blog I decided only to write about places I like. If I go somewhere…a town, city, village or restaurant and I don’t like it I just don’t write about it. I could have been having a bad day, the weather could have been awful and the place just didn’t look its best, or I could have ordered something that I just didn’t like. You can ruin a business’s reputation with negative comments and I don’t think this is fair. Everybody is an expert now and can tell the world their feelings. Some people are just nasty and want to cause trouble.
I take recommendations from people whose opinion I value and don’t listen to the masses.
LikeLike
I love this post – great sense of humor – and I heartily agree, both about restaurant reviews and hotel reviews. When my husband and I travel, we’re just mainly looking for a place to lay our heads for the night (hopefully without bed bugs!), so we’ll overlook a lot. I check for comments about noise (nothing worse than being put in a room over a disco). Generally, we look at all the reviews and toss out the best and worst, then see what the middle has to say. If a really bad review does catch my eye, I look to see if a restaurant owner or hotel manager has tried to make the situation right. If they do, that will often win me over. For restaurants, we tend to ask hotel staff and other guests, people we run into in shops, etc., for their recommendations, and we’ve had really good experiences that way.
LikeLike
Love this comment 😉
LikeLike
When my bF and I use yelp we never pick anything that is less than 4 stars that’s for sure. We try and stick only to the 5 stars when it’s possible.
LikeLike
Reblogged this on amoralegria and commented:
I totally agree! I don’t trust fast food joints in general – they make up the food in advance and then give you as little real sustenance as possible. When traveling, it also depends on why one is traveling. Some people want to be pampered wherever they go, whereas others (like me) just need someplace clean and comfortable, so I can get some sleep, but then be able to get OUT of the hotel and into the life of the place I’m visiting!
LikeLike
I agree with your assessment about In N Out–my daughter went to USC and raved about it, so we stopped there while on a road trip. All I remember is the MESS it was trying to eat it in the car.
LikeLike
So true, so true – and don’t get me started on recipe reviews! I mean shouldn’t you have to follow the recipe before you can review it! Argh!
LikeLike
One problem I’ve heard of with Trip Advisor is rival establishments posting negative comments to try and drive custom away from their rivals – might explain the all-caps abuse!
LikeLike
It’s safe to say that if there is a system that allocates economic benefits someone will find a way to game that system to their advantage. That’s certainly the case with ranking sites. I think it was Amazon that was having a problem with competitors (or even ideological rivals) who would not only spam fake comments but also report valid comments as “abusive” so the valid remarks get removed from the site.
Here’s a specific example I recall where an author made a snarky comment on Twitter about a political pundit and the pundit’s fans retaliated by organizing on Amazon to trash the new book. Check the ratings barbelled at 5 and 1 as each “side” put up fake reviews.
LikeLike
Scary… particularly as I’m an author. Better be careful who I upset! ;0
LikeLike
Glad you approve of Shake Shack! Their Shack Stack is nearest to the Platonic ideal of burgers – meat, fat, cheese, a multitude of flavors – in a portion that is not meant to be a spectacle but just right for a normal person’s hungry appetite.
As for crowd-sourcing Amazon.com seems to have figured out how to present a range of product opinion and rank them by popularity. With something that has received enough feedback there, one’s own new experience is unlikely to be an ‘outlier’ from the range of settled opinion, no?
LikeLike
Hey Charley,
I love that you use Plato’s Theory of Forms as a basis for judging burgers. What’s the word I’m looking for? Classic!
And it’s true; I’ve never had a problem with Amazon’s rankings. But that could be because I don’t really use them to recommend products. I always do my research elsewhere and only go to Amazon to check prices once I’ve decided what I want. That doesn’t mean I won’t consult the reviews to see what people are saying. If a dozen people have a similar, specific complaint, like “the battery life sucks,” it’s a good bet that the battery life sucks.
Oh, and thanks for recommending that burger joint. 😉
LikeLike