Ratings are often the first thing people turn to in deciding on their resort of choice, but these handy shorthands are also fraught with biases and confusion. I thought I would pull back the curtains a bit on these metrics and badges to makes then easier to use and interpret when research your perfect resort.

• Industry star ratings indicate how many boxes a property has ticked against a list of criteria
• Social media star ratings (mostly) indicate how a property has performed against expectations.
• Industry awards are (mostly) just pay-for-cachet shills.

INDUSTRY STAR RATINGS

Traditional “star” ratings (eg. “5-star hotel”) were developed by industry bodies and were determined by a methodical list of criteria. The advantage to this approach is that is objective. The problem was that the checklist reflects quantitative metrics, but not qualitative aspects. It counts things like the number of electrical sockets and whether the bathroom has a bidet, but doesn’t assess the quality of design, materials, aesthetics, etc. Resorts quickly learned to game this system by installing the cheapest versions of anything that would tick the assessor’s boxes to get a coveted “5-star” designation for a fairly chintzy property.

SOCIAL MEDIA REVIEW RATINGS

The Internet and social media introduced the notion of crowd-sourced reviews. The stars that visitors gave were anything but methodical or defined. The reviews were completely haphazard with “1-star: Terrible” reviews going to exceptional properties who made one slip-up during their visit, and “5-star: Excellent” reviews going to mediocre properties visited by people who were just delighted to be on holiday or wanted to boast to the world how amazing it all was.

The notion is that a savvy reader will dismiss the outliers and focus on the shape of the score histogram (eg. shifted more heavily to positive or negative side). Social media does add the richness of two features: (a) the text review itself (so you can drill down into the specifics of the assessment as make your own judgement about whether the attributes focused on concern you or the assessment seems justified), and (b) the authority of the writer (based on reputational scoring like “Helpful” votes).

Seth Godin articulates this dynamic well in this piece “I Hate This Restaurant” (and this is just the inadvertent failure ignoring the deliberate toxic practice of social media extortion where people find tiny failings and demand a big discounts or compensation under threat of them unleashing their condemnations all over social media):

• ·If you look at many 1-star reviews (of books, of music, of restaurants) this is precisely what you’re going to see. A mismatch of expectations. A mismatch that is blamed, completely, on the person who created the work, not the critic. It doesn’t matter that the thing was clearly marked. It doesn’t matter that the thing was extraordinarily well-produced. And it doesn’t matter if just about everyone else experiencing it was thoroughly delighted. Because for this spoiled, under-informed and impatient patron, it failed.”

As a result of this “expectations driven” reviewing, many resorts have shifted the direction of their approach to ratings. Instead of trying to goose their rating as high as possible with covering the official bases as expediently as possible, now many properties voluntarily downgrade the advertised “rating”. So they might officially be a “5 star” property, but they advertise as a “4+ star”. That way, guest come expecting one standard of quality, but find a higher than expected one. Exceeding such expectations is the key to strong social media ratings. Better to be a 4-star on the profile but a 5-star on TripAdvisor, than visa-versa.

INDUSTRY BODY AWARDS

Whatever you do, dismiss the press releases and website merit badges from industry awards (eg. “Best Hotel in the Indian Ocean by the So-So-So Travel Group”). Said industry body charges X-thousand dollars for a resort to buy a table at their award ceremony and pretty much makes sure that everyone who attends, walks away with an award. In fact, in some cases, the more awards a resort flaunts, the more likely they are trying to cover up major inadequacies by buying endorsements (Yes, I know, I have featured some awards on the website and my email signature, BUT I did not pay anything for these and would never).

So with all of these review shortcomings, how is one to assess the quality of a resort in researching a holiday? I do check out the social media ratings (mostly TripAdvisor). I look at the shape of the star distribution (eg. how many 1-stars, how many 2, etc…). I will take a peek at a couple of 1-star reviews our of curiosity to see if they had identified anything truly serious, but in nearly all cases it is just the rambling trolling of a disaffected whinger. I do select for the most highly rated reviewers (eg. most Helpful votes) as these folks are likely to have sensible perspective so that their review will share useful insights.