Twenty-one years ago, Tim and Nina Zagat, two lawyers intimately familiar with New York’s restaurant and fine-dining scene, informally assembled the opinions of 200 of their acquaintances into a self-published survey of local establishments. Between then and now, their handy little red book of names, addresses, rankings, and consumer-driven evaluations has grown into a publishing empire.
The Zagats now put out best-selling guides to restaurants in 45 cities worldwide, including London, Paris, and Tokyo, as well as to U.S. hotels, resorts, and spas and, for New York and Los Angeles, other resources for food and entertaining. (The first Survey of New York City Nightlife has also recently emerged, edited by the Zagats’ son, Ted.) Today, one can gain access to the contents of Zagat surveys via Palm Pilots, Wireless Application Protocol phones, and the Zagat Web site, which recently raised $31 million in equity financing. The overall enterprise is now valued at $120 million, and the Zagats are said to be thinking of an initial public offering next year.
Anyone who has availed himself of a Zagat survey, with its narrow, elongated shape and distinctive red jacket, will hardly be at a loss to explain its appeal. The first thing a reader encounters upon opening this year’s New York survey, for example, is a flagship list of the city’s 50 “Most Popular Restaurants.” This is followed by several other summary lists, including the top 50 restaurants ranked by food alone, top spots by cuisine, most-visited restaurants, and “Top Bangs for the Buck.” At the back are several dozen indices, cross-referencing restaurants not only by type of cuisine but also by neighborhood and special features (e.g., Fireplaces, Outdoor Dining, Wheelchair Access).
But at the heart of the survey, and making up the bulk of its pages, are the capsule descriptions of some 1,800 restaurants, listed in alphabetical order. Here is a typical entry:
UNION SQUARE CAFÉ
F D S C 27 24 26 $55
21 E. 16th St. (bet. 5th Ave. & Union Sq. W.), (212) 243-4020
“Danny Meyer’s masterpiece” and NY’s Most Popular Restaurant for four years running, this New American remains “unsurpassed” as a “fully satisfying dining experience” because its “formula works”: offer “fabulous food” (from Michael Romano) in a “relaxed” yet “classy” space with “gold standard” service for less than your competition; “if you can get in,” odds are it will “steal your heart.”
The numerals along the top indicate, on a scale of one to 30, the rankings achieved by the restaurant in the areas of Food, Décor, and Service, respectively, plus the average Cost of a meal per person. In the body of the review, phrases in quotation marks derive from the comments on readers’ survey forms that have been submitted to the Zagat editors and then strung together by them into a coherent whole. (In the case of the New York City survey, the Zagats themselves are heavily involved; for most other cities, they farm out the editorial work.) The book also contains graphical icons indicating whether a restaurant is open late or on Sundays, and whether it accepts cash only. In short, as a convenient and superbly organized guide, the survey is unparalleled; with a new edition appearing every year, it is timely and comprehensive as well.
With sales in excess of 600,000 copies per year in New York alone, and with nearly 20,000 loyal participants in the New York survey—whose sole recompense for filling out and submitting a form is a free copy of the $11.95 guide—it is unlikely that, in the foreseeable future, anybody will pose a serious challenge to the Zagats’ supremacy. (The few efforts to compete with them have failed emphatically.) And there is also no question that the reverberations of the Zagat enterprise are felt in every quarter of the restaurant world. Even at the most elite levels, owners and chefs know that Zagat rankings are more important to success, not to say survival, than the reviews of all the city’s newspapers combined.
The fundamental premise of the Zagat survey, as stated by the Zagats themselves, is that “rating a restaurant on the basis of hundreds or even thousands of experiences [is] inherently more fair and accurate than relying on one reviewer and just a few meals.” Stated this way, the notion appears reasonable enough: after all, at an average of 1,800 meals per restaurant surveyed, the 2000 guide for New York reflects no fewer than 3.2 million meals consumed. Surely that guarantees greater fairness and accuracy. Or does it?
To begin with, although it might come as a surprise to anyone who has not participated in a Zagat survey, the 30-point ratings in the book do not represent actual choices. In fact, the survey form is much cruder, offering only four choices, 0 through 3. Thus, a conscientious participant who believes that a restaurant’s rank should be, say, 25 on a 30-point scale must choose either a 2 or a 3, each of which represents an inaccurate extreme. These ratings are then averaged and multiplied to give the appearance of a more nuanced evaluation.
Another problem arises with time lag. As any professional critic can testify, accuracy in a restaurant review requires taking careful notes and then writing relatively quickly: after a few days, much specific recollection will be lost (this applies to music and dance reviews as well). But the Zagat survey forms are mailed out once a year, with the specific intent that all restaurants be ranked at once and not necessarily in conjunction with actual meals. As with the simplified four-point scale, this encourages impressionistic rankings at the extremes and, for places that have dramatically improved or dramatically deteriorated, the possibility of wholesale error.
Perhaps most surprising of all, respondents to the Zagat survey are not required to document—through, for example, copies of receipts—that they have actually eaten the meals they claim to be evaluating. The survey’s voting controls consist only of computer programs that scan for major irregularities and discrepancies, which would hardly prevent a shut-in living in Bozeman, Montana from voting in the New York survey. Nor is there anything to prevent determined people from voting multiple times for their friends’ restaurants, evaluating restaurants visited long ago or ranked according to the opinions of others, or engaging in conduct calculated to teach a restaurant a lesson—or make it number one. Online voting, recently introduced, will no doubt make it even easier to cast fictitious and/or ill-considered ballots.
The consequences of this lack of safeguards have been many. In the 1998 survey, for example, Ratner’s restaurant on the Lower East Side of New York was described as offering “chicken soup that will cure anything short of amputations.” Succinct, yes; accurate, no. Ratner’s (a famous establishment that has since regrettably closed) was a kosher dairy restaurant and had never in its long history served a bowl of chicken soup.
In this case, the inaccuracy, though allowed by the editors to sneak through uncorrected, could be chalked up to a single rogue reader. But the editors themselves have also been responsible for misinformation. When, for example, galleys of this year’s Boston survey were mailed to the Boston Herald, the paper’s restaurant reviewer noticed something fishy: one write-up, sounding for all the world as if based on actual visits, was of a restaurant that had not yet opened. Similarly, Brill’s Content reported recently that Zagat employees themselves are sometimes sent to evaluate restaurants that open near to press time, thereby vitiating the whole point of the survey system.
Which brings us to the mini-reviews and their content Admittedly, Zagat reviews are pithy; but pithiness does not preclude substance. Unfortunately, that substance is frequently so bland as to be uninformative. Thus, in the notice given to the Union Square Café, “fabulous food” is the only culinarily germane comment from a reader, while the editors’ notation that the cuisine is New American is hardly meaningful. Throughout the survey, descriptions of restaurants either include no discussion whatsoever of the food or, at best, offer up undifferentiated pap like “amazing paella.”
But bland is the least of it. Although the Zagats are at pains to call their capsule reviews “diner-written,” it is they and their editors who select and assemble the quotations, and these quotations can be made to conform with just about any opinion—even one at odds with what the majority of survey participants have reported. In what may be the most outrageous abuse to date of this editorial prerogative, the 1999 Zagat survey informed its readers that the New York restaurant Patroon turned out food “better than our ratings show.”
A poll’s results can also be influenced by who participates in it. Although anybody can take part in a Zagat survey by requesting a form and sending in a self-addressed stamped envelope, the Zagats also distribute hundreds of thousands of unsolicited surveys to respondents of their own choosing. These include, in Tim Zagat’s words, “white-collar institutions [like law firms and medical practices] where we know people eat out as a way of life” as well as food-and-wine clubs and anybody else the Zagats deem worthy. True, the final ratings are the final ratings, and no evidence has ever emerged of ballot tampering (the raw data have never been made available for evaluation); but at the very best the Zagats’ carefully controlled experiment in culinary democracy raises more questions than it answers.
But let us suppose that the democratic procedures were impeccable. Would they then constitute an appropriate means of rating restaurants?
The scores and rankings in the Zagat survey reflect, by definition, an average of the opinions gathered. But if you want to know how good a restaurant is, averages are seriously misleading. Thus, for four years running, according to Zagat, the number-one restaurant in New York has been Union Square Café. It is number one in the sense that it emerges first in response to this question on the survey: “What are your favorite New York restaurants?” It is true that other restaurants score higher than the Union Square Café in the areas of food, décor, and service, and the closest thing to a “best” restaurant in Zagat, based on scores alone, would be Le Bernardin; but this information appears on no chart.
Union Square Café is, indeed, a very good restaurant, one beloved by many New Yorkers for its compassionate service—it is perhaps the most unintimidating of the city’s better restaurants—and its simple but intensely flavorful food. But with all due respect to that justly popular establishment, it is patently ridiculous to rank it ahead of a dozen other places, and in particular such world-class restaurants as Lespinasse, Jean Georges, and Daniel.
Another example of what can happen when one averages consumers with different levels of taste and (above all) experience was on display in the 1998-99 edition of the Zagats’ America’s Best Meal Deals. On its nationwide list of “Top Delis,” not a single New York delicatessen was to be found; instead, the guide featured places like d’Bronx Deli in Kansas City (which apparently doubles as a pizzeria) and another establishment in Salt Lake City. Even in New York, where the Zagats draw on a presumably more sophisticated base and a far greater number of ballots, the survey disproportionately rewards what might be called “yokel pleasers” like Café des Artistes (the ninth most popular restaurant in the survey) and River Café.
Worse, it is a simple but distorting truth that people tend to prefer the restaurants they already frequent—witness the similarity between the Zagat “Traffic Report” and its list of “Most Popular Restaurants.” The circular nature of this process has been well pinpointed by the critic Seymour Britchky:
Once you learn to hate a restaurant you never go back, [but] since you do not evaluate a restaurant for Zagat unless you have been there in the past year, those who continue to rate a place are, disproportionately, its admirers—fans—while the opinions of detractors go unrecorded.
The New York Times critic William Grimes has labeled this phenomenon “The Zagat Effect,” adding that once a restaurant gets a good rating, “diners flock to it . . . and, convinced that they are eating at a topflight establishment, cannot bring themselves to believe otherwise.”
Finally, what renders a popular survey truly suspect is that one has no sense of the reasoning process (if any) behind Zagat‘s numerals and strung-together quips. There is no way of knowing whether the respondent who spoke of “amazing paella” has any familiarity whatsoever with Valencian cuisine, or is equipped to evaluate a given example of paella. By contrast, a professional restaurant reviewer brings a single voice, a certain degree of accountability, and concrete recommendations. In theory, he also brings expertise—and, by virtue of having dined at many more restaurants, a breadth of experience that offers a meaningful basis of comparison.1
And there is another consideration here. It is the vision of great chefs that ultimately creates educated consumers and hence the demand for better and better cuisine. Under the sway of a Zagat-style survey, however, a restaurant that wants to survive and flourish will find itself pandering to average tastes the way an assistant professor of English saddled with the need to score high on student-evaluation forms inevitably finds himself assigning easier and more popular texts. One does not, after all, gain the accolades of the Zagat constituency by presenting challenging, complex, or advanced cuisine. One gains the love of the Zagat reader by serving tuna burgers (as does the Union Square Café—and very good ones at that).
Of course, the Zagats themselves dismiss all such strictures as the carping of elitists, and defend their survey method with Nader-like appeals to democracy and “the people.” But, their thumb-on-the-scale methods aside, this is specious, and utterly self-serving. It is as if the editors of Consumer Reports were to declare that their patient, meticulous, objective, and very expensive testing of air conditioners and washing machines actually resulted in a less accurate product guide than one based solely on the random and self-selected reports of buyers, whose opinions alone they would henceforth solicit. No, in those 3.2 million meals, bought and consumed not by them but by their unpaid collaborators at an average cost (we are informed) of $33.17 per meal, the Zagats definitely have a good thing going. Whether they are advancing the cause of informed choice is another question entirely.
As their empire has grown, Tim and Nina Zagat have increasingly stepped outside their roles as pollsters and have positioned themselves as power brokers in the industry upon which they report. When they dine out, they expect restaurants to treat them like royalty, however obnoxious their behavior. According to press reports, they are in the habit of appearing unannounced at the most exclusive establishments and demanding to be seated ahead of ordinary, reservation-holding citizens—so much for democracy—or charging into the kitchen without permission in order to talk to the chef. Because they are the two individuals most feared by the American restaurant industry, their demands are invariably accommodated by fawning owners and maître d’s.
Beyond bullying restaurants into doing their bidding, the Zagats now also aspire to impose their views on the industry and the dining public through self-aggrandizing schemes like their new “Diners’ Bill of Rights.” The enumerated rights range from the superfluous (“The right to complain to the manager”) to the picayune (“The right to make special dietary requests”) to the impossible (“The right to be seated within 10 minutes of reservation”) to, inevitably, the politico-cultural (“The right to smoke-free and cellular phone-free seating”). But such are the prerogatives of power. As Huey Long and other successful populists have always known, when you speak in the name of “the people,” you get to call the shots.
1 It is unfortunately true that the credibility of professional restaurant criticism is today at an all-time low (see my “Culinary Correctness” in last month’s COMMENTARY). In this sense, the Zagats have stepped into a vacuum.