 |
| April 1, 2007 Deficient Reviews I’ve received a fair number of e-mails from readers regarding my March Ultra Audio Opinion, "Reviewers and Equipment Loans." Without going into great detail, a conclusion can be drawn from this correspondence: Readers want reliable information presented to them in an honest, forthright fashion. There’s not that much more to it. Is that too much to ask? I don’t think so. There are so many potential pitfalls to reviewing audio equipment -- issues that have nothing to do with biased reviewers or free long-term loans -- that it shouldn’t surprise anyone that readers are skeptical of what is presented by the press. Just as with most things, many examples can be found of both good and bad audio reviews, and bad reviews happen for many different reasons. Here is my partial list of obstacles that can lead to questionable reviewing results: Reviewer deficiencies: Does the reviewer have ample experience with the component type (speakers, amps, cables, etc.) under test, including the particular price range? Too many reviewers, as they gravitate toward higher-priced gear (as they almost always eventually do), become enamored of equipment that is outside their experience. Basically, if someone used to reviewing $700/pair speakers suddenly gets a $4000/pair speaker to review, you have to wonder if their conclusions will be credible. If the reviewer makes an effort to hear, at home, numerous $4000/pair speakers before passing judgment on the review set, then the picture improves. But if the reviewer lacks sufficient experience with products of similar kind and price, the review might not help your buying decision all that much. Room and system deficiencies: Is the reviewer’s room suitable for audio reviews? I’m not suggesting that the reviewer needs a perfect room; those don’t exist. I’m not even implying that the room should be professionally designed. But the room should be suitable for the equipment evaluated. If the reviewer shoehorns a huge speaker into a tiny space and then complains of bass problems, well, then, there you go. Also, is the reviewer’s associated equipment up to the job? Again, the components don’t have to be the best money can buy, but they should be of high enough resolution to allow the component under test to be accurately critiqued. Publication deficiencies: Can the reviewer’s publication obtain the requisite review samples for germane comparisons? I’ve read the reviews published in at least one publication that doesn’t seem able to secure more than a handful of somewhat obscure speakers for review. They claim that one of these brands produces the state of the art -- and they review that brand over and over. But without direct comparisons with established market leaders, how can they know that their favorite brand really is that good? Without a good cross section of experience, there’s no way to accurately assess a product in the context of its peers. And most times, that’s what counts most. Editorial deficiencies: How much official oversight is there of the review process? It’s up to a publication’s editors to assign equipment reviews to staff members who will do a credible job of evaluating components. The components must also fit the system and the reviewer. For instance, I would never send a small bookshelf loudspeaker to an admitted bass fanatic who listens to and likes only large, full-range speakers. This is an obvious example, but it illustrates the need for an editor to be involved in the review process to help ensure a proper match of equipment and reviewer. I’m not trying to tear down an industry that I am entrenched in by pointing out these potential deficiencies. I am trying to drive home the points that publications should be held to high standards, and that we should not be surprised that readers are sometimes skeptical of our conclusions. There’s good reason to be! Once you’ve managed to sort through all the pitfalls, you’ll find some solid, credible reviews that you can rely on. These will generally come from the sources that, over time, have proven themselves worthy of your trust. We at the SoundStage! Network count ourselves part of that group, and hope that you feel the same. If there’s a case in which you feel that we haven’t lived up to your standards, let us know. A little introspection and warranted self-adjustment are always positive things. ...Jeff Fritz
Ultra Audio is part of the SoundStage! Network. |
