From the Editor: Compared to What?

Obviously, this is a complex business and on occasion mistakes were made. But overall, TAS and its reviewers had a remarkable success level developing a shared understanding of what was involved with the reproduction of live orchestral music and of which recordings did the best job of presenting it. For all the variability possible in the whole process, it made sense. And it made sense because there was a shared reference, the actual sound of an orchestra playing in a concert hall. One knew and still knows what audio was aiming at, and so there was a chance of finding which recordings and playback systems were coming closest to the goal. Practical experience with orchestral recording gives a lot of insight into how recording works and why it sometimes doesn’t work. (Steven Stone and I are working on a joint article about our separate experiences recording orchestras. This subject is well worth looking into!)

Not only did most TAS writers like orchestral music, but it was actually one of the best tools for our work of explaining differences among components. But beyond that it was and is excellent for not just telling things apart, but deciding which ones are best. For telling what is best, one must know what one should be hearing. And because orchestral music in concert is a real and quite definite thing, one can tell. If you have heard orchestras a lot (for what it is worth, REG has been playing in orchestras since he was eight years old), then you will know right orchestral sound when you hear it—and wrong orchestral sound, too.

That was then. Turning to the present, examining not just Stereophile and TAS but also the enormous quantity of reviewing online from an exponentially growing number of websites will yield a not entirely surprising but seriously disconcerting conclusion: A great many of these reviews were based on listening to recordings without an acoustic antecedent, recordings which were not a recording of anything but rather were an electronic creation, even if they were generated from material that had had an acoustic existence in some sense at some point. 

There is no reason why a recording of this type could not be thought of as a work of art unto itself and enjoyed on that basis, any more than a movie using CGI images cannot be enjoyed on its merits even though there was no original event being depicted. (The Jurassic Park films are hugely enjoyable to my mind even though no actual dinosaurs were being filmed, and indeed people are not even sure what the dinosaurs looked like exactly.) But there is a fundamental problem in using artificially generated recordings for audio reviewing: You have no way to know what they are supposed to sound like. If a recording engineer is recording an acoustic event and trying to duplicate its sound, there is a way to know what the goal was. An orchestral concert has a sound that is different on different occasions, but the variations are relatively small and are well understood. But with an artificially generated recording or a highly processed recording of an originally acoustic event, there is no telling what happened, sonically. Of course, one can compare the result on two different systems and get information about how the systems differ. But when it comes to deciding which system is more correct—all bets are off with this type of recording. 

In practice, the online reviewers tend to deal with this logical hole in the reviewing process in a way that it would be fair to say is outrageously ambiguous. They tend to judge the system according to whether the recording played on the system sounded “good” in some sense. I invite you to consider the chain here: One supposes that the people making the recording wanted it to sound good and made it do so. Then the reviewer listens with the expectation that it should sound good and judges the playback equipment on whether it does or does not. To say this is full of holes is to understate the case because it requires the reviewer to be in effect a mind reader—to know what the person who made the recording regarded as sounding good.

It’s hard to believe that this mode of audio reviewing is taken seriously. And when one tries this process on the subtle sonic differences that are the province of evaluating good contemporary audio equipment, it seems to us to be truly nonsense. If you have no way to know what you are supposed to hear, how can you possibly know whether what you are hearing is what you are supposed to hear or not?

There is a way to make sense of a review by listening. And it used to be the way people reviewed audio equipment. Listen to a recording of an acoustic event that had a reality of a familiar sort and that you can listen to on a wide variety of equipment to get an idea of what the recording actually does sound like and to what extent it resembles the original event. Then listen to it on whatever piece of equipment you are reviewing. The preparation work must be extensive. One must be quite familiar with one’s reference recordings, having listened to them on a lot of different equipment, and one ideally has an idea of what the original event was like, e.g., it helps if you have encountered the recording venue in person, (or even made your own recordings in it, which is how David Wilson evaluated his early designs). Done right, this turns out to work quite well.

What a particular playback chain does to the recordings can be separated from the recordings themselves. Playback errors and the recording errors tend to be of different types. The original event’s sonic reality varies in different ways from how the recording and playback processes affect the sonic result. One can compare and contrast because there is an original reality, an absolute sound, to refer to as the antecedent to the recording. But without that antecedent, you are looking for the answer to a question that has no answer, trying to judge how much the result resembles…well, nothing: There is no there there to begin with. 

Subjective reviewing can make sense, but it unfortunately often does not do so. Final thoughts about reviewing by listening: As in The Fantasticks, “it really is a pity, it used to be so pretty.” Or at least it used to make sense. But we are in danger of having to say making sense is all over now.

Robert E. Greene and Steven Stone