Viewpoints: To measure or not...
Do test and analysis tools help ­ or hurt ­ the process?

By Frederick J. Ampel

Editor’s Note: Fred starts off the debate on the usefulness of test and measurement, with Doug then responding to Fred’s remarks.

Over the nearly 40 years I’ve been in this business, the measurement of audio systems has evolved from a crude, often hugely inaccurate and generally quite complex “science” into a just-as-complex but supposedly far more accurate “science”

Early measurement devices, clearly primordial and archaic when compared with today’s computer-based, DSP-filter-equipped technology, almost always depended solely upon the skill and knowledge of the user to provide anything remotely viable in the way of useful results.

If lucky, one was able to determine some sort of one-third-octave spectral information, and maybe some reasonably close-to-accurate analog metered voltage and current results. That is, if one had access to some kind of oscilloscope-based (interpretable) measurements. (Try being precise when reading a moving analog meter!)

“One was able to determine information if one had access to some kind of oscilloscope-based measurements.”

PRODUCE ENOUGH DATA Since the introduction of TEF-style measurement in the early ‘80s, the amount of “higher” precision of data available has escalated logarithmically. Add in computer-based simulation and acoustical performance prediction software, and it’s now possible to produce enough data and graphics to convince any client that you know what you’re doing. But the problem I have is that data quantity as currently offered/derived is not usually sufficient grounds to actually believe the information. Just because you have hardware and software that can tell you to within a fraction of a decibel as to exactly how much energy is occurring at some precise point in space doesn’t usually mean squat!

The ready availability of such tools means ­ far more often than not ­ that enormously unqualified people are collecting truly useless data and then applying the data to the “correction” of sound systems with absolutely no understanding of what they’re doing. Further, current systems and methodology doesn’t spell out what the meaning of the information that’s been so accurately collected. The number of systems that suffer from this process is so large that it has become an unspoken rule of thumb: once a system is “measured” and “tweaked,” it often sounds worse than it did before all the technology was applied for “fine-tuning” and “adjustment.”

Don’t assume I’m being a luddite and rejecting analysis tools ­ and the data they produce ­ in favor of just “using your ears” as some have aggressively suggested in the pages of this magazine. It’s just that the number of individuals alive on the planet who can truly perform such work is microscopically small compared to the number who claim they can. It takes years of practice and experience to even know what to listen for, let alone apply that knowledge.

So it’s a given that some kind of measurement tools will have to be applied to the situation. The question is how to use them and whether or not the data derived is helpful.

“Add in computer-based simulation and acoustical performance prediction software, and it’s now possible to produce enough data and graphics to convince any client that you know what you’re doing.”

From my perspective none of the currently offered measurement systems can be used without some kind of training and understanding of the information being presented. Even those tools which largely automate the data collection and interpolation process still require the user to know what they are looking at/for and then what to do with the information. Unfortunately the number of measurement tools in use is far larger than the number of people who have bothered to take the training. And, in the worst cases, so-called professionals don’t even believe they need any additional training. And the fact is that even a minor setup error on some systems, or the choice of the wrong parameter set, can so distort the data as to make it beyond useless. Yet this kind of mistake is so easy to commit that it happens to even the experienced users, and more often than they will admit.

MARKED AS TARGET?

Thank the deities that we’re not in the medical profession, because if this were the case, the simply erroneous data being collected and “interpreted” could literally kill people. Think what would happen if a CAT scan specialist mistook an undefined “blob” for a tumor and immediately called for surgery to have it removed, when all the specialist was really seeing was a defect in the screen or in the imaging software.

I can hear the lawyers salivating now. Luckily, the legal profession hasn’t marked us as a target for these types of “mistake” lawsuits. Yet. However, I believe it’s only a matter of time before some high-profile facility does sue on these grounds. (And the court system being what it is, they just might win.)

We seem to have fallen into the scientific grant/government funded study kind of mindset, wherein the data itself is the goal. Not data that is useful or appropriate or even valid, just mounds and reams of data for it’s own sake.

If you’ve ever attended an Acoustical Society of America (ASA) convention, you can scan the program and see dozens of papers that seem to exist solely because there is data to present. Not data anyone needs to know or even data that could be used in some positive way, just slaughtered forests of data.

FATHOMING THE USE

Many years ago at one such event, a pioneer of precision measurement told me that he could tell there was a 7.32456098 dB bump in the response at 8.2459876 kHz, caused by the knobs on the console in a control room. But, he couldn’t fathom the usefulness of this information, or why I would even want to know about it, since there was nothing that could be done to correct the problem anyway.

Unfortunately he remains in the minority with that view. Useless data, as well as data that is just plain wrong, is produced by the metric ton every day.

I’m glad I don’t have to wade through that swamp of numbers, but the fact remains that the industry does. Or, perhaps more accurately, believes it does. Maybe when all of this technology can provide me with truly meaningful and therefore useful information, I might start believing some of the current conventional wisdom.

In fairness, I should point out that there exist one or two measurement systems that focus on producing data that provides real answers. Regrettably, these systems are often hidden in the tide of hype and bias toward the offending systems, format and thinking that’s currently all too prevalent and undermining.

Fred Ampel has been involved in the design and implementation of audio and A/V systems for decades, and is a noted pro audio editor and writer. He can be reached at www.technologyvisions.com.

Editor’s Note: Regular readers of Live Sound will note that this section has traditionally featured the writings (and rantings) of Jack Alexander. Recently Jack decided to pursue other interests, and we wish him nothing but the best. And we’re all very fortunate that Mr. Ampel is ready, willing and able to put himself into the fray!

October 2003 Live Sound International