@Makmeksam's good measuring vs. good sounding thread

Discussion in 'Audio Science' started by makmeksam, Dec 5, 2019.

  1. Tachikoma

    Tachikoma Almost "Made"

    Contributor
    Joined:
    Oct 4, 2015
    Likes Received:
    429
    Trophy Points:
    63
    I'd be amazed if there were actual trained scientists in the "audio science" group.
     
  2. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,345
    Trophy Points:
    113
    Great chart. These guys never consider that for popular music, almost everything will be compressed, usually multiple times, bringing up the noise floor and those pres will have to be pushed for good mics on most sources, raising the distortion significantly. Not just pushed pres, pushed converters, pushed tape, consoles, everything that sounds better hot on the verge of breakdown and clipping will be pushed. Considering how Raw Power and Welcome to Hell inspired entire genres with productions that might have had -10 db noisefloors and were clear enough, yeah lol.

    They truly have no idea about common gear either. Just look at them blind buying monitors as hifi speakers because some are more "scientifically designed." I see so many ASR members using the JBLs with compression driver tweeters and digital crossovers for music fed by expensive DACs certainly worse than the ones in those monitors. They have zero idea about film vs music split for monitoring tools and the splits among various music genres all gravitating towards different things based on needs, reliability, and what works with a universal respect for ATC that some DSP monstrosity like the Kiis won't ever have because some guitarist or fx guy will fry it hitting the limiter every three seconds for hours. They have no idea why dry as a nun's cunt Yamahas, cheap JBLs, small Proacs, and better KRKs truly are more effective music production tools than Neumanns regardless of any measurements and conformity to straight lines.
     
    Last edited: Dec 9, 2019
  3. makmeksam

    makmeksam New

    Joined:
    Oct 14, 2017
    Likes Received:
    7
    Trophy Points:
    3
    Location:
    Brazil
    One thing is clear when reading all the posts here. Listening should be the #1 evaluation method because measurements can not capture everything. However, even though several of you suggested good ways to navigate the gear space to find what someone likes without depending only on measurements, I think still it is difficult to do it due to personnel biases in published subjective evaluations. I know that there are many people here who are measurement experts. So, has anyone here attempted to come up with any new measurements/metrics to capture what is missing in currently used measurements (many would agree that measurements like THD+N, IMD-CCIF/SMPTE including the plots with limited tones can not tell everything)? If something like that is possible we can universally discuss about audio gear without worrying about personal biases right?

    If you think this is impossible, is it because the precision of currently available technology is limited in some form? If that is the case, hasn’t someone attempted to borrow some technology from a different domain that might be using higher precision at least an electrical measurement technology?
     
    Last edited: Dec 9, 2019
  4. Psalmanazar

    Psalmanazar Most improved member; A+

    Pyrate Slaytanic Cliff Clavin
    Joined:
    Sep 27, 2015
    Likes Received:
    5,345
    Trophy Points:
    113
    That’s total f'ing bullshit that it’s hard to do. Changing the Dangerous Source for MOTU Sabre was a night and day difference in treble clarity despite the conversion being softer. Changing my line level cables to Mogami regular from generic off the shelf mic cables made another big difference. Each of those was easy.

    Now let’s take what I mainly listen on, active nearfield monitors. I mainly listen to those because I listen at my desk, practice guitar direct input into them with amp sims at night, record myself, and do some mixing work for underground metal bands. Let’s look at some measurements of highly regarded stuff: Yamaha HS8 and Neumann KH120. The KH120 measures flatter and has better directivity and all the other shit that ASR and Reddit masturbate to. Regardless of how many loads they have dropped to spec sheets, the HS8 is a better tool than the KH120 no matter how underbuilt in Indonesia the Yamaha is. It doesn’t matter that that Sennheiser controlled directivity with a special computer modeled waveguide, made a special metal tweeter, eqed the drivers to measure flat in the active crossover, and has a much more expensive plate amp. The HS8 is a better speaker anyway despite its limitations. The mids are clearer and the bass is way less compressed mud. The presence is more realistic despite having the veil from an 8” woofer crossed over to a 1” tweeter in the upper mids. It is simply more accurate sounding and easier to produce on than Neumann’s muddy, boxy, veiled flatness. Those aren’t going to come up in simple measurements.

    The only measurements that truly reflect the real world limitations and problems of the Yamahas are those of the limiter circuit. Otherwise they measure very well except for self noise. When the limiter kicks in in terms of gain, how it distorts the sound, and what it does to the sound in terms of impulse response reveal far more than ASR measuring FR and SNR. Only end users and modders have published measurements of the limiter. No reviewer has touched it.

    Meanwhile you can perceive that something is amiss right away with them by hearing everything fit together a bit too cleanly with reference recordings. Things won’t be masked or buried despite the 8” woofer veil. For audio work, you can slap a clean compressor with only 1-2db of gain reduction on recordings, hit bypass, and hear no change. On monitors more revealing of dynamic changes, even cheap ones like KRK Rokits and the mkII JBLs? Apparent. Not on Yamaha HS.
     
    Last edited: Dec 9, 2019
  5. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,134
    Trophy Points:
    113
    Location:
    India
    Have you ever heard of the concept of fractal wrongness? Google.
     
  6. makmeksam

    makmeksam New

    Joined:
    Oct 14, 2017
    Likes Received:
    7
    Trophy Points:
    3
    Location:
    Brazil
    Why not try to correct at least something?
     
  7. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    Well, we focus much more heavily on headphone measurements than anything else. And there's still a ton of debate and research going on in that area. Transducers are going to have the greatest performance limitations, so it makes some sense to start here.

    @purr1n in particular was testing out using sine bursts at varying frequencies to see if that might correlate to subjective listening.

    He also did some experimental DAC measurements to track consistency of multibit DAC output voltage changes (sort of like amplitude jitter).

    I mean, really, we're of the opinion that amp/DAC measurements can only tell us if something is more or less broken. If it's not immediately super messed up, subjectivity plays a role.

    Of course, folks do blind testing without necessarily advertising it here. Why? Because we are comfortable with our sexuality. Or something. Really, because we're not here to pander to certain crowds, nor reiterate what history we have just because someone asks questions without spending the time to do a deep dive on our resources.

    Then when you get things like totally different people, all over the world, coming to similar conclusions on gear listening before they, or anyone else, post(s) anything publicly. This stuff happens in private chats and isn't necessarily seen here. We actually talk to each other as friends in the background, and not all online, if you can believe it.
     
  8. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,134
    Trophy Points:
    113
    Location:
    India
    Do it, then.

    Don't just stand up and cry, "Measurements don't tell us enough, please invent some new ones."

    As if science has stopped. And research wasn't actually going on. Maybe you'll get your wish: the objective hifi rating score, as read direct from machine, one to ten. Or, on the other hand, as and when new measurements are done and tested, you'd still have to actually be a real engineer (at least an amateur one) to understand anything more about them than how the companies use their graphs and diagrams to say "I'm the best!"

    And if you are a real (even amateur) engineer, join the club, and join in the work. Otherwise its just like asking for... dunno... better gravity? Maybe more-understandable gravity.
     
    Last edited: Dec 11, 2019
  9. makmeksam

    makmeksam New

    Joined:
    Oct 14, 2017
    Likes Received:
    7
    Trophy Points:
    3
    Location:
    Brazil
    This happened in 1915 I believe!
     
  10. JustAnotherRando

    JustAnotherRando My other bike is a Ferrari

    Contributor
    Joined:
    Aug 15, 2017
    Likes Received:
    1,373
    Trophy Points:
    93
    Location:
    Hong Kong
    Took me a moment to figure out what that meant, but when I did... that's such a great phrase!

    (Also, kind of appropriate with what's going on with this thread)
     
  11. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,134
    Trophy Points:
    113
    Location:
    India
    It is, isn't it! It goes an infinite number of steps beyond Where do I even start?
     
  12. supertransformingdhruv

    supertransformingdhruv Almost "Made"

    Contributor
    Joined:
    Mar 21, 2018
    Likes Received:
    579
    Trophy Points:
    93
    Location:
    DCish
    This is a little off-topic from the discussion on audio but I think there's something here worth unpacking re: how physicists practice science & by analogy, maybe audio science/measurements. I get really frustrated when people think that science is magic, and I think it's really important to provide a more practical viewpoint on where we stand re: gravity today. Some bullets below to hopefully give a bit of a quick primer:
    • The first thing I want to say is that we don't understand gravity. I believe most professional physicists would agree with me when I say that gravity is the least understood of the fundamental forces, and that gravity is the source of many of the most fundamental open questions in theoretical physics.
    • "But Dhruv," you probably rebut, "we've had GR since 1915! And since LIGO measured gravitational waves, we've even had empirical evidence for GR since 2016! What do you mean we don't understand gravity?"
    • Well, yes. For all practical purposes, we understand gravity for as well as we can measure.
      • In 1915 Einstein wrote down general relativity, and it's a wonderful, beautiful theory with massive impact and successful predictions.
      • We've used relativity to get around the world, with GPS. We've put satellites in space (GRACE, GRACE-FO) to measure the geode that function entirely on the basic math of GR.
      • GR is one of the two fundamental holy pillars of contemporary physics. And yes, it works. Mostly.
    • But here's the rub: we know it's incomplete (and I think this is a key word here: incomplete, not incorrect, but his is a place where someone who lives in this world i.e. a proper theorist in this subfield could debate technicalities all day, so I want to keep my claims relatively high-level).
      • We know GR breaks when it hits black holes. The curvature of spacetime becomes so extreme that the theory fails.
      • We know we can't reconcile GR with the other core pillar of theoretical physics (quantum field theory & the standard model of particles) without some modifications. There's no reason that 3 of the forces work one way, and gravity, another, so we have every reason to believe there should be a unified theory.
      • You might notice that both the places that the theory fails are unmeasureable zones.
      • Though they seem small, the actual impact of these holes in the theory is large enough that thousands of people spend their entire careers studying this topic. (back of the envelope lower bound calculation, there are ~2000 string theorists in the world today, and they've been at it for ~50 years. Not counting all the other disciplines that could also be considered quantum gravities).
    • So. GR doesn't tell the whole story. We know it's lacking. What do we do? Really, there are three core things:
      • First, we keep using GR for our day-to-day lives. The theory's great as long as you're not looking at a strong field and you're not trying to couple a classical gravity field to a quantum (particle) field. Most people aren't trying to do that-- they're trying to launch rockets & stuff, or study how stars & galaxies work. GR's just fine for that.
      • Second, we keep measuring things. LIGO's not done measuring classical gravity waves, there's plenty more data to take there. And even though we can't use these measurements to prove/disprove how the universe definitely works at a quantum scale (a common stat is that to falsify string theories, we'd need a particle accelerator the diameter of the solar system to probe approaching the plank length), we can learn a lot about the classical limits of things. And even at a coarse scale, there's enough to learn about the universe that we don't know yet. It should also be noted that we knew where GR failed even before we came up with the idea for LIGO.
      • Third, we keep making theories that can mathematically reconcile problems with our other theories. Some huge percent of theoretical physicists work on strings these days. While we can't necessarily falsify or confirm theories in this space via measurement, we can make all sorts of interesting discoveries (For example, AdS/CFT correspondence, which kind of fell out of research in quantum gravities but has been used to study nuclear & condensed matter physics via a really neat mathematical duality).
    • In conclusion:
      • We have an understandable theory of gravity. We know that it's not done. Still, we continue using GR.
      • We knew the problems with GR when we started looking for evidence for GR. Still, we continue measuring.
      • We know it might take >1000 years to figure out how to take measurements to confirm the theories we make today. Still, we continue theorizing.
    • Science/Physics/Gravity is not finished. Most people don't even know that these things aren't finished, let alone how little we know. And this is the big stuff, that we spend billions on (LIGO alone cost $1.1 billion).
    • I think the key takeaway is to understand that while we think we know some things, we also know we don't know many things. So we have to keep pushing in every direction. We can't say "we have enough theory, let's focus on measuring things" and we can't say "We won't be able to measure anyways, so let's just sit down and theorize." And we certainly can't say "This is a little wrong, so we can't use GR at all!"
    I try to stay in my lane, but I think my point about the practice of science remains about the same if you transpose my statements from gravity to audio. Sorry this got long.

    (Also, much like in audio, physicists are just as vicious to one another in online forums debating theories of gravity, and even their intrinsic worth as a subject of study-- check out the debate re: non-empirical theory confirmation or the loops people vs. the strings people).
     
  13. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,134
    Trophy Points:
    113
    Location:
    India
    WTF? Who even mentioned gravity?

    Oh... Me. Sorry.

    :oops: :)
     
  14. b0dhi

    b0dhi New

    Joined:
    Nov 20, 2019
    Likes Received:
    32
    Trophy Points:
    13
    Location:
    Sydney
    Without reality giving it hard and regular kicks in the ass, science can and does wander pretty far from truth. Physics has its butt firmly wrapped around reality's foot so it usually works well, but many of the softer sciences don't and are in a low-key crisis at the moment because of it:
    [​IMG]
    I think any rational person who sees these results must develop a strong scepticism of all the softer sciences. Some of those fields are more pseudoscience than science. Scientific fields can, and have, gone decades with all kinds of incorrect BS dominating them. Anyone familiar with Ego Depletion? Long been a "fact" in psychology, 600+ studies on it, but recently been shown to be BS. There are many examples like this from many fields.

    When it comes to audio (which falls into the soft sciences), I think there are two major things holding back progress, but the most immediate of them is that a major mistake was made when the field presumed that because our conscious perception of steady tones is limited to 20-20kHz, that the brain's entire internal audio pipeline must therefore be band limited to 20-20kHz (which in turn imposes certain information-theoretic conclusions about what can and can't possibly be audible). There's good evidence now that refutes this full-pipeline 20kHz band limit, but this should've looked suspicious from the beginning. This is the root of the "measurements tell us everything about how a thing sounds" problem.
     
  15. dematted

    dematted Friend

    Pyrate Banned
    Joined:
    Dec 7, 2019
    Likes Received:
    2,161
    Trophy Points:
    93
    Location:
    Brooklyn, New York
    I think this is really, really important to keep in mind. The entire point of this hobby is ultimately to have a superior experience of music. There are some sciences that deal with human experience - neuroscience, psychology, and the broader interdisciplinary field of cognitive science. But all of these aim ultimately to explain human experience, often in terms that don't include any reference to the experiences themselves that are being explained.

    Good audio equipment, however, is about producing certain kinds of experiences. To do that, one has to understand something about human auditory experience itself. And one of the best ways of understanding human experience is simply having it. It's very difficult to understand how someone who never had an auditory experience, who never knew what it was like to listen to something, could produce quality audio equipment that was enjoyable for others to listen to. Of course, one needs to also know the science - one needs to know how to go about producing a certain sound. But most importantly, you need to know something about the sound that ought to be produced, and to know that, there's not much that can substitute for your own experience of sound.
     
    Last edited: Dec 17, 2019
  16. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,960
    Trophy Points:
    113
    Location:
    Irvine CA
    Indeed the #1 evaluation method is listening. By that it is meant listening to the equipment yourself, not listening to what some "experts" tell you.

    As you correctly noted, personal biases, preferences, and even agendas, will result in different performance evaluations by different people. Similar can be said about measurements. Pick any random NwAvGuy measurement blog entry and one will find all kinds of interpretations of Excellent and Poor performance based on how bad the weather was that day. Measurement interpretation in audio is also a fairly subjective art.

    A complete battery of measurements will give you a fairly good picture of how linear and accurate system is relative to the recording equipment (w/o the car performance analogies, and if you have the background to make sense out of them). Here is what they cannot tell you:
    1. Your personal biases and preferences.
    2. The imperfections in the recording itself.
    Getting an accurate piece of reproduction equipment will in general give you subjectively good results. But it is indeed possible that a less accurate reproduction system will sound better to someone due to the above enumerated reasons.
     
    Last edited: Dec 17, 2019
  17. Cspirou

    Cspirou They call me Sparky

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    8,200
    Trophy Points:
    113
    Location:
    Northwest France
    just going to say Lubos Motl is an asshole
     
  18. Cspirou

    Cspirou They call me Sparky

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    8,200
    Trophy Points:
    113
    Location:
    Northwest France
    This reminds me of some experiment RCA did that led to them declaring that recordings should be filtered past 5000Hz, saying that audience surveys preferred music without the high frequency content. Olson thought this unusual so he setup a classic experiment with live musicians and used acoustic filtering to get the same effect. To no ones surprise the audience preferred full bandwidth live music.

    The reason for the audience preference with recorded sound is because there was a lot of distortion past 5khz. When Olson built an accurate set of very low distortion speakers, the test subjects said they preferred full bandwidth again.
     

Share This Page