APx555 Adventures (and Misadventures)

Discussion in 'Measurement Techniques Discussion' started by Marvey, Jun 16, 2021.

  1. Marvey

    Marvey Super Friend

    Staff Member Pyrate BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,712
    Trophy Points:
    93
    I'm sure many of us may already familiar with the linearity measurements of DACs from Stereophile and Audio Science Review. Basically it's a graph that describes the difference for DAC's actual output level to an intended output level for a set of output levels from loudest (0dbFS) to an arbitrary soft level (say -120dbFS). Somewhere along the way to the softest sounds, a DAC starts to deviate away too much from the ideal. Usually a 1kHz tone is used for this test.

    For 16-bit content, we want to see a straight line down to about -96dbFS. As far what how much deviation is good or bad, that's a tough call. There are various blind test resources on the Internet to see how much of a difference you can hear. At moderate volumes, I can discern 0.2db with ease. At softer volumes, I'm not so sure. And even then, when it comes to music with its ebbs and flows, it's debatable if even relatively large differences such as 2db would be discernable.

    Anyway, psychoacoustics is a tough subject that requires human testing - which could also be subject to individuals' training and abilities. This topic here is about the linearity measurement.

    The proper way to do a precise linearity measurement for DACs on the APx555 is the Bandpass Level Sweep. A standard linearity measurement will not work because of how the analyzer gathers data. The output level is taken as the RMS on what's on the analyzer. Therefore we need to apply a bandpass filter on the generator signal, otherwise at lower levels, the output level will register as too high because of noise and distortion.

    Now let's take a look a typical 1kHz linearity measurement from 0dbFS to -120dbFS with the Bandpass Level Sweep where a 1/24 octave bandpass is applied. This is with the Geshelli JNOG DAC's balanced outputs using USB as the input.

    upload_2021-6-16_18-53-15.png

    Things look pretty good until about -110db where things start to deviate a bit. Even then, at -120dbFS, only the right channel is outputting about 0.5db less than it's supposed to be. I really don't know where to draw the line to what is good or what is bad - that is how much deviation from ideal. Personally, I'd say the result is excellent to -120dbFS.

    So we are done with this measurement right?

    Hold on folks, I ran this a second time (up to about -75dbFS). Wait, what's going on? Why did the behavior change so much?
    upload_2021-6-16_18-58-41.png

    Then I ran it two, three, four, eight more times for a total of 10. Here are the results overlaid on top of each other.
    upload_2021-6-16_19-0-41.png

    So what's the truth? The fact is, they all are the truth. With certain types of measurements, the limitations of the methods we employ to calculate the results, and the randomness of the real world, it is more or less impossible to get a consistent number (or sets of numbers of a graph).

    The reason I wanted to bring this up is since I will be using the amazing capabilities of the APx555, I did not want to mislead people on any specific measurement presentation, in this case, the linearity measurement, in the wrong way. Measurements can "lie". With great power comes great responsibility.

    I'm still not certain how I want to present this measurement. Perhaps taking several sets and overlaying them over each other? There are also custom settings in the APx555 with respect to data collection that I can play with to maybe get more consistent results.
     
    • Like Like x 11
    • Epic Epic x 8
    • List
    Last edited by a moderator: Jun 16, 2021
  2. Marvey

    Marvey Super Friend

    Staff Member Pyrate BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,712
    Trophy Points:
    93
    To explore further, I decided to grab Modi 3+ results.
    upload_2021-6-16_19-47-42.png

    Of course now I'm wondering if the proper method would be to take 10 measurements, take an average and then calculate a standard deviation for each point. This is all purely academic anyway. Unless of course you have music which actually has 19-bits data (microphones can maybe do 14-bits), can hear -110db below whatever peak single tone SPL are you playing your music at, and can discern 0.5db differences at those levels.

    Now if you want the audio equivalent of the Tesla that can do 205+mph even though can't drive anywhere like this, even on most track tracks, just because you can say you can...
     
  3. Marvey

    Marvey Super Friend

    Staff Member Pyrate BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,712
    Trophy Points:
    93
    The AP engineers are brilliant for coming up with this stuff. This is the ADC part of it.

    [​IMG]

    A bandpass is applied in the digital to the fundamental sine to strip it from all the distortion products. This is clear enough. A notch is applied to the input signal to strip out the "everything else". Gain is applied to this "everything else", presumably to increase SNR, before the ADC. Notching the original fundamental also prevents this signal from adding its own distortion products during the ADC process. The analog gain on the "everything else" is then reversed in the digital domain. Not sure what additional digital notch does. Probably there just to make extra sure - kill any remnants left over from the analog notch. Last step is combining the two signals for display on the analyzer screen.

    This high performance sine analyzer is also augmented by a super-duper analog sine generator. Yes, the analog generator is cleaner than the digital generator with respect to distortion. There is probably some magic here that AP isn't revealing. I can show you the results of both generators in a later post.

    The downside: the HPSA only works with single sine wave measurements.
     
    Last edited: Jun 17, 2021
  4. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    Gain linearity variance can be influenced by the power supply residual noise of the DAC.
    Here are a couple of examples to consider.
    These two animations are five measurement sweeps, using identical setup with the exact same cables.
    spdif in and SE out.
    The two sets of five sweeps were performed in the same hour with identical physical setup.

    schiit modius
    modius gain linearity.gif
    reasonably consistent among the measurement sweeps

    Topping D30 (the famous beginning to the lineage)
    D30 gain linearity.gif
    Observe a notably greater variance between measurement sweeps

    I am willing to bet I can reduce the D30 variance by using an LPS instead of the crap SMPS that is included with the DAC.

    dScope has a function to adapt FFT size as the level drops. Above -70 dBFS is typically measured with a 4096 FFT while below will go as high as 64K FFT to reduce the noise of the measurement.
     
    Last edited: Jun 18, 2021
  5. Marvey

    Marvey Super Friend

    Staff Member Pyrate BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,712
    Trophy Points:
    93
    Interesting, as we both know the Modi 3+ and JNOG use switching supplies. I will have to head back and try the Modius (more room on PCB for better switcher implementation) or another DAC with an LPS.

    I also wonder if dithering could screw things up (I need to check to see if I left dithering on in the AP). I want to be cautious before presenting anything. Just because I have an APx555 doesn't mean I should start taking random measurements and proclaiming myself a measurement messiah with each and every measurement presented as The Truth.
     
    Last edited by a moderator: Jun 18, 2021
  6. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    SMPS can be electrically quiet. Look at iFi and the low noise versions of Meanwell. The SMPS that shipped with the D30 is the lowest cost crap rated for 15Vdc with 1A available current. D30 doesn't need 1A but when looking at low cost SMPS they seem to begin around 1A.
     
  7. GoldenOne

    GoldenOne Friend

    Pyrate
    Joined:
    Jul 17, 2020
    Likes Received:
    1,049
    Trophy Points:
    93
    Location:
    England
    It's important for people to know just how 'flexible' so many of the measurements are.

    There is of course just the run to run variation, which quite frankly makes a lot of the stuff like Amir's SINAD chart and giving one dac '1dB' better SINAD just silly.
    The SINAD number is hardly fixed. And just taking the screenshot at the right moment can often mean the difference between getting 120 or 123dB SINAD etc.

    If you wanted to make a particular brand look good/bad it would be incredibly easy to do so....

    Hell even things like using a different USB source, or in some cases literally having your hand near the device or board can change things!
     
  8. GoldenOne

    GoldenOne Friend

    Pyrate
    Joined:
    Jul 17, 2020
    Likes Received:
    1,049
    Trophy Points:
    93
    Location:
    England
    I've made a video discussing just a few ways in which measurements can be misleading, or influenced either by the manufacturer or person doing the measurements. Hopefully it'll be interesting to a decent number of ppl!

     
    • Epic Epic x 25
    • Like Like x 11
    • List
  9. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Fantastic video. Short, to the point, reasonable, and well communicated. I'm glad that you are doing stuff like this because people think that an expensive piece of instrumentation makes things "easy" or "accurate". In some respects, it makes thing worse. We need to pay more attention to not "bork" results.
     
    • Like Like x 8
    • Epic Epic x 3
    • List
  10. GoldenOne

    GoldenOne Friend

    Pyrate
    Joined:
    Jul 17, 2020
    Likes Received:
    1,049
    Trophy Points:
    93
    Location:
    England
    Yep!
    There are so many things that can be different and I think it's important for people to know that.
    And the tricky part is that in most instances, there isn't a 'correct' way. Just a few different ways that make sense for different reasons but will produce different results.

    Having a fancy measurement system does not guarantee consistent results and it's also not a case of education or experience.
    Ironically there is a pretty significant level of subjective choice to how to perform objective tests :p
     
  11. schiit

    schiit SchiitHead

    Pyrate
    Joined:
    Sep 28, 2015
    Likes Received:
    9,937
    Trophy Points:
    93
    Location:
    Texas and California
    Home Page:
    Thank you for doing this!

    We spend an amazing amount of time doubting results, mainly due to cables, custom load boxes, etc. It takes a ton of time to do things right...and sometimes we still get some strange glitches due to settling time--which can change the reported distortion number by a factor of 30X, or show 1dB droop in the audio band frequency response where none exists.

    Of course we may be a weird edge case, because we don't publish the single-screenshot test results, but instead provide a big PDF of analyzer results, including loads not accessible on the AP.

    What's even funnier (to me) is how some products can't be "properly" measured on our "lowly" APx525 in Corpus Christi, and have to be brought to Valencia for the APx555 treatment. Many things measure below the APx525 threshold (about -112dB) when tested at the de facto "standard" levels. What's kinda sad about that is how much time can be wasted trying to get a -115dB design to -120dB.

    Which makes it even super funnier (again, to me) that most of the time in a real system you aren't running the "standard" levels, nor are you full scale on the potentiometer, which (as you noted) will significantly affect measurements due to thermal noise. It's super easy to be running a -80dB levels on a "-120dB" amp. Which is also perfectly fine, and something you may never hear in normal operation.
     
    • Like Like x 15
    • Epic Epic x 4
    • List
    Last edited: Apr 20, 2022
  12. Cryptowolf

    Cryptowolf Repping Chi Town - Friend

    Pyrate Contributor
    Joined:
    Sep 27, 2015
    Likes Received:
    1,450
    Trophy Points:
    93
    Location:
    Rural Illinois
    I think the general population isn’t taught about the precision of measurements and significant digits. I had this beaten into me working in labs. At the start of every day, I had to calibrate the lab scales with certified gram weights (1, 50, 100). These certified weights were kept in a special box and if dropped, had to be discarded. Each day I adjusted the scales to within .001 or better accuracy (to the certified reference weights) and then precision (how close measured vale’s are to each other) against the three weights. And this was for weight measured against know reference values. I can only imagine the difficulty involved with multi-factor measurements of voltage, waveform, or current.

    The point being that expensive equipment that offers ideally higher accuracy AND precision can be easily borked unless reference protocols are strictly followed.
     
  13. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Yup. With super sensitive instrumentation, lots can go wrong.

    I still remember the first engineering class I took in university where we set about into teams and did measurements. Accuracy, consistency, variability varied widely among the teams. I trust @GoldenOne and I trust @atomicbob because of the amount of time they put into their measurements before publishing. Others who crank out measurements based "reviews" at the rate of one piece of gear per day, I would question.
     
  14. Clemmaster

    Clemmaster Friend

    Pyrate Contributor
    Joined:
    Sep 28, 2015
    Likes Received:
    3,268
    Trophy Points:
    113
    Same. I've measured my share of resistors and other passive components to learn about variability, repeatability, tolerance and the likes. A nice gaussian distribution was usually a sign that things went according to God's plan!
     
  15. Ziva

    Ziva Friend

    Pyrate
    Joined:
    Oct 10, 2015
    Likes Received:
    234
    Trophy Points:
    43
    My better half is a neuroscientist. Works with electron microscopes. When you see a trained professional with a PhD throw out 6 months or a year’s worth of research because of a tiny error - calibration, equipment, human, etc. - it really drives this point home. (Not to mention the many brilliant people who cannot work in a lab setting in the first place - too exacting)
     
  16. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    It is not possible to accurately measure and review components at the rate of one per day per individual. That is utter nonsense. Anyone with a modicum of laboratory experience will know how much effort goes into obtaining comprehensive, reliable, repeatable results accurately depicting a complete component performance profile.

    I could buy carpentry tools, but owning the tools doesn't automatically confer the skill and experience of a master carpenter on me.
     
    • Like Like x 7
    • Epic Epic x 2
    • List
  17. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    • Like Like x 4
    • Epic Epic x 1
    • List
  18. GoldenOne

    GoldenOne Friend

    Pyrate
    Joined:
    Jul 17, 2020
    Likes Received:
    1,049
    Trophy Points:
    93
    Location:
    England
    There's currently a thread on ASR about the video.
    I am shadowbanned from ASR (Banned but profile still says 'active user', seemingly to make it seem as though I had run away from the blind test discussion) but posted a response to a few of the points in that thread on an alt.
    This was removed within the span of a couple minutes of course but I saved here:
    https://web.archive.org/web/2022042...udio-measurements-by-goldensound.33222/page-4

    Not sure where else to post it so figured here may be appropriate.
     
    • Like Like x 10
    • Epic Epic x 1
    • List
  19. HeyWaj10

    HeyWaj10 Facebook Friend

    Contributor
    Joined:
    Jan 10, 2022
    Likes Received:
    229
    Trophy Points:
    43
    Location:
    Cary, NC
    I saw your post on ASR, which was wholeheartedly appropriate, and yet they deleted it. Talk about "running away" - they can't even allow someone the opportunity to respond!
     
  20. Merrick

    Merrick A lidless ear

    Pyrate
    Joined:
    Jan 6, 2016
    Likes Received:
    12,569
    Trophy Points:
    113
    Location:
    Portland, OR
    Ah, that's a classic. Shadowban someone and then ding them when they create a second account to inform people of the shadowban. Now they'll say they deleted the second post because you broke forum rules by creating the second account, blah blah blah.

    How anyone can take ASR seriously when they frequently abuse members like this is beyond me.
     

Share This Page