Measurements Are Hard

Discussion in 'Measurement Techniques Discussion' started by purr1n, Sep 29, 2017.

  1. SineDave

    SineDave Friend

    Pyrate
    Joined:
    Apr 18, 2016
    Likes Received:
    862
    Trophy Points:
    93
    Location:
    Houston, TX
    Home Page:
    Amir was one of my partner's boss' actually - from what I've heard he's a good guy, but his first foray into forums (whatsbestforum) is equally full of nuttiness. The story goes that when he retired, he had enough cash to burn so he started a dealer\integrator business called Madrona Digital and co-founded WBF. At some point, WBF and he parted ways, and the new audiosciencereview forum was born. Strange stuff. When you retire wealthy, methinks you need something to stay sane.
     
  2. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,777
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    You mean how @Auditor would travel to our dojo, proclaim his cultural superiority and superior Kung Fu, and then get beaten to a pulp by the groundskeeper?
     
  3. Ringingears

    Ringingears Honorary BFF

    Pyrate
    Joined:
    Sep 26, 2015
    Likes Received:
    3,659
    Trophy Points:
    113
    Location:
    Northern Californium Valley
    I was a VP* for four years. I assure you I was a badass. I know hard to believe. But we had "Kung Fu". Our measurements of knife lengths were accurate to two decimal places. :cool:

    *High School Vice Principal
     
  4. Cspirou

    Cspirou They call me Sparky

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    8,200
    Trophy Points:
    113
    Location:
    Northwest France
    Found another post where he takes measurements of the Modi 2 with power regeneration.

    https://audiosciencereview.com/foru...eview-and-measurements.1829/page-2#post-46217

    Measurements are far better and it’s obvious that bad USB power is the culprit. The next post he goes on to say the Modi 2 is a poor design. Here’s the thing though, the iFi iDAC2 that he regards so highly is $350. A Modi 2 + Wyrd stack is $200 and fixes most problems he’s complaining about. A Modi 2U with separate linear power is $150 and fixes a lot of those issues as well. At the $99 price point of the Modi 2 I don’t know if you can solve the issues of USB power without increasing the price significantly. I mean the USB gen 5 upgrade is $150 alone, which is literally impossible on the Modi 2 since the gen 5 is not USB powered.

    Tried to find Modi 2/2U measurements on SBAF and didn’t see them. Were they on changstar?
     
  5. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,777
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    I never posted Modi 2 measurements (Modi 2 is actually my instrumentation DAC for headphone measurements).
     
  6. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,134
    Trophy Points:
    113
    Location:
    India
    Oh god... Amir on Hydrogenaudio. Don't even think about it: it was horrible.

    I have learned a lot from actual scientists/engineers versus digital-mythologists on the internet, and one of the last things that I learned was not to read those pantomime Oh Yes It Is Oh No It Isn't threads now.
     
  7. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,777
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    The iFi DAC 2 has an internal battery which gives it advantages and disadvantages. The disadvantage is that the battery can run out of juice and the iFi will be relegated to using the power provided by the USB rails. The advantage is that the battery allows the iFi to not be dependent on USB power. This is a great idea, but it does add cost. As you said, the iFi DAC2 is $350. The results of the Modi 2 would likely carry over to other USB powered DACs, O2, SDAC, etc.

    The problem is with USB.

    An USB hub (we are talking inside a computer), that numerous USB will ports attached to, has a only a certain amount of bandwidth and power that must be shared among all of the devices plugged into the USB ports. Sometimes a computer, like a laptop might have one hub with two or three USB ports. A gaming PC might have four hubs with eight or more USB ports available. So to begin with, it's hard of hard for us how to tell WTF which device is plugged into which USB port, and if the juice (or bandwidth) provided to each device is sufficient. For example, I know that plugging in the Focusrite 2i2 with an XMOS USB based DAC doesn't work, from at least one of my computers. Another issue is how USB works. USB doesn't use DMA (direct memory access that bypasses the CPU). USB data transfers still require the CPU to do some work. Finally, we might have have a good idea what the OS might be doing with the power management of USB. I have a SD card reader that seems to turn off and on on its own on one of my computers. A way around this for Windows is to turn off PC power management for each hub.

    Finally, let me illustrate an example. This was the initial measurement I took of the SDAC. I never posted it. You want to know why? Because it looked WRONG. This is where experience comes into play. Maybe for a discrete R2R DAC, it might be believeable. But this was very odd for an AKM chip based DAC. And knowing the guys at Grace, I knew they were not incompetent - that it would be me that was doing it wrong (or something was odd with the computer or Windows).

    bad SDAC.png

    This is exactly why measurements are hard, and why I would hestitate to come up with strong conclusions with such fine measurements, not at least without more corroborating data. One thing I like here is that @Hands, @atomicbob, and I have been doing measurements on the same devices that we can cross compare. Such a system (peer review) should be in place for measurements if we are truly interested in audio science.
     
    Last edited: Sep 30, 2017
  8. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    If we wanted to pursue this a bit further, I propose that a single device, either an amp or DAC be measured by all the participants for cross-calibration. Maybe a magni3 or SDAC. Ship the very same unit around to all participants for measurements. Have at least one common measurement to all, such as THD+N, THD with a spectrum shown. IMD would also be nice.

    Wishing out loud, I wish Quantasylum, HpW and ARTA would consider adding automated THD and THD+N sweeps over frequency measurements with graphs.
     
  9. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,653
    Trophy Points:
    113
    Location:
    On planet
    Here is this morning's ambient SPL shown with Noise Criteria lines. Measured using an ACO Pacific 7046 mic, True Systems mic preamp and
    ADI-2 Pro ADC into Smaart 7. Calibrated with a B&K 4231 mic calibrator.
    201701001-1040 SSL residual noise 16 dBA 7046-28439.png
     
    Last edited: Oct 4, 2017
  10. gixxerwimp

    gixxerwimp Professional tricycle rider

    Pyrate
    Joined:
    Sep 30, 2015
    Likes Received:
    5,773
    Trophy Points:
    113
    Location:
    small island claimed by China
    Just a clarification, the micro iDAC2 is purely USB powered, and it's the micro iDSD(BL) that has an onboard battery.

    I run the iDAC2 at work with a 1st gen iPurifier (came free as a bundle, don't think it does SFA) out of a PC and the iDSD at home on battery only out of an android phone. I can't really hear much difference between them.
     
  11. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,777
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    I thought it did because of its size!
     
  12. Torq

    Torq MOT: Headphone.com

    Pyrate
    Joined:
    Jan 15, 2016
    Likes Received:
    8,193
    Trophy Points:
    113
    Location:
    Seattle, WA
    Based on all the recent chatter about measurements, and per a post in a now-closed-thread, I wanted to experiment with some audio-specific measurements/measurement systems. I've played around with my various audio interfaces for this (still have to rope the RME ADI-2 Pro fs into that), and just picked up a QuantAsylum QA401 (I figured I'd get process, environmental and systems issues sorted there before looking at anything fancier).

    Not generally doing this sort of thing at home (nor in the audio space), I've become accustomed to the relative luxury of ultra-clean power, aerospace-level environmental EM/RF shielding and so on. So it's quite amusing to see what happens when you just connect things up and run a test in a random home environment.

    So I setup the QA401, ran it's initial tests, that's all good. Then I configure a DAC output via ARTA and set it's "Jitter" test running, and kick-off the QA software measurements against it. And ... get this:

    Jitter.png

    Now, I don't know if ARTA's "Jitter" test is an "actual" J-Test or something else. But those little peaks at 250 Hz intervals, and the apparent longer periodicity Rorschach-like "sine wave" they plot, is fascinating. Here they are zoomed in:

    Sine.png

    Can't, for the life of me, currently think of something in my environment that'd produce that. I know it's not the DAC doing it, as every other measurement I've seen of this type for it lacks these artifacts. But at that frequency, nothing off the top of my head explains those little peaks. I could theorize about ALL sorts of odd interactions, from running the software in a VM, something funky about the USB 3.0 hub, not-understanding what ARTA is outputting, a mismatch between ARTA's output and the QA401's capture, or some borked setting(s).

    Out of interest, I ran a different test, using a simple, pure, 12 kHz sine wave (also via ARTA) and got this:

    12 kHz.png

    That's more like the inferred jitter plots I've seen/would expect. There are two tiny spurs equidistant from the fundamental (see zoom, below), and the spikes at 250 Hz intervals are gone (ignore the change in output level, I tried both tests both ways and the results are identical). USB packet-tick is not apparent. No mains artifacts.

    NOW I get to figure out what's going on here ... because I'm not reckless enough to start throwing mud at things when clearly the issue is NOT the DUT. Something that's obvious to me even though I'm not practiced in making audio measurements.

    12 kHz Zoom.png
     
  13. Elnrik

    Elnrik Super Friendly

    Pyrate
    Joined:
    Jan 10, 2017
    Likes Received:
    8,973
    Trophy Points:
    113
    Location:
    Denver CO
    Home Page:
    Is the input selector switch on the right setting? Oh wait... You aren't Amir. Never mind.
     
  14. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,960
    Trophy Points:
    113
    Location:
    Irvine CA
    ARTA manual (page 26):

    "Note 2: Jitter test signal is a sine signal with a frequency equal to 1/4 of the sampling rate, and with a LSB bit toggled with a frequency equal to 1/192 of the sampling rate."

    Maybe that's the culprit.
     
  15. Torq

    Torq MOT: Headphone.com

    Pyrate
    Joined:
    Jan 15, 2016
    Likes Received:
    8,193
    Trophy Points:
    113
    Location:
    Seattle, WA
    Oooh!

    Well, that'd definitely map to the 250 Hz peaks! (48,000 / 192 = 250).

    THIS is a perfect example of why measurements from those practiced in doing them, and in a specific domain, are far more useful than those from people that just plug shit in, hit "run" and assume the output is golden.

    It's also an excellent illustration as to why one should RTFM!

    Thanks Sir!
     
  16. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,777
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Those hairs 250Hz apart are actually supposed to be there. They are inherent in the J-Test file. Another way to examine them is to output the ARTA file and examine the spectrum in something like Adobe Audition.

    In some instances, those hairs might not make it through because of dithering, noise shaping, filters, upsampling, etc.
     
  17. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    Ideally, a 16-bit/44.1KHz JTest result should look something like this:

    [​IMG]

    With the 24-bit test looking like this:

    [​IMG]
     
  18. Torq

    Torq MOT: Headphone.com

    Pyrate
    Joined:
    Jan 15, 2016
    Likes Received:
    8,193
    Trophy Points:
    113
    Location:
    Seattle, WA
    @purr1n and @Hands and @ultrabike

    The plot makes perfect sense based on what the actual ARTA jitter test is outputting. It's a good example of bad assumptions going in (i.e. what the ARTA jitter test is actually doing vs. what I thought it was doing) and not being experienced in making such measurements when confronted with the results.

    Thanks!
     
  19. ultrabike

    ultrabike Measurbator - Admin

    Staff Member Pyrate MZR
    Joined:
    Sep 25, 2015
    Likes Received:
    8,960
    Trophy Points:
    113
    Location:
    Irvine CA
    This is the ARTA Jtest result for the Focusrite 2i2 (differential I/O):

    jtest.png

    This was done using the latest 2i2 ASIO driver which only supports 24 bit I/O. The Line Out potentiometer is set at full range, but the level is set to -3.2 dBFS according to ARTA, which I don't fully trust. Let's say it's highish. Note this test was done at 48 kHz sampling rate (not 44.1 kHz).

    These are a set of loopback measurements performed by @OJneg on a Lynx Hilo (higher end interface). The last plot is the ARTA Jtest:

    https://www.superbestaudiofriends.org/index.php?threads/hilo-loopback-arta-measurements-baseline.38/

    I prefer the QA401 visualization.

    Based on what I see, it seems ARTA only supports 16th bit toggle, which I think is in agreement with the original test. But I could be wrong.
     
    Last edited: Jul 30, 2018
  20. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    That could be. I don't recall ever getting the sidebands to disappear using the ARTA JTest for any DAC or ADC.
     

Share This Page