Soncoz SGD1 ASR Measurement Discrepancies

Discussion in 'Digital: DACs, USB converters, decrapifiers' started by GoldenOne, Jun 17, 2021.

  1. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    MODERATOR ADD:

    This is a new thread split off from here where @purr1n initially found some discrepancies with the measurements of SINAD chart topping Soncoz SGD1 DAC on Audio Science Review. These discrepancies were significant to the point of downgrading the Soncoz DAC from the one of the "leaders of the pack" to one the laggards behind the leaders. Some people felt the initial discrepancies were because of differences in measurement gear, first a AverLAB, and then an APx525 vs. the state of the art APx555 (ASR).

    Further investigation with two additional samples of the SGD1 DAC using the APx555 have confirmed @purrin's initial findings. One positive aspect is that the "IMD hump" fix has been corrected. We do not know if the significantly worse SINAD measurements are the result of this fix or if ASR received a cherry-picked sample from the manufacturer. However, we think it is necessary to let the buying public know that the spectacular SINAD results could not be duplicated with three off-the-shelf production samples, especially to those who rely on measurements to make a determined buying decision. In addition, a third sample showed worse measurements in one channel compared to the other.

    ORIGINAL POST STARTS HERE:


    So I just put the SGD1 on the APx555 and @purr1n looks like you were right, this is NOT measuring the same as ASR is showing.


    This is SPDIF in, dac vol at max

    0dbfs:

    upload_2021-6-17_14-58-43.png

    -0.5dbfs (to match voltage shown on ASR measurements):

    upload_2021-6-17_14-57-8.png

    -3dbfs:

    upload_2021-6-17_14-56-22.png

    -60dbfs:

    upload_2021-6-17_14-53-40.png

    -90dbfs:

    upload_2021-6-17_14-54-44.png




    I think there are two most likely explanations:

    1) ASR got a unicorn/cherry picked unit, though given the huge variance and stupidly low harmonics shown in the ASR measurements I imagine this isn't the most likely explanation

    2) The poorer SINAD/Higher harmonics are as a result of their 'IMD Hump Fix'.
    Other manufacturers like Gustard I believe have said that they've not fixed the IMD hump because it would reduce SINAD.
    Another manufacturer I spoke to recently said that the ESS chip can be run in voltage or current mode. When in voltage mode, you get better THD+N but do have the IMD hump.
    Whereas in current mode it gets slightly poorer THD+N but no IMD hump.
    There are likely other ways of addressing the issue given as some manufacturers have 'suppressed' the IMD hump instead of getting rid of it, but this seems to be the easiest fix and so it could simply be Soncoz has done that.

    Also, It was suggested to me that for any device with a 9038 chip in it, measurements be taken after an hour or so of use. Apparently cooling the 9038 is not all that easy and many designs do not do so adequately which results in worse performance after a while.
    How true this is I don't know but would be interesting to check. (SGD1 uses the mobile variants though so likely not an issue here)
     

    Attached Files:

    Last edited by a moderator: Jun 19, 2021
  2. purr1n

    purr1n Burned out

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    82,449
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    I may pick up another unit for sample #3 and run the IMD sweep. This is disturbing and probably deserves its own thread. I may move some posts around.
     
  3. Woland

    Woland Friend

    Friend
    Joined:
    Jan 13, 2021
    Likes Received:
    1,065
    Trophy Points:
    93
    Location:
    a friendly land
    Sorry @GoldenOne It seems you're saying something important but I don't follow.... could you explain your conclusion more simply.. you're measuring a bit / a lot better / worse than ASR?
     
  4. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    This one definitely has no IMD hump which is great.
    I'm thinking the diff in performance is because of the fix.

    upload_2021-6-17_15-15-54.png
     
  5. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    Significantly worse than ASR.

    I'd actually measured an SGD1 not too long ago just using my ADI-2 Pro ADC and it had shown significantly worse results than the one on ASR.
    I figured that I must have been doing something wrong, and wouldn't be able to measure harmonics as low as shown on ASR without something like an APx555 anyway. So just didn't post anything.

    Then @purr1n said that he'd also found his unit to have significantly worse performance (not that it's bad, just nowhere near what ASR is showing).

    Both of us have now tested SGD1s on APs and it shows that the measurements ASR is showing is nowhere even close to representative of what a consumer will be buying.

    This could be the result of ASR getting an uber cherrypicked unit, or it could be the result of a change in design to fix the IMD hump (which has disappeared on my unit, but was prominent on ASR's).

    In any case, Amir should add a PROMINENT warning to the review, remove it, or re-measure the device as what he is showing is NOT what consumers are buying.
     
    • Like Like x 8
    • Epic Epic x 2
    • List
  6. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    Also worth noting that I'm getting much worse jitter measurements than Amir did
    I don't know what method Amir is using to actually test jitter in his reviews. I'm just playing a J-Test file manually to the dac and recording the FFT atm. (APx J-Test utility has not yet been updated for APx 6.1 so can't use that)

    (Also am I allowed to invoke my own beg-a-thon? XD I'm going to be sad when I have to give this back)

    upload_2021-6-17_15-30-51.png

    upload_2021-6-17_15-30-56.png
     
  7. dasman66

    dasman66 Self proclaimed lazy ass - friend

    Friend
    Joined:
    Mar 13, 2018
    Likes Received:
    1,711
    Trophy Points:
    93
    Location:
    NW Pennsylvania
    ROFL, LMAO
     
    • Like Like x 4
    • Epic Epic x 2
    • List
  8. Merrick

    Merrick A lidless ear

    Friend
    Joined:
    Jan 6, 2016
    Likes Received:
    10,565
    Trophy Points:
    113
    Location:
    Portland, OR
    [​IMG]
     
  9. purr1n

    purr1n Burned out

    Staff Member Friend BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    82,449
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    I don't mind people using measurements as a method to choose what gear they want to buy. In the morass of many varied opinions, it's hard to get a sense how something may actually sound like. Good measurements are one way to ensure one is getting something that is actually decent.

    What I do find troubling is outfits like Soncoz handing off to Amir cherry-picked units for Amir to review. I would otherwise believe that the fix for the IMD hump was the cause of worse SINAD at 0dbFS, except that the tradeoff would have been so extreme (20db worse for H2, 25db worse for H3) as for it not to be worth it.

    What's worse is that they allow a member (the other guy in China with the APx555) to post progress on the IMD hump, and then also apparently from the manufacturer themselves. However, no one bothers to post a very simple 1kHz FFT at 0dbFS - which is the easiest thing to do with one has an APx525/555 (when we start the AP program, this measurement screen is literally just set up right there to run at the touch of a button).

    So I don't know what's going on. If it's a cherry picked unit, then shame on Soncoz. If the IMD hump fix turned a chart-topping 120db SINAD DAC into a 108db SINAD DAC, then that's cherry picking measurements while hiding certain others. I dunno. I'm kind of upset about this. ASR looks like they've turned into another HF with a bunch of manufacturer sock-puppets gaming the system. Except worse since Amir seems to actively allow or even encourage this - especially from Chinese manufacturers. Does anyone notice how the China brands are always given a free-pass, the benefit of the doubt, until it is no longer possible (e.g, after exploding five headphones)?

    If an NA or EU brand did this on ASR, they would be crucified.
     
    Last edited: Jun 19, 2021
  10. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    @purr1n So I have the other Soncoz Dac on hand and did some tests on that.

    The 1khz SINAD test is...mostly the same, though one channel isn't as good:

    ybqCVAcFUp.png

    But other tests are quite different from the results amir got. Linearity on Amir's tests for example was "textbook perfect" whereas on this unit it looks like this:

    t39ILqxfuC.png

    Fairly certain this isn't a signal path setup issue. Holo may linearity for comparison:

    nRx2xDFEfy.png

    Again on Amir's unit, Jitter was as perfect as you like. Whereas on this unit both 44.1khz and 48khz Jitter are mediocre:

    WUfswJFi1K.png
    38LngTt4ql.png
    [EDIT: Turns out that on coax to the APx It's way better:
    [​IMG]
    Seems perhaps the LA-QXD1 is sensitive to noise on the USB source? I'll play about with a few USB sources and see what happens]

    [EDIT 2: Nope.avi
    LA-QXD1 on SMS200 Ultra Neo (dac fed by roon):
    [​IMG]
    Different than PC but still certainly not great. Seems the USB input is not so ideal here.]

    IMD and Dynamic range measurements match Amir's though.
    But yeah, yet another device where the measurements of the unit given to Amir do not match that of a normal unit (I bought this new from Amazon).

    And to be clear that isn't me accusing Amir of having falsified something, I'm saying that perhaps getting units from a manufacturer isn't the best idea.
    For someone with as much swing as Amir, a better way to do it would be to tell a company that you're going to buy one yourself and have the company reimburse you, rather than them send one directly.
    That way you know you're not getting a cherrypicked unit.
     
    Last edited: Jun 18, 2021
  11. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,693
    Trophy Points:
    93
    Jitter results are interesting because usually, USB has better jitter than coax SDPIF.

    Are you running the Bandpass Level Sweep for linearity (sequence mode)?
    upload_2021-6-18_22-8-22.png
     
  12. TheTalbotHound

    TheTalbotHound Rando

    Joined:
    Mar 17, 2018
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Earth

    i think L7 mentioned that amir does his linearity tests somewhat differently. Your linearity looks like L7's i think, but idk what exactly it is that amir does differently.
     
  13. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,693
    Trophy Points:
    93
    Amir does his linearity tests differently after Jude with an AP engineer's help schooled Amir on how he did it wrong with respect to ASR's borked Yggdrasil measurements. The test needs to be run through a bandpass filter because otherwise the AP takes the RMS reading, which at lower levels will be subject to +error because of distortion and noise. It's not totally clear from the AP interface, so it's easy to make this mistake (note that ASR has not corrected this mistake nor removed the borked results - I'm surprised Schiit hasn't sued ASR). The function with the bandpass actually isn't available in Bench Mode and only appears to be available in Sequence Mode.

    https://www.superbestaudiofriends.org/index.php?threads/apx555-adventures-and-misadventures.11108/
     
    Last edited by a moderator: Jun 19, 2021
  14. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    Thank you for this, I'd not known about this.
    Just ran both and can see the differences below:

    "Stepped Level Sweep":
    upload_2021-6-19_16-58-16.png

    "Bandpass Level Sweep":
    upload_2021-6-19_16-59-38.png

    Seems to be a pretty clear example of why the contents of the graph are fairly useless without the context of how the analyzer is configured.

    I did also downgrade software to 6.0 and run the official APx J-Test utility. Got the same jitter results as before. Excellent on Coax, mediocre on USB. Same thing happened on SGD1.
     
  15. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    So I had a very good conversation with a manufacturer yesterday, and we discussed 'misleading measurements' quite extensively.

    One of the ideas we came up with is to create a 'standardised' APx test project file for community members to use. This would ofc only apply to those with access to an AP rather than other test equipment, but would still be quite beneficial.

    What it would do is ensure that no matter if I, @purr1n , or anyone else tests a device, it is being tested with EXACTLY the same config. No borked measurements, no 'unorthodox' analyzer configs. Everyone is running the same agreed upon APx project sequence and config.
    This project file can be locked, and the report will show that it was indeed used, so it shouldn't be possible to falsify this.
    And then the report file will show exactly the analyzer config and 'full' test results for a device.

    They said that they would be quite happy to use this when publishing measurements for their devices so that the community can trust them as well.
    This would be good for everyone if done properly.
     
    Last edited: Jun 19, 2021
  16. Marvey

    Marvey Super Friend

    Staff Member Friend BWC
    Joined:
    Jun 6, 2021
    Likes Received:
    1,693
    Trophy Points:
    93
    This is a great idea in theory, and one I am willing to subscribe to. However, I do think it's a bit too AP-centric. As much as I like the AP analyzers and respect their engineers, I would hate to give them a monopoly over this. There are a few other analyzers on the market which can be just as effective. I would also be a bit concerned about sending the wrong message with respect to measurements. We've already seen the big-schlong waving with respect to SINAD from companies like Soncoz and Topping, and personally I think this is unhealthy for what otherwise is a subjective hobby. And really, an APx525 is good enough for this. We don't need an APx555 (As an aside, it also makes no sense when I see people use transducers with terrible linear and non-linear distortion with these high SINAD topping amps, pun intended).

    Finally, I doubt ASR, the largest player in the measurement forum universe, would subscribe to this. What would ASR do if he could not cherry pick measurements, accept cherry picked gear, to opportunistically find results so as to market gear from manufacturers he probably has a business relationship with or crap on gear where a relationship may have gone sour? How would ASR easily and simply write a concluding narrative of recommended / not recommended. Such a proposal would require nuance - an understanding of many different factors into what determines good sound - whereas ASR is a few shades of grey (or Pink Panther poses).

    We've seen Amir react positively to criticism of this methods, e.g. "Amir-bits", where he eventually went on to SINAD from his prior improperly conducted ENOB measurements. However, I doubt he is one to adhere to a standardized comprehensive process. Such a process may take too long. Part of ASR's appeal is an insane pace of two reviews (measurements) a day.

    I've had the APx555 for a week now and have still not posted any reviews based on it. There's a reason for this. With great power (or measurement gear) comes great responsibility. I don't think ASR has the same moral compass. ASR just makes it up as he goes along, press the play button the AP with nary a care in the world so long as it suits his ego and brings followers. We've already seen how his "praetorian guard" protects him firsthand.
     
    Last edited by a moderator: Jun 19, 2021
  17. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    Oh I agree. An AP is absolutely not 'needed'. And I absolutely wouldn't want to give any sort of impression that 'if you don't have an apx555 then your measurements are meaningless' cause that's simply nonsense.
    You can get great results even with consumer stuff like an ADI-2 Pro ADC to be honest if you use it properly or go further and build your own analog notch.

    In terms of the hyper-focus on SINAD. It is most definitely a problem, especially when manufacturers are openly avoiding fixing other issues like IMD humps specifically to protect a few dB of SINAD. But I think the best way to fix this problem would be to have fully comprehensive measurements available, and a good dialog about them.
    Have a proper discussion and put out good accompanying subjective discussion to get a better understanding of how some measurements might impact what we experience subjectively. Rather than just more of the 'this dac has 1dB more SINAD so it's better end of story and if you disagree we'll ban you'.
    I trust this forum to do that. There are some great people here who know their stuff and the content such as what @atomicbob puts out is absolutely stellar.

    Amir would never subscribe to it no, but there's no point in getting hung up over it. Amir won't change, the people who are not open to having their minds changed in any fashion won't either, and so there's no point trying. Lets focus on putting out something constructive for those who are curious and want something more thorough than just a SINAD test.


    In the case of those who don't have AP's its tricky cause you can't have the same project sequencing feature.
    So perhaps to fix that, there should be an accompanying written set of guidelines so that anyone can replicate the test conditions and configuration regardless of their measurement gear.

    In terms of pacing though, that would be the other benefit of this.
    If the sequence is constructed properly with the correct automation then you can plug device in, hit 'go' and just wait for it to spit out the full report.
     
  18. atomicbob

    atomicbob dScope Yoda

    Friend BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    16,780
    Trophy Points:
    113
    Location:
    On planet
    So from the forgoing I read the community that can play are listed as follows:
    @purr1n
    @schiit
    @GoldenOne
    Jude
    Amir

    Players with other measurement equipment are excluded when the standard requires a specific analyzer to use a specific project file. Standards were developed to allow diversity of equipment to follow a consistent application of measurement methods.

    I would like to point out the consistency for which I've reached with my technical measurements and always include
    1. list of equipment used including type of signal cables and power cables
    2. common analyzer settings such as analyzer frequency, signal generator levels, etc.
    3. setup picture
    4. standard measurement suite common to all such that fair comparisons may be made between components
    5. corrections when errors have occurred
    Additionally I have followed AES, IEC and ANSI standards where they exist and are applicable.

    But if the trend is towards APx555 centric measurements, then I can conclude my efforts are becoming a waste of time and bandwidth.
     
    • Like Like x 3
    • Respectively Disagree Respectively Disagree x 1
    • Epic Epic x 1
    • List
  19. GoldenOne

    GoldenOne Friend

    Friend
    Joined:
    Jul 17, 2020
    Likes Received:
    895
    Trophy Points:
    93
    Location:
    England
    This is not what I'm intending to imply at all.
    In fact your measurements are quite frankly some of the best and most thorough around. And you ALWAYS provide detailed descriptions of your setup which is exactly what's needed and a huge thank you for doing that.

    I am NOT wanting to make any sort of implication that anything other than an AP isn't good enough. That isn't true and anyone who thinks that is the case doesn't really understand what the differences between an AP and other hardware is.

    This is just meant to be an easy way for those that DO have an AP to put out a comprehensive set of results in a way that shows they haven't screwed stuff up.

    This is NOT intended to exclude anyone and the message is NOT that an AP is needed. But I don't think it's a bad thing to have a test sequence that people who DO have an AP can agree upon and use to ensure measurements are identical.
     
    Last edited: Jun 19, 2021
  20. gepardcv

    gepardcv Almost "Made"

    Contributor
    Joined:
    Nov 20, 2015
    Likes Received:
    382
    Trophy Points:
    63
    Location:
    Terra, Sol System
    Unless these file locks involve the use of cryptographically-secure signatures, they absolutely can and will be falsified.
     

Share This Page