Soncoz SGD1 ASR Measurement Discrepancies

MODERATOR ADD:

This is a new thread split off from here where @purr1n initially found some discrepancies with the measurements of SINAD chart topping Soncoz SGD1 DAC on Audio Science Review. These discrepancies were significant to the point of downgrading the Soncoz DAC from the one of the "leaders of the pack" to one the laggards behind the leaders. Some people felt the initial discrepancies were because of differences in measurement gear, first a AverLAB, and then an APx525 vs. the state of the art APx555 (ASR).

Further investigation with two additional samples of the SGD1 DAC using the APx555 have confirmed @purrin's initial findings. One positive aspect is that the "IMD hump" fix has been corrected. We do not know if the significantly worse SINAD measurements are the result of this fix or if ASR received a cherry-picked sample from the manufacturer. However, we think it is necessary to let the buying public know that the spectacular SINAD results could not be duplicated with three off-the-shelf production samples, especially to those who rely on measurements to make a determined buying decision. In addition, a third sample showed worse measurements in one channel compared to the other.

ORIGINAL POST STARTS HERE:


So I just put the SGD1 on the APx555 and @purr1n looks like you were right, this is NOT measuring the same as ASR is showing.


This is SPDIF in, dac vol at max

0dbfs:



-0.5dbfs (to match voltage shown on ASR measurements):



-3dbfs:



-60dbfs:



-90dbfs:






I think there are two most likely explanations:

1) ASR got a unicorn/cherry picked unit, though given the huge variance and stupidly low harmonics shown in the ASR measurements I imagine this isn't the most likely explanation

2) The poorer SINAD/Higher harmonics are as a result of their 'IMD Hump Fix'.
Other manufacturers like Gustard I believe have said that they've not fixed the IMD hump because it would reduce SINAD.
Another manufacturer I spoke to recently said that the ESS chip can be run in voltage or current mode. When in voltage mode, you get better THD+N but do have the IMD hump.
Whereas in current mode it gets slightly poorer THD+N but no IMD hump.
There are likely other ways of addressing the issue given as some manufacturers have 'suppressed' the IMD hump instead of getting rid of it, but this seems to be the easiest fix and so it could simply be Soncoz has done that.

Also, It was suggested to me that for any device with a 9038 chip in it, measurements be taken after an hour or so of use. Apparently cooling the 9038 is not all that easy and many designs do not do so adequately which results in worse performance after a while.
How true this is I don't know but would be interesting to check. (SGD1 uses the mobile variants though so likely not an issue here)

 

Sorry @GoldenOne It seems you're saying something important but I don't follow.... could you explain your conclusion more simply.. you're measuring a bit / a lot better / worse than ASR?

 

This one definitely has no IMD hump which is great.
I'm thinking the diff in performance is because of the fix.

 

Significantly worse than ASR.

I'd actually measured an SGD1 not too long ago just using my ADI-2 Pro ADC and it had shown significantly worse results than the one on ASR.
I figured that I must have been doing something wrong, and wouldn't be able to measure harmonics as low as shown on ASR without something like an APx555 anyway. So just didn't post anything.

Then @purr1n said that he'd also found his unit to have significantly worse performance (not that it's bad, just nowhere near what ASR is showing).

Both of us have now tested SGD1s on APs and it shows that the measurements ASR is showing is nowhere even close to representative of what a consumer will be buying.

This could be the result of ASR getting an uber cherrypicked unit, or it could be the result of a change in design to fix the IMD hump (which has disappeared on my unit, but was prominent on ASR's).

In any case, Amir should add a PROMINENT warning to the review, remove it, or re-measure the device as what he is showing is NOT what consumers are buying.

 

Also worth noting that I'm getting much worse jitter measurements than Amir did
I don't know what method Amir is using to actually test jitter in his reviews. I'm just playing a J-Test file manually to the dac and recording the FFT atm. (APx J-Test utility has not yet been updated for APx 6.1 so can't use that)

(Also am I allowed to invoke my own beg-a-thon? XD I'm going to be sad when I have to give this back)

 

ROFL, LMAO

 

@purr1n So I have the other Soncoz Dac on hand and did some tests on that.

The 1khz SINAD test is...mostly the same, though one channel isn't as good:



But other tests are quite different from the results amir got. Linearity on Amir's tests for example was "textbook perfect" whereas on this unit it looks like this:



Fairly certain this isn't a signal path setup issue. Holo may linearity for comparison:



Again on Amir's unit, Jitter was as perfect as you like. Whereas on this unit both 44.1khz and 48khz Jitter are mediocre:



[EDIT: Turns out that on coax to the APx It's way better:

Seems perhaps the LA-QXD1 is sensitive to noise on the USB source? I'll play about with a few USB sources and see what happens]

[EDIT 2: Nope.avi
LA-QXD1 on SMS200 Ultra Neo (dac fed by roon):

Different than PC but still certainly not great. Seems the USB input is not so ideal here.]

IMD and Dynamic range measurements match Amir's though.
But yeah, yet another device where the measurements of the unit given to Amir do not match that of a normal unit (I bought this new from Amazon).

And to be clear that isn't me accusing Amir of having falsified something, I'm saying that perhaps getting units from a manufacturer isn't the best idea.
For someone with as much swing as Amir, a better way to do it would be to tell a company that you're going to buy one yourself and have the company reimburse you, rather than them send one directly.
That way you know you're not getting a cherrypicked unit.

 

i think L7 mentioned that amir does his linearity tests somewhat differently. Your linearity looks like L7's i think, but idk what exactly it is that amir does differently.

 

Thank you for this, I'd not known about this.
Just ran both and can see the differences below:

"Stepped Level Sweep":


"Bandpass Level Sweep":


Seems to be a pretty clear example of why the contents of the graph are fairly useless without the context of how the analyzer is configured.

I did also downgrade software to 6.0 and run the official APx J-Test utility. Got the same jitter results as before. Excellent on Coax, mediocre on USB. Same thing happened on SGD1.

 

So I had a very good conversation with a manufacturer yesterday, and we discussed 'misleading measurements' quite extensively.

One of the ideas we came up with is to create a 'standardised' APx test project file for community members to use. This would ofc only apply to those with access to an AP rather than other test equipment, but would still be quite beneficial.

What it would do is ensure that no matter if I, @purr1n , or anyone else tests a device, it is being tested with EXACTLY the same config. No borked measurements, no 'unorthodox' analyzer configs. Everyone is running the same agreed upon APx project sequence and config.
This project file can be locked, and the report will show that it was indeed used, so it shouldn't be possible to falsify this.
And then the report file will show exactly the analyzer config and 'full' test results for a device.

They said that they would be quite happy to use this when publishing measurements for their devices so that the community can trust them as well.
This would be good for everyone if done properly.

 

Oh I agree. An AP is absolutely not 'needed'. And I absolutely wouldn't want to give any sort of impression that 'if you don't have an apx555 then your measurements are meaningless' cause that's simply nonsense.
You can get great results even with consumer stuff like an ADI-2 Pro ADC to be honest if you use it properly or go further and build your own analog notch.

In terms of the hyper-focus on SINAD. It is most definitely a problem, especially when manufacturers are openly avoiding fixing other issues like IMD humps specifically to protect a few dB of SINAD. But I think the best way to fix this problem would be to have fully comprehensive measurements available, and a good dialog about them.
Have a proper discussion and put out good accompanying subjective discussion to get a better understanding of how some measurements might impact what we experience subjectively. Rather than just more of the 'this dac has 1dB more SINAD so it's better end of story and if you disagree we'll ban you'.
I trust this forum to do that. There are some great people here who know their stuff and the content such as what @atomicbob puts out is absolutely stellar.

Amir would never subscribe to it no, but there's no point in getting hung up over it. Amir won't change, the people who are not open to having their minds changed in any fashion won't either, and so there's no point trying. Lets focus on putting out something constructive for those who are curious and want something more thorough than just a SINAD test.


In the case of those who don't have AP's its tricky cause you can't have the same project sequencing feature.
So perhaps to fix that, there should be an accompanying written set of guidelines so that anyone can replicate the test conditions and configuration regardless of their measurement gear.

In terms of pacing though, that would be the other benefit of this.
If the sequence is constructed properly with the correct automation then you can plug device in, hit 'go' and just wait for it to spit out the full report.

 

So from the forgoing I read the community that can play are listed as follows:
@purr1n
@schiit
@GoldenOne
Jude
Amir

Players with other measurement equipment are excluded when the standard requires a specific analyzer to use a specific project file. Standards were developed to allow diversity of equipment to follow a consistent application of measurement methods.

I would like to point out the consistency for which I've reached with my technical measurements and always include
1. list of equipment used including type of signal cables and power cables
2. common analyzer settings such as analyzer frequency, signal generator levels, etc.
3. setup picture
4. standard measurement suite common to all such that fair comparisons may be made between components
5. corrections when errors have occurred
Additionally I have followed AES, IEC and ANSI standards where they exist and are applicable.

But if the trend is towards APx555 centric measurements, then I can conclude my efforts are becoming a waste of time and bandwidth.

 

This is not what I'm intending to imply at all.
In fact your measurements are quite frankly some of the best and most thorough around. And you ALWAYS provide detailed descriptions of your setup which is exactly what's needed and a huge thank you for doing that.

I am NOT wanting to make any sort of implication that anything other than an AP isn't good enough. That isn't true and anyone who thinks that is the case doesn't really understand what the differences between an AP and other hardware is.

This is just meant to be an easy way for those that DO have an AP to put out a comprehensive set of results in a way that shows they haven't screwed stuff up.

This is NOT intended to exclude anyone and the message is NOT that an AP is needed. But I don't think it's a bad thing to have a test sequence that people who DO have an AP can agree upon and use to ensure measurements are identical.

 

Unless these file locks involve the use of cryptographically-secure signatures, they absolutely can and will be falsified.