DAC Blind Test Series

Discussion in 'Digital: DACs, USB converters, decrapifiers' started by purr1n, Feb 28, 2019.

  1. Josh83

    Josh83 Friend

    Pyrate
    Joined:
    Mar 1, 2017
    Likes Received:
    1,375
    Trophy Points:
    93
    Location:
    Charlottesville
    I’ve interacted with people on other sites who, honestly or not, claim they conducted rigorous blind tests (specifying procedures, posting pics, etc.) and found two pieces of gear sound the same. What they don’t seem to understand is that a blind test might be a refutation of differences if someone who said they heard clear differences after sighted listening couldn’t pass it. But if someone who not only claims there are no differences, but who also didn’t even attempt to hear differences through sighted listening, “fails” a blind test, it doesn’t really prove anything besides that a person who knew nothing still knows nothing.
     
  2. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    @purr1n The way you describe the MOS16 relative to the Amethyst is reminiscent of how I would describe a good TDA1541 NOS implementation, or so I would assume. I’m thinking of the upper end Audial stuff in particular...

    I believe the guy that made the MOS used to work a ton with the 1541 in the DIY community, so this would make sense from a targeted voicing perspective. But just guess work on my end, so I may be wrong.

    Granted, I’m still of the opinion that a laser trimmed ladder chip DAC > pretty much any discrete resistor ladder DAC. That includes the TDA1541, which on paper should be good for a solid, true 15 bits if used right. Not bad for old school shit...

    The Metrum stuff is laser trimmed too, per the designer, but I’m not sure if anyone has cracked open a module to verify if that’s true. I think the problem is Metrum designs from the top down. The Pavane and Adagio are excellent. Anything below feels like a half baked parts bin product in comparison, and still too expensive at that. All the secrecy on top of that, like closed door modules without full specs on internals or scratched off chip faces in first gen, rubs me the wrong way. (Not to mention their new marketing and PR team is atrocious.)

    In light of that, I appreciate what the RDAC and MOS are bringing to the table. Sound in the same vein as Metrum at an approachable price point. Scandals aside, that is...

    I am also curious if the insistence on toslink is holding the MOS back. I don’t know if it’s possible to make a “good” toslink input. Maybe his custom USB to optical stick is magical. But it’s like for all the work people put into making USB inputs sound (or measure) good, it still seems like a no win situation relative to good SPDIF (new Schiit stuff TBD). And good USB > any toslink I’ve heard.
     
  3. murphythecat

    murphythecat GRU-powered uniformed trumpkin

    Pyrate Banned
    Joined:
    Apr 4, 2016
    Likes Received:
    1,201
    Trophy Points:
    93
    Im too curious about his UPL16. supposedly, it makes a amazing source and raise the MOS16 to excellent levels.

    UPL16 ordered. Id be happy to ship to Marv
     
  4. dubharmonic

    dubharmonic Friend

    Pyrate
    Joined:
    Mar 30, 2017
    Likes Received:
    3,032
    Trophy Points:
    93
    Location:
    Evanston, IL
    Last weekend I was switching back and forth between a few DACs as well, with similar results. These differences really don’t show up in any measurements that DAC makers could publish?

    The number of “objectivists” who clearly haven’t listened to real hardware, spreading bad advice, makes me nervous about the future of this hobby. Building a DAC to produce clean sine waves for the objectivists reminds me of teachers being forced to teach to standardized tests in public schools.
     
  5. mkozlows

    mkozlows Friend

    Pyrate
    Joined:
    Oct 6, 2015
    Likes Received:
    512
    Trophy Points:
    93
    Yeah, this is the problem with most blind tests: The people who do them are the people who are pretty sure they can't hear a difference in the first place, whereas the people who are all "oh, it's night and day" don't often bother. (Which I understand; blind tests are unpleasant and stressful, and why bother if you already are sure you have the answer.)

    Really, the most meaningfully negative blind test that I've seen recently was IF's "Big Sound" stuff, when skilled listeners who were sure they could tell differences in gear were unable to do so blind.

    I think it's great that Purrin is doing these blind tests. The blind test I'd really love to see is between FLAC and 320kbps MP3, because every single time this comes up in conversation elsewhere, I get linked to Hydrogenaudio threads about how it's totally impossible to tell the difference, but... well, Hydrogenaudio is exactly the kind of place where people would be doing those tests in order to prove that they failed them.
     
  6. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    320kps and flac can be hard to tell the difference with certain kinds of music. Usually the busier stuff with sawtooth wave synth will be easier to tell. Little girl with guitar music (e.g., Jewel, Nora Jones, etc.), and it's hard to tell the difference. It's also a matter of knowing what to listen for. Younger people with extended hearing to 20kHz can almost instantly tell the difference.

    There is an ABX comparator for foobar.
     
  7. dmckean44

    dmckean44 In a Sherwood S6040CP relationship

    Pyrate
    Joined:
    Nov 17, 2016
    Likes Received:
    1,425
    Trophy Points:
    93
    Location:
    Peoria, IL
    I was a test listener for development builds of the 8hz MP3 encoder back in 1996 when I was an 18 year old and had better ears. There's a number of ways to tell MP3 files from lossless but MP3s biggest weakness (in 2019) is it's reconstruction filters.

    MP3 files will have more upper frequency energy compared to lossless files, this will come across as "enhanced detail" when listening and the lossless file will sound softer in comparison.
     
  8. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,129
    Trophy Points:
    113
    Location:
    India
    Oh dear. Some idiots claim that everything sounds the same. The intelligent, even the most objective, rarely do.

    Blind testing is the answer to audiophools, or those freak occasions when two components that really, really should not or cannot sound different appear to do so, not to prove that two things that sound different sound different. Blind test is the answer to rip off merchants who sell components to the bessotted, misguided, gullible wealthy. They have a right to spend their money on what they want, but the rip-off merchants should be prosecuted. Blind testing is the answer to rocks, and adapters stuffed with sand.

    There are even people who claim that all "objectivists" sound the same. They are the biggest idiots of all

    I put "objectivists" in quotes, because close examination usually shows them to be made of straw.
     
  9. Thad E Ginathom

    Thad E Ginathom Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    14,129
    Trophy Points:
    113
    Location:
    India
    The worst kinf of subjectivists: those who can't be persuaded that a lovely knowb and $,$$$s (and a winch?) does not sound better.

    The worst kind of Objectivists: One name that cannot be spoken: A**r
     
  10. Dzerh

    Dzerh Friend

    Pyrate
    Joined:
    Apr 1, 2018
    Likes Received:
    425
    Trophy Points:
    63
    Location:
    San Ramon, CA, US
    I can't see much difference between most "objectivists" and "subjectivists" - one just chooses what to believe - hype or couple of digits.
    People are lazy and learning how it sounds, comparing, testing, measuring, or even understanding what measurements mean, - it is work, meticulous work that takes a lot of time. It much easier to believe in ready answers. Especially reinforced by your money spent.
    I'm guilty as well but at least I'm recognizing it. :)
    Sorry for offtopic.
     
  11. mscott58

    mscott58 Friend

    Pyrate
    Joined:
    Oct 13, 2015
    Likes Received:
    2,028
    Trophy Points:
    113
    Location:
    Philly - Yo
    Yeah, people are lazy and like to live inside their own echo chambers, self-selecting into groups that agree with their POV and believing everyone else is wrong. IMHO the 24-hour news channels are the devil's children in this regard.

    Wish more people could achieve intellectual humility, being open to where they are wrong and seeking to understand other's input as valid. Here's a good (and short) article on the topic - https://www.characterlab.org/thought-of-the-week-naive-realism

    PS - Angela's "Thoughts of the Week" (such as the above) are usually pretty good, and unusually for an academic they are concise and actionable! Guess that's why she won a MacArthur Foundation Fellowship (otherwise known as a "genius grant").

    PPS - I'm 100% right about this and everyone else is wrong. So go piss-off if you don't agree...
     
  12. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    I compared the MOS16 against the Modi Multibit today (later, "fixed" iteration). I used both the Eitr and DigiOne Signature as my source. Straight coax for the Modi, but I had to use a $15 coax->toslink box for the MOS16. I did use a spare LPSU with the converter box to be safe. I made sure to give the MOS16 a full charge and let it play music for several hours straight before critical listening. No sonic changes observed from early to late warmup.


    MOS16 Strengths:

    - Solid, powerful, dynamic low end. Kick drums in particular sound forceful and full. Bass pitch, texture, and overall quality are slightly better than Modi Multibit's. Modi sounds a bit weak and hollow in comparison.

    - Smooth, balanced transition from bass into upper-midrange. Very cohesive. Tone and timbre are good, though top-end is rolled and it has a slightly dark tilt overall due to the NOS nature. It is handled well for what it is.

    - Very black background. Sounds clean. I'm sure the battery helps, but I've heard similarly "black" DACs without batteries.

    - Fairly liquid and wet sounding. A sort of thick sound overall. Slight pervasive bloom. (Sometimes tasteful.)

    - Generally fairly lively. Has good macro-dynamics. Engaging to listen to.

    - Macro details are OK, I guess?

    - Non-fatiguing but also not boring.


    MOS16 Weaknesses:

    - Arguably a little too thick. Slight pervasive bloom.

    - I am not hearing the resolution Marv mentioned. It is significantly less resolving than the Modi Multibit (and the RDAC was more or less on tier with the Modi).

    - In combination with the above, MOS16 seems to "whitewash" nuances like reverb, room decay and other spacial acoustic elements, etc. I have noticed this on many of the discrete multibit DACs I've heard. It's a "clean" sound but more akin to having the black level, contrast, and brightness set too high on your TV so it "pops" or looks "deep," when in reality you've obliterated all those nice little details in the shadows and highlights. (Give me a properly calibrated LED over a poorly calibrated OLED, if those are my only options.)

    - With headphones, I find it to generally be more forward and aggressive, in terms of staging, than the Modi Multibit. It has a smaller, closer stage, and in general sounds more claustrophobic; harder to separate out each piece from the mix. Despite this, it still sounds cohesive for what it's doing. It's not too narrow, not too wide, not too forward, not too distance if considered within the confines of its unique sound traits.

    - Combine the lower resolution + a definite lack in good spacial characteristics, and you lose a lot of magic when listening to the MOS16. It's fun, and it sounds good in general...but pop on "Lose Yourself to Dance," and listen to the weird ambient shit flying around at the start. It's kind of a cool effect on the Modi Multibit, which gives it a sort of "flying around the head in 3D" feel. It's just "there" on the MOS16. Queue up the live "Hotel California." One feels like you're in a live venue (Modi), and the other...doesn't (MOS16). These things all go hand it hand, and the MOS16 doesn't have them.

    - Feels and looks like a DIY garage project.


    MOS16 Random Thoughts:

    - I guessed it might sound like a decent TDA1541A DAC (based on my whopping sample size of one experience). It does not. It sounds more like something Audio-GD would have released between 2010-2015. I'm thinking the AGD Wolfson and 1704 DACs.

    - See my rant in the other thread about design aspects that irk me. See closing remarks here too for some of the same, and some new, thoughts when tied back to sound quality, value, etc.


    Some other comparison notes:

    - Modi Multibit sounds tighter and faster overall, even though it's less clean, more grey, hazier, etc.

    - And though this may sound contradictory to the above, the Modi Multibit sounds a bit more bloated/slower in the low end due to the mid-bass presentation. However, the MOS16 sounds very "thicc" and juicy in comparison, even though it does tend to have a tighter low end overall (only area where it's "tighter" and faster than Modi Multibit, yet, yes, still thicker too).

    - Modi Multibit is a bit strained and rough around the edges in the upper-mids/lower-treble. The MOS16 definitely has a sense of ease or flow in comparison. Nitpicking because the Modi still has great tone/timbre, and some of this is to be expected given the OS vs NOS nature of the DACs. Despite this, I'd say the Modi is a relatively neutral DAC. It does not sound as colored as the early unit I owned long ago.

    You are free to leave now if you only wanted sound comparisons, but I think the MOS16 is a lot more complicated than that.


    Closing Remarks and Other Rants:

    I have no doubts the MOS16 is source sensitive. I could only work with what I had. Maybe the toslink stick from a good USB port would be better. I'd be wiling to try it, though I don't have any great USB ports on any of my machines.

    I've heard great things about the dedicated player too (UPL16 or 24), but, again, am skeptical about the MOS16's inherent performance limitations and the player's, uh, "usability" angle. See post in other thread for full comments on that topic. Still, I'd give it a shot if offered. Change my mind!

    I really, really think it should have just come with a dang coax port. I get the whole misguided reason for using toslink, but let me make my own choices/mistakes if I want, yeah?

    I'm not sure why the MOS24 is priced so much higher than the MOS16. They share the same board, housing, etc., but the MOS24 just has empty areas populated. Perhaps the MOS24 uses higher quality/tolerance resistors? This too may be what is needed to realize the "vision" ECdesigns laid out, but I don't have enough information to make an educated guess on the 24-bit version.


    Thoughts on NOS, MOS16 Value

    I don't think the MOS16 is a good representation of or good introduction to the NOS sound, and mostly feel that way for reasons not directly related to its sound quality. The low-end is great, as is the cohesive tone and timbre. It's not perfect, but it's unique and works that angle well. I actually really dig it myself.

    However, the DAC really lacks when it comes to nuances. The strong low-end, black background, and interesting flavor are appealing at first (and remain appealing to some degree) but lose luster fast in the face of noticeable technical shortcomings.

    For the price, I would normally say this is totally acceptable. It's got it's own, cool thing going for it given the price point. I can recommend it for what it is in terms of sound representation.

    But with the toslink only thing comes the pressure to have to rethink your digital sources (yes, the start of the slippery slope of audiophool nervosa), inevitably leading you to consider investing in the whole ECdesigns ecosystem, just to try to maximize what is, at best, an overall OK DAC, and also being saddled with other "quirky" design choices in your gear stack (WAV only! 99 CDs and up to 9999 tracks, but you gotta number all the files!) that play into the sort of "mission" ECdesigns seems to be on for their audio gear (the sorts of "missions", "messages, and "visions" from audiophile witch doctors that audiophiles are quick to latch on to, like their own tiny audio gear religion)...

    God damn, the more I think about the DAC and the whole ECdesigns product lineup, with our common underlying drive to always want to synergize and have moar, and how the product lineup leverages the inherent consumer-nervosa angle, stuffing buyers into an extremely niche and, frankly, kooky environment, the angrier it makes me.

    (Yes, this isn't totally unique to ECdesigns, but they are an example on one far end of the spectrum. Just go read forums posts about these new products, how quick people are to snatch up the UPL16/24 out of near necessity just for the MOS16/24, and how it's suddenly a revelation in sound quality. Legitimately scary to me.)

    And I do like the MOS16! I genuinely enjoy listening to it and have reason to want to listen to it over the Modi Multibit enough to consider that, hey, maybe I'd want something like this myself.

    But the more I think about it, consider what it does and does not do well, and start to peek in that "what if door," which is still an urge that pulls at my core as an audio enthusiast, the more I feel the urge to back out, hard. I've been bitten before, will get bitten again, and see people fall prey all the time still. I'm a lot smarter, more cautious, and, frankly, less patient today, so ECdesigns as a whole to me sets off a whole lot of alarms I've setup over the years to try to stay safe and sane.

    (Because I know there's a high chance someone will reply to this, or comment on this in some other forum, and say, "Well if only he tried it with the UTOS or, better yet, UPL...")


    On discrete multibit DACs and increasingly less focused, sensible ramblings:

    STOP MAKING THEM (for the most part). I know they are hot shit right now, because discrete is always cool and better, right? But they generally suck in critical listening tests. The RDAC is the only one I've felt good about lately. Cheap, NOS-ish sound, generally matches performance level of Modi Multibit (which MOS16 doesn't, despite Modi's own, notable shortcomings). The powers that be know I raised concerns in private about Massdrop's plans for a discrete R2R DAC until I heard it. I'm willing to change my mind, but the trend I'm seeing and hearing with discrete multibit DACs is not so good.

    Anyway, I really want to mention how I don't think the MOS16 compares favorably to Metrum. I would much rather spend the extra cash on an Amethyst than a MOS16. Maybe Metrum is pulling a PR stunt, but they claim to use laser trimmed multibit DACs. Granted, the whole DAC module is huge (maybe hiding a discrete design), but they seem to rail on fully discrete multibit DACs because of all the engineering gymnastics you need to take just to overcome the fact you can get, what, 0.01% tolerance resistors at best?

    I mean, a partial discrete portion of the DAC is fine. I think UltraAnalog did that in their modules. But if you're going all the way, it better be 1) cheap, 2) use an architecture that helps address resistor tolerance shortcomings, 3) use resistors of the highest tolerance you can find, 4) have a shit ton of resistors/DAC modules for some parallelization, 5) not be toslink only. Still, seems like more effort than it's worth, but I guess it's still "cool" for customers and thus sells.

    The Amethyst is overpriced, no doubt, but it at least has the level of nuances I'd want from a DAC. I would have handily taken it over the Modi Multibit, but I ended up with the latter because I needed the cash. And, while also way overpriced, so far none of the discrete multibit DACs get anywhere close to matching the performance of the Pavane/Adagio. Those little things that get swept under the rug, they matter. Metrum seems to have figured that out. (Insert Schiit MB DACs in place of Metrum if you want oversampling and remove references to being overpriced, same idea.)

    I'd rather see limited runs of new DACs using old chips. I'd rather see DS DACs with custom designed, user selectable filters. I'd rather see more companies try to take on the same effort Schiit did by getting their choice of multibit IC to work. Hell, the Metrum stuff is still OEM, as far as I know. There are so many things I'd rather see than more discrete multibit DACs. The MOS16 further reinforces that, primarily but enticing me with a fair price point at the start while claiming I really need all their other shit to get my money's worth.

    Put another way: When I start looking at random, questionable, Chinese multibit DACs, which use old school chips, on eBay and think that, "Modding this to sound proper is probably a better use of my time and money than dealing with this discrete multibit DAC," that's a bad sign.

    In Closing, For Real

    I really do enjoy the MOS16, but it's more on the, eh, "musical" and "euphonic" side than it is a good performer. You can get both without necessarily paying much more, and without having to consider a weird, custom, toslink audio environment just for a very particular and peculiar DAC. It's not all bad, but given the whole picture, I'd say pass on the ECdesigns stuff for now.
     
  13. murphythecat

    murphythecat GRU-powered uniformed trumpkin

    Pyrate Banned
    Joined:
    Apr 4, 2016
    Likes Received:
    1,201
    Trophy Points:
    93
    thanks for the review.

    Gotta wonder how relevent it is to compare two DAC using a different source. I should have shipped the MOS16 with the UTOS from ecdesign, to at least listen to their DAC with one of their dedicated source.

    if you want, I can ship to you the UTOS.
     
  14. atomicbob

    atomicbob dScope Yoda

    Pyrate BWC MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    18,652
    Trophy Points:
    113
    Location:
    On planet
    Very well said.

    Meticulously conducted ABX testing is difficult and challenging. Even with a great ABX test system such as QSC or Van Alstine. Very few know how to conduct the tests properly. By comparison measurement suites are much easier. So is extended listening evaluations with only one system at a time being evaluated. So much bullshit on the internet at large, so little time.
     
  15. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    @Hands:

    I am wondering if the toslink to coax adapter is responsible for the differences in resolution and fine nuance. I used toslink direct from my CD transports.

    But yeah, point taken about lack of coaxial input.
     
  16. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    If you have the patience to let me hold onto the DAC until you can ship it, I am absolutely willing to give it a clean-slate try and eat crow if anything noticeably improves with their USB dongle! (It won't fix my other frustrations about the DAC or product lineup, but I'm willing to have my mind changed.)

    @purr1n It very well may be, but I did mention how frustrating it is to me that I'd ever have to consider "maybe it needs a good toslink source," inevitably leading me down the path of wondering about the other ECdesigns stuff. Then you start reading about folks raving about how the ECdesign player is the best thing they've ever heard, yada yada. It's a dangerous road and not one I usually want to walk.

    The toslink out on my X-Fi SB1240 (external USB card) sounded noticeably, eh, muddier and more congested than Eitr/DO Sig into coax converter. So that's one example of a straight USB->toslink source, though it may be a garbage toslink source.

    I have a couple other things in-house I may be able to scrounge together in the meantime for additional toslink source testing.

    The tinkerer in me noticed the digital input circuit is fairly easy to trace. Looks like there are through holes on the PCB one could use if they wanted to wire up straight I2S connectors.
     
  17. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,771
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    FWIW, toslink from either of my CD transports is actually decent. As in I wouldn't go into a murderous rage if someone switched it on me from the AES or coax. LOL, that is I may not even notice until two weeks later.

    I get you though. I believe the selling point behind the toslink on the EC designs site was galvanic isolation. It's frustrating that coax SPDIF input was not offered because it's what's offered on any self-respecting DAC out there. Personally, I think all this galvatranonic isolation shit is utter bullshit:
    1. Coax (or AES) SPDIF always sounded better. It was made for audio and up until recently was used by studios until it was replaced my AOIP - even then, the last leg is still Coax or AES, not toslink.
    2. Galvanic isolation can be accomplished with transformers (in fact, many SPDIF coaxial implementations do this).
    3. Audiophiles are retards - they lock on to concepts/terms such as "galvanic isolation", "feed-forward", and "inter-sample overs" and make them out to be way more than they ought to be.
    4. It just smells like too much like HF or DIYA (which has really turned into shit over the years).
     
    Last edited: Mar 31, 2019
  18. Hands

    Hands Overzealous Auto Flusher - Measurbator

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    12,285
    Trophy Points:
    113
    Location:
    Colorado
    Home Page:
    Funny you say that. This guy is a prominent figure on DIYA, as far as I know. More for his work on TDA1541 DACs, but as of the last several years for ECdesigns.
     

Share This Page