Schiit (MultiBit) Bifrost

Yea, I've experienced that audio quality can be more of an integral function as well. I've had the experience many times where I've been at first disappointed at an audio upgrade - and only after getting used to the new setup through a lot of listening and hearing the old again gain perspective on the upgrade. I'm going to guess I put over 25 hours in listening to the Bifrost Multibit over four days since I work at home - the switch box mostly gave me very prolonged blind listening and not necessarily just quick A/B (since I can flip a switch and not touch cables). I could listen for a few hours and switch every track or sometimes every 20 minutes to the other DAC without knowing which was which and try to pin which I generally preferred. In any case, I learned a lot through this trial and hopefully my experience is helpful to some. I definitely urge folks to do some careful blind listening on their own.

 

I have a Bifrost Multibit in transit to me right now, but I'm honestly glad to hear your ABX report. I feel this hobby desperately needs more sombre reflection. In all likelihood there is no perceivable difference between quality DACs nowadays. I doubt I'll be able to tell between my Modi 2 Uber and Bifrost Multibit. I'm in the process of building an automated ABX box powered by an Arduino to verify. Regardless, I'm glad to hear of your ABX results as it helps inform others where to spend their money. In reality, there seems to be little reason to invest heavily in anything other than transducers in this day and age.

 

Likewise I didn't expect anyone to pick it out! I didn't know the song or the album until only a few months back, boy o boy... What was most interesting for me during the test with The Glass Hall (especially against the La Voce) was the amount of glasses heard and the actual articulation of someone "hitting" the glasses (first 30 seconds). Up until this point it sounded to my like it was glistering. Even for just this neat effect I'd advise anyone to use this song in any form of testing, transducers, sources, amplifiers. The start of The White Winds is also exceptional as you noted.

Haha this worries me a bit, I'm now contemplating whether the tests done were actually sub 0.1db. To be fair, I feel that if we need to go through that level of trial and error then it takes the fun out of the testing. I personally love testing, it can be a lot of fun, but if so much time is needed to simply get to 100% accurate tests, it can become a drag. Also I feel speakers are better transducers in testing DACs.
Anyhow, I value your tests a lot, so really thanks for the impressions.

@JewBear
Random interjection here, yesterday I tried to reproduce my test results with the Shure 1540 (iirc you and I are two of its biggest fans), with the 1540 I found it notoriously more difficult. It lacked certain micro details that I was focusing on before, now gone. Against the ODAC 5/10 and 5/10 against the Modi 2, with a few I know were more guessing than anything else.

In the end I'm really happy with my purchase, it saved me a ton, for me it feels like a major upgrade and I'm looking forward in testing it against the Lynx, QB9 and a friend of mine has a loaner Hugo TT up.

Edit: @paranoidroid I'm determined to give your exact test a try. I have the Bifrost Multibit for over a week now with close to at very least 60-100 hours spent on it. I have my ODAC right here and I will get the switch box and see if we can get below 0.1dB to make sure it's accurate. Then do both a long term listening test and quick switch and see how I fair. I feel that 10 wouldn't cut it so I'll bump it to 15 songs to use.

 

It really isn't, though. Because the point of that is that one popular style of blind comparison -- short tests done quickly -- is really super-hard to tell subtle differences with, and that you can often find more differences if you do casual long-term blind listening, switching over periods of hours. Which is exactly what paranoidroid did, after careful level-matching got them failing the short-term tests.

I am relatively agnostic about the merits of DAC upgrades -- there's enough going on that it's totally plausible they could have an audible effect on the sound, but at the same time human ears super-suck at hearing things so it's also plausible that there's not -- but do paranoidroid the favor of accepting that they did some really great high-quality listening tests. I think it would be super-awesome if someone else did a test as careful and as detailed and came to a different conclusion, and in many ways I think the most interesting/surprising part of paranoidroid's test was how easy it was to set the test up such that it made it easy/possible to hear differences that they couldn't hear on more closely volume-matched testing.

 

Good luck! It is a huge PITA but I personally found it fun, especially since I knew I was comparing and listening to two very different methods of D/A conversion. To get it level matched to 0.1dB or better you'll need a way to measure the output directly. Either play a test tone and measure using multimeter or (what I did) measure the frequency response. Measuring the frequency response also got me to see that it slightly changed if I changed the output rate so I made sure to set both devices the same. If you're able to finally identify (like I eventually did) try re-level matching and switch attenuators around just to mix it up a bit and confirm impressions.

I hope you or someone else is able to do a carefully set up test and come to different conclusions than I did. I also know I'm missing a good portion of audio having ears that roll off after 16khz. Also I don't need any convincing to know the Schiit Multibit implementation is technically the superior way to do the D/A rendering, so despite not being able to hear it myself I'd rest easier knowing that someone else went through the rigor and was able to hear the superior DAC.

 

There is also the factor that if each link in the system has a small improvement - one that is just on the edge of audibility - the sum total of all the improvements can be clearly audible.

mkoslows wrote: "I am relatively agnostic about the merits of DAC upgrades" Does that include between " One dollar chip DAC built into the motherboard " versus $500 DAC ?

 

Nope! In fact, when I got the Multibit Bifrost, the first test I did was comparing it to the Chromecast Audio's analog output. The difference was very, very obvious, one of those changes where you don't even think about blind-testing anything, because it's really super-clear that you're hearing an undeniable difference. Which should be zero percent surprising to anyone -- the Chromecast Audio is a $35 USB-powered Wifi-enabled Linux computer that happens to have a DAC thrown in.

I also compared it to the Headroom Micro DAC very briefly. It seemed like the Bifrost was better, but this was a much subtler "better", something more like what paranoidroid was mentioning as their sighted impressions (and could easily have been level-matching differences, since I didn't do anything that way). I didn't do any more detailed testing, because there was never a chance I was going to keep using the Micro DAC (it doesn't visually match the Lyr, and the cabling would be an ugly tangle) and there's not a real benefit to collecting information when your decision is already made, you know?

So I'm uncertain as to whether Bifrost Multibit really sounds better than other quality DACs (I'll be thrilled if SKiring's closely level-matched listening still shows the Bifrost sounding better); but as to whether it can sound better than ultra-cheapies, I'm satisfied to a fairly high confidence level.

 

I would still be interested in seeing an ABX for Chromecast vs Multibit. I have a sneaking suspicion that we may actually find they're not so noticeably difference in a blind test. Placebo is a very strong effect.

 

Placebo and probably also volume differences. And there's a third trap with the Chromecast, which is that by default its analog out applies dynamic range compression and you need to check the "High Dynamic Range" setting to make it the same as the digital (which I had done). So yeah, I wouldn't swear to the difference.

At the same time, it really wasn't a subtle difference; it was about the same magnitude of difference between driving the HD-650 straight out of a phone vs. out of an amp. So while it's completely possible the difference is illusory, this is a situation where I'd actually be surprised if that turned out to be the case.

 

You do realize you are saying " There are no difference between any manufactured DACs in the world " ?

 

Alright, as promised more blind test results. My friend can do proper measurements so he helped me make sure we were absolutely hitting sub 0.1dB . In this test we used the ODAC vs. Bifrost Multibit to recreate @paranoidroid's test. We used my HD800 + HDVA600 but also did some tests with my friend's Confidence C1s + Passlabs Int 150s for fun.
This at some point went from fun to exhausting because A) it really was hard as hell to distinguish between and B) because we expanded the total amount of tests to make them more significant. We went through fast switch ups and through long time testing, the tests took two days to be completed.

HD800 + HDVA600 + ODAC/Bifrost Multibit
My fast switching results: 9/20
My friend's fast switching results: 6/15
My long listening results: 14/20
My friend's long listening results: 9/15

C1s + Pass Labs Int 150 + ODAC/Bifrost Multibit
My results: 11/15
My friend's results: 11/15

One more thing, we also did some tests with the O2, let me go ahead and tell you right now I'd be impressed if anyone will hear differences using the O2. It never showed either DACs capabilities.

Needless to say this was much, much harder... It took almost half a day of switching, measuring and just getting accustomed to each note, making sure we're going in knowing what to look for. And once again, speakers are really easier to find nuances blind. One of the main things we went for were highly detailed tracks and acoustic tracks. Keep in mind that this wasn't to make the test easier, merely to make sure we were getting the most out of Bifrost Multibit. Also to make sure we knew beforehand, we tested our ears, I can hear up to 19khz, my friend is stuck to 17khz.

Another interesting note, my friend is selling his QB9 to get a Bifrost Multibit or Gungnir Multibit, to give you an idea on how impressed he was haha. It was a clear cut win for the Bifrost Multibit, subjective non blind impressions.

Tracks used:
Andreas Vollenweider - The Glass Hall
Andreas Vollenweider - The White Winds
Yanni - For All Seasons
Yanni - Play Time
Lang Lang - Scherzo No. 3 in C-Sharp Minor, Op 39
The London Assemble - Lux Aeterna
Marc Cary - Beehive
Miles Davis - So What
Joe Pass - Walkin' Up
Ana Torroja - Hijo de la Luna
Amber Rubarth - Strive
Marcus Miller - Blast
Amy Duncan - Your Very Soul
Queen - Love Kills (Ballad)
Dire Straits - The Telegraph Road
Dire Straits - Your Latest Trick
Raul Midon - Sunshine
Mahsa Vahdat & Mighty Sam McClain - Silent Song
Fabrizio De Andre ft. Mina - La Canzone di Marinella
Tracy Chapman - Fast Car

 

How does he reconcile the test results with the non blind impressions? I've always wondered if it would help if people have some sort of reference to look at while they A-B'ed. For example, hook up the switch to a projector so that when it is on A, it projects a large A on the wall, and vice versa for B. I have a theory that people need a visual anchor in order to process audio differences properly.

 

Well he found it very interesting how much harder it is and especially how significant this is. However he was inclined that in the end what matters is, what his ears tell him.

I mean it could be that a visual anchor can change many aspects of listening, but the doubts, the accuracy of your auditory memory, things that take your mind of the actual listening. I think those are the biggest problem, blind you're focused on all kinds of things, maybe too focused, sighted you hear a sudden nuance and that's what pops as being different. Or in this case how the instruments sound more natural, the voices sound more natural, yet with so many micro details it's just simply impressive.

I mean I have now been through multiple blind condition tests, perfectly measured and all, knowing it's hard, sometimes inaudible to discern, but in the end I'm never listening like that. I'm sitting back, with no stress or doubts listening to a song which is why the nuances are so much easier to hear. I feel this is the basic thing, I mean here I am nearing the 3rd week and I'm still awestruck hearing things in recordings I've never heard before: claps, snares, the damn piano pedal (funny this was something that made the blind test easier, hearing the pedals was something inaudible with any D/S I've tried), the actual strumming, hell even a damn triangle is now suddenly crystal clear. Also the presentation.

I'm confident that understanding what is audible, what nuances to look for and most importantly having no worries is the biggest factor, but who knows? Maybe you're right and even a minor visual anchor could make a major change. All I know is that I've been rediscovering music that I have listened to for years and of all things a DAC is doing it, at a fraction of the costs of what I've spent on DACs so far. That alone, whether objectively insignificant or not, is what truly matters to me.

 

Excellent minimally mic'd recordings producing better ability to discriminate is my experience as well, particularly noticeable with binaural recordings using either Neumann KU100 dummy head or Jecklin disc stereo recording methods. The group experience using level matched A/B blind also have more difficulty it appears when A/B is short vs long in time spent on each DAC.

 

On behalf of both yours and @paranoidroid's efforts, this is really great work. You guys really went all out to put together a great experiment. Blind testing is such hard work to set up and do.

My take away is that blind testing is a very different experience than just listening, even when you give yourself extended time on each stack (although I believe extended time helps). I would also say that the results show that it is just plain-ol' difficult to reliably tell the differences between gear during blind A/B switching. (It does seem that you guys did a little better with the longer listening, and better still with the speakers.)

The thing is that both you and your friend (as well as most others that attempt this) still firmly believe that the differences you hear are quite significant during normal listening. Based on this and what I have read of other peoples' efforts, and my own tests, I believe that this is actually true. That for whatever reason, the human auditory apparatus does not distinguish subtle auditory information well in A/B tests. We do know that auditory memory is on the order of a second or less, unlike how our visual memory works, which to me means that our brains can't maintain a concrete, solid reference of an auditory "image" between switching stuff around. And so the error/fallacy is to extrapolate the failure of blind A/B testing to the conclusion that we therefore don't actually hear real differences at all. It doesn't seem like it needs to be controversial that A/B switching might not be an effective tool for this purpose given what we know about auditory memory.

I have also experienced the dramatic difference that the Bifrost Multibit achieves compared to the Modi 2, but I cannot A/B it (at least not with my Valhalla 2). I have tried several informal A/B tests, even sighted, and I just suck at it. But give me a week with Modi 2, and then switch to Bifrost Multibit, and I see god.

 

Very nice report @SKiring! I found the short term testing to be exhausting as well, and enjoyed the long term more since I could just listen how I normally do and just hit the switcher now and then to switch it up. It was good you guys tested your hears since perhaps it's an important factor - and really impressive you can go up to 19khz. Interesting you guys found speakers to be a bit easier, I was going to try that next with my Paradigm Studio 20s & Mackie HR824 monitors but with only a limited time on the loaner and also being tired enough from my own blind trials it didn't happen. Perhaps I'll buy a Bifrost Multibit sometime and put some real long term hours on it. You guys seemed to get some statistically valid results on speakers. A really good and interesting writeup!

 

Because I was curious (and have no intuition with this sort of thing), I went to see how statistically significant these are, and the answer is: very!

Well, okay, not the short tests (25% likely to be based on real differences for you, 15% for your friend), but the long tests give you a 94% chance of having heard a real difference, and your friend 70%. Both of you have a 94% chance of having heard real differences with the C1/Pass combo.

(All numbers from this phenomenally hideous web page, which I assume is not lying, even if their web designer is a sadist.)

Very cool, and very much appreciated effort.

 

I would be cautious about drawing too many inferences from such a a limited sample size. @SKiring back to the ABX mines for you!

 

I mean, one doesn't want to be credulous, but there's such a thing as an abundance of skepticism, too. If your priors are such that you strongly believe audible effects from differences in quality DACs don't exist, sure, this won't convince you. (Though it's worth asking yourself what would, before finding a way to dismiss every result.)

But if your priors are such that you're merely unsure... well, strongly suggestive evidence is strongly suggestive, and ought to sway you in that direction at the very least. And it's worth noting that even paranoidroid's "failed" test had them going 10/10 on a level-matched-to-0.1dB comparison, when all the stuff I can find from ABX advocates says that 0.3dB is the lower limit of detectability (and official ABX switchgear matches to 0.1dB).

At this point, you've got a) a plausible theory about how the different DAC affects the sound differently, b) a significant positive result on long-term blind tests from two people (and negative-leaning but mixed results from another person), and c) lots of subjective impressions of a difference.

If your position is that there's no difference, I'd argue that you're leaning against the actual evidence to hand. So I mean, yes, be open to more information, and resist the urge to speak with the iron certainty so beguiling on audio forums, that's all good advice. But at the same time, be willing to accept, provisionally, that a difference exists.

 

Just wanted to clear up what the statistics mean here. mkozlows' numbers do indeed take into consideration the limited sample size. In the case of the headphone long listening results, if we assume that SKiring was just randomly guessing, there is only a probability of about 6% that SKiring would have gotten 14 or more guesses correct out of 20. So we say we are 94% sure that SKiring was not just randomly guessing (which implies SKiring could identify some difference between the DACs). Similar story (and numbers) with the 11/15 correct on the speakers.

That being said, these numbers may or may not be statistically significant depending on your criteria. The test does not pass the standard 95% level