Merv's Heptagons and Presentation Techniques

Discussion in 'Blind Testing and Psychoacoustics' started by Boops, Dec 1, 2015.

  1. Boops

    Boops Friend

    Pyrate
    Joined:
    Nov 8, 2015
    Likes Received:
    3,179
    Trophy Points:
    93
    Location:
    New York
    I was looking the spider chart @purrin made and thought a companion graphic could be useful. I brought the chart into Illustrator and worked up a quick mockup of a "spider small multiple." Here it is:

    [​IMG]

    We all know that summing the individual scores is bad, but it occurred to me that a comparison of the relative area enclosed by the individual webs would be a useful thing to see, and this format makes it much easier to do that. For example, @purrin's comment about the TH-X00 being a better all-arounder than the Omni is born out by the more bulbous (for lack of a better term) shape of it's web which covers more area in comparison.

    It's also clear that the Fostex compares favorably to the HD 650, which covers a similar amount of visual area even though the overall shape of the web is quite different. Interesting also to see how the 650's excellence in three categories compensate for its weaknesses from a web surface area standpoint.

    All in all, the spider charts are great for diving into the details and comparing cans against each other in the individual categories in an visually efficient way. But I was feeling that it was a bit difficult to get a more "big picture" view of each headphone relative to the others with the spider chart alone.

    I agree wholeheartedly that there's nothing useful to be gleaned from summing up the individual categories to see which headphone got the most "points". But I think the itch to total up the categories hints at something that the combined spider chart isn't optimized for but that the "spider multiples" may help address.

    If you all think these are useful, I'd be happy to make more for the other spider sets.
     
  2. sorrodje

    sorrodje Carla Bruni's other lover - Friend

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    2,812
    Trophy Points:
    113
    Location:
    Dijon / Burgundy / France / EU
    As some of us highly suspected , charts and notations will overwhelm everything else and will be exploited wrongly. Something went wrong somewhere .

    I'm waiting the moment when an idiot will come and "prove" with oversimplified charts how a Porta pro is "better" than a SR009. Dunno when, but it will happen . no doubt.

    Sad times.
     
  3. Boops

    Boops Friend

    Pyrate
    Joined:
    Nov 8, 2015
    Likes Received:
    3,179
    Trophy Points:
    93
    Location:
    New York
    That's a lot of gloom and doom @sorrodje for a simple data visualization. There's actually nothing exploitative in the spider multiple I made. It's the same data in the original chart just laid out horizontally rather than overlaid.

    Maybe a spider chart comparison of a PortaPro and an SR009 would make you feel better about the future? ;-)
     
  4. joch

    joch Friend

    Pyrate Contributor
    Joined:
    Oct 8, 2015
    Likes Received:
    1,397
    Trophy Points:
    93
    Location:
    the other side of the big ocean
    Well, on a portability scale, the porta-pro...

    Horses for the courses

    Anyway, there are lies, damned lies, and twisted measurements
     
    Last edited: Dec 1, 2015
  5. batriq

    batriq Probably has made you smarter

    Pyrate
    Joined:
    Oct 31, 2015
    Likes Received:
    1,041
    Trophy Points:
    93
    @Boops is touching on an alternative to (or different visualization of) spider/radar charts, which are sometimes called "petal charts". They are usually displayed on one graph with each petal emanating from the center (each quadrant here would contain all comparison dimensions).
     
  6. Boops

    Boops Friend

    Pyrate
    Joined:
    Nov 8, 2015
    Likes Received:
    3,179
    Trophy Points:
    93
    Location:
    New York
    Petal charts are another interesting variation. The advantage of the spider multiple (especially as a companion to the overlaid spider) is that the nodes maintain their orientation as you look across the "webs." In the typical petal, it's harder to compare across "webs" because the nodes don't line up as clearly.
     
  7. blackships

    blackships Friend

    Pyrate
    Joined:
    Oct 2, 2015
    Likes Received:
    130
    Trophy Points:
    43
    @purrin, I really want to thank you for including the symbols for each data point for those of us who are colorblind. I can't tell the HD650 and TH-x00 lines apart, but the symbols make it completely manageable. It's a small thing, but I really appreciate it.
     
  8. OJneg

    OJneg The Most Insufferable

    Pyrate BWC
    Joined:
    Sep 25, 2015
    Likes Received:
    3,923
    Trophy Points:
    113
    Location:
    Grand Rapids, MI
    I projected all of the curves onto a Ouija board, took the sum of the surface integrals, multiplied by Cardas Golden Ratio, and ended up with this:
    [​IMG]
    Coincidence?
     
  9. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    It won't happen in my stead. I've always been very careful with how my measurements and visualizations have been used*; and have done what I could to stop the gestation of mindless minions or sycophants. Contrary to the image of what folks like Stillhart are trying to paint of me, I do not want minions or sycophants; I want smart friends who can think for themselves. Many of the visualizations (the new-old CSDs, percentage distortion, etc.) and measurement techniques I use were actually developed over time, driven by knowledgeable folks who provided useful critique. Those around in the early years of Changstar (most them are not around anymore as they have crossed the veil after dinging audiophile level 100) can most certainly attest to this.

    As far was Boops offered, I think it's a great idea. It's hard to see the individual shapes when they are overlaid on top on each other. And yes, people who add up the numbers will be shot.

    * And this is one of the reasons that I never posted the details of my measurement system for such a long time. With great power comes great responsibility.
     
    Last edited: Dec 1, 2015
  10. kapanak

    kapanak Canucklehead - Friend

    Pyrate
    Joined:
    Sep 27, 2015
    Likes Received:
    259
    Trophy Points:
    63
    I actually really like this, and consider it a very good extension to the heptagon. Well done :D
     
  11. knerian

    knerian Friend

    Pyrate
    Joined:
    Sep 25, 2015
    Likes Received:
    454
    Trophy Points:
    63
    Location:
    San Francisco
    The visuals are very helpful, as long as you look at them through the ears of purrin. It would be almost impossible to translate this into something that had 10 people's scores on it.
     
  12. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    That would not a good idea unless we mind-melded.
     
  13. spoony

    spoony Spooky

    Pyrate
    Joined:
    Sep 28, 2015
    Likes Received:
    651
    Trophy Points:
    93
    Is it possible to embed an interactive chart using SVG or smth to display these graphs? (making possible to highlight areas, isolate outlines, etc) I might volunteer to code it.
     
  14. spoony

    spoony Spooky

    Pyrate
    Joined:
    Sep 28, 2015
    Likes Received:
    651
    Trophy Points:
    93
  15. sorrodje

    sorrodje Carla Bruni's other lover - Friend

    Staff Member Pyrate MZR
    Joined:
    Sep 27, 2015
    Likes Received:
    2,812
    Trophy Points:
    113
    Location:
    Dijon / Burgundy / France / EU
    Yes. You're more a messiah than ever Marv. Minions are all over the place and watch you as such a Messiah. f**k, even if you opened a thread to tell us you farted under your shower, you'd receiver shitloads of "like". So you have a big influence on audio world. Maybe not at the level of Tyll or Jude but pretty darn big byu the way.

    You 're unleashing a monster! You can maybe control some peole here in a bunch of threads and avoid bad exploitation of your charts but what about elsewhere when all people will decide that charts are really "cool" How much time before Jude will put charts in HF guide ? and how much time for Headphone.guru ? Do you see like me those reviews with pics , charts in fancy colors and FB like buttons everywhere ? If you want to shot all the fools who will adds notes and create rankings, I advice you to buy shitloads of ammo Right now.

    We all know how charts destroyed politics and corporate management . It will destroy all what you and other people tried to build : bing some more intelligence and subtlety in audio impressions. No doubt about that. It's just a matter of time.

    I'd like to read Tyll's opinion. He probably already thought about notations and charts himself.
     
  16. gbeast

    gbeast Mighty Moral Power Ranger

    Pyrate
    Joined:
    Nov 8, 2015
    Likes Received:
    1,723
    Trophy Points:
    93
    Location:
    Phoenix
    Home Page:
    The thing about it all though is that no matter who the person you won't always agree.
    Charts are great but look how much warning purrin has put in his prefacing and even so it's taken as a law of sorts. It is subjective at that. I think that's where the responsibility comes in.
    To caution the reader not to fully buy in or out of what the reviewer is saying.


    Also if others start with charts etc it doesn't mean they got the idea from him (could be a good chance they did).That's nothing new outside of consumer products. It'd be a bit rash to proprietize it all to one person as if others couldn't think of it. In fact I have thought of charts for reviews myself though nothing similar to the spider chart. Me seeing it and appreciating it is partly because I get to see someone fill a void I felt was missing.

    I'm freaking tired of having to read /write lengthy reviews that are written as to not hurt the Hp manufacturers. Feeling they have to validate themselves with well written narratives as if it's a freakin magazine article for plane flights. Just tell me, draw me, measure me, how the thing sounds and if you like it or not. I'm guilty myself of trying to write all cool like that and I suck at it.
    I'll be more to the point now.

    If anything all of this is an example of how it should be done. Thanks SBAF
     
  17. thune

    thune Friend

    Pyrate
    Joined:
    Sep 26, 2015
    Likes Received:
    50
    Trophy Points:
    18
    I do admire the attempt to assign objective merit scores to performance components without asserting their relative value to the total impression, leaving the total impression and 'artistic merit' as something to be considered independently and judged separately. But obfuscating numbers from a simple table with fancy graphs is just insulting to everybody, not only the fools it is trying to waylay.
     
    Last edited: Dec 2, 2015
  18. Boops

    Boops Friend

    Pyrate
    Joined:
    Nov 8, 2015
    Likes Received:
    3,179
    Trophy Points:
    93
    Location:
    New York
    I think this is off the mark. Simple tables are great when they make sense, but an appropriate visual -- like Purrin's spider charts -- helps communicate things more efficiently than a table of numbers and can even help readers make connections that wouldn't be as clear just looking at the numbers alone.

    The important thing to remember that underlies everything, is that there is a big difference between an appropriate visual and a chart-for-chart's sake. I've found all of the types of charts in use here to be carefully chosen and considered. Anyone can dump a spider chart out of excel, but it takes some careful thought to choose which aspects of a piece of gear you are going to score, and which pieces of gear you are going to include in a comparison. Those are the actual hard decisions and the imitators who don't understand that won't make charts that are worth looking at.
     
  19. Bobcat

    Bobcat Friend

    Pyrate
    Joined:
    Oct 3, 2015
    Likes Received:
    400
    Trophy Points:
    63
    Location:
    Texas
    Home Page:
  20. purr1n

    purr1n Desire for betterer is endless.

    Staff Member Pyrate BWC
    Joined:
    Sep 24, 2015
    Likes Received:
    89,778
    Trophy Points:
    113
    Location:
    Padre Island CC TX
    Nooooooooo! I refuse to believe that. I don't think of myself as a messiah and I don't want to be. I've always seen my role as one of function and service.

    I don't think this will happen. Jude already has "TOTL" headphone measurement equipment, but he hasn't really done jack with it. It also might not be a good idea for Jude to show strange measurement behaviors of gear he's currently shilling. And if he tweaked the graphs (with smoothing or axis manipulation, he'd be called so hard on it by not only me, but many other people.)

    The other factor is that neither Jude or the reviewers at Headphone.guru tend to do comparative reviews. Besides if Jude did one, it would probably look like this:
    pentagram.png

    It's what @Boops said: there has been a lot of thought put behind these (ideas sitting around for years). I for one would love to see other people do stuff like this. I did learn two important things via the spider plots (things that I already knew inside, but the presentations made them that much more explicit): 1) Different headphones bring different things to the table; 2) The headphones that have been recently discussed are all pretty good and there isn't a wide gap between them (upper-mid-fi purgatory).

    And in the end, I think the presentations are a lot better than the old Changstar three tier rating system (five stars, not-shit, POS).
     
    Last edited: Dec 2, 2015

Share This Page