Discussion:
Do all lossless file types sound the same? (MEASUREMENTS)
Archimago
2013-05-22 15:47:23 UTC
Permalink
A straight forward DiffMaker test... I think the conclusions are quite
clear!

http://archimago.blogspot.ca/2013/05/measurements-do-lossless-compressed.html


------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Julf
2013-05-22 15:52:10 UTC
Permalink
Post by Archimago
A straight forward DiffMaker test... I think the conclusions are quite
clear!
I am relieved to see that Claude Shannon (and pretty much every
information scientist after him) was right after all. News at 11... :)


------------------------------------------------------------------------
Julf's Profile: http://forums.slimdevices.com/member.php?userid=42050
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Archimago
2013-05-22 16:06:33 UTC
Permalink
Post by Julf
I am relieved to see that Claude Shannon (and pretty much every
information scientist after him) was right after all. News at 11... :)
Claude Elwood Shannon (April 30, 1916 – February 24, 2001), RIP.

Hey, it's 2013 we need to be reminded of first principles once awhile.
:-)


------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Mnyb
2013-05-22 16:13:35 UTC
Permalink
... And your "diff" is very likely limitation in the test method and
some random noise.

You forgot to mention one cause of confusion in the blog , replay gain
tags ! WAV does not have but other formats can have it .


------------------------------------------------------------------------
Mnyb's Profile: http://forums.slimdevices.com/member.php?userid=4143
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Archimago
2013-05-22 16:25:49 UTC
Permalink
Post by Mnyb
... And your "diff" is very likely limitation in the test method and
some random noise.
You forgot to mention one cause of confusion in the blog , replay gain
tags ! WAV does not have but other formats can have it .
Yes. There will always be a bit of noise measuring the analogue signal.
I'll update the ReplayGain as well...


------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Mnyb
2013-05-22 19:50:31 UTC
Permalink
Post by Archimago
Yes. There will always be a bit of noise measuring the analogue signal.
I'll update with ReplayGain as well...
But it is great that you actually try , and this with playback
environments that sometimes seriusly are considered sub optimal :) not
like a squeezebox that is further isolated from potential computer audio
issues.


------------------------------------------------------------------------
Mnyb's Profile: http://forums.slimdevices.com/member.php?userid=4143
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
netchord
2013-05-28 18:12:08 UTC
Permalink
your title is very odd. how does one apply definitively objective
criteria to an inherently subjective phenomenon?


------------------------------------------------------------------------
netchord's Profile: http://forums.slimdevices.com/member.php?userid=21002
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Chunkywizard
2013-05-28 19:13:43 UTC
Permalink
Hi net chord,

You may think you can hear a difference but can you prove it? this
thread tries to proves you can't. You can try and prove you can by doing
a double blind test and publishing the results. Are you up for it?

CW


------------------------------------------------------------------------
Chunkywizard's Profile: http://forums.slimdevices.com/member.php?userid=25868
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Julf
2013-05-28 19:20:17 UTC
Permalink
Post by netchord
your title is very odd. how does one apply definitively objective
criteria to an inherently subjective phenomenon?
It is subjective, and it isn't. You can use objective criteria to show
that the resulting sound waves in your listening room are exactly the
same, but how you hear the two identical sound waves is of course
entirely subjective. That is what controlled, double-blind ABX is for -
it can determine if the subjective sensation is consistent and
repeatable.


------------------------------------------------------------------------
Julf's Profile: http://forums.slimdevices.com/member.php?userid=42050
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
JerryS
2013-05-28 21:10:39 UTC
Permalink
Lots of references to ABX testing in this forum. Is there any
theoretical reason why this should be more robust than triangle testing?
i.e. picking the odd one out from 3 unknowns played in random order,
two are A and one is B or one is A and two are B. Somehow, this makes
more sense to me intuitively and is the method I have used to confirm my
belief that I have cloth ears and can't distinguish flac from decent
mp3.
Jerry S
------------------------------------------------------------------------
JerryS's Profile: http://forums.slimdevices.com/member.php?userid=38998
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
aubuti
2013-05-28 21:23:55 UTC
Permalink
Lots of references to ABX testing in this forum. Is there any
theoretical reason why this should be more robust than triangle testing?
I'd be interested to see the science behind it, because there is a
subtle difference in the methodology. But either way you're picking the
odd one out. The only difference with ABX is that the randomness is
limited to third sample (X). You're always getting either ABA or ABB.

To turn the tables, why should the possibility of getting BBA or AAB
make more sense intuitively?

Note also that in terms of identifying the different one (or the
matching one), the triangle's BAA and BAB are equivalent to ABX's ABB
and ABA, respectively.


------------------------------------------------------------------------
aubuti's Profile: http://forums.slimdevices.com/member.php?userid=2074
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
JerryS
2013-05-28 22:10:23 UTC
Permalink
Post by aubuti
I'd be interested to see the science behind it, because there is a
subtle difference in the methodology. But either way you're picking the
odd one out. The only difference with ABX is that the randomness is
limited to third sample (X). You're always getting either ABA or ABB.
To turn the tables, why should the possibility of getting BBA or AAB
make more sense intuitively?
Note also that in terms of identifying the different one (or the
matching one), the triangle's BAA and BAB are equivalent to ABX's ABB
and ABA, respectively.
Just that with ABX you know in advance that A and B are different? With
triangle it could be AAB, BBA, BAB, ABA, ABB or BAA so you don't have
the expectation bias? Or do I misunderstand ABX?


------------------------------------------------------------------------
JerryS's Profile: http://forums.slimdevices.com/member.php?userid=38998
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
aubuti
2013-05-29 10:33:16 UTC
Permalink
Why is there any expectation bias because you know that A and B are
different? That they are different is a given in ABX testing, so that
"expectation" is always correct. The task is to determine whether X is A
or B, and whether X is A or B is completely random.

ABX is very simple. Listen to A, then listen to B, knowing that it is
different from A. Then listen to X and choose whether it is A or B.
Repeat N times, with X being randomly A or B each time. Choose correctly
a high enough percentage of the time and you can be confident that you
didn't choose correctly merely by guessing.


------------------------------------------------------------------------
aubuti's Profile: http://forums.slimdevices.com/member.php?userid=2074
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Julf
2013-05-29 10:50:00 UTC
Permalink
Lots of references to ABX testing in this forum. Is there any
theoretical reason why this should be more robust than triangle testing?
i.e. picking the odd one out from 3 unknowns played in random order,
two are A and one is B or one is A and two are B. Somehow, this makes
more sense to me intuitively and is the method I have used to confirm my
belief that I have cloth ears and can't distinguish flac from decent
mp3.
I think the major concern with triangle testing is that if you have a
person assisting in the testing (switching cables or whatever) who knows
which test sample is which, it is much easier to (inadvertently) signal
"this one is the special one" (for example by a changed breathing
pattern). Signalling "this one is the same as A, not B" is much harder.


------------------------------------------------------------------------
Julf's Profile: http://forums.slimdevices.com/member.php?userid=42050
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
SBGK
2013-05-29 11:20:17 UTC
Permalink
Interestingly this is the methodology used by JRMC to prove that JPlay
has no affect on the sound.

Don't think there are many people who would say that JRMC sounds the
same as JPlay so there must be something that is not being measured that
affects the sound.

I don't have any view on different lossless formats, I use Wav files
because I don't want the noise associated with decoding flacs at the
same time as rendering the pcm data.


------------------------------------------------------------------------
SBGK's Profile: http://forums.slimdevices.com/member.php?userid=52003
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Julf
2013-05-29 11:55:39 UTC
Permalink
Post by SBGK
Interestingly this is the methodology used by JRMC to prove that JPlay
has no affect on the sound.
Which methodology? ABX or the difference tests of Archimago?
Post by SBGK
I don't have any view on different lossless formats, I use Wav files
because I don't want the noise associated with decoding flacs at the
same time as rendering the pcm data.
Have you verified that the decoding actually affects the sound?


------------------------------------------------------------------------
Julf's Profile: http://forums.slimdevices.com/member.php?userid=42050
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
JerryS
2013-05-29 12:55:26 UTC
Permalink
I don't want to turn this into an argument because I just don't know
enough about it. My point is that if the brain knows that A and B are
different file types then it is going to be working pretty hard to
detect audible differences. If it finds such differences then it is
easier to discern them in X because the brain now knows where to listen
for them (different phrases, difference notes, different instruments
etc). If on the other hand the first two plays in triangle testing are
from the same file, then the brain has not prelearnt the cues to pick up
the differences on the third play. It would not be surprising if the
sensitivity and specificity of ABX and triangle testing are different
because of this prelearning.


------------------------------------------------------------------------
JerryS's Profile: http://forums.slimdevices.com/member.php?userid=38998
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
aubuti
2013-05-29 14:33:12 UTC
Permalink
Post by JerryS
I don't want to turn this into an argument because I just don't know
enough about it. My point is that if the brain knows that A and B are
different file types then it is going to be working pretty hard to
detect audible differences. If it finds such differences then it is
easier to discern them in X because the brain now knows where to listen
for them (different phrases, difference notes, different instruments
etc). If on the other hand the first two plays in triangle testing are
from the same file, then the brain has not prelearnt the cues to pick up
the differences on the third play. It would not be surprising if the
sensitivity and specificity of ABX and triangle testing are different
because of this prelearning.
I think that "pre-learning" often takes place long before anyone sits
down to a particular ABX, triangle, or any other type of comparative
listening session. Indeed, there are people who consciously train
themselves to identify the differences in lossless versus lossy, and
even more casual comparative listeners learn to pick out certain
instruments that reveal differences. And it seems there are a few who
train themselves in identifying the differences between lossless and
lossless (misguided in my opinion, but that's just my opinion).

I don't see why having those cues about where the listener _thinks_ they
hear a difference between A and B is a bad thing at all. It most
certainly does not bias the test results. Keep in mind that often there
are differences that exist but that listeners cannot hear (see
Archimago's recent thread and blog on comparing lossless and lossy). If
they really can hear a difference, then that will be borne out in their
assignments of X to A or B.


------------------------------------------------------------------------
aubuti's Profile: http://forums.slimdevices.com/member.php?userid=2074
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
JerryS
2013-05-29 14:48:23 UTC
Permalink
Well, good place to leave it I think? I can't think of an easy way of
demonstrating whether the two methods do or do not give identical
results and I am not convinced that you have demonstrated that they
theoreticaly should give the same results. Doesn't matter that much
anyway, given my cloth ears .... 8-)


------------------------------------------------------------------------
JerryS's Profile: http://forums.slimdevices.com/member.php?userid=38998
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
aubuti
2013-05-29 15:02:35 UTC
Permalink
Post by JerryS
Well, good place to leave it I think? I can't think of an easy way of
demonstrating whether the two methods do or do not give identical
results and I am not convinced that you have demonstrated that they
theoreticaly should give the same results. Doesn't matter that much
anyway, given my cloth ears .... 8-)
Sure, we can leave it there as far as I'm concerned. I would not claim
that they give identical results, because the testing methods are
different. Maybe there are times when the triangle method is more
appropriate than ABX. But I think for the cases in which the basic
premise is that there is something different (different codecs,
different cables, what have you), then ABX is perfectly appropriate.

And, ABX is easier to implement for comparisons of different codecs on
the same hardware, especially with the ABX component in foobar2000.


------------------------------------------------------------------------
aubuti's Profile: http://forums.slimdevices.com/member.php?userid=2074
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
adamdea
2013-05-29 13:51:01 UTC
Permalink
Post by SBGK
Interestingly this is the methodology used by JRMC to prove that JPlay
has no affect on the sound.
Don't think there are many people who would say that JRMC sounds the
same as JPlay so there must be something that is not being measured that
affects the sound.
I don't have any view on different lossless formats, I use Wav files
because I don't want the noise associated with decoding flacs at the
same time as rendering the pcm data. How is the world of international competitive non sequitur generation
coming on?


------------------------------------------------------------------------
adamdea's Profile: http://forums.slimdevices.com/member.php?userid=37603
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
ralphpnj
2013-05-29 19:57:19 UTC
Permalink
Post by SBGK
Don't think there are many people who would say that JRMC sounds the
same as JPlay so there must be something that is not being measured that
affects the sound.
I don't have any view on different lossless formats, I use Wav files
because I don't want the noise associated with decoding flacs at the
same time as rendering the pcm data.
Two seemingly harmless statements filled with utter nonsense. Come on
SBGK behind those cryptic initials you are actually one of the top
reviewers/writers/editors for one of the high end audio magazines.
Please let us in on your little secret, we promise we won't tell.

But just for fun:

"something that is not being measured that affects the sound" - must be
the use of USB cables which cost less than $500.

"the noise associated with decoding flacs" - I agree! The abacus in my
computer can often be heard as the little beads are slid back and
forth.

In fact, if I may so bold, if you are NOT already a
reviewer/writer/editor for one of the high end audio magazines then I
will made a compilation of some your finest posts from this forum and
forward it to a few of the editors at these magazines. I think that a
very bright future awaits!


------------------------------------------------------------------------
ralphpnj's Profile: http://forums.slimdevices.com/member.php?userid=10827
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Mushroom_3
2013-05-30 17:10:04 UTC
Permalink
Maybe he writes for Hi-Fi News and Record Review (the self proclaimed
oldest hi-fi magazine in the world).

This month's issue has a group test of USB cables. Needless to say the
most expensive cable, Crystal Absolute Dream (a snip at £6480/m) came
out top.

Some extracts from the review:
Inserted into the test rig,our 1m length of Absolute Dream showed the
lowest deterministic jitter and the fastest data edge risetime of
11.0nsec. - whatever that means.
You could almost reach out and touch the atmosphere.

At the back are 2 opinion pieces basically backing up the review
criteria.

Stuff like that more than justifies justifies my decision years ago to
stop buying those magazines.


------------------------------------------------------------------------
Mushroom_3's Profile: http://forums.slimdevices.com/member.php?userid=13434
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
ralphpnj
2013-05-30 20:42:48 UTC
Permalink
Post by Mushroom_3
Maybe he writes for Hi-Fi News and Record Review (the self proclaimed
oldest hi-fi magazine in the world).
This month's issue has a group test of USB cables. Needless to say the
most expensive cable, Crystal Absolute Dream (a snip at £6480/m) came
out top.
Inserted into the test rig,our 1m length of Absolute Dream showed the
lowest deterministic jitter and the fastest data edge risetime of
11.0nsec. - whatever that means.
You could almost reach out and touch the atmosphere.
At the back are 2 opinion pieces basically backing up the review
criteria.
Stuff like that more than justifies justifies my decision years ago to
stop buying those magazines.
As I've stated several times on this forum I only subscribe to
Stereophile and The Absolute Sound for a few reasons:

1) The subscriptions are cheap at about $1 per issue

2) The laughs I get from reading all the nonsense published by these so
called "experts" are priceless. For example any one with a decent
understanding of digital audio knows that USB cables cannot add to or
subtract from jitter, which basically proves that the clowns who write
for the high end audio magazines are not the "experts" they claim to be.


------------------------------------------------------------------------
ralphpnj's Profile: http://forums.slimdevices.com/member.php?userid=10827
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Archimago
2013-05-31 04:38:50 UTC
Permalink
Post by Mushroom_3
Maybe he writes for Hi-Fi News and Record Review (the self proclaimed
oldest hi-fi magazine in the world).
This month's issue has a group test of USB cables. Needless to say the
most expensive cable, Crystal Absolute Dream (a snip at £6480/m) came
out top.
Inserted into the test rig,our 1m length of Absolute Dream showed the
lowest deterministic jitter and the fastest data edge risetime of
11.0nsec. - whatever that means.
You could almost reach out and touch the atmosphere.
At the back are 2 opinion pieces basically backing up the review
criteria.
Stuff like that more than justifies justifies my decision years ago to
stop buying those magazines.
If 11ns Crystal was the fastest, what was the slowest, Mushroom?

Curious what's the "cheapest" cable they'd test...


------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Mushroom_3
2013-05-31 08:00:30 UTC
Permalink
Archimago,

The cheapest cable was QED Performance @ £18/m. The best "value for
money" was Chord Silver Plus @ £55/m.
The other prices were: £48; £139; £182; £60; £95; £70; £125
From what was stated the slowest was 13.5. There were some graphs that I
couldn't understand that may have shown other figures.

(I love reading your tests and conclusions, but don't begin to pretend I
understand all your accompanying graphs!!!!)


------------------------------------------------------------------------
Mushroom_3's Profile: http://forums.slimdevices.com/member.php?userid=13434
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Archimago
2013-06-01 04:40:50 UTC
Permalink
Post by Mushroom_3
Archimago,
The other prices were: £48; £139; £182; £60; £95; £70; £125
From what was stated the slowest was 13.5. There were some graphs that I
couldn't understand that may have shown other figures.
(I love reading your tests and conclusions, but don't begin to pretend I
understand all your accompanying graphs!!!!)
Thanks Mushroom & Julf.

11ns rise time vs. 13ns... Clearly veils are being lifted by that 2ns
difference ;-). I guess it's similar to the "test" in the ad I showed
here with the square wave response:
http://archimago.blogspot.ca/2013/05/musings-audiophile-digital-cables.html
Unless we know what they're measuring, this all seems quite meaningless
and impossible to put into any kind of context!

Disappointed it doesn't seem like they measured the audio output itself.


------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Julf
2013-05-31 08:35:38 UTC
Permalink
Post by Mushroom_3
Maybe he writes for Hi-Fi News and Record Review (the self proclaimed
oldest hi-fi magazine in the world).
This month's issue has a group test of USB cables. Needless to say the
most expensive cable, Crystal Absolute Dream (a snip at £6480/m) came
out top.
Indeed - I was really disappointed. Hi-Fi news is the only hi-fi mag I
still subscribe to, mainly because they have been pretty good to
highlight upsamplings and other sheenigans in their music reviews, but I
don't think I'll renew.

In the USB cable test, they do publish eye diagrams, but the rankings
are purely subjective. And while they state that the reviewers were able
to tell the differences between the cables under blind conditions to a
statistically significant degree, they don't properly divulge their test
setup, or actual blind listening statistics.


------------------------------------------------------------------------
Julf's Profile: http://forums.slimdevices.com/member.php?userid=42050
View this thread: http://forums.slimdevices.com/showthread.php?t=98828
Loading...