Discussion:
INTERNET TEST: 24-bit vs. 16-bit Audio - Closed and Results Being Posted Soon...
Archimago
2014-06-21 23:57:24 UTC
Permalink
Just a quick note guys. Finished the test yesterday with a total of 140
respondents to the survey. Not bad IMO!

Just put up the PROCEDURE post to describe how the test was conducted as
well as preparation of the test files.

Also, the "answer" for which samples were 24-bits: - Bozza A - Vivaldi B
- Goldberg B. Did you get it right?

Results of the survey to follow in the next week or so!



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Wombat
2014-06-22 16:40:39 UTC
Permalink
Two things about the used dither. Some people claim that even truncating
hard the last 8bit should not be audible with music playing. Using a low
amount of dither with music should be sufficient because no matter how
you try there is some own noise there. In this case 0.5 bit are used.
With synthetic music or signals like some sinuids it may be different.
Archimago also used stupid, flat dither to make it not easily visible. A
noise-shaped dither used with 96kHz easily could have archieved 20dB
more content in the audible range up to 20kHz.
So IF there already are no clear results showing the inferiority of the
16bit files you should wonder what all the talk of different dither
approaches and magic sound of dither is worth.



Transporter (modded) -> RG142 -> Avantgarde Acoustic based 500VA
monoblocks -> Sommer SPK240 -> self-made speakers
------------------------------------------------------------------------
Wombat's Profile: http://forums.slimdevices.com/member.php?userid=4113
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
garym
2014-06-22 17:30:56 UTC
Permalink
Post by Archimago
Just a quick note guys. Finished the test yesterday with a total of 140
respondents to the survey. Not bad IMO!
Just put up the PROCEDURE post to describe how the test was conducted as
well as preparation of the test files.
Also, the "answer" for which samples were 24-bits: - Bozza A - Vivaldi B
- Goldberg B. Did you get it right?
Results of the survey to follow in the next week or so!
I got them all exactly wrong. My guesses were B, A, A. Then I again I
did report that I had no confidence in the decisions (essentially
guessing)



*Location 1:* VortexBox 4TB (2.2) > LMS 7.8 > Transporter, Touch, Boom,
Radio w/Battery (all ethernet except Radio)
*Location 2:* VBA 3TB (2.2) > LMS 7.8 > Touch > Benchmark DAC I, Boom,
Radio w/Battery (all ethernet except Radio)
*Office:* Win7(64) > LMS 7.8 > Squeezelite
*Spares:* VBA 4TB, SB3, Touch (3), Radio (3), CONTROLLER
*Controllers:* iPhone4S & iPad2 (iPeng7 & Squeezepad), CONTROLLER, or
SqueezePlay 7.8 on Win7(64) laptop
Ripping (FLAC) - dbpoweramp, Tagging - mp3tag, Streaming - Spotify
------------------------------------------------------------------------
garym's Profile: http://forums.slimdevices.com/member.php?userid=17325
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
jimmypowder
2014-06-24 12:50:34 UTC
Permalink
Post by garym
I got them all exactly wrong. My guesses were B, A, A. Then I again I
did report that I had no confidence in the decisions (essentially
guessing). Edit: by guessing, I mean I was trying to guess which was
24/96. And my rule-of-thumb is that I picked the one I *thought* sounded
better as 24/96. So it seems from the test that I prefer 16/44.1.
That's good, cause that's what I mostly have!
I guess Gary doesn't have the Golden Ears!

Too many rock concerts.


------------------------------------------------------------------------
jimmypowder's Profile: http://forums.slimdevices.com/member.php?userid=61215
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
garym
2014-06-24 12:58:49 UTC
Permalink
Post by jimmypowder
I guess Gary doesn't have the Golden Ears!
Too many rock concerts.
True! Front row at The Who in 1971 is the one that likely did the most
damage. It was LOUD. :cool:
Im just happy I don't have tinnitus.



*Location 1:* VortexBox 4TB (2.2) > LMS 7.8 > Transporter, Touch, Boom,
Radio w/Battery (all ethernet except Radio)
*Location 2:* VBA 3TB (2.2) > LMS 7.8 > Touch > Benchmark DAC I, Boom,
Radio w/Battery (all ethernet except Radio)
*Office:* Win7(64) > LMS 7.8 > Squeezelite
*Spares:* VBA 4TB, SB3, Touch (3), Radio (3), CONTROLLER
*Controllers:* iPhone4S & iPad2 (iPeng7 & Squeezepad), CONTROLLER, or
SqueezePlay 7.8 on Win7(64) laptop
Ripping (FLAC) - dbpoweramp, Tagging - mp3tag, Streaming - Spotify
------------------------------------------------------------------------
garym's Profile: http://forums.slimdevices.com/member.php?userid=17325
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
jimmypowder
2014-06-26 12:19:54 UTC
Permalink
Post by garym
True! Front row at The Who in 1971 is the one that likely did the most
Im just happy I don't have tinnitus.
Your right about that. Add in a 4 hour Led Zeppelin concert during the
70's and some of those high end frequencies you used to hear are gone.


------------------------------------------------------------------------
jimmypowder's Profile: http://forums.slimdevices.com/member.php?userid=61215
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Archimago
2014-06-26 05:34:42 UTC
Permalink
Post by garym
I got them all exactly wrong. My guesses were B, A, A. Then I again I
did report that I had no confidence in the decisions (essentially
guessing). Edit: by guessing, I mean I was trying to guess which was
24/96. And my rule-of-thumb is that I picked the one I *thought* sounded
better as 24/96. So it seems from the test that I prefer 16/44.1.
That's good, cause that's what I mostly have!
Actually, garym, you got them right - golden ears for you :-)

The answer is BAA as per my blog post - I guess I must have been a
little distracted that evening putting this post above and have
corrected the answer!

I'll be posting the results up soon with all 140 responses tallied up...



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Mnyb
2014-06-26 05:47:00 UTC
Permalink
Your bloggpost there is to many "A" in the description , so you have to
edit it some more .

BTW i've missplaced my notes on what I voted so I have not the slightest
idea :D



--------------------------------------------------------------------
Main hifi: Touch + CIA PS +MeridianG68J MeridianHD621 MeridianG98DH 2 x
MeridianDSP5200 MeridianDSP5200HC 2 xMeridianDSP3100 +Rel Stadium 3
sub.
Bedroom/Office: Boom
Kitchen: Touch + powered Fostex PM0.4
Misc use: Radio (with battery)
iPad1 with iPengHD & SqueezePad
(in storage SB3, reciever ,controller )
server HP proliant micro server N36L with ClearOS Linux

http://people.xiph.org/~xiphmont/demo/neil-young.html
------------------------------------------------------------------------
Mnyb's Profile: http://forums.slimdevices.com/member.php?userid=4143
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Archimago
2014-06-26 07:40:31 UTC
Permalink
Post by Mnyb
Your bloggpost there is to many "A" in the description , so you have to
edit it some more .
BTW i've missplaced my notes on what I voted so I have not the slightest
idea :D
Oi, yer right... I knew I shouldn't be "cut & pasting" filenames with
scotch in hand... :-)

All good now, I hope! :o



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
lrossouw
2014-06-26 08:29:45 UTC
Permalink
Great effort. I was wondering how you were limiting cheats.



Louis
'Last.fm' (http://www.last.fm/user/lrossouw)
------------------------------------------------------------------------
lrossouw's Profile: http://forums.slimdevices.com/member.php?userid=3416
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Mnyb
2014-06-26 08:38:14 UTC
Permalink
Post by lrossouw
Great effort. I was wondering how you were limiting cheats.
There could possibly be other ways to cheat anyway like totally
unnatural listening conditions , repeat the last very very faint tail
ends of a track (- 90dB stuff ) on incredible volume in headphones or
put your ear to the speaker .
Just like putting your nose on your HDTV you probably see pixels but
that's not how you use the device normally and it's not really a problem
.

But I do think natural source noise or dither would thwart even this
aproach , but Archimago had to choose a very simple algorithm for other
reasons as explained



--------------------------------------------------------------------
Main hifi: Touch + CIA PS +MeridianG68J MeridianHD621 MeridianG98DH 2 x
MeridianDSP5200 MeridianDSP5200HC 2 xMeridianDSP3100 +Rel Stadium 3
sub.
Bedroom/Office: Boom
Kitchen: Touch + powered Fostex PM0.4
Misc use: Radio (with battery)
iPad1 with iPengHD & SqueezePad
(in storage SB3, reciever ,controller )
server HP proliant micro server N36L with ClearOS Linux

http://people.xiph.org/~xiphmont/demo/neil-young.html
------------------------------------------------------------------------
Mnyb's Profile: http://forums.slimdevices.com/member.php?userid=4143
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Archimago
2014-06-26 15:29:12 UTC
Permalink
Post by Mnyb
There could possibly be other ways to cheat anyway like totally
unnatural listening conditions , repeat the last very very faint tail
ends of a track (- 90dB stuff ) on incredible volume in headphones or
put your ear to the speaker .
Just like putting your nose on your HDTV you probably see pixels but
that's not how you use the device normally and it's not really a problem
.
But I do think natural source noise or dither would thwart even this
aproach , but Archimago had to choose a very simple algorithm for other
reasons as explained
Indeed. There will always be a way to cheat since the files have to
reflect a true 24-bit vs. 16-bit resolution ultimately... There were
ideas like putting 24-bit decay trails on 16-bit files to thwart efforts
but ultimately, I didn't want to end up confused with too many
machinations.

Also, if I manipulated the files too much, there would be outcries about
excessive processing happening or criticisms that the editing changed
the sound, etc.

I think you'll see that what was done was good enough when I put up the
results - likely tomorrow...



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Mnyb
2014-06-27 05:52:09 UTC
Permalink
Post by Archimago
Indeed. There will always be a way to cheat since the files have to
reflect a true 24-bit vs. 16-bit resolution ultimately... There were
ideas like putting 24-bit decay trails on 16-bit files to thwart efforts
but ultimately, I didn't want to end up confused with too many
machinations.
Also, if I manipulated the files too much, there would be outcries about
excessive processing happening or criticisms that the editing changed
the sound, etc.
I think you'll see that what was done was good enough when I put up the
results - likely tomorrow...
Yeah that's the power of numbers :) 140 testers that's quite ok .

The possible flaw is that the group is self selected , but that depends
it could be a point !



--------------------------------------------------------------------
Main hifi: Touch + CIA PS +MeridianG68J MeridianHD621 MeridianG98DH 2 x
MeridianDSP5200 MeridianDSP5200HC 2 xMeridianDSP3100 +Rel Stadium 3
sub.
Bedroom/Office: Boom
Kitchen: Touch + powered Fostex PM0.4
Misc use: Radio (with battery)
iPad1 with iPengHD & SqueezePad
(in storage SB3, reciever ,controller )
server HP proliant micro server N36L with ClearOS Linux

http://people.xiph.org/~xiphmont/demo/neil-young.html
------------------------------------------------------------------------
Mnyb's Profile: http://forums.slimdevices.com/member.php?userid=4143
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
kalessin
2014-06-27 11:52:52 UTC
Permalink
Hi,

I came across this test when a member in a different forum (Pianoteq)
mentioned it. They also claimed to be able to pick out the 24bit samples
without guessing with 100% certainty, and this made me curious. So I
acquired the test data and had a look. After doing my own (failed)
listening test, I could however not just disregard the claim of the
other forum member, as they absolutely ruled out any guessing was
involved. So... I looked closer.

To cut a long story short, I found certain flaws in the test data,
likely due to a problem in processing. Because of this, I would not be
surprised if a statistically significant number of listeners were able
to hear differences between the files. For example, I can discern very
clearly between the files when looking at their spectra even after
downsampling to 44/16. And this is something I should not be able to
do.

You can try this for yourself. Take the Vivaldi sample (the real 24bit
one), and export it to 16bit using Audacity. Be sure to use the
high-quality settings and e.g. the triangle dither. Then re-import it
into Audacity. Now pick e.g. a two-second window and run a FFT analysis.
You can see the differences directly; however, the dithering noise is
very low and only really visible in the high frequency range (>20kHz),
where the original file has virtually no spectral power. This is to be
expected.

Now comes the trick. Convert both samples (the original and the 16bit
conversion) to 44.1kHz and look at them again. When I do this here
(since Audacity -does- the conversion correctly, at least in its
highest-quality mode), all differences between the spectra more or less
vanish completely, especially when looking at frequencies of 16kHz and
below; this is also to be expected. Thus when I do these steps in
Audacity, the dithering indeed only affects the highest frequencies,
since they are the weakest.

Soo... and now we have a look at that 'B' sample. Again choose an
analysis window, e.g. 1:23 to 1:25. Take care to shift it 1ms to the
left to compensate for the 1ms cut as documented. Look at it in 96kHz.
Already -lots- of differences between A and B, even in low frequencies
and at relatively high levels. Convert it down to 44.1kHz. Be amazed at
how different the spectra still look. The differences are subtle, but
they -are- present, and as I mentioned they cover the whole spectral
band, including the lower frequencies. Although -my- ears might be
rubbish, I am absolutely not surprised that someone with good hearing
can hear those differences.

The problem I have with this is that when I accept the sampling theorem,
which I do (both as a physicist and a programmer), then any
'improvement' a 96/24 or 192/24 recording offers has to happen either at
very low levels or at high frequencies, since -everything- from -96 to
0dB and from 0 to 22kHz is encoded losslessly. So it is indeed a very
important baseline test that both the original 24bit and the dithered
files in a comparison test yield a (virtually) identical result when
converting down to 44/16. Audacity manages this quite well. The used
version of Adobe Audition does not, it seems.


------------------------------------------------------------------------
kalessin's Profile: http://forums.slimdevices.com/member.php?userid=63059
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Wombat
2014-06-27 13:26:07 UTC
Permalink
There are several ways to detect this with analysis, not listening. Just
extract the last 8 bit and compare them, lol. The files are not flawed.
Time alligned they null to -90dB and more.

One way your playing with resampling is may be that going to 16/44.1
doesn't use a big enough stop band and difference become obvious but as
mentioned before there are several ways identifying these differences.

It was the point to be honest with oneself therefore "faith in
humanity". With audiophools that want try to get some attention for
being something special (at least in one way) cheating in such tests
will always be a point. Especially on the internet they don't have to
wurry about someone coming home and asking for prove.



Transporter (modded) -> RG142 -> Avantgarde Acoustic based 500VA
monoblocks -> Sommer SPK240 -> self-made speakers
------------------------------------------------------------------------
Wombat's Profile: http://forums.slimdevices.com/member.php?userid=4113
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
kalessin
2014-06-27 14:59:01 UTC
Permalink
Post by Wombat
There are several ways to detect this with analysis, not listening. Just
extract the last 8 bit and compare them, lol. The files are not flawed.
Time alligned they null to -90dB and more.
You know, when you try to taunt someone for not knowing what they are
talking about, maybe you should first actually read and understand what
they are saying before you do so.

I -said-, and I will repeat this exactly this once, that there are
significant differences in the files covering the
_whole_frequency_spectrum_. This includes frequencies <4kHz and levels
as high as -30dB. And yes, I have aligned the two A and B samples, just
-as I already wrote-. The downsampling step is not strictly necessary,
as the discussed differences are visible even before. It just increases
their visibility. In other words: those spectral differences are
significant, they are most probably audible, they are a very sure sign
that the audio files are flawed, and they have absolutely nothing to do
with the "last 8 bits". I also said that those differences do not happen
when using another editor (Audacity). I'm done here. I know enough about
physics, math and data analysis to predict what the results of this
'test' will be, and what to make of them.


------------------------------------------------------------------------
kalessin's Profile: http://forums.slimdevices.com/member.php?userid=63059
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Archimago
2014-06-27 16:05:09 UTC
Permalink
Post by kalessin
Hi,
I came across this test when a member in a different forum (Pianoteq)
mentioned it. They also claimed to be able to pick out the 24bit samples
without guessing with 100% certainty, and this made me curious. So I
acquired the test data and had a look. After doing my own (failed)
listening test, I could however not just disregard the claim of the
other forum member, as they absolutely ruled out any guessing was
involved. So... I looked closer.
To cut a long story short, I found certain flaws in the test data,
likely due to a problem in processing. Because of this, I would not be
surprised if a statistically significant number of listeners were able
to hear differences between the files. For example, I can discern very
clearly between the files when looking at their spectra even after
downsampling to 44/16. And this is something I should not be able to
do.
You can try this for yourself. Take the Vivaldi sample (the real 24bit
one), and export it to 16bit using Audacity. Be sure to use the
high-quality settings and e.g. the triangle dither. Then re-import it
into Audacity. Now pick e.g. a two-second window and run a FFT analysis.
You can see the differences directly; however, the dithering noise is
very low and only really visible in the high frequency range (>20kHz),
where the original file has virtually no spectral power. This is to be
expected.
Now comes the trick. Convert both samples (the original and the 16bit
conversion) to 44.1kHz and look at them again. When I do this here
(since Audacity -does- the conversion correctly, at least in its
highest-quality mode), all differences between the spectra more or less
vanish completely, especially when looking at frequencies of 16kHz and
below; this is also to be expected. Thus when I do these steps in
Audacity, the dithering indeed only affects the highest frequencies,
since they are the weakest.
Soo... and now we have a look at that 'B' sample. Again choose an
analysis window, e.g. 1:23 to 1:25. Take care to shift it 1ms to the
left to compensate for the 1ms cut as documented. Look at it in 96kHz.
Already -lots- of differences between A and B, even in low frequencies
and at relatively high levels. Convert it down to 44.1kHz. Be amazed at
how different the spectra still look. The differences are subtle, but
they -are- present, and as I mentioned they cover the whole spectral
band, including the lower frequencies. Although -my- ears might be
rubbish, I am absolutely not surprised that someone with good hearing
can hear those differences.
The problem I have with this is that when I accept the sampling theorem,
which I do (both as a physicist and a programmer), then any
'improvement' a 96/24 or 192/24 recording offers has to happen either at
very low levels or at high frequencies, since -everything- from -96 to
0dB and from 0 to 22kHz is encoded losslessly. So it is indeed a very
important baseline test that both the original 24bit and the dithered
files in a comparison test yield a (virtually) identical result when
converting down to 44/16. Audacity manages this quite well. The used
version of Adobe Audition does not, it seems.
Interesting comment. I must admit I'm not much of an Audacity user so am
not familiar with what Audacity is doing in those 24 <--> 16-bit
conversion steps.

However, I just downloaded Audacity to this workstation and had a quick
look. As far as I can tell, Audacity uses a noise-shaped dithering by
default even when I set it to triangular mode in the preferences and
export to 16-bits. Also, I don't know if I can set the strength of
dithering being applied in Audacity. Like I said, I'm not an Audacity
guy but am suspicious that you're seeing the difference between a very
low 0.5 bit flat triangular dithering (what I did with Audition) with
the noise shaping characteristics of Audacity which would start showing
differences below 20kHz when converted to 16/44...



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
kalessin
2014-06-27 16:13:03 UTC
Permalink
Hi,

I am sorry, forget everything I said: your test data is okay. I have
redone my analysis and now it matches. The big noticeable differences
between A and B I saw in the first run have vanished. I have no
conclusive idea why, other than maybe I was asleep when I did the first
analysis. :(

Anyway, your results are impressive. :) And from the point of view of a
physicist -and- coder, my belief in both mathematics and physics is
reassured. :D


------------------------------------------------------------------------
kalessin's Profile: http://forums.slimdevices.com/member.php?userid=63059
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Wombat
2014-06-27 16:21:51 UTC
Permalink
I have no conclusive idea why, other than maybe I was asleep when I did
the first analysis. :(
Good morning and welcome to the forum :)
Sorry if i came over as bada** Normaly we love each other here!



Transporter (modded) -> RG142 -> Avantgarde Acoustic based 500VA
monoblocks -> Sommer SPK240 -> self-made speakers
------------------------------------------------------------------------
Wombat's Profile: http://forums.slimdevices.com/member.php?userid=4113
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Archimago
2014-06-27 16:24:16 UTC
Permalink
Post by kalessin
Hi,
I am sorry, forget everything I said: your test data is okay. I have
redone my analysis and now it matches. The big noticeable differences
between A and B I saw in the first run have vanished. I have no
conclusive idea why, other than maybe I was asleep when I did the first
analysis. :(
Anyway, your results are impressive. :) And from the point of view of a
physicist -and- coder, my belief in both mathematics and physics is
reassured. :D
Glad to hear! Thanks for the update.

:cool:



Archimago's Musings: (archimago.blogspot.com) A 'more objective'
audiophile blog.
------------------------------------------------------------------------
Archimago's Profile: http://forums.slimdevices.com/member.php?userid=2207
View this thread: http://forums.slimdevices.com/showthread.php?t=101742
Continue reading on narkive:
Loading...