Discussion:
[Portaudio] Resampling in PortAudio
aCOSwt
2011-01-06 09:10:53 UTC
Permalink
Hello,

As it appears that (At least under Linux implementations), Portaudio does not
resample if the sampling rate of the device differs from the sampling rate of
the data to output.
I find this particularly annoying when the Jack's API is concerned.
As a matter of fact, I believe it is on behalf of the client to adapt itself
to the jack's sampling rate and not the other way around.
That is by the way what the vast majority of jack's clients does.

I am interested in enabling portaudio to resample because I do not want to
constraint my entire sound chain to 22K just because I use espeak.

After a quick glance to the code, I think I will go with libsamplerate and
implement the necessary code within pa_process

Before starting, (Simply to know if I achieve this the quick & dirty way or If
I should take more care...) I would like to know :

1/ Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.

2/ Am I the only one interested in this /vs/ this development would be seen
interesting and eventually integrated in further releases.

3/ In the case of official devs interested, what would be their requirements ?

Regards,

aCOSwt
Bjorn Roche
2011-01-06 15:06:31 UTC
Permalink
This isn't really my thing anymore, but at least under ALSA you need
to setup DMIX, which is very unintuitive and, for some reason, not
installed by default on most systems. If you google DMIX and ALSA you
should be able to find help.

bjorn
Post by aCOSwt
Hello,
As it appears that (At least under Linux implementations), Portaudio does not
resample if the sampling rate of the device differs from the
sampling rate of
the data to output.
I find this particularly annoying when the Jack's API is concerned.
As a matter of fact, I believe it is on behalf of the client to adapt itself
to the jack's sampling rate and not the other way around.
That is by the way what the vast majority of jack's clients does.
I am interested in enabling portaudio to resample because I do not want to
constraint my entire sound chain to 22K just because I use espeak.
After a quick glance to the code, I think I will go with
libsamplerate and
implement the necessary code within pa_process
Before starting, (Simply to know if I achieve this the quick & dirty way or If
1/ Is this a lost cause ? Some already tried this without
succeeding /vs/
This evolution is already in the official development pipe.
2/ Am I the only one interested in this /vs/ this development would be seen
interesting and eventually integrated in further releases.
3/ In the case of official devs interested, what would be their requirements ?
Regards,
aCOSwt
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
Phil Burk
2011-01-06 17:45:19 UTC
Permalink
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want to
constraint my entire sound chain to 22K just because I use espeak.
1/ Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be seen
interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio.
It is often needed and seems like an obvious feature.

I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.

Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.

That has been the policy so far. But things can always change. What do
others think?

Phil Burk
Dmitry Kostjuchenko
2011-01-06 18:36:06 UTC
Permalink
Hi Guys!

Phil, I agree with your opinion about sample rate conversion and other
additional stuff. PortAudio is really good because it is very slim
middleware and is just an adapter to OS specific audio interface
rather than audio processing library/engine. Adding additional code
will simply grow the size of binary while many users I believe are
using own ways of coping with sample rate and already got used to the
fact that PortAudio does not offer re-sampling.

If libsamplerate is desired anyway for some reason than it can be very
simply used inside the user-side PortAudio callback or in some other
asynchronous processing stage before callback call.

Dmitry.
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want to
constraint my entire sound chain to 22K just because I use espeak.
1/ Is this a lost cause ? Some already tried this without succeeding ?/vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be seen
interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio. It
is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal interface
to native audio devices. It would handle all the host specific issues and
leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions on
how those things should be done. There are a number of ways to do sample
rate conversion ranging from fast and cheesy linear interpolation to
expensive and "correct" sinc interpolation. So we just leave that sort of
portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Oliver Rettig
2011-01-06 20:05:26 UTC
Permalink
Hi Phil,
I agree with you not to add sample rate conversion into portaudio. To do this
creates the need for a plugin concept to change different versions of sample
rate conversion implementations. And if we would have a plugin concept someone
would like to add other effects ... and we need a chain of plugins ... and so
on. I think, it is better to do this application specific.

best regards
Oliver
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want
to constraint my entire sound chain to 22K just because I use espeak. 1/
Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be
seen interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio.
It is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Bjorn Roche
2011-01-06 21:19:47 UTC
Permalink
While I don't disagree, SRC is becoming more and more common at lower
levels as quality improves dramatically and CPUs speed up. If a
decision is made to add it, here are some references:

https://ccrma.stanford.edu/~jos/resample/Free_Resampling_Software.html

Microsoft Public License: http://naudio.codeplex.com/
LGPL: http://shibatch.sourceforge.net/

bjorn
Post by Oliver Rettig
Hi Phil,
I agree with you not to add sample rate conversion into portaudio. To do this
creates the need for a plugin concept to change different versions of sample
rate conversion implementations. And if we would have a plugin
concept someone
would like to add other effects ... and we need a chain of
plugins ... and so
on. I think, it is better to do this application specific.
best regards
Oliver
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want
to constraint my entire sound chain to 22K just because I use espeak. 1/
Is this a lost cause ? Some already tried this without
succeeding /vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be
seen interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in
PortAudio.
It is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear
interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
Tim Bunnell
2011-01-06 21:35:39 UTC
Permalink
Just to add a slightly different perspective here, sample rate, unlike equalization or other forms of difficult to specify signal modification, is already a parameter one directly specifies (at least in an aspirational sense) in the process of opening a stream for input or output. To my way of thinking, it would add great utility and virtually no change to the API calls to make sample rate a specification that the I/O interface adheres to.

t
Post by Bjorn Roche
https://ccrma.stanford.edu/~jos/resample/Free_Resampling_Software.html
Microsoft Public License: http://naudio.codeplex.com/
LGPL: http://shibatch.sourceforge.net/
bjorn
Post by Oliver Rettig
Hi Phil,
I agree with you not to add sample rate conversion into portaudio. To do this
creates the need for a plugin concept to change different versions of sample
rate conversion implementations. And if we would have a plugin concept someone
would like to add other effects ... and we need a chain of plugins ... and so
on. I think, it is better to do this application specific.
best regards
Oliver
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want
to constraint my entire sound chain to 22K just because I use espeak. 1/
Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be
seen interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio.
It is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Director, Center for Pediatric Auditory and Speech Sciences (CPASS)
Head, Speech Research Lab | Adjunct Associate Professor
duPont Hospital for Children | Comp. and Info. Sciences / Linguistics
Voice: 1+302+651-6835 | University of Delaware
Fax: 1+302+651-6895 | URL: www.asel.udel.edu/speech/
Phil Burk
2011-01-06 22:26:45 UTC
Permalink
Hello Tim,
sample rate is already a parameter one directly specifies
Good point. That makes it easier to draw the line at SRC and not get
sucked into providing a plugin chain of DSP modules. The API already
provides a way to request a sample rate. So we are just honoring that
request.

If we did this we could add a paConvertSampleRateIfNeeded flag to
PaStreamFlags.

It could use native SRC (eg CoreAudio) if available or our own basic
sinc interpolator. If the caller wanted a better or cheaper SRC then
they could skip ours and provide their own.

A sinc interpolator is easy. We already have a block adapter to help
with the different rates on each end. It would be good to have our own
code so we are not mixing software licenses.

We should also need to provide way to tell the user what hardware sample
rate they actually got.

Hmmm....

Phil Burk
Just to add a slightly different perspective here, sample rate, unlike equalization or other forms of difficult to specify signal modification, is already a parameter one directly specifies (at least in an aspirational sense) in the process of opening a stream for input or output. To my way of thinking, it would add great utility and virtually no change to the API calls to make sample rate a specification that the I/O interface adheres to.
t
Post by Bjorn Roche
https://ccrma.stanford.edu/~jos/resample/Free_Resampling_Software.html
Microsoft Public License: http://naudio.codeplex.com/
LGPL: http://shibatch.sourceforge.net/
bjorn
Post by Oliver Rettig
Hi Phil,
I agree with you not to add sample rate conversion into portaudio. To do this
creates the need for a plugin concept to change different versions of sample
rate conversion implementations. And if we would have a plugin concept someone
would like to add other effects ... and we need a chain of plugins ... and so
on. I think, it is better to do this application specific.
best regards
Oliver
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want
to constraint my entire sound chain to 22K just because I use espeak. 1/
Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be
seen interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio.
It is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
Ross Bencina
2011-01-07 01:40:52 UTC
Permalink
A sinc interpolator is easy. We already have a block adapter to help with
the different rates on each end. It would be good to have our own code so
we are not mixing software licenses.
With all due respect Phil, a sinc interpolator is not easy. There are many
ways to window it, there are many ways to implement it. Not to mention the
choice of data points. And what if I want a least-squares optimised
interpolator instead of a windowed sinc interpolator?

More comments in my next post...

Ross.
aCOSwt
2011-01-06 23:09:04 UTC
Permalink
I cannot agree with you Oliver.

My point is *not* to add some extra fancy optional feature to PortAudio.
PortAudio claims supporting JACK doesn't it ?
But It is only a semi-support.
All the JACK's clients I can know just resample when the sample rate of the
stream differs from JACK's sampling rate.
They do not return paInvalidSampleRate period !
You cannot request from the application, which relies on PortAudio in order to
abstract the backend, to jack_get_sample_rate(jackHostApi->jack_client) prior
to calling portaudio's routines.
If this is actually needed to be done on the application side, then the
application would better dealing directly with the JACK's API rather than
going through portaudio.

Regards,

aCOSwt
Post by Oliver Rettig
Hi Phil,
I agree with you not to add sample rate conversion into portaudio. To do
this creates the need for a plugin concept to change different versions of
sample rate conversion implementations. And if we would have a plugin
concept someone would like to add other effects ... and we need a chain of
plugins ... and so on. I think, it is better to do this application
specific.
best regards
Oliver
Post by aCOSwt
Hello,
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want
to constraint my entire sound chain to 22K just because I use espeak.
1/ Is this a lost cause ? Some already tried this without succeeding
/vs/ This evolution is already in the official development pipe.
Technically you should be able to easily add sample rate conversion.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be
seen interesting and eventually integrated in further releases.
I can understand why you would want sample rate conversion in PortAudio.
It is often needed and seems like an obvious feature.
I think the philosophy is that PortAudio would try to be a minimal
interface to native audio devices. It would handle all the host specific
issues and leave all the portable math code to the applications.
Sample rate conversion, music synthesis, EQ and other effects can all be
done in portable code. It's just math. Also people have strong opinions
on how those things should be done. There are a number of ways to do
sample rate conversion ranging from fast and cheesy linear interpolation
to expensive and "correct" sinc interpolation. So we just leave that
sort of portable audio processing up to the application.
That has been the policy so far. But things can always change. What do
others think?
Phil Burk
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Richard Ash
2011-01-06 22:18:58 UTC
Permalink
Post by aCOSwt
As it appears that (At least under Linux implementations), Portaudio does not
resample if the sampling rate of the device differs from the sampling rate of
the data to output.
I find this particularly annoying when the Jack's API is concerned.
As a matter of fact, I believe it is on behalf of the client to adapt itself
to the jack's sampling rate and not the other way around.
That is by the way what the vast majority of jack's clients does.
That's certainly Jack's intention. Remember that Jack aims to route
audio around with a low enough latency and high enough quality that it
can be used like a studio patch bay, including for recording live
performances. They aren't going to achieve this if resamplers start
getting added as they (unavoidably) have a significant latency.

For the same reason, people using portaudio-based applications with Jack
want in many cases to preserve the latency performance of Jack so they
can (for instance) sync a midi synth with a multitrack recorder and then
record playing along with the output without needing to manually re-sync
the tracks.

A hardware digital studio would impose the same constraints in terms of
sample rate being matched between devices unless sample rate converter
units are explicitly coupled into the signal chain, which would normally
be avoided for quality reasons.

Ultimately if you are playing back multiple streams at different sample
rates through a single sound device, and don't need the ability to
couple one application's output to another's input, then Jack is
probably not the right tool for your problem. You might find ALSA's
output capture facility useful if you want to record what is being
played by an application into a file:
http://wiki.audacityteam.org/wiki/Recording_audio_playing_on_the_computer#Using_the_ALSA_PCM_file

Richard
Ross Bencina
2011-01-07 01:58:33 UTC
Permalink
Post by aCOSwt
As it appears that (At least under Linux implementations), Portaudio does not
resample if the sampling rate of the device differs from the sampling rate of
the data to output.
I find this particularly annoying when the Jack's API is concerned.
As a matter of fact, I believe it is on behalf of the client to adapt itself
to the jack's sampling rate and not the other way around.
That is by the way what the vast majority of jack's clients does.
Yes. Keep in mind that your program is the client of JACK, not PortAudio.
PortAudio just makes it easy for you to talk to JACK the same way you talk
to other APIs.

PortAudio is best thought of as an abstraction and wrapper to allow single
API access to multiple host audio APIs. If the host audio API requires you
to resample, then you need to be able to resample. There is nothing that
says that PortAudio must do everything for you.

It is easy to argue that PortAudio should offer this capability -- but it is
not a responsibility that PortAudio has ever taken on board and we have
argued successfully for 10 years to keep it out of PortAudio. But, as others
have said, there is no reason why we shouldn't consider it again.

There are a few of good arguments against PortAudio including resampling
code: the best is that the choice of resampling quality is highly dependent
on the application your are writing and it would be inappropriate to include
a one-size-fits all solution. There are many quality/efficiency trade offs
that can be made.

We have trouble enough deciding how to convert ints to floats, and
resampling algorithm design has so many more degrees of freedom.
Post by aCOSwt
I am interested in enabling portaudio to resample because I do not want to
constraint my entire sound chain to 22K just because I use espeak.
After a quick glance to the code, I think I will go with libsamplerate and
implement the necessary code within pa_process
Why not just use libsamplerate in your client? I can pretty much guarantee
this will be easier than hacking PortAudio.

If you do want to experiment with modifying PortAudio: pa_process.c is
already rather complicated -- from an architectural point of view I think it
would be preferable to add sample rate conversion as a separate module
either before or after pa_process rather than within it. But I could be
wrong, perhaps you'll come up with a solution I can't envisage...
Post by aCOSwt
Before starting, (Simply to know if I achieve this the quick & dirty way or If
1/ Is this a lost cause ? Some already tried this without succeeding /vs/
This evolution is already in the official development pipe.
There are a few things I can say with reasonable confidence:

- We'd be reluctant to introduce a dependency on a 3rd party resampling
library. Especially if it used a different licence.

- We definitely wouldn't agree to using libsamplerate, its licence is not
compatible with PortAudio's.
Post by aCOSwt
2/ Am I the only one interested in this /vs/ this development would be seen
interesting and eventually integrated in further releases.
I can't speak for the other devs, but personally I would be interested if we
had our own resampling code (eg licence is the same PortAudio licence and
code was part of the PortAudio distro), and we offered a variety of
quality/efficiency settings including a high-quality transparent one that
was as good or better than the Apple one we use in the CoreAudio
implementation. That would be my criteria for acceptance. This is a _lot_ of
work imho.

That said, given that we don't even have a full suite of sample format
converters in pa_converters.c and no one has stepped forward to help fix
that situation I would personally be reluctant to encourage extension into
new areas of development.

Another argument against it is that I can't see a reason why this
functionality can't be implemented just as easily in the client. (perhaps a
library that makes this very easy would be appropriate though, like one that
acts as an adapter between the real PA callback and a client callback after
resampling.

As I mentioned in my previous post, resampling is a very technical and
complex area (many different approaches with different
design/quality/efficiency tradeoffs) and there are a number of benefits to
not handling it inside PortAudio:

- Clients can make design trade offs that best suit their application
- We don't have to take responsibility for making design decisions that are
not (in my view) the business of PortAudio
- We don't have to provide high-quality resampler implementations

There is also the issue of platform consistency: on some platforms where it
is available (eg CoreAudio) we use the host's resampler, if we use our own
on other platforms then we will be explicitly offering inconsistent
behavior.

Another thing I can think of is that resampling will add complexity to
host/client callback buffer size negotiations -- the kind of CPU usage
issues we get with ASIO buffer size adaption will potentially be much worse
with SRC and more difficult to negotiate.
Post by aCOSwt
3/ In the case of official devs interested, what would be their requirements ?
See above.

In short: I would personally be happy to see a very high quality
implementation of SRC in PortAudio. But I would not be interested in a
half-baked solution, and I am not confident that a high-quality solution
could be achieved given the available resources -- there is a lot of
behavior to specify and corner cases to negotiate. I hope what I've written
above gives some impression of why I think that.

Best wishes

Ross
Chris
2011-01-07 02:27:49 UTC
Permalink
My two cents' worth:

WASAPI support is almost finished. Finalize version 19 with WASAPI
support, then debate the issue of sample rate conversion ad infinitum
for version 20.
TJF
2011-01-07 14:59:33 UTC
Permalink
Hi,

beware of doing that (integration of src)! There will be many questions,
like:

- do you prefer pre-ringing or post-ringing?
- or do you prefer no pre-ringing at all against linear phase?
- or do you prefer anything between?

and so on. Currently I am comparing (listening to) all this
possibilities. You can use i.e. libsamplerate, where you cannot choose
anything ... I prefer sox. Here you can choose among everything. Have
also a look at this: http://src.infinitewave.ca/

If you want to make an own resampler, you will walk into an endless
story. I think it would be useful to talk about an effective way how to
integrate such a resample library in any callback of a program, because
a high cpu-load will be the result (at higher samplerates). I didn't
have success in using the sox input and output handlers (see sox -
eample1.c ) this way...

Regards
Thomas
Bjorn Roche
2011-01-07 16:03:05 UTC
Permalink
and we offered a variety of quality/efficiency settings including a
high-quality transparent one that was as good or better than the
Apple one we use in the CoreAudio implementation.
That is a tall order. While I have not done listening tests, the
analyses I've seen show that Core Audio (Tiger and beyond) is one of
the best ones out there. Very CPU efficient to boot.

bjorn

-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
Ross Bencina
2011-01-08 01:40:17 UTC
Permalink
Post by Bjorn Roche
and we offered a variety of quality/efficiency settings including a
high-quality transparent one that was as good or better than the Apple
one we use in the CoreAudio implementation.
That is a tall order.
I realise that.
Post by Bjorn Roche
While I have not done listening tests, the analyses I've seen show that
Core Audio (Tiger and beyond) is one of the best ones out there. Very CPU
efficient to boot.
I don't see how we can legitimately offer this feature in PortAudio without
having something that is as good as the industry standard.

Ross.
Phil Burk
2011-01-08 02:40:55 UTC
Permalink
Hi Ross,
Post by Ross Bencina
There are a few of good arguments against PortAudio including
resampling code: the best is that the choice of resampling quality is
highly dependent on the application your are writing and it would be
inappropriate to include a one-size-fits all solution. There are
many quality/efficiency trade offs that can be made.
I agree that we cannot please everybody with a one-size-fits-all
resampler. So whatever we provide, if we provide it, should only be an
option that is disabled by default.

Most folks do not know or care about the difference between the various
sinc windows or other SRC details. If we provided a pretty good SRC,
better than most apps need, but not too expensive, then many folks would
be happy to use it.

The DSP purists who do care about SRC implementations can always do
their own resampling in their application. No one will be stuck with
using the PortAudio resampler.

I think the best argument against NOT providing SRC is that we are too
busy to add features that could be implemented at the application level.

Phil Burk
Chris
2011-01-08 03:14:53 UTC
Permalink
<< I think the best argument against NOT providing SRC is that we are
too
busy to add features that could be implemented at the application
level. >>

With your double negative you are saying that "we are too busy" is the
best argument FOR adding SRC. That doesn't make sense.
Phil Burk
2011-01-08 17:44:33 UTC
Permalink
Oops. Right. Double negative.

Make that:
I think the best argument AGAINST providing SRC is that we are too
busy to add features that could be implemented at the application level.

Sorry for the confusion.
Phil
<< I think the best argument against NOT providing SRC is that we are too
busy to add features that could be implemented at the application level. >>
With your double negative you are saying that "we are too busy" is the
best argument FOR adding SRC. That doesn't make sense.
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Ross Bencina
2011-01-10 02:46:48 UTC
Permalink
Post by Phil Burk
I think the best argument AGAINST providing SRC is that we are too
busy to add features that could be implemented at the application level.
Indeed.

Except aCOSwt was potentially offering to do the work, and may not be too
busy. I don't think we can afford to be too busy to accept patches that meet
our design criteria.

Ross.
aCOSwt
2011-01-10 10:37:19 UTC
Permalink
Post by Ross Bencina
It is easy to argue that PortAudio should offer this capability
It is indeed so... I will not.

It is even easier for me not to argue that, that, per se, my point was *not*
to add resampling capabilities in PortAudio.

My point was, per se, to enhance PortAudio's JACK support.
I am not playing with words. If the goal is different, the issues that need to
be addressed are different.

For sure if the goal is to add resampling capabilities, then all the issues
you enumerate are true questions.
But, If adding resampling capability proceeds from the need to make JACK's
support useful, then, one thinks of resampling as a solution to a specific
need => in this particular case, resampling is to be seen as a default
solution, some kind of fallback.
And the first quality of a fallback is to KISS !
=> Lowest possible power, lowest possible cost, resampling solution !

It's a bit like the case when a device is not specified ! If some code claims
ALSA support then it will naturally select ALSA's default and neither return
ENODEV nor fire some sophisticated algorithm searching for some self-supposed
best possible device.
Post by Ross Bencina
We have
argued successfully for 10 years to keep it out of PortAudio.
Geee !! Pardon me. I had not imagined I would wake a ten years old sea-snake
up !
Post by Ross Bencina
There are a few of good arguments against PortAudio including resampling
code: the best is that the choice of resampling quality is highly dependent
on the application.
I totally disagree with this :

The final quality (the only one that, at the end
of the day, matters) is *totally* depending on the end user who is supposed to
know what he does. (Particularily if he his using JACK)

When I define the sampling rate of my jackd instance, I am aware that the
output of any source on a different sample rate will get degraded.
If I do care about the quality of a given source, I make so the sample rates
match period !
If I don't make so that sample rates match, then It *means* that I do not care
about quality from this source compared with other jackd's clients.

=> True aesthetic in this matter is entirely governed by technical
compatibility and coherence not by the intrinsic qualities of the component.
Resampling quality is, in this particular case, second. A very accessory,
incidental question, even from the application point of view.

BTW PortAudio devs are, as far as I can read in pa_front.c perfectly in
accordance with this philosophy :
"defaultHighOutputLatency is used below instead of defaultLowOutputLatency
because it is more important for the default stream to work reliably than it
is for it to work with the lowest latency"
Post by Ross Bencina
We have trouble enough deciding how to convert ints to floats, and
resampling algorithm design has so many more degrees of freedom.
I can understand and appreciate this perfectly on a project leader point of
view and would even be ready to share it if I had shared your troubles.
Post by Ross Bencina
Why not just use libsamplerate in your client? I can pretty much guarantee
this will be easier than hacking PortAudio.
Indeed ! I had simply imagined that implementing in PortAudio would
be helpful to others apps. That's the purpose of a library isn't it ?
Post by Ross Bencina
- We definitely wouldn't agree to using libsamplerate, its licence is not
compatible with PortAudio's.
Fair enough. This is once more perfectly understandable as well as sensible on
a project leader point of view.
Post by Ross Bencina
Another argument against it is that I can't see a reason why this
functionality can't be implemented just as easily in the client.
For sure it can be. And, as it seems I am the only one interested in this and
as I get a unique existing PortAudio based application, this is probably what
I will do.
But, in this case, (IMHO) PortAudio is not very helpful :

Why do we need to wait for the callback sequence to be rolling, to discover
that the samplerates do not match ?
I read that ValidateOpenStreamParameters "Checks for absurd sample rates".
As far as JACK is concerned, a sample rate different from JACK's sampling rate
sounds (IMHO) as "absurd" as a samplerate > 200.000 ! It just won't work
either !
Why cannot we get informed that paInvalidSampleRate as soon as the return of
the Pa_OpenStream call ?
This would be far more convenient from an application point of view.

(BTW... while this is not convenient from an application point of view, I
would have done the same programming within PortAudio... if... I had had the
intention to resample later on...)
Post by Ross Bencina
- We don't have to provide high-quality resampler implementations
Fair enough. Once again my intention was not regarding a high quality
resampling feature, but was regarding a low power low cost fallback solution
enabling true JACK support from PortAudio.
Post by Ross Bencina
I hope what I've written
above gives some impression of why I think that.
Absolutely and I thank you very much for this.
All this makes sense and your position is understandable and sensible from a
project's lead point of view.

Thank you all for your answers and thanks to all those who develop and
maintain PortAudio.

Regards,

aCOSwt
Ross Bencina
2011-01-11 02:53:08 UTC
Permalink
Post by aCOSwt
My point was, per se, to enhance PortAudio's JACK support.
I am not playing with words. If the goal is different, the issues that need to
be addressed are different.
OK, understood. Sorry I misconstrued your original post. It could be argued
however that WASAPI is now in a similar situation.

[snip]
Post by aCOSwt
Post by Ross Bencina
There are a few of good arguments against PortAudio including resampling
code: the best is that the choice of resampling quality is highly dependent
on the application.
The final quality (the only one that, at the end
of the day, matters) is *totally* depending on the end user who is supposed to
know what he does. (Particularily if he his using JACK)
My point was that if SRC is in done it PortAudio, it is no longer possible
for the client to take control of the SRC quality (unless PortAudio offers a
number of options from high-quality, to low-CPU).
Post by aCOSwt
When I define the sampling rate of my jackd instance, I am aware that the
output of any source on a different sample rate will get degraded.
If I do care about the quality of a given source, I make so the sample rates
match period !
If I don't make so that sample rates match, then It *means* that I do not care
about quality from this source compared with other jackd's clients.
I disagree with this attitude as the only possible approach. This is indeed
one attitude you could take -- but SRC can be extremely good, and there is
no reason for the user to always equate "different sample rates" with
"degraded quality" -- at least not with "really degraded quality".

In any case, I would not be very happy with an outcome where users who use
JACK with PortAudio apps at mismatched sample rates have no choice but to
get "degraded quality". But I think we need input from other JACK users here
since I am quite ignorant of JACK usage and culture.
Post by aCOSwt
=> True aesthetic in this matter is entirely governed by technical
compatibility and coherence not by the intrinsic qualities of the component.
Resampling quality is, in this particular case, second. A very accessory,
incidental question, even from the application point of view.
I'm not sure what you're saying here. Do you mean that it's more important
to make it work than to make it work well?

You can already make it work very well: do the SRC in your client callback
(using secret rabit code or whatever you like).
Post by aCOSwt
BTW PortAudio devs are, as far as I can read in pa_front.c perfectly in
"defaultHighOutputLatency is used below instead of defaultLowOutputLatency
because it is more important for the default stream to work reliably than it
is for it to work with the lowest latency"
That's not a valid comparison: PortAudio defaults to default high latency,
but also supports _very_low_latency_. Right now PortAudio does not support
SRC at all. As I have already said, if PA did support SRC it would need to
support both efficient and high-quality modes. You seem to be arguing that a
low quality mode would be enough.

But perhaps this is a good point for starting a discussion. You could do
your low quality SRC for now and we could add a flag
paNoSampleRateConversion that users could use if they wanted Pa_OpenStream()
to fail with "invalid sample rate" instead of performing SRC. What do you
think?
Post by aCOSwt
Post by Ross Bencina
We have trouble enough deciding how to convert ints to floats, and
resampling algorithm design has so many more degrees of freedom.
I can understand and appreciate this perfectly on a project leader point of
view and would even be ready to share it if I had shared your troubles.
Post by Ross Bencina
Why not just use libsamplerate in your client? I can pretty much guarantee
this will be easier than hacking PortAudio.
Indeed ! I had simply imagined that implementing in PortAudio would
be helpful to others apps. That's the purpose of a library isn't it ?
Post by Ross Bencina
- We definitely wouldn't agree to using libsamplerate, its licence is not
compatible with PortAudio's.
Fair enough. This is once more perfectly understandable as well as sensible on
a project leader point of view.
Post by Ross Bencina
Another argument against it is that I can't see a reason why this
functionality can't be implemented just as easily in the client.
For sure it can be. And, as it seems I am the only one interested in this and
as I get a unique existing PortAudio based application, this is probably what
I will do.
Why do we need to wait for the callback sequence to be rolling, to discover
that the samplerates do not match ?
I read that ValidateOpenStreamParameters "Checks for absurd sample rates".
As far as JACK is concerned, a sample rate different from JACK's sampling rate
sounds (IMHO) as "absurd" as a samplerate > 200.000 ! It just won't work
either !
Why cannot we get informed that paInvalidSampleRate as soon as the return of
the Pa_OpenStream call ?
This would be far more convenient from an application point of view.
I agree with you on this. I did not implement the JACK PortAudio
implementation and I can't really comment on the decisions that went in to
this. But of course we can look at changing/improving the PortAudio/JACK
implementation. I think this is perhaps a broader discussion than simply
adding SRC and we would need input from others who are using JACK with
PortAudio to discuss details -- certainly I am not expert on JACK at all.
Post by aCOSwt
(BTW... while this is not convenient from an application point of view, I
would have done the same programming within PortAudio... if... I had had the
intention to resample later on...)
Post by Ross Bencina
- We don't have to provide high-quality resampler implementations
Fair enough. Once again my intention was not regarding a high quality
resampling feature, but was regarding a low power low cost fallback solution
enabling true JACK support from PortAudio.
I see.

Well, one possibility might be to implement it in such a way that it can be
upgraded to high-quality later (some kind of modular approach).
Post by aCOSwt
Post by Ross Bencina
I hope what I've written
above gives some impression of why I think that.
Absolutely and I thank you very much for this.
All this makes sense and your position is understandable and sensible from a
project's lead point of view.
Thank you all for your answers and thanks to all those who develop and
maintain PortAudio.
No problems :-)

Ross.
Phil Burk
2011-01-11 19:20:24 UTC
Permalink
Hi Ross,
Post by Ross Bencina
My point was that if SRC is in done it PortAudio, it is no longer
possible for the client to take control of the SRC quality (unless
PortAudio offers a number of options from high-quality, to low-CPU).
I agree that we must allow callers to have control over any SRC.
If we add SRC then I think it should be a non-default option. The user
can set psConvertSampleRateIfNeeded if they want our SRC. Otherwise
they can just do their own SRC. They should never be stuck with our SRC.

Phil Burk
Robert Bielik
2011-01-11 19:27:08 UTC
Permalink
Post by Phil Burk
I agree that we must allow callers to have control over any SRC.
If we add SRC then I think it should be a non-default option. The user can set psConvertSampleRateIfNeeded if they want our SRC. Otherwise they can just do their own SRC. They should never be stuck with our SRC.
I think it would be reasonably easy to adapt PA so that the user can supply a ptr to a SRC function (much like the PA callback), then PA can also supply a reference implementation of such a function (of good enough quality) to start from.

/Rob
Robert Bielik
2011-01-11 21:40:26 UTC
Permalink
Dear all,

I've gotten a request to port an application which pretty much is like a DSP-filter that inserts itself in the default device
output stream. On Windows, its a virtual audio device (plus a streaming application) and on Mac OS X pretty much the same.

But I have no idea how to do it on Linux ? Can someone give me pointers/hints ? ;) (I'd like to do it as "natively" as possible,
which pretty much rules out Jack)

TIA
/Rob
Richard Ash
2011-01-11 22:24:43 UTC
Permalink
Post by Robert Bielik
I've gotten a request to port an application which pretty much is like
a DSP-filter that inserts itself in the default device
output stream. On Windows, its a virtual audio device (plus a
streaming application) and on Mac OS X pretty much the same.
But I have no idea how to do it on Linux ? Can someone give me
pointers/hints ? ;) (I'd like to do it as "natively" as possible,
which pretty much rules out Jack)
ALSA-lib PCM plugin - in other words, some code that can be loaded into
alsa-lib, which then ends up in the audio path for all programs that
play via a given sound device (and yes, you could put it in the default
audio device with some care).

http://wiki.audacityteam.org/wiki/Recording_audio_playing_on_the_computer#Using_the_ALSA_PCM_file
uses and existing plug-in to send the audio to a file, but the same
process could be used to put your filter in.

Doing it in ALSA means it ends up underneath either PulseAudio or JACK
if they are in use, which catches most things.

Richard
Robert Bielik
2011-01-12 05:14:53 UTC
Permalink
Thnx guys for the input, extremely helpful!
Post by Richard Ash
Doing it in ALSA means it ends up underneath either PulseAudio or JACK
if they are in use, which catches most things.
This pretty much sounds exactly what I'm after :)

/Rob

Brad Fuller
2011-01-11 23:17:04 UTC
Permalink
I assume you looked at LADSPA? If not
http://www.ladspa.org/

or use VST with:
http://wiki.linuxaudio.org/apps/all/dssi

just some ideas... which you may have already considered.
Post by Robert Bielik
Dear all,
I've gotten a request to port an application which pretty much is like a
DSP-filter that inserts itself in the default device
output stream. On Windows, its a virtual audio device (plus a streaming
application) and on Mac OS X pretty much the same.
But I have no idea how to do it on Linux ? Can someone give me
pointers/hints ? ;) (I'd like to do it as "natively" as possible,
which pretty much rules out Jack)
TIA
/Rob
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Phil Burk
2011-01-11 22:16:59 UTC
Permalink
Post by Robert Bielik
I think it would be reasonably easy to adapt PA so that the user can
supply a ptr to a SRC function (much like the PA callback), then PA can
also supply a reference implementation of such a function (of good
enough quality) to start from.
I like that idea.

Phil
Tim Bunnell
2011-01-11 22:25:55 UTC
Permalink
This really does seem like a great idea, giving people who simply want a turn-key good-enough solution something to use, real experts a hook to hang their preferred approach(es) from, and people in between a hook to hang themselves from, errr, I mean a place to start if they want to learn and fiddle around with SRC alternatives.

t
Post by Phil Burk
Post by Robert Bielik
I think it would be reasonably easy to adapt PA so that the user can
supply a ptr to a SRC function (much like the PA callback), then PA can
also supply a reference implementation of such a function (of good
enough quality) to start from.
I like that idea.
Phil
_______________________________________________
Portaudio mailing list
Portaudio at music.columbia.edu
http://music.columbia.edu/mailman/listinfo/portaudio
Director, Center for Pediatric Auditory and Speech Sciences (CPASS)
Head, Speech Research Lab | Adjunct Associate Professor
duPont Hospital for Children | Comp. and Info. Sciences / Linguistics
Voice: 1+302+651-6835 | University of Delaware
Fax: 1+302+651-6895 | URL: www.asel.udel.edu/speech/
Bjorn Roche
2011-01-11 22:35:55 UTC
Permalink
Post by Phil Burk
Post by Robert Bielik
I think it would be reasonably easy to adapt PA so that the user can
supply a ptr to a SRC function (much like the PA callback), then PA can
also supply a reference implementation of such a function (of good
enough quality) to start from.
I like that idea.
There are many questions to answer: since there is a ratio of input
samples to output samples, who manages buffering so that the user can
still be presented with the buffer size they requested? what happens
if the SRC library and PA disagree about the ratio due to rounding or
other error? What about requirements of memory alignment that SRC
libraries are likely to make, can PA accommodate them, or is this up
to the user writing the wrapper function? How much does it really
simplify things for a user to write a wrapper around an existing
library and hook it into PA than for us to support an existing library
(albeit optionally)?

Many of these answers may seem obvious, but I suspect that this is a
project.

Another point worth making is that SRC would only need to be provided
to fill a few cases: ALSA where DMIX is missing, JACK, and maybe some
cases on windows. Another question is: is it worth having some
solution here, even if it's not perfect? Ross suggests that In these
cases, SRC needs to be as good as the best, but maybe something is
better than nothing.

bjorn

-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
Loading...