Discussion:
Need help debugging SuperCollider's PortAudio on Linux
(too old to reply)
Hanno Zulla
2016-04-01 12:57:13 UTC
Permalink
Hello,

I would appreciate some help debugging / improving the PortAudio
interface of SuperCollider.

When using SuperCollider's PortAudio driver on Linux, I have two
problems:

- it works with PulseAudio's "default" device,
but there are occasional buffer underruns

- it does NOT work with pure ALSA's "default" device,
(very choppy audio, see [1])

- it works with a hardware device like "hw:0,0" in pure ALSA,
no buffer underruns on my system (but not sure)

I don't know if the "buffer underrun" issue and the "ALSA default device
choppy audio" are the same or not.

After digging into the code, I found that SuperCollider unnecessarily
allocates all channels reported back by PortAudio, but my patch [2]
for that wouldn't fix the issues above.


Some background to explain the problem, please bear with me for this
writeup.


SuperCollider [3] is a programming language for audio synthesis, or
in other words, a programmable electronic synthesizer.

SuperCollider is also the synthesizer inside of Sonic Pi [4], a
Live Coding synthesizer used in schools to teach coding. Sonic Pi is
using a ruby interpreter as a more kids-friendly programming language
and converts the commands to SuperCollider's language while hiding its
complexity from the users.

Sonic Pi is mainly developed for Raspberry Pi, but just like
SuperCollider, it is distributed for OS X, Windows and generic Linux
distributions.

I'm a casual contributor to the Sonic Pi team and our Linux users
experience recurring problems with SuperCollider's default JACK driver.
Well, it works fine, but most Linux desktop distributions ship with
PulseAudio and for an average user, it's fairly complicated to get JACK
and PulseAudio cooperate. (You can use one of them at a time, but
they'll stomp each other's feet once you try to run both.) So a simple
way to use SuperCollider without JACK would be a major advantage for
Sonic Pi users who don't want to fiddle with their audio backends.

The SuperCollider scsynth backend comes with three different
AUDIOAPI drivers:

- jack (default for Linux)
- portaudio (default for Windows)
- coreaudio (default for OSX).

The portaudio driver SC_PortAudio.cpp works fine on Windows. The
JACK driver SC_JACK.cpp works fine on Linux. Both drivers use more or
less the same code structure and flow.

It is possible to build SuperCollider on Linux using the PortAudio
driver:

git clone https://github.com/supercollider/supercollider.git
cd supercollider
# see README_LINUX.md, install build dependencies, then:
git submodule init
git submodule update
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DAUDIOAPI=portaudio ..
make
sudo make install
/usr/local/bin/sclang

After this, you can enter SuperCollider code:

s.options.device="default"
s.boot
x = {PinkNoise.ar(0.1)}.play
// should result in pink noise on one channel
x.free
// leave with ^D

Try this sclang code with and without PulseAudio [5] and you'll
see the problems described above.


Help would be appreciated. The goal is to get SuperCollider's
SC_PortAudio.cpp working with pure ALSA's "default" device and get
rid of the buffer underruns. I've spent some time looking at this
problem now and haven't found an obvious way to debug/fix this.


Kind regards,

Hanno


[1] https://github.com/supercollider/supercollider/issues/1944
[2] https://github.com/supercollider/supercollider/pull/1943
[3] http://supercollider.github.io
[4] http://sonic-pi.net
[5] http://askubuntu.com/a/394872/274072
Phil Burk
2016-04-01 15:36:18 UTC
Permalink
Hello Hanno,

I have been using PortAudio on an RPi 2. It is a great machine. I just
ordered an RPi 3.

There is a Wiki page for Portaudio on Raspberry Pi here:

https://www.assembla.com/spaces/portaudio/wiki/Platforms_RaspberryPi

Note the bug at the bottom. The default low latency is too low and causes
glitches or "choppy audio".

I increased the suggested latency in my test program to 0.030 seconds and
was able to get smooth audio. You can probably go lower but for live coding
you don't need very low latency.

To set the latency you need to call Pa_OpenStream() and set
suggestedLatency in the outputParameters.

http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#a443ad16338191af364e3be988014cbbe

Let me know if increasing the latency does not work.
Post by Hanno Zulla
http://sonic-pi.net
Sonic Pi sounds like a great project. I'll check it out.

Phil Burk
Spencer Russell
2016-04-01 17:19:30 UTC
Permalink
Does the supercollider portaudio code do the audio processing directly
in the audio callback or does it pass data to the main task through some
other buffers?

I'm debugging some dropout issues with portaudio->alsa->pulseaudio->alsa-
hardware for my Julia wrapper, but haven't really troubleshot enough
to know where the issue is. In my case the callback reads/writes from
an intermediate buffer that is fed/consumed by the main task, so I
suspect there could be an issue where my portaudio buffer size is
smaller than the pulseaudio buffer size, so the callback is being
called back-to-back. Reading the docs more carefully it looks like
maybe I should be giving zero for my buffer size when I open the stream
to let portaudio choose.

Not sure if this is helpful, but I figured our issues might be related.

-s
Hello Hanno,
I have been using PortAudio on an RPi 2. It is a great machine. I just
ordered an RPi 3.
https://www.assembla.com/spaces/portaudio/wiki/Platforms_RaspberryPi
Note the bug at the bottom. The default low latency is too low and
causes glitches or "choppy audio".
I increased the suggested latency in my test program to 0.030 seconds
and was able to get smooth audio. You can probably go lower but for
live coding you don't need very low latency.
To set the latency you need to call Pa_OpenStream() and set
suggestedLatency in the outputParameters.
http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#a443ad16338191af364e3be988014cbbe
Let me know if increasing the latency does not work.
http://sonic-pi.net[1]
Sonic Pi sounds like a great project. I'll check it out.
Phil Burk
_________________________________________________
Portaudio mailing list
https://lists.columbia.edu/mailman/listinfo/portaudio
Links:

1. http://sonic-pi.net/
Ross Bencina
2016-04-03 07:35:14 UTC
Permalink
Post by Spencer Russell
Does the supercollider portaudio code do the audio processing directly
in the audio callback
Yes. I had something to do with that code many years ago when I worked
on the windows SC port.

Ross.
Hanno Zulla
2016-04-04 12:59:04 UTC
Permalink
Hi,
Post by Spencer Russell
Not sure if this is helpful, but I figured our issues might be related.
Yes, I stumbled over this in your discussion with Phil Burk:

<https://lists.columbia.edu/pipermail/portaudio/2016-March/000351.html>
Post by Spencer Russell
Also not that if you are doing both audio input and output then you
should start with the input empty and the output full. That way if
the app gets hung up the input will fill and the output will drain
symmetrically.
Indeed, SuperCollider does input, audio processing and output in its
callback loop. I wonder what to code so that one can "start with output
full" and if this might be an explanation for SuperCollider's audio issue.

Regards,

Hanno
Hanno Zulla
2016-04-04 14:18:24 UTC
Permalink
Hi,
Post by Hanno Zulla
Indeed, SuperCollider does input, audio processing and output in its
callback loop. I wonder what to code so that one can "start with output
full" and if this might be an explanation for SuperCollider's audio issue.
Ok, if I tell SuperCollider to skip the input channels and only compute
output audio...

<https://github.com/supercollider/supercollider/issues/1944#issuecomment-205299557>

...the problems go away almost completely, using default latency values.

So it's something in the way input and output are being handled in the
main loop

<https://github.com/hzulla/supercollider/blob/audiodriver/server/scsynth/SC_PortAudio.cpp#L219-L276>

Yet I still don't know what it is causing this.

Thanks,

Hanno
Hanno Zulla
2016-04-04 14:57:12 UTC
Permalink
Hi,

ok, sorry to flood the mailing list, but I have now found a fairly
minimal source example for you to reproduce the problem on your box,
without having to build a SuperCollider binary.

The original paex_sine.c from the PortAudio documentation was doing
audio output, only.

I have modified the original code minimally, to reflect the behaviour
from SuperCollider's code.

- the stream format is using paFloat32 | paNonInterleaved

- the program opens a stereo input channel on the default input device

Bingo, opening the input channel triggers the problem on my box. (*)

Once the input channel is added to the program, the exact same problem
occurs as experienced with SuperCollider. Replace &inputParameters with
NULL in Pa_OpenStream() and the problem goes away.

The callback here is really minimal, so it's not any audio processing
inside the callback look causing this.

What can be done to fix this?

Kind regards,

Hanno

(*) Ubuntu 15.10, amd64, desktop CPU.
PulseAudio turned off, using pure ALSA's "default" device.
Phil Burk
2016-04-04 15:27:34 UTC
Permalink
Hello Hanno,

Thanks for the test program. I will try that when I get on my Linux machine.

Good job narrowing down the issue. I agree this looks related to
input/output stream synchronization. If the output is not primed properly
then there can definitely be issues with output underruns.

Note that there is a flag, paPrimeOutputBuffersUsingStreamCallback, that
you can OR with paClipOff.

http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#ad7b862a223edd680d7985c33c847e31d

Also note that paPrimeOutputBuffersUsingStreamCallback may not be fully
implemented on ALSA.

http://www.portaudio.com/docs/proposals/status.html

The problem may be related to block size adaptation. Try setting
framesPerBuffer to paFramesPerBufferUnspecified, 0, when you open the
stream.

Also note that the callback is passed a set of flags.
http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#a55a005924bcfa0424594f4f65cd4ae82

Are any of those being set?

Thanks,
Phil Burk
Post by Hanno Zulla
Hi,
ok, sorry to flood the mailing list, but I have now found a fairly
minimal source example for you to reproduce the problem on your box,
without having to build a SuperCollider binary.
The original paex_sine.c from the PortAudio documentation was doing
audio output, only.
I have modified the original code minimally, to reflect the behaviour
from SuperCollider's code.
- the stream format is using paFloat32 | paNonInterleaved
- the program opens a stereo input channel on the default input device
Bingo, opening the input channel triggers the problem on my box. (*)
Once the input channel is added to the program, the exact same problem
occurs as experienced with SuperCollider. Replace &inputParameters with
NULL in Pa_OpenStream() and the problem goes away.
The callback here is really minimal, so it's not any audio processing
inside the callback look causing this.
What can be done to fix this?
Kind regards,
Hanno
(*) Ubuntu 15.10, amd64, desktop CPU.
PulseAudio turned off, using pure ALSA's "default" device.
_______________________________________________
Portaudio mailing list
https://lists.columbia.edu/mailman/listinfo/portaudio
Hanno Zulla
2016-04-04 15:47:45 UTC
Permalink
Hi,

thanks.
Post by Phil Burk
Note that there is a flag, paPrimeOutputBuffersUsingStreamCallback,
that you can OR with paClipOff.
No, didn't help when I added that.
Post by Phil Burk
The problem may be related to block size adaptation. Try setting
framesPerBuffer to paFramesPerBufferUnspecified, 0, when you open the
stream.
No, that didn't help, either.
Post by Phil Burk
Also note that the callback is passed a set of flags.
http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#a55a005924bcfa0424594f4f65cd4ae82
Are any of those being set?
Yes, I get paOutputUnderflow (value 4) with my paproblem.c once it opens
an input channel.
Post by Phil Burk
If the output is not primed properly then there can definitely be
issues with output underruns.
Ok, how does one prime the output properly?

Is there some example code to look at for this?

Kind regards,

Hanno
Hanno Zulla
2016-04-05 07:20:41 UTC
Permalink
Hi,
Post by Phil Burk
Also note that paPrimeOutputBuffersUsingStreamCallback may not be
fully implemented on ALSA.
Indeed, it isn't.
Post by Phil Burk
/* XXX: Ignore paPrimeOutputBuffersUsingStreamCallback
untill buffer priming is fully supported in pa_process.c */
Kind regards,

Hanno
Ross Bencina
2016-04-05 11:42:14 UTC
Permalink
Post by Hanno Zulla
Indeed, it isn't.
Post by Phil Burk
/* XXX: Ignore paPrimeOutputBuffersUsingStreamCallback
untill buffer priming is fully supported in pa_process.c */
Ross Bencina
2016-04-07 11:48:38 UTC
Permalink
Post by Phil Burk
/* XXX: Ignore paPrimeOutputBuffersUsingStreamCallback
untill buffer priming is fully supported in pa_process.c */
Hanno Zulla
2016-04-07 13:25:25 UTC
Permalink
Hi,
Post by Phil Burk
Thanks for the test program. I will try that when I get on my Linux machine.
I tried to look into the ALSA code of PortAudio, but franky, I'm already
much deeper into the guts of audio programming than expected on this
journey. However, I'd love to help out. Is there anything I can do to help?

Kind regards,

Hanno
Neil C Smith
2016-04-04 16:37:28 UTC
Permalink
Post by Hanno Zulla
- the program opens a stereo input channel on the default input device
Have you tried other sample rates? I see the code you sent defines
sample rate as 44.1kHz. Does 48kHz work? IIRC the default ALSA
device might be resampling to/from 48kHz.

Best wishes,

Neil
--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk
Ross Bencina
2016-04-05 04:20:32 UTC
Permalink
Post by Hanno Zulla
What can be done to fix this?
It may or may not help, but please make sure that you are testing
against the latest PortAudio SVN version, not whatever Debian is
distributing.
Hanno Zulla
2016-04-05 07:13:21 UTC
Permalink
Post by Ross Bencina
Post by Hanno Zulla
What can be done to fix this?
It may or may not help, but please make sure that you are testing
against the latest PortAudio SVN version, not whatever Debian is
distributing.
I have removed Debian/Ubuntu's version 19+svn20140130-1 on my system and
compiled & installed the current nightly tarball.

Problem remains the same.

Thanks,

Hanno
Alan Horstmann
2016-04-08 16:29:15 UTC
Permalink
Hi,
Post by Hanno Zulla
ok, sorry to flood the mailing list, but I have now found a fairly
minimal source example for you to reproduce the problem on your box,
without having to build a SuperCollider binary.
Thanks for that, it is a very helpful step.

I have tried that problem test code on my workstation here and it ran fine.
This is an older Suse Linux that pre-dates Pulse-audio altogether.

The Alsa config uses dmix for the default device, which has a mixing system
running at 48000 with a buffer fixed at 1024 frames. Thus the Alsa system
effectively puts a minimum on the buffer size of 21ms. At that level it is
well safe against glitching. Using instead a direct hw: pcm-device the
default latency here is 8ms which also does not glitch. I had to change the
requested latency to below 3ms to manifest problems, and this is the sort of
level I would expect on a reasonably setup Linux.

I do not know what the Alsa config is like on current 'Pulse-enabled'
desktops. But something is not right if glitching occurs with latencies
(==buffer sizes) in the order of 0.1 sec or more, as you are seeing.

Later I may be able to repeat testing on a recent Debian.

Regards

Alan
Alan Horstmann
2016-04-08 21:45:02 UTC
Permalink
Post by Alan Horstmann
Later I may be able to repeat testing on a recent Debian.
And did. Interestingly that *does* show the problem. The audio output is
just short bursts of 20, 40, or 60 ms approx separated by gaps mostly also
multiples of 20-21ms, with irregular size and spacing. With Portaudio debug
enabled it is showing "Input underflow" which perhaps suggests the capture is
not running? This is followed by handling Xruns and Alsa restart then
capture poll timeouts.

However, changing the sample rate to 48000, which is the native rate for dmix
(and hence for the default device) the issue does not occur. Anyway, 48000
is a better rate on these systems as it avoids resampling to the dmix rate.
So that is one possible workaround to try.

To test without Portaudio I tried:

arecord -traw -Ddefault -d1 -ffloat_le -c2 -r44100 test_float32.raw

On my working system that was fine, but on the Debian it continued
indefinitely and created hundreds of empty files! This is all with
Pulseaudio running. It would appear that there are wider issues with the
audio capture, but more tracking down is needed. This is just posting my
interim info.

Regards

Alan
Hanno Zulla
2016-04-08 21:56:41 UTC
Permalink
Thanks,
Post by Alan Horstmann
However, changing the sample rate to 48000, which is the native rate for dmix
(and hence for the default device) the issue does not occur. Anyway, 48000
is a better rate on these systems as it avoids resampling to the dmix rate.
So that is one possible workaround to try.
My target application will be Sonic Pi, using SuperCollider as its
synthesizer backend.

All sample files in Sonic Pi are 44100, so that is the desired sample
rate to be used by the system.

Kind regards,

Hanno
Alan Horstmann
2016-04-09 20:51:46 UTC
Permalink
Post by Hanno Zulla
Thanks,
Post by Alan Horstmann
However, changing the sample rate to 48000, which is the native rate for
dmix (and hence for the default device) the issue does not occur.
Anyway, 48000 is a better rate on these systems as it avoids resampling
to the dmix rate. So that is one possible workaround to try.
My target application will be Sonic Pi, using SuperCollider as its
synthesizer backend.
All sample files in Sonic Pi are 44100, so that is the desired sample
rate to be used by the system.
At this stage the primary goal is to understand the problem. If is only at
non-dmix rates, that implies different issues to occurring at all rates.
Each test or experiment gives clues about the perpetrator!

As an alternative, try changing '/usr/share/alsa/alsa.conf' where it has a
line like:

defaults.pcm.dmix_rate 48000

to 44100 and see if it is any different. (Probably have to restart Alsa.)
Post by Hanno Zulla
That's why I mentioned this the other day. I've seen that issue about
dmix mentioned a lot, and have experienced it myself. That was
without PortAudio too - I have a feeling there's nothing to be done on
the PortAudio side that could address it.
Can you say any more about those issues? I tend also to the suspicion that
the cause is outside Portaudio.

Regards

Alan
Neil C Smith
2016-04-11 08:49:25 UTC
Permalink
Hi Alan,
Post by Alan Horstmann
Post by Neil C Smith
That's why I mentioned this the other day. I've seen that issue about
dmix mentioned a lot, and have experienced it myself.
Can you say any more about those issues? I tend also to the suspicion that
the cause is outside Portaudio.
Nothing much to add really, except that it replicated the OP's issue -
connecting through dmix using a non-default sample rate causing
glitchy audio when doing input and output, although output only often
OK. It's like the input and output resampling doesn't quite sync up.
IIRC it's possible to configure alternative resampling plugins for
dmix - don't know if that helps? I assume dmix does actually resample
input?!

None of that was PortAudio specific.

This is all a bit sketchy and a few years back now - my current
machine has Pulse enabled, and unlike the OP I've not had problems
between it and JACK for a long time.

Best wishes,

Neil
--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk
Hanno Zulla
2016-04-11 09:00:21 UTC
Permalink
Hi,
Post by Neil C Smith
This is all a bit sketchy and a few years back now - my current
machine has Pulse enabled, and unlike the OP I've not had problems
between it and JACK for a long time.
Yes, using PulseAudio as a JACK audio sink with PortAudio fixes this.

However, the Raspberry Pi ships without PulseAudio and I'd like to find
a solution that works out of the box with pure ALSA.

Kind regards,

Hanno
Neil C Smith
2016-04-11 09:19:38 UTC
Permalink
Hi Hanno,
Post by Hanno Zulla
However, the Raspberry Pi ships without PulseAudio and I'd like to find
a solution that works out of the box with pure ALSA.
Yes, sorry, I do understand that - it was more that I haven't
personally experienced this issue for a few years so info is a bit
sketchy.

Unless you can run SuperCollider at 48kHz or get the user to change
their dmix config to 44.1kHz, then I'd suggest your only pure ALSA
solution is to do what JACK does and use the HW device directly.

You may want to check this issue is still replicatable with a
non-PortAudio application just to be sure, if you haven't already?

Best wishes,

Neil
--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk
Neil C Smith
2016-04-08 22:01:38 UTC
Permalink
Hi,
Post by Alan Horstmann
However, changing the sample rate to 48000, which is the native rate for dmix
(and hence for the default device) the issue does not occur. Anyway, 48000
is a better rate on these systems as it avoids resampling to the dmix rate.
So that is one possible workaround to try.
That's why I mentioned this the other day. I've seen that issue about
dmix mentioned a lot, and have experienced it myself. That was
without PortAudio too - I have a feeling there's nothing to be done on
the PortAudio side that could address it.

Best wishes,

Neil
--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk
Phil Burk
2016-04-09 20:01:53 UTC
Permalink
Post by Hanno Zulla
All sample files in Sonic Pi are 44100, so that is the desired sample
rate to be used by the system.
If we can't get 44100 working because of these dmix issues then you may
want to do your own resampling from 44100 to 48000. Then run PortAudio at
48000.

Phil Burk
Post by Hanno Zulla
Hi,
Post by Alan Horstmann
However, changing the sample rate to 48000, which is the native rate for
dmix
Post by Alan Horstmann
(and hence for the default device) the issue does not occur. Anyway,
48000
Post by Alan Horstmann
is a better rate on these systems as it avoids resampling to the dmix
rate.
Post by Alan Horstmann
So that is one possible workaround to try.
That's why I mentioned this the other day. I've seen that issue about
dmix mentioned a lot, and have experienced it myself. That was
without PortAudio too - I have a feeling there's nothing to be done on
the PortAudio side that could address it.
Best wishes,
Neil
--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net
Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk
_______________________________________________
Portaudio mailing list
https://lists.columbia.edu/mailman/listinfo/portaudio
Hanno Zulla
2016-04-04 10:20:15 UTC
Permalink
Hi everyone,

thanks.

I'm not talking about the Raspberry Pi. The RPi is a target platform,
yes, but the problems I described happen on my desktop machines.
Post by Phil Burk
https://www.assembla.com/spaces/portaudio/wiki/Platforms_RaspberryPi
Note the bug at the bottom. The default low latency is too low and
causes glitches or "choppy audio".
Is this a problem with the Debian portaudio package? It's used on
Raspbian, but it also affects me on my Ubuntu box.

If it is, could it be patched there?


I have looked at the paex_sine sample source code and compared it to the
SuperCollider source, saw no obvious problems. The paex_sine code works
without buffer underruns and without choppy audio, SuperCollider does
things more or less the same way but audio is broken there.

As the only difference, SuperCollider uses paNonInterleaved, but even
when I modified paex_sine to do the same, the problem didn't appear there.

Kind regards,

Hanno
Hanno Zulla
2016-04-04 11:46:41 UTC
Permalink
Hi,
Post by Phil Burk
Note the bug at the bottom. The default low latency is too low and
causes glitches or "choppy audio".
The default latency is set here:

<https://github.com/hzulla/supercollider/blob/audiodriver/server/scsynth/SC_PortAudio.cpp#L361-L364>

I have changed suggestedLatencyIn and suggestedLatencyOut to
"defaultHighInputLatency", but still choppy audio on pure ALSA's default
device.
Post by Phil Burk
I increased the suggested latency in my test program to 0.030 seconds
and was able to get smooth audio. You can probably go lower but for live
coding you don't need very low latency.
Let me know if increasing the latency does not work.
Yes, increasing the latency did work. Smooth audio began around a
suggested latency value of 0.5, but there were still occasional audio
drops. At value 1.0, things were fine.

I really don't think 1.0 is a good value, though.

Kind regards,

Hanno
Loading...