Code snippet of the audio API and wrapper
C++ is quite the language. Obtuse, unforgiving, fast, powerful. Its chunkiness makes me want to avoid it in most cases, but there's some cases where using it is almost mandatory.
One such case, and the topic of this blog, is audio processing. A typical audio stream runs at 44100 samples per second, per audio channel, with 16 bit depth, in stereo audio. That's a bitrate of roughly 172 kB/s, which is fairly high. on higher end setups, it can get towards 281, or even higher. And, in order to process that much data in real time without significant delays... you pretty much need a language like C(++). Sure, you could use a scripting language to process that, but you'd be spending a huge amount of the system's processing time on it, and add in further latency.
Miniaudio is a pretty good library, created by David Reid. It's multiplatform, open source, and provides direct PCM sample access. Basically, it removes the need to bother with annoying OS-specific audio code. This is quite nice when you're a bit of an idiot like me, and find Windows.h to be a nightmare. So used the library i did. Of course, you can't just call C++ code from a scripting language, first you have to build an API around it. but, we'll get there.
The first step to continuous audio playback is to set up a buffer for both input and output. This smoothes over timing inconsistencies, and means you don't have to be running all of the time. For this, a ring buffer is used. Ring buffers are interesting, occupying a fixed memory space that is cycled through with each write and read. They're incredibly fast and memory efficient, which is perfect for audio. Another benefit is that running behind means reading old data, which in most cases is undesirable, but in audio, will loop the currently playing sound, resulting in a less jarring mode of failure.
Ok, now that you have buffers, and some functions to push and pull data from them, you can now start thinking about making the API. The Lua API is... a bit clunky in some ways, but perfectly servicable, and you only need a couple of functions anyways. In all i made 5. One to pull microphone data, one to clear the playback buffer, one to modify the playback buffer, one to set the output volume, and one to check the buffer sample position. using these 5, a fairly robust and simple playback system can be made.
But now what? what do you do with all this audio routing? Well simple. send it over a network to players. All you have to do is create another packet type and send it over to the server and boom, you can send voice data to other players. Oh, but if only it were that simple. Because now with a network, you have to deal with packet loss and misorderings, not to mention latency to boot. So, for this we use... can you guess? another buffer! - but not a ring buffer. just a standard array buffer. This is because a ring buffer would not be very useful in this case, since the main issue isn't speed but rather reliability, and an ordered array lets us be a lot more structured than a ring.
This network-layer buffer seeks to do a few things. handle missing packets, reorder packets for correct playback, and maintain reliable timing with unreliable inputs. The buffer can accomplish most of this fairly well. The array is filled with packets in the correct order when they are received, and during each playback tick, the first element of the array is removed, and the other elements shifted forwards. If there is no avalible packet, the last good one will be repeated up to 3 times. And, if the incoming packet timing is too far ahead, it'll force jump the array forward to avoid latency drifts.
And there you go, now you've implemented voice chat in your game. As easy as that. And a project i've had on the backburner for too long will include it. Of course, the audio engine in that does not run at 44100hz, for both speed and networking reasons. instead, the quality has been significantly cut down, to 8000hz 8bit mono. This sounds pretty crunchy, but in a good way in my opinion, kinda like the xbox live lobbies of old. Such is the beautiy of limitations, they can be used for creative purposes.