Please help the improvement by saying your thoughts about this DEMO
1st Stage Implementation Test Methodology
In this post, I will tell you the method I’ve used to create the 3D effect and also the problems found on the way. A more technical detailed post will follow in the near future.
To create the 3D effect, the main manipulated elements are the level and time differences. To create the shadowing effect (head filtering), essential for sound localization, which effects also the intensity of the sound (level), it was necessary to manipulate the phase.
Because there are no multichannel recordings, only mono and stereo sounds, split into 16 virtual channels using the “ambix Encoder” plug-in, phasing becomes a big problem even for the mono files, because they are spread into 16 channels. When phasing occurs, all the stereo and 3D spatial image are distorted (only left side, or no spatial location at all).
At this stage, I’ve found one solution for this to be fixed. On the audio track, insert “Oculus Spatializer” for a more accurate sound control (2in/2out), followed by any processing needed (EQ, Reverb, Compression etc.) and the last one in the chain, the “ambix Encoder”(2in/16out). This chain ordering will give more accuracy for the sound to be placed in the 3D space, but not fix all the issues. For the sound to remain in the 3D space where is needed and to be perfect in phase, the azimuth(rotation) and elevation controls on the “ambix Encoder” have to be manipulated until the sound gets clear and stays where is needed in the 3D space (pure listening). Not the best solution, as it makes the automation more difficult, but a good one for this stage of the project (research in progress). After this stage, the sound it’s sent on the master and also on a Submix (BUS).
The BUS track
On the BUS channel, it was inserted the “ambix Converter” plug-in which converts the ambix format into Universal Ambisonics (UA) which means it makes it compatible with 2 channels media. This is also sent to the master track. At the mixing stage, the direct sounds from the ambix encoder, where mixed with the converted sounds, for a better 3D effect.
The Master track
On the master track, are two decoders. The first one is “ambix Binaural Decoder” which distributes the sound on 24 virtual speakers sphere (8 bottom, 8 middle, 8 top) to create the full 3D effect.
The second decoder is the “ATK FOA Decode Binaural” which emulates different HRTF’s (Head-Related Transfer Function) which is a binaural recording technique, using dummy heads or actual humans, with two omnidirectional microphones placed at ear level (they look like headphones) when record using a person, or inside the year when a dummy head it’s used. This will make the sound compatible with any headphones.
The audio track plug-in chain has to be inserted on every single track, and because multiple audio layering is required, of course, we end up with a lot of tracks.
And here it comes another issue: CPU Overload
I use a Windows PC with an Intel Core i7-3770 @ 3.4GHz (4 cores, 8 threads) with 8GB of RAM. Not the best computer in the world, but still it’s not getting overloaded so easy.
The CPU “gave up” at the stage were Reaper was doing video processing, encoding/decoding the audio and handling 151 VST plug-ins over 43 audio channels. I think it handled more than enough. The solution to this is to freeze the tracks that are already placed on the timeline and processed, and keep unfroze only the tracks that need processing/manipulation. It’s easy to go back and edit the frozen tracks by simply unfreeze them when needed.
Although the 3D audio effect was achieved using this method, I don’t think that is the best way to do it. The main reason being the processing needed, because of the number of plug-ins used. Another issue being the use of two plug-ins for sound placement in the 3D space.
Full technical details about this 1st method will be shown in a different post.
- Next target is to find a new way of implementing by keeping this method but reduce the number of VST’s or, find a better and more accurate way of implementation, but stick with the plan of using ONLY freeware plug-ins and software.
- Work on generating a more immersive and bigger 3D effect by trying to emulate the Dolby Atmos Technology.