Have you ever wondered how to transcode an audio file of mp3, wav and flac formats into USoundWave at runtime? If yes, then this article will be helpful for you. Let's start from the basics.
All the information provided is mainly theoretical in nature, inside of which there is still some practical knowledge, but not to the full extent required by Unreal Engine. The plugin that I made, resting on the specified information, is presented at this link. It is also available through the Marketplace.
USoundWave as the main sound object in Unreal Engine
The most powerful and perhaps most sensible way to interact sound with player actions is to use an object such as USoundWave. You do not need to manually calculate the sound frequency spectrum and amplitude at runtime in order to produce some events based on these parameters - Unreal Engine has a way how to do this much optimized. And Unreal Engine uses an object called USoundWave as sound, and as a result, this is the most digestible option for connecting sound with a scene. For more information on how to do this, see this site.
Method for transcoding audio files from device memory to USoundWave object
First, you need to decode the audio file into wave data and the size of that wave. This is a rather difficult process associated with close work with memory, which depends on the specific audio file format and codec taken. I recommend using any libraries for this, such as dr_libs, which make this process much easier. The next step is to create a static USoundWave object. After that, you need to fill in all the necessary variables and USoundWave buffers (such as audio length, sample rate, etc.). The only way to do this is to use the FWaveModInfo object, which has a function to read the sound wave from the sound wave data and its size that we already got. This function will automatically generate the necessary variables for the USoundWave object, after which the only thing left is to fill in the USoundWave with these variables. It also requires a little work with memory, but it is not as significant and complex as in the case of decoding an audio file into rough data.