Using Fmod Doppler on Unity without a rigidbody

As I mentioned on this article, Fmod includes a built-in doppler feature but unfortunately it requires two things in order to work on Unity:

• A rigidbody on the game object where the emitter is.

• The game object needs to be moving via the physics engine, like for example using Rigidbody.AddForce or just using gravity.

This limitation has prevented me from really using the doppler effect since, on the games I have worked on, we usually move objects by code or using animations which means that Unity’s physics engine has no clue what the object’s velocity is, so in turn, Fmod doesn’t know either.

Additionally, we would also need to know the listener’s velocity, since in nature the doppler effect always takes into consideration the relative velocity between the listener (you and your ears driving in a car) and the emitter (another car’s horn).

So let’s jump into Unity and try to find a solution for this problem. I took most of the code from this thread on the Fmod forums.
Note: this work was done on the following versions so as time goes by, it could get more and more outdated:

• Fmod Studio & Fmod Unity implementation plugin version: 2.02.04

• Unity version: 2020.3.19f1

You can get this project on GitHub so you can copy the code more easily or play around with it. Here is the zipped project in case you prefer that.

Using Doppler with a rigidbody and the physics engine

Before we do anything else, let’s see how the system was intended to be used by default. As you can see in the video below, I have a very basic setup: I created a new, clean Unity project, integrated Fmod into it and created an Fmod project.

The Fmod project simply contains a 3D event with a looping testing sine wave tone. On the Unity project I just have a camera with the Fmod studio listener, a plane with no collisions and a cube. On the cube, I added a rigidbody and an Fmod emitter. As you can hear, at first we hear no doppler effect but activating the doppler feature on the event macro section and re-building banks makes the doppler effect work.

So now that we can see that Doppler is working when using gravity, let’s try switching the movement to just animation. As you can see, I disabled gravity, added an Animator to the cube and made an animation so it falls in a similar way as it does with the physics engine. You can clearly hear that there is no doppler in this case and we only get it back when we switch the gravity back on and disable the animator. So this is basically the problem we need to solve.

How Fmod receives velocity information

If we look at Fmod’s documentation, we can see that: “The Doppler property requires velocity in order to calculate how much Doppler adjustment needs to be applied. Velocity can be provided by the game engine or by calling EventInstance::set3DAttributes. In the case of the Unity and Unreal Engine integrations, this is usually set automatically to match the velocity of the associated Rigidbody or RigidBody.”

So this is how we are currently sending the velocity to Fmod, via the rigibody. But as the docs say, we can also provide this information manually which means we need to calculate the velocity ourselves but at least we don’t depend on the physics engine anymore. Let’s call this manually calculated velocity, “kinematic velocity”, since it doesn’t care about the forces applied, just the movement itself.

Let’s adapt the default Fmod emitter. I personally don’t use it since I have my own emitters but the default one is a good example to learn what’s going on.

Modifying StudioEventEmitter

First, let’s think about when we want to use kinematic velocity. In theory, everytime there is no rigidbody, we want to potentially use it but it would be a waste to do it if the event in Fmod is not even using doppler. On the other hand, sometimes we will have a rigidbody and we still want to use kinematic velocity since the object could be moved by just animation or code.

First, I thought about reading the event description to see if the event in question had doppler activated and turn on kinematic velocity based on that. The problem with this approach is that sometimes we may have doppler turned off, but we still want to send kinematic velocity information to Fmod to use it as a parameter for other things.

So my solution was to add a new option to StudioEventEmitter where you can set if you want to calculate kinematic velocity. This would be independent of having a rigidbody and also independent of the event having doppler activated. So firstly, let’s add a new bool variable to the public settings within StudioEventEmitter:

Since this class uses a custom made editor, we need to modify the editor class too:

As you can see, the class now has an additional option on the inspector.

View fullsize

We then declare a couple of variables on StudioEventEmmiter that we need to calculate the game object’s velocity:

As you can see, I’m using a VelocityVector3 class that is new to both Unity and Fmod. Let’s create this class on RuntimeUtils:

Next, let’s create a method to calculate the velocity on a given frame and update our velocity related variables:

On Update(), we want to call the previous method, but we only do it if we are using kinematic velocity for this particular event:

We are almost finished with StudioEventEmitter. The last thing we want to do is make sure that we attach the Fmod instance to the gameobject in a slightly different way when we want to use kinematic velocity. This is done within the PlayInstance() method.

So we modify the code to be:

As you can see, we are making sure we only attach the rigidbody if it exists AND we don’t want to use kinematic velocity. Otherwise, we use our new kinematic velocity. Here we are using a new overload on AttachInstanceToGameObject() that we don’t have yet. We will fix this in the next section.

RuntimeManager

This script takes care of updating things at runtime. We need to do a few modifications for our kinematic velocity to work. We first add two new variables to the AttachedInstance class. We do this so we can keep track of the velocities without a rigidbody.

Next, we create the overload we were missing before. As you can see, this attaches the Fmod instance to the game object and uses our kinematic velocity class instead of a rigidbody.

We now need to make sure we are updating the values on each frame. As mentioned earlier, this is accomplished by setting the 3D attribute by hand. So we look for the Update() method and we modify it like so:

Essentially, we are making sure we only update the attributes based on the rigidbody when it exists AND we are not using kinetic velocity. Otherwise, we update the values including kinetic velocity.

Notice that to be able to do this we need to overload the To3DAttributes() method on RuntimeUtils like so:

StudioListener

The last thing we need to do is also modify the Fmod listener. We need this since the Doppler effect is always based on the relative velocity of the emitter and listener. So we need to tweak StudioListener (or you could also create a child from it) to be able to measure its kinematic velocity.

We first add the same variables we added to StudioEventEmiiter. We also make sure we initialize the kinematic velocity variable if no rigidbody is found. You may need to modify this if your listener lives on a game object that has a rigidbody and you still want to use kinematic velocity.

We now copy the same method we used before for calculating speed in real time and we make sure we call it each frame if we are using kinematic speed.

The last thing we need to do here is to modify SetListenerLocation() so we can also use kinematic velocity as an option:

To make this work, we also need to overload SetListenerLocation() on the RuntimeManager:

And that’s all the code work finished!

Final Results

If everything went well, we should now be able to hear the doppler effect with no rigidbody and just moving the object via animations. As you can see, Fmod can now also get velocity values, which could also be useful to modify the audio based on them.

Success! These modifications could be done in many different ways depending on your needs so take my take as inspiration. If this doesn’t work for you or you have questions, feel free to contact me.

Remember that you can get this project on GitHub to play around with. Here is the zipped project in case you prefer that.