Note: This tutorial was written for students at the Technical University of Munich attending the "Interaction Methods and Devices" course. Additionally, the tutorial can be used by anyone with some Unity experience.

Introduction

A key feature of game engines is the range of tools they offer to help developers create their games. However, developers often need specialized tools that are not included by default in the engine. This tutorial offers insights into tool development in Unity, demonstrating how to create a custom tool through a practical example. In this tutorial, we redevelop the Unbound Rotation Tool.

Create Your Own Unity Package

Before starting with our tool, we need to set up our folder structure correctly. Instead of developing our tool inside the Assets folder, we want to work directly in the Packages folder. That way, Unity interprets our code as a package and also shows errors that would occur if we installed the package from an outside source. Here's the first few steps for our package:

1. Navigate to your project's Packages folder in your file explorer, create a new folder and give it a meaningful name. Then navigate to the newly created folder.

2. Add a new file package.json to this folder. Then, look at the Unity Documentation regarding the layout and required information in the package manifest. Fill out at least the required and recommended properties. This file defines the folder as a package. A new package should now show up in your Unity editor with the name you gave the package in its manifest. If there are errors, Unity will show them to you.

3. Whenever we write editor code, we usually create an Editor folder and everything that is related to editor features is created inside of this folder. Editor scripts are linked via an assembly definition asset inside the Editor folder. For our package, create a new Editor folder and then create a new assembly definition asset inside of this folder. Make sure that the assembly definition is valid for editor code (see section further down on the website).

For now, these are the only required steps for our package. Make sure that whenever we add something you add it inside the Packages folder and not the Assets folder of your project.

Create an Editor Tool in Unity

In this chapter, we will implement the interface of our tool. We want to have our own icon in Unity's transform toolbar of the scene window that already contains other tools (Hand, Move, Rotate, etc.).

These are the steps to add your own icon to the Unity transform toolbar:

1. Make your own icon or select a suitable icon. The icon should be a .png file. It does not require a big resolution. We used a 16x16 image as seen in the screenshot. Save it inside the package and make note of its location.

2. Next, we create the script for our tool. Inside the Editor folder of our package, create a new C# script and give it a meaningful name.

3. Inside the script, delete all the dependencies of MonoBehavior (including the Start and Update method).

4. Then, inherit from the class EditorTool. We want our tool to be available for all objects, so we want a global tool. Add the attribute EditorTool with your tool name as its attribute. You can find how to do so here. As soon as this works, you should see a new button in the toolbar of the scene window.

5. Now, we want to add our own icon to the toolbar. The EditorTool class provides a public property toolbarIcon which we will override. This property is accessed frequently. Therefore, we will load and unload the icon in the OnEnable and OnDisable methods. The Unity documentation's sample code provides a good baseline. When setting the icon in the OnEnable method, we want to use our custom icon. Here you can find more information on loading textures inside a package. Load the texture and set the toolbarIcon to our icon.

After these steps, the scene toolbar should show your new icon and you can select the new tool via this icon. For even more functionality, let us add a shortcut for our tool as well.

1. Add a new static method to your class and add the ShortcutAttribute to this method. With this attribute, we register a new shortcut to the Unity editor and when the specified key(s) are pressed, this method will be executed. The ShortcutAttribute could look like this: [Shortcut("Activate Arcball Selection Tool", KeyCode.U, ShortcutModifiers.Control)]

2. The code inside the method is responsible for setting our tool as the currently active tool. The ToolManager is responsible for that kind of logic. Find the right method and call it in the newly created method.

Now, whenever the user presses the shortcut key(s), Unity should automatically select our tool.

Note: It might be the case that when you first use the shortcut, Unity recognizes a conflict if you chose a key-combination that is already a registered shortcut in Unity. You can always change all keybinds later on in Edit > Project Settings > Input.

Implementing the Arcball Behavior

The implementation of our rotation tool follows the Arcball method as described in the lecture and by (Shoemake, 1992). However, we made a slight adaptation: unlike the default Arcball implementation, which is fixed to a static camera or modifies only the camera's orientation, our version adjusts the object's orientation directly as seen from the scene camera.

Setup

• The initial mouse position on the 2D screen when pressing the left mouse button.

• The 3-dimensional vectors v0 and v1.

• A boolean indicating whether we are currently rotating or not.

Implementing the Arcball Logic

When the mouse is being dragged, our tool rotates the selected GameObject. This is the main part of the implementation and just like the Arcball implementation in the lecture, we will put the logic in three methods: Calculate3DVectorFromGaussianImage, CalculateRotation, and RotateViaArcball.

The first method calculates our 3D vector from the gaussian image (i.e., our two mouse positions). The method should have the following signature: public Vector3 Calculate3DVectorFromGaussianImage(Vector2 newMousePosition). Here's what we need to do in that method:

1. We need to calculate the delta of our current mouse position and the initial mouse position. For the x direction, we invert the result (otherwise, our tool rotates in the wrong direction).

2. Calculate the length of that position change.

3. The yaw based on the mouse position equals the arctan of the delta on the y axis to the delta on the x axis. You can use the Atan2 method of the Mathf class for this.

4. For the pitch, we take the arccosine of the length calculated in step 2 and divide it by a number representing the maximal length. The number defines how much change in mouse position we allow before the value of the arccosine becomes invalid (remember that the arccosine is only defined in the range of [-1, 1]). You can define the max value based on Screen.height and Screen.width.

5. For the z coordinate of our 3D vector, we take the sine of the pitch.

6. Before calculating the x and y coordinate, we calculate a scaling factor. The factor equals the cosine of the pitch.

7. The x and y coordinate of our 3D vector equal the cosine and sine of the yaw, respectively. Scale the result with the scaling factor calculated in step 6.

8. Before we can return the vector, we need to apply the camera rotation of the current scene view that is being drawn. Look here or use your IDE’s IntelliSense to find that rotation.

9. The resulting 3D vector is our return value.

Now we can map a 2D mouse movement to a 3D vector. Next, we want to calculate our rotation based on the Arcball method. For this we need a helper method that calculates a rotation. This method returns a Quaternion rotation based on two 3D input vectors (our v0 and v1). The method should have the following signature: public Quaternion CalculateRotation(Vector3 v0, Vector3 v1).

1. Calculate the rotation angle. This is the dot product between the two input vectors.

2. We define the rotation axis as the cross product of the two input vectors. After normalizing this cross product, you need to apply the inverse rotation of the target’s GameObject's Transform. You can access the target via the target property of EditorTool. You can cast the target to a GameObject - we will introduce the required sanity checks in the OnToolGUI method later. Remember to apply the calculation in the correct order: var rotaxis = Quaternion.Inverse(((GameObject)target).transform.rotation) * Vector3.Cross(v0, v1).normalized;.

3. We want our return value to equal the rotation of the value we got in step 1 around the rotation axis we got in step 2. The Quaternion class provides the AngleAxis method to do exactly that.

4. Return this value.

Last, we need a method that uses the other two methods we have already implemented to calculate the arcball rotation. We'll later call this method whenever the mouse is dragged. The method should have the following signature: public void RotateViaArcball(Vector2 newMousePosition).

1. Set v0 to the old value of v1. Update v1 to the new value calculated via Calculate3DVectorFromGaussianImage (mapping mouse position to a 3D vector).

2. Get the new rotation change for the target by calculating the rotation with CalculateRotation(v0, v1). You also need to take into account the rotation of the target itself! Quaternion newRotation = ((GameObject)target).transform.rotation * CalculateRotation(v0, v1);

3. Apply the rotation to our target.

Putting It All Together

We now have all the puzzle pieces to our tool. In order for our tool to work, we need to put it all together. Our implementation reacts to three events: When the left mouse button is first pressed MouseDown, while dragging the mouse MouseDrag, and when the left mouse button is lifted MouseUp. We implement the described behavior in the OnToolGUI method. public override void OnToolGUI(EditorWindow window)

1. First, check whether the parameter window is actually a SceneView. If not, we do not want to execute any code, as our tool should only work in the SceneView, but not in the directory view, for instance. if (!(window is SceneView))

2. We do some sanity checks for the target: We only want to be able to rotate if the target is not null and it is a GameObject. if (target == null || !(target is GameObject))

3. Because we want to override the default mouse behavior of Unity when the tool is selected, we need to add a default control like this: HandleUtility.AddDefaultControl(GUIUtility.GetControlID(FocusType.Passive));

4. In total, we have three states that we need to take care of: MouseDown, MouseDrag, and MouseUp. Get the current event and set up a switch case statement with these three events. This includes the switch switch (Event.current.GetTypeForControl(controlID)) and the three cases case EventType.MouseDown:, case EventType.MouseDrag:, and case EventType.MouseUp:. For all of them, check that the left mouse button is the one that triggers the event. Remember additional checks (such as MouseDrag requiring that we are already rotating).

5. Implement the functionality for MouseDown, MouseDrag, and MouseUp:

   • MouseDown Behavior When the left mouse button is clicked, we need to (i) set that rotating has started, (ii) set the initial mouse position, and (iii) based on that initial mouse position calculate our 3D vector on the sphere and set it for both v0 and v1.

   • MouseUp Behavior When the left mouse button is lifted, we need to update our rotation indicator boolean to indicate that the rotation has stopped.
   • MouseDrag Behavior When the mouse is being dragged, our tool rotates the selected GameObject by invoking RotateViaArcball.

Now, our tool is ready to be used: When you select a GameObject in the scene view (preferably something with a Renderer so you can see the changes), select the tool, and then drag your mouse around, the selected object should rotate.

Tool Visuals

Although the tool’s functionality is fully implemented, adding visual feedback would enhance the user experience. To achieve this, we can draw a sphere around the object, similar to Unity’s rotation tool, and visually indicate its rotation. Use the Handles class and its built-in drawing methods.You can choose how to represent the sphere and rotation axes. This can be done by defining fixed lengths and radii or calculating them dynamically based on the object's size (e.g., using its Mesh Filter to determine dimensions).

Note: When adding new references to your scripts, you might encounter an error. This happens because the required package is a dependency that isn't included by default in your project. To resolve this, go to your assembly definition in Unity and add the necessary reference under Assembly Definition References. Once the reference is added, Unity will recompile your project.

Extending the Tool: Random Rotation Duplication

We aim to enhance our rotation tool by overriding the default duplication shortcut. Instead of simply duplicating the selected object, this feature will create a duplicate with a random rotation. You can choose which axis or axes to apply the random rotation. This demonstrates how to override default shortcuts and implement custom logic for specific tools. The functionality and code are adapted from this Reddit thread.

1. First, create a method that returns a random rotation around the specified axes. It is fine if you hard-code which axes you rotate around for this exercise, but you could also pass three booleans that determine for which axes a random rotation should be set.

2. Next, we write the method that actually does the duplication. In it, first check whether the currently selected active object is a GameObject. If not, we want to execute the default behavior of Unity. You can do so with this line of code: EditorApplication.ExecuteMenuItem("Edit/Duplicate");

   If the selected object is a GameObject, instantiate a new GameObject that is a copy of the selected object, except for the rotation. For the rotation, we want to use a random rotation.

   The basic functionality itself is implemented. However, we also want to mimic behavior of Unity's default duplication. This includes setting the currently active GameObject to the newly created GameObject as well as registering the action to the undo list. If not registered, we can not undo the duplication.

Next, in the OnToolGUI method, we want to catch all events that correspond to the Unity shortcut of duplication. For now, we'll just assume that the user did not change the default keybind for duplicating (Default: CTRL/CMD + D). So, when the current event's keys contains the keys required for the duplication, we want to execute our "override" of the duplication. After that, tell the EventSystem that the current event is used. Otherwise, the default duplication would be executed as well.

This completes our implementation of our Unbound Rotation tool. By zipping the package file, you can distribute and install it easily via the Package Manager.

Conclusion

This tutorial gave you a short insight into tool development. We implemented a custom arcball rotation tool.

Note: You can find an implementation of the tool above here. Also, try adding it via Git and the Package Manager. Also take note of the Licensing, Readme, and Changelog files. As mentioned in the exercise, these files are important when you want to share your own packages.

References

• Shoemake, K. (1992, May). ARCBALL: A user interface for specifying three-dimensional orientation using a mouse. In Graphics interface (Vol. 92, pp. 151-156).