Hi,

Thanks for using The Machinery. We’re excited to share with the world what we have cooked up and happy to have you among the people that are trying out the engine. This book is here to give you a little bit of background and information about what you’re looking at.

Besides this book we have several other resources which might be good to checkout:

Also feel free to checkout our internal

The purpose of this Programming Guidebook is to lay down principles and guidelines for how to write code and work together at Our Machinery.

Enjoy!

The Machinery Team

Introduction to the Engine

What is The Machinery?

The Machinery is a framework for building different kinds of 3D software: editors, tools, pipeline components, games, visualizations, simulations, toys, experiments, etc. You can think of it as a game engine, but it’s intended use stretches beyond games, covering a wide range of applications. What makes The Machinery special is that it is lightweight and completely plugin-based. That means that you are not limited to a single editor and runtime. Rather, you can mix and match components as you need (or write your own) to create your own unique experience. The Machinery can also be stripped down and run embedded, as part of a larger application.

License

There are three different licenses that you can use for The Machinery:

  • The Indie Free license is completely free for indie developers and includes all of The Machinery, except for the source code.

  • The Indie Pro license is exactly the same, but also includes the full source code of the engine and all associated tools. For this version we charge $100 $50 / user / year.

  • The Business version is aimed towards professional users. It includes the same stuff as the Indie Pro version except your support tickets will be prioritized. It sells at $900 $450 / user / year.

We define as an “indie”, any company who’s yearly revenue or funding is less than $100K/year. Until the end of the year 2021, we're offering all the paid licenses at 50 % off. If you buy a paid license now, you will keep this low price for the next five years.

All of the licenses allow for full royalty-free use of The Machinery. You can build and sell commercial games, tools, applications, and plugins. For more details, see our pricing page.

When you download The Machinery, you will be on the Indie Free license. If you do not fall in the "indie" category, you can still use this license to evaluate The Machinery, but you need to buy a business license to put it into production.

Glossary

The following list provides the most common terms used in The Machinery.

TermDescription
The TruthThe Truth is a system that holds an authoritative data state and allows it to be shared between various systems. In particular, it supports the following features:

- Objects with properties (bools, ints, floats, strings).
- Buffer properties for holding big chunks of binary data.
- Subobjects (objects of other types belonging to this object).
- References to other objects.
- Sets of subobjects and references.
- Prototypes and inheritance.
- Change notifications.
- Safe multi-threaded access.
- Undo/redo operations (from multiple streams).

More information is available in the documentation.
Truth AspectAn “aspect” is an interface (struct of function pointers) identified by a unique identifier. The Truth allows you to associate aspects with object types. This lets you extend The Truth with new functionality.
Creation GraphThe creation graph is (as the name implies) a graph based tool for creating assets. It allows developers to define and alter various types of assets using visual scripting. More broadly speaking, the creation graph can be used for arbitrary data processing tasks. More
Creation Graph GPU NodesCreation Graphs contain nodes that can execute on both the CPU and GPU. The GPU nodes usually become part of a shader program when the creation graph is compiled. There are nodes to transition from the CPU to GPU.
Entity GraphThe Machinery’s Visual Scripting Language for game play. More
PrototypeOur prototype system allows entity assets to be used inside other entities. Therefore, you can create an entity asset that represents a room and then create a house entity that has a bunch of these room entities placed into it. For more information on Prototypes, check out its Prototypes. (Also called Prefabs in other Engines)
Properties TabWhenever an object gets focused in the asset browser or entity tree, properties of it can be edited from the properties tab.
Asset BrowserShows all assets in your project in an explorer-style fashion. An asset can be thought of as the equivalent of a file in the OS. Some assets such as entities can be dragged and dropped from the asset browser into the scene tab.
Graph Tab/ViewA tab that can either represent an Entity Graph or a Creation Graph.
Entity TreeHierarchical view of the Entity currently being edited.
In The Machinery, every entity has a parent except for the root entity of the world. (which is generally called “world”).
This Tab Allows the user to quickly reason and re-organize the hierarchy by adding, removing, expanding and reordering entities.
Simulation EntryIs a generic concept to provide functionality for the game play to start, stop and update (tick). This concept consists of two components: The Simulation Entry Interface (tm_simulation_entry_i) as simulation entry component. The interface is there to provide functionality, while the component is the connection with the simulation.
Engine (ECS)An engine is an update function that runs on all entities that have certain components. (Jobified)
System (ECS)A system is an update function that runs on the entire entity context.
Entity ContextIt’s the core runtime Entity-Component-System of The Machinery.
It’s main two components are:

- Entities (see Entity)
- Systems and Engines
EntityIn The Machinery a game world is built out of different entities which are defined through different components. Those components create the building blocks of each entity and give them their meaning.
ComponentsA building block for entities. Moreover a component is a type of data that can be associated with an entity.
DCC AssetDigital Content Creation Assets. Any type of assets created by any digital content creation tool such as Blender or Maya.
PluginThe Machinery is built around a plugin model. All features, even the built-in ones, are provided through plugins. You can extend The Machinery by writing your own plugins. When The Machinery launches, it loads all the plugins named tm_*.dll in its plugins/ folder. If you write your own plugins, name them so that they start with tm_ and put them in this folder, they will be loaded together with the built-in plugins.
Plugin assetsA special type of asset that contains the library (executable code) for a plugin. This makes it possible to add compiled plugins without have to copy the DLL into a specific folder, instead you can just drag n drop DLLs into the project. The user will be asked for consent to run the code in the plugin.
ProjectIt is a self-contained directory or database file that holds all the game data used during the development.
tmsl.tmsl stands for The Machinery Shader Language and is essentially a data-driven json front-end to create a tm_shader_declaration_o. The tm_shader_declaration_o can be anything from a complete shader configuration (all needed shader stages, any states and input/output it needs, etc) that can be compiled and used when rendering a draw call (or dispatching a compute job), to just fragments of that. Typically a .tmsl file only contains fragments of what's needed and our shader compiler combines a bunch of them to build a functional shader.

Troubleshooting

This section addresses common problems that can arise when using The Machinery.

Windows 10 Editor

When it comes to crashes on Windows which you cannot debug yourself. You can enable a full crash dumb via the following file utils/enable-full-dumps.reg. The dumps can be found in the folder AppData\Local\The Machinery and then in the CrashDumps folder. In case of an error report it can be very helpful to provide access to the crash dump. You can submit bugs on our public GitHub issues page. Please do not forget to mention your current Engine version.

In order to obtain log files you have to go to the same folder where you can find the CrashDumps (AppData\Local\The Machinery) but they till instead be in the Logs subfolder.

Windows 11

We have not tested windows 11 yet therefore possible problems are to be expected. Please report them to us on the public GitHub issues page so we can keep track of them.

tmbuild cannot find build tools

In case of a non typical installation of Visual Studios you have to provide to tmbuild the correct environment variables: TM_VS2017_DIR or TM_VS2019_DIR. They need to point to the root directory of your Visual Studio installation.

tmbuild cannot find environment variables

Before we can build any project, we need to set up our environment. You need to set the following environment variable: (If this one has not been set the tool will not be able to build)

  • TM_SDK_DIR - This is the path to find the folder headers and the folder lib

If the following variable is not set, the tool will assume that you intend to use the current working directory:

  • TM_LIB_DIR - The folder which determines where to download and install all dependencies (besides the build environments)

Make sure you have added the needed environment variables. Follow this guide on tmbuild.

clang-format pollutes my git commits

Make sure that you are using the clang-format version the engine downloads for you and add to the TM_LIB_DIR. The engine uses the version 6.0. In case you are using visual studio or visual studio code make sure it points to the right executable!

Graphics

In case of a crash you will get an error message, this will give some information about the details of the crash. The first step in a Vulkan related crash is to update your graphics drivers. If this didn’t help then please report the issue to us with the following information.

  • The error message you got when the crash happened, this should include file information and a Vulkan error code, it’s vital to share these.
  • The log file, see the previous section on how to obtain this.
  • A crash dump file, see the previous section on how to obtain this.

You can submit bugs on our public GitHub issues page. Please do not forget to mention your current Engine version.

Where to report bugs or feedback

  1. If you have any problems running the software, encounter crashes, etc, report them on our public bug tracker as soon as possible. We will fix bugs as soon as we can and provide updated executables for download on the website.
  2. If you have other feedback or questions, post them on our forum. We appreciate candid, honest opinions.

Note that you need to create a separate login to log in to the forum. (We might unify the logins in the future.) You can also drop in on our Discord Server. You will frequently find us there, answering questions. We'll pay special attention on Thursdays.

Getting started

To run The Machinery you need:

  • A 64-bit Windows 10 machine with the latest Vulkan drivers
  • Or a 64-bit Ubuntu 20.04 Linux machine with the latest Vulkan drivers (ArchLinux should also work, no guarantees are made for other distros)
  • And an ourmachinery.com account. Sign up here!

On Linux, you also need to install the following packages for the runtime:

sudo apt-get install libxcb-ewmh2 libxcb-cursor0 libxcb-xrm0 unzip

This does not work on your distro? No problem, visit our Linux installation process across distributions guide.

Getting up and running

Quick steps to get up and running:

  1. Download The Machinery at https://ourmachinery.com/download.html.

  2. Sign up for an ourmachinery.com account here. (It's free!)

  3. Unzip the downloaded zip file to a location of your choosing.

  4. Run bin/the-machinery.exe in the downloaded folder to start The Machinery.

  5. Login with your ourmachinery.com account at the login screen and approve the EULA.

  6. To find some samples to play with go to Help > Download Sample Projects in the main menu.

  7. Pick one of the sample projects (for example Physics), click Get and then Open.

  8. Play around with it, try some other samples and read the rest of this document to find out what else you can do with The Machinery.

If you get errors that mention Vulkan or if you see weird rendering glitches, make sure to update your GPU drivers to the latest version. If that doesn't work, post an issue with our issue tracker or ping us on Discord and we will help you.

Related videos to these topics are:

Source Code access

You can make use of tmbuild (from the binary build) to download the engine (source code) and install all needed dependencies as well. This can be done via tmbuild --install in this case you may want to use --github-token as well and provide your token. As alternative you just clone the repo as you are used to from any other git repositry.

I signed up for source code but didn't get access.

Make sure your GitHub account is correctly entered on the Profile page. It should be your account name, not your email.

GitHub invites frequently end up in the Spam folder. Check there or go to the repository to see your invite.

What is in this package?

In the distribution package you will find this:

FolderContent
headers/
lib/
bin/The headers, libraries, and DLLs that make up The Machinery SDK.
bin/the‑machinery.exeThe Machinery's main editor.
doc/SDK documentation.
samples/Sample code that shows how to extend the editor and SDK with plugins, as well as how to build new executables on top of The Machinery.
code/For reference: code for our utility programs.
bin/simple‑draw.exeA simple drawing program built on top of the SDK.
bin/simple-3d.exeA simple 3D viewport built on top of the SDK.
bin/tmbuild.exeOur build tool, that can be used to build samples and plugins.
bin/docgen.exeOur tool for generating documentation.
bin/hash.exeOur tool for generating static hash strings.
bin/localize.exeOur tool for generating localization data.
bin/runner.exeAn executable than can load a The Machinery project and do what Simulate Tab does, but stand-alone. It is copied whenever you Publish a project from within The Machinery.
utils/.clang-formatThe clang-format settings we use to format our code.
utils/pre-commitThe pre-commit hook we use in our git repository to auto-format the code.
utils/enable-full-dumps.regA registry file that enables full crash dumps (see below).

You can use the Download tab inside The Machinery to download sample projects for the engine.

Here's a list of the sample projects that are available:

ProjectDescription
AnimationA sample project that features an animated character.
Creation GraphsSample use of creation graphs.
Gameplay First PersonA sample first-person game.
Gameplay Interaction SystemA sample first-person game with intractable entities.
Gameplay Third PersonA sample third-person game.
Modular Dungeon KitA sample modular project that lets you compose dungeon scenes out of modular components.
PhysicsA sample project that demonstrates the use of physics.
PongA sample visual scripting and gameplay project.
Ray Tracing: Hello TriangleA sample project showing how to use the ray tracing APIs.
SoundA sample project demonstrating sound playback.
All Sample ProjectsA zip containing all sample projects.

The All Sample Projects download contains all these projects, and also some sample engine plugins that can get you started with extending the engine.

Note that some of the content in these projects was created by other people and licensed under Creative Common or other licenses. See the *-license.txt files in the projects for attribution and license information.

The editor that is included allows you to:

  • Import various types of DCC assets (FBX, GLTF, etc).
  • Create simple entities/scenes by placing and arranging assets.
  • Add scripted behaviors to entities using the visual scripting language in the Entity Graphs.
  • Add physics collision to objects and modify their physical properties.
  • Import animations and create advanced animation setups as an Animation State Machine.
  • Import WAV files and play them or place them on entities.
  • Run and simulate these behaviors using the Simulate tab.
  • Extend the engine and the editor with your own plugins that implement new editor tabs, entity components, gameplay code, etc.
  • Write your own applications, using The Machinery as an SDK.

Logging in

When you first run the-machinery.exe, you will encounter a login screen:

Login screen.

To use the editor, you must log in with an Our Machinery account. If you haven't done so already, press the Sign Up button to create an account, or go directly to https://ourmachinery.com/sign-up.html.

In addition, you also need to agree to our EULA.

Project Setup

Let us talk about the concept behind projects first. In The Machinery, we have two kinds of projects: A database project and a directory project. It is possible to save a database project as a directory project and vice versa.

Note: If resaving a Database project as a Directory project or vice versa, be aware that these are two different projects. Hence changes to one will not apply to the other.

The Directory Project

A directory project saves all your assets, etc., in a specified project folder. This file format is better suited for source control.

view of a directory project in the Windows explorer

The Asset database Project

The difference between both is that a database project results in one file rather than multiple files. It has the file ending .the_machinery_db. This database will contain all your assets.

database project in the file explorer

Project Management

Can I directly add Assets to my project from the File Explorer of my OS?

No, the editor will not import assets directly added to the project folder via your OS File Explorer. However you can modify The Machinery files (in a directory project) during runtime or before. If you do this the Engine will warn you.

Changes to the project on disk where detected: [Import] [Ignore]

More information on this topic here.

You handle the main project management steps through the File Menu. Such as Create and Save. By default, (1) The Machinery will save your project as a directory project. However you can save your current project as an Asset Database (2).

Possible folder structure for a project

The following project shows a possible folder structure of a game project. This is not a recommendation just a suggestion. At the end it depends on your needs and your workflows how your projects should be structured.

  1. game_project contains a The Machinery Directory Project
  2. plugins contains the dll of your plugins (should not be checked in into source control)
  3. raw_assets may contain the raw assets your DCC tool needs to process. Your the Machinery project can point here and you maybe able to just reimport things from there if needed
  4. src contains the source code of your plugins. They can be in multiple sub folder depending on your liking and need.

Example src folder

In here we have one single premake file and a single libs.json as well as the libs folder. This allows you to run tmbuild just in this folder and all plugins or the ones you want to build can be built at once. Besides it will generate one solution for Visual Studio. In this example all plugins will copy their .dll/so files into the ../plugins folder.

A possible .gitignore

plugins/*
src/libs/*

as well as the default Visual Studio, Visual Studio Code and C/C++ gitignore content.

Getting Started with a New Project

This walkthrough shows how to create a new project. It will also show you what comes by default with the Engine.

This part will cover the following topics:

  • Project Pipeline
    • What is the difference between a directory project and a database project?
    • What is a Scene in The Machinery?
    • What comes by default with a project?

Table of Content

About Scenes

In The Machinery, we do not have a concept of scenes in the traditional sense. All we have are Entities. Therefore any Entity can function as a scene. All you need is to add child entities to a parent Entity. The Editor will remember the last opened Entity. When publishing a Game, the Engine will ask you to select your "world" entity. You can decide to choose any of your entities as "world" Entity.

For more information on publishing, check here.

New Project

After the Machinery has been launched for the first time and the login was successful, the Engine will automatically show you a new empty project. If you do not have an account, you can create an account for free here, or if you had any trouble, don't hesitate to get in touch with us here.

At any other point in time, you can create a new project via Files → New Project.

A new project is not empty. It comes with the so-called "Core," a collection of valuable assets. They allow you to start your project quickly. Besides the Core, the project will also contain a default World Entity, functioning as your current scene. By default, the world entity includes 2 child entities: light and post_process. Those child entities are instances of prototypes.

A prototype in The Machinery is an entity that has been saved as an Asset. Prototypes are indicated by yellow text in the Entity Tree View.

For more information about Prototypes, click here.

the content of the world entity

The Core

Let us discuss the Core a bit. As mentioned before, the Core contains a couple of useful utilities. They are illustrating the core concepts of the Engine, and they are a great starting point. They can be found in the Asset browser under the core folder:

A content overview of the core folder and its subfolders may looks like this:

  • A light entity

  • A camera entity

  • A post-processing stack entity (named post-process in the world entity)

  • A default light environment entity

  • A post-processing volume entity

  • A default world entity (the blueprint of the world entity)

  • A bunch of helpful geometry entities. They are in the geometry folder.

    • A sphere entity
    • A box entity
    • A plane entity
    • as well as their geometry material
  • A bunch of default creation graphs is in the folder creation_graphs

    • import-image
    • DCC-mesh
    • editor-icon
    • drop-image
    • DCC-material
    • DCC-image

How to add some life to your project

All gameplay can be written in C or a C in any binding language such as C++ or Zig. You can also create gameplay code via the Entity Graph. The Entity Graph would live inside of a Graph Component. You can add that to an Entity.

You can find more information about gameplay coding in the "Gameplay Coding" Section

Project Structure Recommendation

It is recommended to separate your gameplay source code, plugin code from the actual project and store them in a separate folder.

my_project/game_project // the directory project
my_project/game_plugins // the main folder for all your gameplay code, plugins

First Gameplay Project

This walkthrough shows how you make a simple scene via the Entity Graph.

This part will cover the following topics:

  • How to create with the visual scripting language (The Entity Graph) a simple playable

Table of Content

How to add some life to your project

Let us create a new project and than add some gameplay to it. Before this, let us define a goal: This part aims to create a plane and a cube that we can move with W on this plane.

It requires multiple steps:

  • Add a plane to the Scene
  • Add a box to the Scene
  • Add a camera
  • Make the box and the plane physical so the box cannot fall through
  • Add an Entity graph for the movement

The first one is to add a plane to our Scene. As we remember in Core in the folder geometry, we have a plane entity. All we need to do is drag it into the Scene and scale it a bit up.

Open the core/geometry folder and add the plane.entity to the scene by dragging it into the scene

The next step is as straightforward as the first step. We repeat what we did before and drag and drop the box from the geometry folder into the scene tab. After this, we can see what happens in the simulation. To start the simulation and open the simulation tab, you need to click on the "play" button.

The moment the simulation tab opens, the simulation starts. As you might have guessed, nothing happens. You can navigate in the tab by using the AWSD keys if no custom camera is selected. Also, we can pause, reset the current simulation or increase the simulation speed.

Pause, reset and increase simulation speed

Let us go back to the scene tab and add some more exciting things. The next point of our task list is to add physics. To make our entities aware of physics, we need to add the Physics Shape and Physics Body Component to the corresponding entities. In this case, the plane should have the Physics shape component that we set as a plane shape, and the box should have the physics shape component and the physics body component. Adding a component to an entity can be done either through right-click on and then Add Component via the Entity tree or the property tab and the Add component button. It is also possible to select an entity in the Entity Tree and use Space.

box entity plane entity

To visualize the box shape or the plane shape, we can use the visualization menu in either the scene or simulation tab.

When the simulation starts, the box - if placed above the plane - will fall on the plane. Isn't this already more exciting?

Add some movement

The next step is to make the box move. There are two approaches we could do a physicals-based movement or just a placement change. We are starting with the simpler option: Placement Change.

We need to add Graph Component to the box entity. (We could also add it to any other entity or the world entity, but we add it to the box for simplicity and organization's sake add it to the box).

Add Menu opened via right click “Add Component”

When the Graph component has been added, all we need is to double-click the Component, and the Graph Editor opens.

The Graph Editor View is empty. We can add new nodes via pressing Space or right-click a new node. It opens the node adding a menu. In there, we can search for nodes.

The Entity Graph is an event-based Visual Scripting Language. It means events will trigger actions.

There are three main events we should use for a particular type of action:

  • Init Event - This gets called when the Graph component gets initialized. The perfect place to set up variables etc.
  • Tick Event - For all actions that need to happen on Tick (Be aware that this might influence your game performance if you do expensive things always on tick)
  • Terminate Event - For all actions that need to happen when the Component gets removed. It mainly happens when the Entity gets destroyed.

You can also create custom events and listen to them. You can define those events in the graph itself or in C.

Our first iteration will use changing the placement of our box whenever we use the key W. The Machinery input system is based on polling rather than on events. We can check if key was pressed via the "Key Poll" node needs to be checked every frame. Therefore a tick event is a need. The "Key Poll" node requires the specify the key and returns a Boolean (true/false) if a key has been pressed, is down, released, or up. It leads to the following setup:

In this setup on tick, it's being checked if the key "w" is down. If that is the case, the graph will execute the logic.

What is the actual logic? The actual logic is

  1. to get the Entity
  2. get its transform
  3. get the components of the transform (split the vector in x,y,z)
  4. manipulate the x value
  5. set the new transform to the Entity

In the Entity Graph, any entity of the current Scene can be referenced by name. It happens via the "Scene Entity" node.

If no name is provided, it will return the graph components Entity.

The return type is an Entity. When we drag the wire into the void, the Add Node menu will suggest only nodes which take the Entity as input:

With expanded Entity Category

We should connect the entity wire with a Get Transform node to get the world transform. Then the transform position should be split into its components. After this, component X should be modified by adding 10 * delta time.

To get the delta time, all that is needed is to add the delta time node and multiple its float value with 10.

When all this is complete, we should set the position of the Entity to the new value. First, a new position vector needs to be constructed by adding the old values to it and then the new modified value.

The next step is to tell the Entity to use this position instead of the old one. This happens through the Set Transform node.

Note that we leave all the other things as they were. It means they remain as before the change.

If we simulate the Scene now, the box moves, but it won't fall into the void if we move it to beyond the plane. It was to be expected because the physics system has not been updated, and therefore the velocity has not changed.

Add physics-based movement

To make the box move with the physics system, we can use the node "PhysX Push":

The node will move the box by applying velocity to it.

What is next?

If you are more interested in physics, download the physics sample via the download tab in that project, you will learn more about the use of physics in the Engine.

Sample Projects

You can download any sample project you like from our website: OurMachinery: Samples. Alternatively you find them under Help > Download Sample Projects in the main menu. After you have downloaded them you can open them via the Download Tab or directly from the hard drive.

A few sample projects are coming by default with the Engine, you find them in the root folder under samples/. If you are interested in how our tools work you can look their source code up in the code/ folder.

The Machinery for Unity Dev's

When migrating from Unity to The Machinery, there are a few things that are different.

Table of Content

Quick Glossary

The following table contains common Unity terms on the left and their The Machinery equivalents (or rough equivalent) on the right.

UnityThe Machinery
GameObjectsAre composed of Entities and Components and Systems
PrefabsPrototypes
Materials, Shaders, Textures, Particle Effects, Mesh, Geometry, Shader Graph, Material EditorCreation Graphs
UIUI
Hierarchy PanelEntity Tree
InspectorProperties Tab
Project BrowserAsset Browser
Scene ViewScene Tab
ProgrammingProgramming
BoltEntity Graph
C#C

UI Differences

Unity

The Machinery

  1. The Main Menu: It allows you to navigate through the Engine, such as opening new tabs or import assets
  2. The Entity Tree shows a tree view of the entity you are editing. It shows the entity's components and child entities. You start editing an entity by double-clicking it in the asset browser.
  3. The Scene shows an editable graphic view of the edited entity. You can manipulate components and child entities by selecting them. Use the Move, Rotate, and Scale gizmos for your desired action.
  4. The Simulate current scene button will open the Simulate tab that lets you "run" or "simulate" a scene.
  5. The Properties tab shows the properties of the currently selected object in the Scene. You can modify the properties by editing them in the properties window.
  6. The Console tab shows diagnostic messages from the application.
  7. The Asset Browser shows all the assets in the project and enables you to manage them.
  8. The Preview shows a preview of the currently selected asset in the asset browser.

The Editor is a collection of editing Tabs, each with its specific purpose. You can drag tabs around to rearrange them. When you drag them out of the window, a new window opens. Use the View menu to open new tabs.

Questions you might have

Where are my GameObjects?

The Machinery has no concept of GameObjects in the sense as Unity does. The Engine is based around Entities and Components. In the game world, not the editor, everything lives within the Entity Component System (ECS). To be exact, it lives within the Entity Context, an isolated world of entities. Your GameObjects are split into data and Behaviour.

You would usually couple your logic together with data in your C# MonoBehaviour scripts. In The Machinery, you separated them into Components and Systems / Engines. They represent your data and Systems or Engines that represent your Behaviour. They operate on multiple entities at the same time. Each Entity Context (the isolated world of Entities) has several systems/engines registered to them.

What is the difference between a System and an Engine?

An Engine update is running on a subset of components that possess some set of components. Some entity component systems are referred to as systems instead, but we choose Engine because it is less ambiguous.

On the other hand, a system is an update function that runs on the entire entity context. Therefore you can not filter for specific components.

Where are my Prefabs?

What Unity calls Prefabs is more or less what we call Prototypes. Our prototype system allows entity assets to be used inside other entities. Therefore, you can create an entity asset that represents a room and then creates a house entity that has a bunch of these room entities placed into it. For more information on Prototypes, check out its Prototypes.

How do I script?

The Machinery supports two ways of gameplay coding by default:

  1. using our Visual Scripting Language (Entity Graph)
  2. using our C API's to create your gameplay code. This way, you can create your Systems/Engines to handle your gameplay.

You do not like C? Do not worry! You can use C++, Zig, Rust, or any other language that binds to C.

Where are my Materials, Shaders, Textures, Particle Effects?

All of these can be represented via the Creation Graphs.

Project data?

The Machinery supports two types of Project formats:

  1. The Directory Project (Default)

A Source control and human-friendly project format in which your project is stored on Disk in separate files (text and binary for binary data)

  1. The Database Project

A single binary file project. It will contain all your assets and data. This format is mainly used at the end to pack your data for the shipping/publishing process.

Where do I put my assets?

At this point in time, you can only drag & drop your assets via the Asset Browser as well as via the Import Menu. See more in the section about importing assets. How to import assets

Import difference between Unity and The Machinery: .dcc_asset

When importing assets created in e.g. Maya they will be imported as dcc_asset. A dcc_asset can hold all types of data that was used to build the asset in the DCC-tool, such as objects, images, materials, geometry and animation data.

During the import step, The Machinery only runs a bare minimum of data processing, just enough so that we can display a visual representation of the asset in the Preview tab. Imports run in the background so you can continue to work uninterrupted. When the import finishes the asset will show up in the Asset Browser.

Note that import of large assets can take a significant amount of time. You can monitor the progress of the import operation in the status bar.

For more information see How to import assets or checkout our Example Workflow for Importing an Asset and create an Entity

What are common file formats supported?

Asset TypeSupported Formats
3D.fbx, .obj, .gltf
Texture.png, .jpeg, .bmp ,.tga, .dds
Sound.wav

Our importer is based on Assimp. Therefore we support most things assimp supports. (We do not support .blend files)

See the Import Chapter for more details

Where do my source code files go?

In the Machinery, all we care about is your plugins. Therefore if you want your plugins (tm_ prefixed shared libs.) to be globally accessible, please store them in the /plugins folder of the Engine. An alternative approach is to create plugin_asset in the Engine then your plugin becomes part of your project.

Please check out the introduction to the Plugin System as well as the Guide about Plugin Assets.

Using Visual Scripting

Visual Scripting is a perfect solution for in-game logic flow (simple) and sequencing of actions. It is a great system for artists, designers, and visually oriented programmers. It is important to keep in mind that the Visual Scripting language comes with an overhead that you would not pay in C (or any other Language you may use for your gameplay code).

The Machinery for Unreal 4 Dev's

When migrating from Unreal Engine 4 (UE4) to The Machinery, there are a few things that are different.

Table of Content

Quick Glossary

The following table contains common UE4 terms on the left and their The Machinery equivalents (or rough equivalent) on the right.

UE4The Machinery
Actor, PawnAre composed of Entities and Components and Systems
Blueprint Class (only the inheritance aspect and that they can represent actors)Prototypes
Material Instance, Shaders, Textures, Particle Effects, Static Mesh, Geometry,Skeletal Mesh, Material EditorCreation Graphs
UIUI
World OutlinerEntity Tree
Details PanelProperties Tab
Content BrowserAsset Browser
ViewportScene Tab
ProgrammingProgramming
BlueprintsEntity Graph
C++C

Questions you might have

Where are my Actors?

The Machinery has no concept of Actors in the sense as UE4 does. The Engine is based around Entities and Components. In the game world, not the editor, everything lives within the Entity Component System (ECS). To be exact, it lives within the Entity Context, an isolated world of entities. Your Actors are split into data and Behaviour.

You would usually couple your logic together with data in your Actor classes. In The Machinery, you separated them into Components and Systems / Engines. Different behaviour can be achieved via composition rather than via Inheritance.

Components represent data while Systems or Engines represent your behaviour. They operate on multiple entities at the same time. Each Entity Context (the isolated world of Entities) has several systems/engines registered to them.

Where are my Blueprints?

The Machinery supports two ways of gameplay coding by default:

  1. using our Visual Scripting Language (Entity Graph)
  2. using our C API's to create your gameplay code. This way, you can create your Systems/Engines to handle your gameplay.

You do not like C? Do not worry! You can use C++, Zig, Rust, or any other language that binds to C.

What is the difference between a System and an Engine?

An Engine update is running on a subset of components that possess some set of components. Some entity component systems are referred to as systems instead, but we choose Engine because it is less ambiguous.

On the other hand, a system is an update function that runs on the entire entity context. Therefore you can not filter for specific components.

Where are my Material Instance, Shaders, Textures, Particle Effects, Static Mesh, Geometry, Skeletal Mesh, Material Editor?

All of these can be represented via the the Creation Graphs.

Project data?

The Machinery supports two types of Project formats:

  1. The Directory Project (Default)

A Source control and human-friendly project format in which your project is stored on Disk in separate files (text and binary for binary data)

  1. The Database Project

A single binary file project. It will contain all your assets and data. This format is mainly used at the end to pack your data for the shipping/publishing process.

Where do I put my assets?

At this point in time, you can only drag & drop your assets via the Asset Browser as well as via the Import Menu. See more in the section about importing assets. How to import assets

What are common file formats supported?

Asset TypeSupported Formats
3D.fbx, .obj, .gltf
Texture.png, .jpeg, .bmp ,.tga, .dds
Sound.wav

Our importer is based on Assimp. Therefore we support most things assimp supports. (We do not support .blend files)

Where do my source code files go?

In the Machinery, all we care about is your plugins. Therefore if you want your plugins (tm_ prefixed shared libs.) to be globally accessible, please store them in the /plugins folder of the Engine. An alternative approach is to create plugin_asset in the Engine then your plugin becomes part of your project.

Please check out the introduction to the Plugin System as well as the Guide about Plugin Assets.

Using Visual Scripting

Visual Scripting is a perfect solution for in-game logic flow (simple) and sequencing of actions. It is a great system for artists, designers, and visually oriented programmers. It is important to keep in mind that the Visual Scripting language comes with an overhead that you would not pay in C (or any other Language you may use for your gameplay code).

The Machinery for Godot Dev's

When migrating from Godot to The Machinery, there are a few things that are different.

Table of Content

Quick Glossary

The following table contains common Godot terms on the left and their The Machinery equivalents (or rough equivalent) on the right.

GodotThe Machinery
NodesAre composed of Entities and Components and Systems
Materials, Shaders, Textures, Particle Effects, Mesh, Geometry, Shader Graph, Material EditorCreation Graphs
UIUI
SceneEntity Tree
InspectorProperties Tab
FileSystemAsset Browser
ViewportScene Tab
ProgrammingProgramming
VisualScriptEntity Graph
C#, GDScript, C++C

Questions you might have

Where are my Nodes?

The Machinery has no concept of Nodes in the sense as Godot does. The Engine is based around Entities and Components.

Everything within the Game World lives within the Entity Component System (ECS). To be exact, it lives within the Entity Context, an isolated world of entities. Your Nodes are split into data and behaviour.

You would usually couple your logic together with data when working in Godot. This coupling happens because Godot uses the Object-oriented approach while The Machinery uses the Data-Oriented approach. Hence you would inherit classes to compose different kinds of behaviour.

In The Machinery, you separated them into Components and Systems / Engines. They represent your data and Systems or Engines that represent your Behaviour. They operate on multiple entities at the same time. Each Entity Context (the isolated world of Entities) has several systems/engines registered to them. In the following image we broke down an example player in Godot into separate parts (components) and its methods into separate Engines.

Important to understand is that the Player Class on the left does not equal the Entity on the left! Since the Entity on the left is just a weak reference to the components, it does not own the data, unlike the Player Class. The Components together form the player and the Systems/Engines on the far right just consume a few of those components, they do not need to understand them all!

This setup allows you to compose entities that are reusing the same engines/systems! For example all your other entities that can move can use the Movement Engine and Jump Engine to get the jump function. All you need to do is compose entities with the:

  • Transform Component
  • Physics Mover Component
  • Jump Component
  • Movement Component

The Movement Engine and Jump Engine will pick them up and apply the same logic to them!

What is the difference between a System and an Engine?

An Engine update is running on a subset of components that possess some set of components. Some entity component systems are referred to as systems instead, but we choose Engine because it is less ambiguous.

On the other hand, a system is an update function that runs on the entire entity context. Therefore you can not filter for specific components. For more information see the chapter about the Entity Component System.

How do I script?

The Machinery supports two ways of gameplay coding by default:

  1. using our Visual Scripting Language (Entity Graph)
  2. using our C APIs to create your gameplay code. This way, you can create your Systems/Engines to handle your gameplay.

You do not like C? Do not worry! You can use C++, Zig, Rust, or any other language that binds to C.

Where are my Materials, Shaders, Textures, Particle Effects?

All of these can be represented via the Creation Graphs.

Project data?

The Machinery supports two types of Project formats:

  1. The Directory Project (Default)

Source control and human-friendly project format in which your project is stored on Disk in separate files (text and binary for binary data)

  1. The Database Project

A single binary file project. It will contain all your assets and data. This format is mainly used at the end to pack your data for the shipping/publishing process.

Where do I put my assets?

At this point in time, you can only drag & drop your assets via the Asset Browser as well as via the Import Menu. See more in the section about importing assets. How to import assets

What are common file formats supported?

Asset TypeSupported Formats
3D.fbx, .obj, .gltf
Texture.png, .jpeg, .bmp ,.tga, .dds
Sound.wav

Our importer is based on Assimp. Therefore we support most things assimp supports. (We do not support .blend files)

Where do my source code files go?

In the Machinery, all we care about is your plugins. Therefore if you want your plugins (tm_ prefixed shared libs.) to be globally accessible, please store them in the /plugins folder of the Engine. An alternative approach is to create plugin_asset in the Engine then your plugin becomes part of your project.

Please check out the introduction to the Plugin System as well as the Guide about Plugin Assets.

Using Visual Scripting

Visual Scripting is a perfect solution for in-game logic flow (simple) and sequencing of actions. It is a great system for artists, designers, and visually oriented programmers. It is important to keep in mind that the Visual Scripting language comes with an overhead that you would not pay in C (or any other Language you may use for your gameplay code).

Introduction into C Programming

This page will link to useful resources about the C Programming language.

Table of Content

Reference

Tutorials

Videos

The Machinery specials

These are things that differ from std C practices.

Memory Management

While programming plugins for the Machinery, you will encounter the need to allocate things on the heap or generally speaking. In standard C code, you might tend to use malloc, free or realloc. Since we try to be as allocator aware as possible, we pass allocators actively down to systems. This means that wherever you need a long-life allocator (such as malloc), we give a tm_allocator_i object down. This allows you to allocate memory like you would with malloc. Like in std C you need to free the memory allocated via a tm_allocator_i at the end of its use. Otherwise you may leak. Using our built-in allocators gives you the benefits of automatic leak detection at the end of your program. Since all allocations are registered and analyzed at the end of the application, you will be notified if there is a leak.

Note: more about leak detection and memory usage check the chapter about the Memory Usage Tab

Child Allocators

In case you check our projects you will find that we are making extensive use of child allocators. This allows us to log the use of their memory in our Memory Usage Tab.

In case you check our Write a custom Tab example you will find in its create function this code:

static tm_tab_i* tab__create(tm_tab_create_context_t* context, tm_ui_o *ui)
{
    tm_allocator_i allocator = tm_allocator_api->create_child(context->allocator, "my tab");
    uint64_t* id = context->id;

    static tm_the_machinery_tab_vt* vt = 0;
    if (!vt)
        vt = tm_global_api_registry->get(TM_CUSTOM_TAB_VT_NAME);

    tm_tab_o* tab = tm_alloc(&allocator, sizeof(tm_tab_o));
    *tab = (tm_tab_o){
        .tm_tab_i = {
            .vt = (tm_tab_vt*)vt,
            .inst = (tm_tab_o*)tab,
            .root_id = *id,
        },
        .allocator = allocator,
    };

    return &tab->tm_tab_i;
}

static void tab__destroy(tm_tab_o* tab)
{
    tm_allocator_i a = tab->allocator;
    tm_free(&a, tab, sizeof(*tab));
    tm_allocator_api->destroy_child(&a);
}

The tm_tab_create_context_t context gives you access to the system allocator. This one again allows you to create a child allocator. We can now use the new allocator in our tab. This is why we store it within our tm_tab_o object. In the end, we need to destroy our tab and, therefore free the tab object and destroy the child allocator. tm_allocator_api->destroy_child(&a); If we now shut down the engine and we forgot to free any of the allocations done between create and destroy, we will get a nice log:

D:\git\themachinery\plugins\my_tab\my_tab.c(100): error leaked 1 allocations 4112 bytes
D:\git\themachinery\foundation\memory_tracker.c(120): error: Allocation scope `application` has allocations

Rule of the thumb

Like in std C any allocation done with tm_alloc or tm_alloc_at and tm_realloc should be followed by a tm_free!

Temporary Allocator

Sometimes we need to allocate data within a function. Sadly we do have access to the default allocator or do not want to give access to an allocator. Do not worry. The temp allocator comes to the rescue and the frame allocator. These are two concepts for quick allocations that you can forget about because the memory will free them at the end of the function or the frame!

How to use the temp allocator?

The temp allocator is part of the foundation and lives in its header file: foundation/temp_allocator.h Some APIs require temp allocators as the input. They will use them to allocate data that is needed for processing. At first, we need to create an object by using the following macro: TM_INIT_TEMP_ALLOCATOR(ta)

A temp allocator is created and can be used now! Importantly do not forget to call TM_SHUTDOWN_TEMP_ALLOCATOR(ta) at the end; otherwise, you have a memory leak! Do not worry. You won't be able to compile without calling this function! Back to our example. Let's ask the Truth for all objects of a specific type:

TM_INIT_TEMP_ALLOCATOR(ta);
tm_tt_id_t* all_objects = tm_the_truth_api->all_objects_of_type(tt, type, ta);
// do some magic
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

The truth API will now use the temp allocator to create the list. We do not need to call tm_free anywhere, this is all done at the end by TM_SHUTDOWN_TEMP_ALLOCATOR!

Sometimes APIs require a normal tm_allocator_i, but you are not interested in creating an actual allocator or have access to a memory allocator! No worries, we have your back! The Temp Allocator gives you the following macro: TM_INIT_TEMP_ALLOCATOR_WITH_ADAPTER(ta, a); It generates a normal [tm_allocator_i](https://ourmachinery.com/apidoc/foundation/allocator.h.html#structtm_allocator_i) uses the temp allocator as its backing allocator. Hence all allocations done with the allocator will be actually done via the taallocator! Again at the end, all your memory is freed byTM_SHUTDOWN_TEMP_ALLOCATOR(ta);`

Note: You also have tm_temp_alloc() they expect a tm_temp_allocator_i instead of the tm_allocator_i.

What is the Frame Allocator?

The tm_temp_allocator_api has a frame allocator. A frame allocator allocates memory for one frame and then at the end of the frame the memory is wiped out. This means any allocation done with it will stay in memory until the end of the frame! The way of using it is: tm_temp_allocator_api.frame_alloc() or tm_frame_alloc()

Great use case: Formatted Strings

Both the frame as well as the temp allocator are great for using when you need to have a string with formatting! Infact the tm_temp_allocator_api provided an extra function for this: tm_temp_allocator_api.printf() /tm_temp_allocator_api.frame_printf()

Note: About formatting checkout the tm_sprintf_api or the logging chapter

Arrays, Vectors, Lists where is my std::vector<> or List<>

In the foundation we have a great header file or better inline header file the foundation/carray.inl which contains our solution for dynamic growing arrays, lists etc. When used you are responsible for its memory and have to free the used memory at the end! Do not worry, the API makes it quite easy.

Let us create an Array of tm_tt_id_t. All we need to do is declare our variable as a pointer of tm_tt_id_t.

tm_tt_id_t* our_ids = 0;

After this we can for example push / add some data to our array:

tm_carray_push(our_ids,my_id,my_allocator);

Now the my_id will be stored in the our_ids and allocated with my_allocator! In the end when I do not need my array anymore, I can call: tm_carray_free(our_ids,my_allocator), and my memory is freed!

This doesn't look very pleasant when I am working with a lot of data that only needs to be a temporary list or something. For this case, you can use our temp allocator! Every tm_carray_ macro has a tm_carray_temp_ equivalent.

Note: It is also recommend to make use of tm_carray_resize or tm_carray_temp_resize if you know how many elements your array might have. This will reduce the actual allocations.

Going back to our previous example:

TM_INIT_TEMP_ALLOCATOR(ta);
tm_tt_id_t* all_objects = tm_the_truth_api->all_objects_of_type(tt, type, ta);
// do some magic
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

tm_the_truth_api->all_objects_of_type actually returns a carray and you can operate on it with the normal C array methods: e.g. tm_carray_size() or tm_carray_end(). Since it is allocated with the temp allocator you can forget about the allocation at the end as long as you call TM_SHUTDOWN_TEMP_ALLOCATOR.

How to access an element?

You can access a carray element normally like you would access it in a plain c array:

TM_INIT_TEMP_ALLOCATOR(ta);
tm_tt_id_t* all_objects = tm_the_truth_api->all_objects_of_type(tt, type, ta);
if(all_objects[8].u64 == other_objects[8].u64){
    // what happens now?
}
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

Iterate over them

You can iterate over the carray like you would iterate over a normal array:

TM_INIT_TEMP_ALLOCATOR(ta);
tm_tt_id_t* all_objects = tm_the_truth_api->all_objects_of_type(tt, type, ta);
for(uint64_t i = 0;i < tm_carray_size(all_objects);++i){
    TM_LOG("%llu",all_objects[i].u64);// we could also use TM_LOG("%p{tm_tt_id_t}",&all_objects[i]);
}
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

An alternative approach is a more for each like approach:

TM_INIT_TEMP_ALLOCATOR(ta);
tm_tt_id_t* all_objects = tm_the_truth_api->all_objects_of_type(tt, type, ta);
for(tm_tt_id* id = all_objects;id != tm_carray_end(all_objects);++id){
    TM_LOG("%llu",id->u64);// we could also use TM_LOG("%p{tm_tt_id_t}",id);
}
TM_SHUTDOWN_TEMP_ALLOCATOR(ta);

The Editor

After opening the Engine, you should see the Editor's interface with menus along the top of the interface, and the basic tabs opened. The following image will show you the default engine layout. Here's a brief description of what you can see:

  1. The Main Menu: It allows you to navigate through the Engine, such as opening new tabs or import assets
  2. The Entity Tree shows a tree view of the entity you are editing. It shows the entity's components and child entities. You start editing an entity by double-clicking it in the asset browser.
  3. The Scene shows an editable graphic view of the edited entity. You can manipulate components and child entities by selecting them. Use the Move, Rotate, and Scale gizmos for your desired action.
  4. The Simulate current scene button will open the Simulate tab that lets you "run" or "simulate" a scene.
  5. The Properties tab shows the properties of the currently selected object in the Scene. You can modify the properties by editing them in the properties window.
  6. The Console tab shows diagnostic messages from the application.
  7. The Asset Browser shows all the assets in the project and enables you to manage them.
  8. The Preview shows a preview of the currently selected asset in the asset browser.

The Editor is a collection of editing Tabs, each with its specific purpose. You can drag tabs around to rearrange them. When you drag them out of the window, a new window opens. Use the View menu to open new tabs.

Note that you can have multiple tabs of the same type. For example, you can open various Asset Browser tabs to drag assets between them easily. Tabs docked in the same window work together. Therefore if you dock a Preview tab in the same window as an Asset Browser, it will show a preview of the selected asset in that browser. You can create multiple windows to view numerous assets simultaneously:

Editor layouts

You can rearrange the Editor layouts by dragging tabs around in the Editor. The best structure for the Editor depends on what you are doing and your personal preferences. You can save layouts via the Window Menu. It is also possible to restore your layout to the default one or load a custom-defined one in this menu.

About Tabs

The Machinery is based around a collection of editing Tabs, each with its specific purpose. You can drag the tabs around to rearrange them. Use the Tab menu to open new tabs. It is possible to have multiple tabs of the same type.

Table of Content

About Tab-Wells

Windows in The Machinery have a root tab-well covering the whole window. A tab-wells are rectangular areas containing one or more tabs. You can split them either horizontally or vertically to form two-child tab-wells. Also, you can switch around tabs within a tab-well via the keyboard using Ctrl + 1-9 or via Ctrl Page Up/Down.

Pinning tabs

You can also pin tabs to the current content or other settings with the pin icon.

It is also possible to use the context menu if you click on the tab label:

right click on the tab label will show this.

Pinning options

In the context menu, you have more options for pinning. These allow you to manage and arrange the window layout in a way that suits your workflow. The following table will show all the possible options:

OptionDescription
Pin via icon ⚲Will pin the tab to the current shown content
Pin to View 🗖This Pins the tab's content to another tab view that is currently open in the current window. You can pin a tab to multiple other tabs at the same time.
Pin to Window 🗗It pins the current tab to the selected window. For example, if you pin the Properties tab to Window 2. and choose an asset from the Asset Browser, the properties tab in Window 1. will display the selected asset.

Besides, it is possible to extend the Engine with custom tabs. You can do this via the File → New Plugin → Editor Tabs. How to write your custom tab is out of the scope of this article but is covered here.

Keyboard bindings

KeyDescription
Ctrl + TabSwitch between tabs
Ctrl + 1-9Switch between tabs in current tab well
Ctrl + Page up/DownSwitch between tabs in current tab well

Entity Tree Tab

The Entity Tree shows the hierarchy of the entity you choose to edit. This view allows you to organize and add new entities as well as new components to the entities.

Note: Just be aware that the Entity Tree does not reflect the current runtime state.

Besides, it is essential to remember is that The Machinery does not have specific Scene assets. A scene in The Machinery is just an entity with a lot of child entities.

Table of Content

How to edit a Scene / Entity

In this tab, you can edit any entity from the current project. To start editing, you can either use Right Click → Open or Double Click the entity to open them. This action will update the Scene Tab and the entity tree view tab. If you close the Scene Tab, the Entity Tree Tab will display an empty tree.

Managing Entities

A Scene is composed of child Entities and Parent Entities. When you create a new project, it starts with a "world" entity. This entity can be edited and lives as world.entity in your Asset Browser. You can create other Entities that function as your Scene. Any entity in the Asset Browser can serve as your Scene. Later, when publishing your game, you can choose the world entity you like.

Click a parent entity's drop-down arrow (on the left-hand side of its name) to show or hide its children. This action is also possible via the Keyboard: Arrow keys down and up let you navigate through the tree, while Arrow keys left and right allow you to show or hide the entity children.

Prototypes in the Entity Tree View

The Entity Tree view, prototype instances are shown in yellow to distinguish them from locally owned child entities (which are shown in white).

Prototypes: Prototypes are just entities that live in the Asset Browser as assets (.entity and the current entity in the Entity Tree View is based upon those entity assets. You can overwrite them in the Entity Tree View and make them unique, but any change to the asset will propagate to the entities based on the original prototype. More here

If you expand an instance, you will notice that most of its components and child entities are grayed out. They cannot be selected because they are inherited from the prototype, and the prototype controls their values.

The light entity is an instance of a prototype and the Render Component is inherited

If the prototype is modified — for example, if we scatter some more props on the floor — those changes reflect everywhere the prototype is placed.

Adding new child entities

When you want to reorder one entity as a child to another, you can drag and drop them on them onto each other.

You can add new children to an Entity through right-click and "Add Child Entity" or dragging an Entity Asset from the Asset Browser into the Entity Tree View.

Searching and changing the visibility of Entities

You can search for entities via the filter icon, and you can hide all the components with the little gear icon next to it. You can also change the visibility of each entity via the little eye icon next to its name. If you change the visibility, you will hide the entity in the Scene View.

The Lock Icon makes sure you cannot select the entity in the Scene Tab. It can help to avoid miss-clicking.

Keyboard bindings

KeyDescription
Arrow Up & DownNavigate through the tree
Arrow Left & RightExpand or collapse the selected Entities
FSelected Entity will be framed in Scene Tab
HSelected Entity will be hidden in Scene Tab
F2Start renaming the selected Entity
F3Adds child entity to parent
SpaceOpens Add Component menu to selected Entity
Space + ShiftOpen Add Entity Menu to add child entity
CTRL + LMoves selected entities one level up in the Entity Hierarchy
CTRL + FOpens the filter / search menu
CTRL + HDon’t show components in the Entity Tree
CTRL + DDuplicates selected Entities
CTRL + CCopies Entities
CTRL + VPastes Entities
CTRL + XCuts Entities

Scene Tab

The Scene tab is the view into your world. You can use the Scene tab to select, manipulate assets, and position new ones in your world.

Table of Content

The Scene Tab allows for different ways of navigating through your world. The primary method of navigating through the Scene is via the mouse and the keyboard.

Movement

  • Middle Mouse Button: Keep pressed down to move through the scene.
  • Left Mouse Button: Keep pressed down to rotate in the Scene by rotating the mouse. If you keep the mouse pressed so you can also use WASD to move through the Scene. To increase or decrease the movement speed, you need to move the mouse wheel.

Zoom in

  • Mouse Wheel: To zoom in, you can zoom in or out via the mouse wheel.

Frame Entities or the scene

  • Press F: To frame the currently selected entity or if you have nothing selected the Scene. Alternatively, you can double click on an entity the Entity Tree Tab.

Opening an Entity Asset

Through a double click on an Entity Asset in the Asset Browser, you will load the asset. If you want to move between previously loaded entities, the toolbar provides a back and forth navigation option.

Alternatively, you can use the context menu of the tab label and navigate through the previously focused entities.

Working in the Scene

The Scene tab comes with tools that allow for editing and moving entities in the current Scene.

The main tools you will be working with in to edit the scene are the

  • Select Tool: To select Entities in the Scene
  • Move Tool: For moving Entities in the Scene
  • Rotate Tool: Rotates selected Entities
  • Scale Tool: Scales Entities
  • Snapping: Enable or disable snapping and the snap distance

You can manipulate the Grid via Main Menu → Scene → Grid Settings.

If you do not like the layout of the current toolbar, you can change its layout by dragging them around.

Box Select in the Scene Tab

The Machinery now supports a long awaited feature — box selection.

To select multiple items in the scene, simply drag out a selection rectangle with the mouse:

Box dragging to select multiple entities.

The touched entities will become selected in the scene:

The resulting selection.

Simulate your Scene or change visualization modes

You can simulate your current Scene and manipulate the way your Scene's visualization with this toolbar:

  • The simulation button ▶: Simulates your scene in a new tab if no simulation tab is open.
  • The camera button 📷: Allows you to change the camera in your viewport.
  • The light button 💡: Use Lighting Environment Asset. Will create a lighting environment in the scene. Automatically enabled for assets with no light.
  • Visualize button: Allows to enable more visualization modes.
    • Lighting Model
      • Visualize Albedo
      • Visualize Normals
      • Visualize Specular
      • Visualize Roughness
      • Visualize Shadow Maps
      • Visualize Pixel Velocity
      • Visualize NaN / INF
      • Show as Overlay
    • Exposure
      • Visualize EV100
      • Visualize Histogram

Keyboard bindings

KeyDescription
FFrames either the current scene if nothing is selected or the selected objects
GEnables and disables the Grid
ESCDeselect objects
CTRL + DDuplicates selected objects
Shift + Drag with mouseDuplicates selected objects
CTRL + CCopies object
CTRL + VPastes object
CTRL + XCuts object

Properties Tab

The Properties tab shows the properties of the currently selected object. You can modify the properties by editing them in the properties window.

Use mathematical expression

We’ve also have support for mathematical expression to our property editor. So you can now type both numerical values and expressions.

You can use x in the expression to mean whatever value the property had before, so if you type x + 1 you will increase the current value by 1.

Using expressions in the property editor.

Multiple tabs with different properties

You can have multiple tabs of different properties open if you wish. In this case, it comes very handily that you can pin Properties Tabs to a specific Object. Otherwise, the property tab will reflect the next selected object, and you would have multiple times the same thing open. You can pin content by clicking the Pin icon on Properties Tab. It will bind the current object to this instance.

Pin the properties tab for the prop floor barrel (module dungeon kit example)

Moreover, if you dock a Preview tab in the same window as an Asset Browser, it will show a preview of the selected asset in that browser.

About Prototypes

The Machinery has a prototype system that allows entity assets to be used within each other. Therefore you can create an Entity-Asset that represents a room and then create a house Entity with a bunch of these room entities placed within. We call the room asset a prototype, and we call each placed room entity an instance of this prototype.

Any Entity-Asset can be a prototype, with instances of it placed in another entity asset. Note that prototypes are not special assets. More about this here.

The overridden entities and components are drawn in blue. We change the x and z components of the position to move the box. Note how the changed values are shown in white, while the values inherited from the prototype are shown in grey.

Missing

  • link somehow somewhere the prototype content

Asset Browser

The Asset Browser shows all your project's assets and enables you to manage them. It has multiple views at your exposal, which can make managing assets easier. The asset browser also allows you to search for assets in the current folder.

The asset browser’s grid view

As the image shows, the asset browser contains two components: The Directory Tree View of your project and the actual Asset View Panel.

Table of Content

Structure

The Directory Tree View reflects all directories and subdirectories of your project. You can use it to navigate quickly through your project. The navigation works either via mouse or via keyboard. You have the same management functionality as in the Asset View Panel:

  • Add a new folder or assets to a folder
  • Rename, Delete, Copy, Cut, Paste, Duplicate folder
  • Drag Drop folder and reorganize the folder structure
  • Show in explorer if the current project is a directory project
  • Copy the path if the current project is a directory
  • Change Views: Change to Grid, Details or List view
  • Change the Filter & Sorting

All of those actions can be either done via the Main Menu or via the context menu.

The Asset View Panel reflects all directories and sub directories as well as their assets in your project. This is the main place to organize your assets and folders. You have the same management functionality as in the Directory Tree View.

The Asset View Panel comes in three different views: Grid (default), Detail, and List-View. You can change the views via the context menu or the Change View button next to the search field. Besides, there are also shortcuts to change the views: Shift + C will switch to the Grid View, Shift + L will change to the List view, and Shift + D will change to the details view.

Detail View

Different View Modes

The Grid View will display your assets in a grid, which will shrink or grow depending on the tab size. In Details View allows you to see the details of your assets: What type are they easily? How big are they, and where on the disc are they located (if it's a directory project). The List-View will display your project's content in a list form.

About filtering, sorting, and changing the view There you have the option to filter via file extension or Asset-Labels. You can filter all assets by file extension type / Asset-Labels via the little filter icon or the context menu. You can also mix and match, as you require.

You can sort them via the context menu or the sort arrow up/down buttons (▲▼). Besides, there are also shortcuts to change the views: Shift + N will sort by Name Shift + S sort by Size, and Shift + T sort by type. The sorting will be applied in all view modes.

You can search in your project, and the default search is local. If you want to search globally, you want to click on the button with the Globe. This search will search for you in the entire project. Be aware your filters will influence your search!

Switch from local to global view

When searching globally, you can right-click on any search result and open the location of the selected asset.

Asset management

Editing specific assets Double clicks on an asset may open the asset in their corresponding tab or window. Not all assets have a specific action associated with them.

AssetAction on double click
Animation State Machine (.asm)Opens a new window with the Animation state machine layout.
Creation (.creation)Opens a graph tab if no graph tab is open. If a graph tab is open it will load this graph. Whenever we have a .creation file the graph is called Creation Graph and is used for working on graphics related workflows such as materials.
Entity (.entity)Opens a scene tab if no scene tab is open. If a scene tab is open it will change the view to this entity.
Entity Graph (.entity_graph)Opens a graph tab if no graph tab is open. If a graph tab is open it will load this graph. Whenever we have a .entity_graph file the graph is called Entity Graph and is used to make Entity Graph functionality reusable and shareable between multiple graphs.

A single click will always focus an associated Properties Tab on the selected asset.

Dragging assets into the Scene

You can drag assets around in the asset browser. It allows for quick reorganization of assets. It is also possible to drag assets from the asset browser directly into the Scene. Assets can be dragged from the Asset Browser Tab to other tabs if they support the asset type. You can also drag assets from the Windows-Explorer into your project. This action supports the same formats as the regular importer via File → Import Assets.

Asset Labelling

To organize your project, you can use Asset Labels. An asset can be associated with one or more different asset labels. You can use them to filter your asset in the asset browser or plugins via the asset label api.

There are two types of Asset labels:

  • System Asset Labels: They are added by plugins and cannot be assigned by the user to an asset.
    • User Asset Labels: You add them, and they are part of the project with the file extension: .asset_label and can be found in the asset_label directory.

The user can manage them like any other asset and delete user-defined Asset Labels via the asset browser. There you can also rename them, and this will automatically propagate to all users of those asset labels.

Add an Asset Label to as Asset

You can add asset Labels via the property view to any asset:

  • You select an Asset.
  • You expand the Label View in the Property View Tab.
  • You can type in any asset label name you want. The system will suggest already existing labels or allow you to create the new one.

1. Asset Label View in the Asset Property View Tab 2. The home of all your asset labels.

Keyboard bindings

KeyDescriptions
Arrow KeysAllow you to navigate through the Asset browser
EnterIf selected a folder or an asset will open the folder or asset
F2Will rename asset or folder in Asset View Panel and Directory Tree View
Ctrl + FSearch in the current project
CTRL + DDuplicates selected Asset
CTRL + CCopies Assets
CTRL + VPastes Assets
CTRL + XCuts Assets
Ctrl + Alt + OOpens Location of the asset in the asset browser view if in search view.
Ctrl + Shift + NNew folder
Ctrl + Shift + EOpen in Explorer
Shift + DChange to Details View
Shift + LChange to List View
Shift + CChange to Grid View
Shift + NSort by Name
Shift + SSort by Size
Shift + TSort by File Extension
Shift + OOpens a Directory or Asset

Simulate Tab

In The Machinery, we make a distinction between simulating and editing. When you are editing, you see a static view of the scene. (Editing the scene with everything moving around would be very tricky.) all the runtime behaviors like physics, animation, destruction, entity spawning, etc., are disabled. If you are building a game, the simulation mode will correspond to running the game. In contrast, when you are simulating or running, all the dynamic behaviors are enabled. It allows you to see the runtime behavior of your entities. To simulate a scene, open a scene in the Simulate tab.

Control over your simulation While your simulation is running, you can Stop, reset or speed up the simulation.

If your scene contains multiple cameras, you can pick between them via the camera toolbar.The default camera is a free flight camera.

In the same toolbar, you can enable Debug-Rendering-Tags from various components. For example, it will render a box around the Volume Component from the Volume Component if enabled.

Within this toolbar, you also find the statistic button to open several overlays, such as Frame Time.

Besides those options, you have the Render option, which allows for the same options as in the Scene Tab.

Preview Tab

The Preview Tab displays selected objects for you.

Camera Controls It allows for the same Free-Camera controls as the Scene Tab.

Movement

  • Middle Mouse Button: Keep pressed down to move through the Scene.
  • Left Mouse Button: Keep pressed down to rotate in the Scene by rotating the mouse. If you keep the mouse pressed so you can also use WASD to move through the Scene. To increase or decrease the movement speed, you need to move the mouse wheel.

Zoom in

  • Mouse Wheel: To zoom in, you can zoom in or out via the mouse wheel.

Interface Customizations

In this guide, you will learn about how to customize the Engine's interface.

  • Change a theme
  • Export/Import a theme
  • Change the window scale.
  • Add layouts.
  • Modify the Global Grid settings.

Change Theme

Sometimes we do not like the default theme or, based on reasons such as color blindness, and we cannot use the default theme. In the Machinery, you can change the default theme via Window -> Theme. You will find a list of themes the user can select and use.

The Engine comes by default with some base themes you can build your themes on top of:

Theme
Dark
Light
High Contrast Dark
High Contrast Light

Custom Theme

If you like to customize the default themes or create a new theme, click on the "New Theme" menu in the same menu as the theme selection. After clicking this, the current Theme will be used as your base, and the Theme Editor Tab opens.

If you do not like the base, you can choose a different theme as a base.

change base of theme

All changes are applied and saved directly. All changes will be immediately visible since your new Theme is selected as your current Theme.

changes are directly visible

Export / Import a theme

You can export a custom theme from the Window -> Theme and later import it there as well. The Theme will be saved as a .tm_theme file. These files are simple json like files.

Change the Scale of the UI

In the Window menu, you have a menu point Zoom.

zom option

This allows you to zoom in or out. You can also use the key bindings:

MeaningKeys
Zoom InCTRL + Equal, CTRL + Num + Plus
Zoom OutCTRL + Minus, CTRL + Num + Minus

Custom Layout

In case you do not like your current layout, you can always restore the default layout by using the Window -> Restore Default Layout menu point.

restore default layout

If you want to store your current layout, it would be very useful for the later time you can save your current window layout.

save current layout

You can create a new window or workspace with a layout in case you need it.

Note: The Engine should restore the last used layout when it shutdown.

If you need to change some details of your Window layout you can do this via the Edit layout menu.

edit layout

This will open the settings of the current Layout:

World Grid

In case you need to adjust the World Grid, you can do this at two places:

  1. Via the Application Settings

  1. Via any Scene or Preview Tab Application Menu entry.

Changes made there will only be applied to the specific tab.

Basic editing workflow

The basic scene editing workflow in The Machinery looks something like this:

  1. Import some asset files into the Asset Browser using File > Import… If you don't have any models to work with you can find free ones at Sketchfab for example.

  2. Organize the files by right-clicking the Asset Browser and choosing New > New Folder and by dragging and dropping.

  3. Any imported model, for example fbx or gltf, will appear as a dcc_asset.

  4. Rig an entity from the dcc_asset by selecting it and clicking Import Assets in the property panel. This also imports the materials and images inside the dcc_asset.

  5. Double click the imported entity to open it for editing.

  6. Add components for physics, animation, scripting, etc to the entity.

  7. Open a scene entity that you want to place your imported entity inside. A new project has a scene entity called world.entity that you can use. Or you can create your own scene entity by right clicking in the Asset Browser and choosing New > New Entity.

  8. Drag your asset entities into the scene entity to position them in the scene.

  9. Use the Move, Rotate, and Scale tools in the Scene tab to arrange the sub-entities.

  10. Holding down shift while using the move tools creates object clones.

  11. Select entities or components in the Entity Tree and Scene tabs to modify their properties using the Properties Tab.

  12. Drag and drop in the Entity Tree Tab to re-link entities.

  13. Each tool (Move, Rotate and Scale) has tool-specific settings, such as snapping and pivot point, in the upper-left corner of the scene tab.

  14. Use Scene > Frame Selection and Scene > Frame Scene to focus the camera on a specific selected entity, or the entire scene. Or just use the F key.

  15. When you are done, use File > Save Project… to save the scene for future work.

Known issues: When you drag a tab out of the current window to create a new one, you don’t get any visual feedback until the new window is created.

Entities

The Machinery uses an entity-component based approach as a flexible way of representing “objects” living in a “world”.

An entity is an independently existing object. An entity can have a number of components associated with it. These components provide the entity with specific functionality and features. The components currently available in The Machinery are:

ComponentDescription
Animation Simple PlayerPlays individual animations on the entity.
Animation State MachineAssigns an Animation State Machine to the entity which can be used to play and blend between animations.
CameraAdds a camera to the entity. The Scene > Camera menu option lets you view the scene through this camera.
Cubemap CaptureUsed to capture cubemaps to be used for rendering reflections.
Dcc AssetRenders an asset from a DCC tool. For more advanced rendering you would use the Render component.
Entity RiggerUsed to "rig" an imported DCC Asset as an entity. See below.
GraphImplements a visual scripting language that can be used to script entity behaviors without writing code.
LightAdds a light source to the entity.
Physics BodyRepresents a dynamic physics body. Entities that have this component will be affected by gravity.
Physics JointRepresents a physics joint, such as a hinge or a ball bearing.
Physics MoverRepresents a physics character controller. Used to move a character around the physics world.
Physics ShapeRepresents a physics collision shape. If the entity has a physics body, this will be a dynamic shape, otherwise a static shape.
RenderRenders models output by a Creation Graph.
Scene TreeRepresents a hierarchy of nodes/bones inside an entity. Entities with skeletons, such as characters have scene trees.
SculptComponent used to free-form sculpt with blocks.
Sound SourceComponent that will play a looping sound on the entity.
SpinSample component that spins the entity.
TagAssigns one or more "tags" to an entity, such as "bullet", "player", "enemy", etc. When implementing gameplay, we can query for all entities with a certain tag.
Tessellated PlaneSample component that draws a tessellated plane.
TransformRepresents the entity’s location in the world (position, scale, rotation). Any entity that exists at a specific location in the world should have a Transform Component. Note that you can have entities without a Transform Component. Such entities can for example be used to represent abstract logical concepts.
VelocityGives the entity a velocity that moves it through the world.

In addition to components, an entity can also have Child Entities. These are associated entities that are spawned together with the entity. Using child entities, you can create complex entities with deep hierarchies. For example, a house could be represented as an entity with doors and windows as child entities. The door could in turn have a handle as a child entity.

In The Machinery entity system, an entity can't have multiple components of the same type. So you can’t for example create an entity with two Transform Components. This is to avoid confusion because if you allowed multiple Transform Components it would be tricky to know which represented the “actual” transform of the entity.

In cases where you want multiple components (for example, you may want an entity with multiple lights) you have to solve it by creating child entities and have each child entity hold one of the lights.

Note that The Machinery does not have specific Scene assets. A scene in The Machinery is just an entity with a lot of child entities.

Import assets

This walkthrough shows how to import assets into a project and how to use them.

This part will cover the following topics:

  • How to import Assets into the Editor
  • How to import Assets into the Editor via url
  • Import Asset pipeline

Table of Content

Import differences between Meshes and Textures

Assets that are created by e.g. Maya will be of type after they are imported. Where DCC stands for Digital Content Creation. Materials wont be imported untill the dcc_asset has been dragged into the scene or used otherweise.

A dcc_asset can hold all types of data that was used to build the asset in the DCC-tool, such as objects, images, materials, geometry and animation data.

During the import step, The Machinery only runs a bare minimum of data processing, just enough so that we can display a visual representation of the asset in the Preview tab. Imports run in the background so you can continue to work uninterrupted. When the import finishes the asset will show up in the Asset Browser.

Note that import of large assets can take a significant amount of time. You can monitor the progress of the import operation in the status bar.

Textures on the the otherhand will be imported as creation graphs.

Major supported formats

At the time of writing this walkthrough, The Machinery is supporting the following formats:

Note: Not all formats have been tested that extensivly.

FormatFile Ending
fbx.fbx
GLTF Binary.glb
GLTF.gltf
wav.WAV
dae.dae
obj.obj
stl.stl
jpeg.jpeg
pg.pg
png.png
tga.tga
bmp.bmp
Windows Shared lib dll.dll
Linux shared lib so.so
The Machinery Theme.tm_theme
The Machinery Spritesheet.tm_spritesheet
The Machinery database project.the_machinery_db
The Machinery Directoy Project.the_machinery_dir
zip.zip
7z.7z
tar.tar

A complete list can be found on the bottom of this page

How to import assets into the project

The Machinery has three different ways of importing assets. The Import local files, Import remote files, Drag and Drop.

Import via the file menu

The first method of important an asset is via the File menu. There, we have an entry called Import File, which opens a file dialog. There you can import any of the supported file formats. Import File allows for importing any supported asset archive of the type zip or 7zip. This archive will be unpacked and recursively checked for supported assets.

Import from URL

It is possible to import assets from a remote location. In the File menu, the entry Import from URL allows for importing any supported asset archive of the type zip or 7zip. This archive will be unpacked and recursively checked for supported assets.

Note: The url import does not support implicitly provided archives or files such as https://myassetrepo.tld/assets/0fb778f1ef46ae4fab0c26a70df71b04 only clear file paths are supported. For example: https://myassetrepo.tld/assets/tower.zip

Drag and drop

The next method is to drag and drop either a zip/7zip archive into the asset browser or an asset of the supported type.

Adding the asset to our scene

In The Machinery a scene is composed of entities. The engine does not have a concept of scenes like other engines do. A dcc_asset that is dragged into the scene automatically extracts its materials and textures etc. into the surrounding folder and adds an entity with the correct mash etc. to the Entity view.

Another way of extracting the important information of a DCC asset is it to click on the DCC asset in the asset browser and click the button "Extract Assets" in the properties panel. This will exactly work like the previous method, but the main difference is that it creates a new entity asset that is not added to the scene.

Entity assets define a prototype in The Machinery. They are distinguished in the Entity Tree with yellow instead of white text. This concept allows having multiple versions of the same entity in the scene but they all change if the Prototype changes.

About Import Setting

You can define the import creation graph prototype there as well.

  • For Images
  • For Materials
  • For Meshes

Every DCC asset allows changing of the extraction configuration. Therefore it is possible to define the extraction locations for outputs, images and materials.

Instead of importing assets and change their the configuration per asset , it is possible to define them per folder. All you need to do is add an "Import Settings Asset" in the correct folder. This can be done via the asset browser. Right Click -> New -> Import Settings

Note: worth noting that this is somewhat of a power-user feature and not something you need to have a detailed understanding of to get started working with The Machinery.

Video about importing and creating an Entity

Complete list of supported file formats

At the time of writing this walkthrough, The Machinery is supporting the following formats:

Note: Not all formats have been tested that extensivly.

formatfile ending
3d.3d
3ds.3ds
3mf.3mf
ac.ac
ac3d.ac3d
acc.acc
amf.amf
ase.ase
ask.ask
assbin.assbin
b3d.b3d
bvh.bvh
cob.cob
csm.csm
dae.dae
dxf.dxf
enff.enff
fbx.fbx
glb.glb
gltf.gltf
hmp.hmp
ifc.ifc
ifczip.ifczip
irr.irr
irrmesh.irrmesh
lwo.lwo
lws.lws
lxo.lxo
m3d.m3d
md2.md2
md3.md3
md5anim.md5anim
md5camera.md5camera
md5mesh.md5mesh
mdc.mdc
mdl.mdl
mesh.mesh
mesh.xml.mesh.xml
mot.mot
ms3d.ms3d
ndo.ndo
nff.nff
obj.obj
off.off
ogex.ogex
pk3.pk3
ply.ply
pmx.pmx
prj.prj
q3o.q3o
q3s.q3s
raw.raw
scn.scn
sib.sib
smd.smd
stl.stl
stp.stp
ter.ter
uc.uc
vta.vta
x.x
x3d.x3d
x3db.x3db
xgl.xgl
xml.xml
zae.zae
wav.WAV
ddsexrjpg.ddsexrjpg
jpeg.jpeg
pg.pg
png.png
tga.tga
bmp.bmp
psd.psd
gif.gif
hdr.hdr
pic.pic
Windows Shared lib dll.dll
Linux shared lib so.so
The Machinery Theme.tm_theme
The Machinery Spritesheet.tm_spritesheet
The Machinery database project.the_machinery_db
The Machinery Directoy Project.the_machinery_dir
zip.zip
7z.7z
tar.tar

Import Projects

The Machinery allows to share and remix the content of projects made within the Engine via the import project feature.

Project Import provides an easy way to import assets from one The Machinery project to another. To use it, select File > Import File… and pick a The Machinery project file to import. The project you select is opened in a new Import Project tab and from there, you can simply drag-and-drop or copy/paste assets into your main project’s Asset Browser.

Importing assets from another project.

Importing assets from another project.

When you drag-and-drop or copy-paste some assets, all their dependencies are automatically dragged along so that they are ready to use.

Here is a video showing this in action. We start with a blank project, then we drag in a level from the physics sample and a character from the animation sample, put them both in the same scene, and play:

To make it even easier to share your stuff, we’ve also added File > Import from URL… This lets you import any file that The Machinery understands: GLTF, FBX, JPEG, or a complete The Machinery project directly from an URL. You can even import zipped resource directories in the same way.

For example, in the image below, we imported a Curiosity selfie from NASA (using the URL https://www.nasa.gov/sites/default/files/thumbnails/image/curiosity_selfie.jpg ) and dropped it into the scene we just created:

JPEG imported from URL.

JPEG imported from URL.

Have you made something interesting in The Machinery that you want to share with the world? Save your project as an Asset Database and upload it to a web server somewhere.

Other people can use the Import from URL… option to bring your assets into their own projects.

Note: Be aware when you download plugins from the internet, they might contain plugin assets. Only Trust them if you can trust the source! More on this See Plugin Assets

The asset pipeline by Example

This section we focus on how to set up a simple entity from an imported dcc_asset.

You will learn the basics about:

  • What the creation graph is
  • How the asset pipeline works
  • How to create (rigg) an Entity from an imported asset

Introduction

There are lots of things you might want to do to an imported asset coming from a DCC-tool. For example, extracting images and materials into a representation that can be further tweaked by your artists or rigging (create) an entity from the meshes present in the asset. In The Machinery, we provide full control over how data enters the engine and what data-processing steps that get executed, allowing technical artists to better optimize content and set up custom, game-specific asset pipelines.

This is handled through Creation Graphs. A Creation Graph is essentially a generic framework for processing arbitrary data on the CPUs and GPUs, exposed through a graph front-end view. While we can use Creation Graphs for any type of data processing.

For more information visit the Creation Graphs section.

Tip: if you wish to see other use cases such as particle systems, sky rendering and sprite sheets, then have a look in the creation_graphs sample that we provide.

Importing a DCC asset

You can import an asset by selecting File > Import... in the main menu, pressing Ctrl-I, or dropping a DCC file on the Asset Browser tab. When you do this, it ends up in our data-model as a dcc_asset.

For a more in detail explanation about how to import assets checkout the Asset Import Part.

Basic entity rigging, with image and material extraction

If you click on a dcc_asset that contains a mesh in the Asset Browser, you will be presented with importer settings in the Properties tab:

Inspecting a dcc_asset

The Preview tab does just enough to show you what the dcc_asset contains, but what you probably want is an entity that contains child entities representing the meshes found inside the dcc_asset. Also, you probably want it to extract the images and materials from the dcc_asset so you can continue to tweak those inside The Machinery. There are two ways to do this. Either you drag the DCC asset onto the Scene Tab, or you click the Import Asset button. The Import Assets button will automatically create a prototype Entity Asset in your project, while dropping it into the scene will rig the entity inside the scene, without creating a prototype.

In either case, we will for our door.dcc_asset get a door.resources directory next to it. This directory will contain materials and images extracted from the DCC asset. If you prefer dropping the assets into the scene directly, but also want an entity prototype, then you can check the Create Prototype on Drop check box.

Each image and material in the resources folder is a Creation Graph, which is responsible for the data-processing of those resources. You can inspect these graphs to see what each one does. They are described in more detail below.

Creation Graphs for dcc_asset import

In The Machinery, there are no specific asset types for images or materials, instead, we only have Creation Graphs (.creation assets). To extract data from a dcc_asset in a creation graph, the dcc_asset-plugin exposes a set of helper nodes in the DCC Asset category:

  • DCC Asset/DCC Image -- Takes the source dcc_asset together with the name of the image as input and outputs a GPU Image that can be wired into various data processing nodes (such as mipmap generation through the Image/Filter Image node) or directly to a shader node.

  • DCC Asset/DCC Material -- Takes the source dcc_asset together with the name of the material as input and outputs all properties of the material model found inside the dcc_asset. This material representation is a close match to GLTF 2.0's PBR material model. Image outputs are references to other .creation assets which in turn output GPU Images.

  • DCC Asset/DCC Mesh -- Takes the source dcc_asset together with the name of the mesh as input and outputs a GPU Geometry that can be wired to an Output/Draw Call output node for rendering together with the minimum and maximum extents of the geometry that can be wired to an Output/Bounding Volume output node for culling.

The steps for extracting images and material .creation assets from a dcc_asset involve deciding what data-processing should be done to the data before it gets wired to an output node of the graph. This can either be done by manually assembling creation graphs for each image and material, or by building a generic creation graph for each asset type and use that as a prototype when running a batch processing step we refer to as Resource Extraction.

Here's an example of what a generic creation graph prototype for extracting images might look like:

Simple image processing.

To quickly get up and running we provide a number of pre-authored creation graphs for some of the more common operations:

  • import-image-- Operations applied to images imported directly into the editor (not as part of a dcc-asset).

  • dcc-image -- Operations applied to images extracted from an imported dcc_asset.

  • dcc-material -- Shader graph setup to represent materials extracted from an imported dcc_asset.

  • dcc-mesh -- Operations for generating a draw call that represents a mesh from an imported dcc_asset.

  • drop-image -- Operations for generating a draw call to display the image output of another creation graph in the context of an entity, making it possible to drag-and-drop images into the Scene Tab.

These pre-authored creation graphs are shipped as part of our Core project which is automatically copied into new projects. How they are used when working with assets is exposed through the import_settings asset:

Default Import Settings.

By exposing these settings through an asset it is possible to easily change the default behavior of how imported assets are treated when placed under a specific folder.

Note: It's worth noting that this is somewhat of a power-user feature and not something you need to have a detailed understanding of to get started working with The Machinery.

Meshes / Materials / Shaders

In the Machinery Creation Graphs represent Meshes, Materials and Shaders all at the same time.

The creation graph is (as the name implies) a graph based tool for creating assets. It allows developers to define and alter various types of assets using visual scripting. More broadly speaking, the creation graph can be used for arbitrary data processing tasks.

A creation graph thus defines the asset’s pipeline into its final state. For instance, an image asset will have a file that defines the data of the image, but the creation graph asset specifies how that data should be processed. Should it generate mipmaps, should it be compressed, do we need a CPU copy of it, etc.

A more familiar example might be a material shader. In The Machinery this is also defined using a creation graph. This case maps very well to Unity’s shader assets and Unreal’s material assets. In the image below you can see a simple default material with a saturated red base color.

Simple red material

Image loading and material creation are just a few examples of what can be achieved with the creation graph. The table below shows when a creation graph is used compared to the tools one could use in Unity and Unreal.

Asset TypeUnityUnrealThe Machinery
ImagesTextureTextureCreation graphs
MaterialsShaderMaterialCreation graphs
ParticlesParticle EffectCascadeCreation graphs
Post processingShaderMaterialCreation graphs
Procedural materialsProcedural MaterialsMaterialCreation graphs
MeshesMeshStatic MeshDCC Asset +
Creation graphs

Another example for the creation graph is mesh processing. A graph like this will define how the mesh should be draw or traced against. The graph below takes two inputs, a material creation graph and the mesh data from a DCC asset. This data is than imported and passed to the various outputs of our mesh pipeline. In this case those are: a ray tracing instance, a normal draw call, a bounding volume, and a physics shape. Note that not all of these outputs have to be used, rather the code that uses this creation graph might only look for the rendering outputs and ignore the physics shape, whilst some other code might only care about the physics shape output.

Mesh processing

Like the entity graph, the creation graph can executes nodes in sequence from an event. Some examples of this are the Tick, Init, and Compile events which are executed at known intervals. Most of the creation graphs however work with a reverse flow, compiling the graph into a sequence of nodes for a specific output. The two examples presented earlier show this workflow. Some outputs are: Draw Call, Shader Instance, Image, and Physics Shape. Note that these outputs are just blobs of data, an implementation can define more output type in code.

Prototypes

The Machinery has a prototype system that allows entity assets to be used inside other entities.

Table of Content

So you can for example create an entity asset that represents a room, and then create a house entity that has a bunch of these room entities placed into it:

Three Room entities placed to form a House entity.

Difference between a Prototype, an Instance and an Asset

We call the room asset a prototype, and we call each placed room entity an instance of that prototype. Note that prototypes are not special assets, any entity asset can be used as a prototype, with instances of it placed in another entity asset.

In the Entity Tree tab, prototype instances are shown in yellow to distinguish them from locally owned child entities (which are shown in white).

Instance Properties: Inspect

If you expand an instance you will notice that most of its components and child entities are grayed out and can't be selected. This is because they are inherited from the prototype, and the prototype controls their values. If the prototype is modified — for example if we scatter some more props on the floor — those changes are reflected everywhere the prototype has been placed. In the example above, all three room instances would get the scattered objects.

Instance Properties: Override

If we want to, however, we can choose to override some of the prototype’s properties. When we override a property, we modify its value for this instance of the prototype only. Other instances will keep the value from the prototype.

To initiate an override, right-click (or double click) the component or child entity whose properties you want to override and choose Override in the context menu. In addition to modifying properties, you can also add or remove components or child entities on the overridden entity.

Note: If you override the property of some node deep in the hierarchy of the placed entity, all its parents will automatically get overridden too. Let's modify the position of the barrel in the front-most room:

Barrel position overridden.

The overridden entities and components are drawn in blue. We change the x and z components of the position to move the barrel. Note how the changed values are shown in white, while the values that are inherited from the prototype are shown in gray.

If you look back to the first picture you will see that the Link Component was automatically overridden when the prototype was instanced. This is needed because if we didn’t override the Link Component, it would use the values from the prototype, which means all three room instances would be placed in the same position.

When we override something on an instance, all the things not explicitly overridden are still inherited from the prototype. If we modify the prototype — for example change the barrel to a torch — all instances will get the change, since it was only the x and z positions of the object that we changed.

Instance Properties: Reset or Propagate

The context menus can be used to perform other prototype operations. For example, you can Reset properties back to the prototype values. You can also Propagate the changes you have made to an overridden component or entity back to the prototype so that all instances get the changes. Finally, you can Remove the overrides and get the instance back to being an exact replica of the prototype.

Other forms of prototypes: e.g. Graphs

Prototypes can also be used for other things than entities. For example, if you have a graph that you want to reuse in different places you can create a Graph Asset to hold the graph, and then instantiate that asset in the various places where you want to use it. Just as with the entity instances, you can override specific nodes in the graph instance to customize the behavior.

What is next?

Simulation

In The Machinery, we make a distinction between simulating and editing. When you are editing, you see a static view of the scene. All the runtime behaviors like physics, animation, destruction, entity spawning, etc are disabled. (Editing the scene with everything moving around would be very tricky.)

In contrast, when you are simulating or running, all the dynamic behaviors are enabled. This allows you to see the runtime behavior of your entities. If you are building a game, the simulation mode would correspond to running the game.

To simulate a scene, open a scene in the Simulate tab.

Simulate tab.

You can use the controls in the tab to pause, play, or restart the simulation or change the simulation speed. Note that if you haven't added any simulation components to the scene, the Simulate tab will be just as static as the Scene tab. In order to get something to happen, you need to add some runtime components.

The Entity Graph gives you a visual scripting language for controlling entity behavior. It will be described in the next section.

You can launch the engine in simulation mode from the command line:

the-machinery.exe --load-project project.the_machinery --simulate scene

Here project.the_machinery should be the name of your project file and scene the name of the entity you want to simulate. This will open a window with just the Simulate tab, simulating the scene that you specified.

Entity Graphs

The Entity Graph implements a visual scripting language based on nodes and connections. To use it, right-click on an entity to add a Graph Component and then double click on the Graph Component to open it in the Graph Editor:

Graph editor.

The visual scripting language uses Events to tick the execution of nodes. For example, the Tick Event node will trigger its out connector whenever the application ticks its frame. Connect its output event connector to another node to run that node during the application's update.

Nodes that just fetch data and don't have any side-effects are considered "pure". They don't have any event connectors and will run automatically whenever their data is needed by one of the non-pure nodes. Connect the data connectors with wires to pass data between nodes.

In addition to connecting wires, you can also edit input data on the nodes directly. Click a node to select it and edit its input data in the properties.

There are a lot of different nodes in the system and I will not attempt to describe all of them. Instead, here is a simple example that adds a super simple animation to an entity using the graph component:

What is next?

For more information go and checkout the Gameplay Coding / Visual Scripting Chapter

For more examples, check out the pong and animation projects in the samples.

Physics

The Machinery integrates Nvidia's PhysX toolkit and uses it for physics simulation of entities. This section will not attempt to describe in detail how physics simulation works, for that we refer to the PhysX documentation. We will only talk about how physics is set up in The Machinery.

Table of Content

The physics simulation system

The physics simulation system introduces two new assets: Physics Material and Physics Collision as well as four new components: Physics Shape Component, Physics Body Component, Physics Joint Component, and Physics Mover Component.

Physics Assets

A Physics Material asset specifies the physical properties of a physics object: friction (how "slippery" the object is) and restitution (how "bouncy" the object is). Note that if you don't assign a material to a physics shape it will get default values for friction and constitution.

A Physics Collision asset describes a collision class. Collision Classes control which physics shapes collide with each other. For example, a common thing to do is to have a debris class for small objects and set it up so that debris collide with regular objects, but not with other debris. That way, you are not wasting resources on computing collisions between lots of tiny objects. (Note that the debris objects still need to collide with regular objects, or they would just fall through the world.)

In addition to deciding who collides with who, the collision class also decides which collisions generate callback events. These events can be handled in the Entity Graph.

If you don't assign a collision class to a physics shape, it will get the Default collision class.

Physics Components

The Physics Shape Component can be added to an entity to give it a collision shape for physics. Entities with shape components will collide with each other when physics is simulated.

A physics shape can either be specified as geometry (sphere, capsule, plane, box) or it can be computed from a graphics mesh (convex, mesh). Note that if you use computed geometry, you must press the Cook button in the Properties UI to explicitly compute the geometry for the object.

Convex shape.

If you just give an entity a Physics Shape Component it will become a static physics object. Other moving objects can still collide with it, but the object itself won't move.

To create a moving physics object, you need to add a Physics Body Component. The body component lets you specify dynamic properties such as damping, mass, and inertia tensor. It also lets you specify whether the object should be kinematic or not. A kinematic object is being moved by animation. Its movement is not affected by physics, but it can still affect other physical objects by colliding with them and pushing them around. In contrast, if the object is not kinematic it will be completely controlled by physics. If you place it above ground, it will fall down as soon as you start the simulation.

Note that parameters such as damping and mass do not really affect kinematic objects, since the animations will move them the same way, regardless of their mass or damping. However, these parameters can still be important because gameplay code could at some point change the object from being kinematic to non-kinematic. If the gameplay code never makes the body non-kinematic, the mass doesn't matter.

The Physics Joint Component can be used to add joints to the physics simulation. Joints can tie together physics bodies in various ways. For example, if you tie together two bodies with a hinge joint they will swing as if they were connected by a hinge. For a more thorough description of joints, we refer to the PhysX documentation.

The Physics Mover Component implements a physics-based character controller. If you add it to an entity, it will keep the entity's feet on the ground, prevent it from going through walls, etc. For an example of how to use the character controller, check out the animation or gameplay sample projects.

Physics scripting

Physics can be scripted using the visual scripting language in the Entity Graph.

We can divide the PhysX scripting nodes into a few categories.

Nodes that query the state of a physics body:

  • Get Angular Velocity
  • Get Velocity
  • Is Joint Broken
  • Is Kinematic

Nodes that manipulate physics bodies:

  • Add Force
  • Add Torque
  • Break Joint
  • Push
  • Set Angular Velocity
  • Set Kinematic
  • Set Velocity

Event nodes that get triggered when something happens in the scene:

  • On Contact Event
  • On Joint Break Event
  • On Trigger Event

Nodes that query the world for physics bodies:

  • Overlap
  • Raycast
  • Sweep

Note that the query nodes may return more than one result. They will do that by triggering their Out event multiple times, each time with one of the result objects. (In the future we might change this and have the nodes actually return arrays of objects.)

From C you can access those features via the tm_physx_scene_api.

Missing Features

Note that The Machinery doesn't currently support all the features found in PhysX. The most glaring omissions are:

  • D6 joints and joint motors.
  • Vehicles.

We will add more support going forward.

For an example of how to use physics, see the Physics Sample Project.

Tutorials

For more information and guides checkout out the The Machinery Tutorials Book as well as our Physics Sample.

Animation

The Animation system lets you play animations on entities. You can also create complicated animation blends, crossfades, and transitions using an Animation State Machine.

The animation system adds two new assets to The Machinery: Animation Clip and Animation State Machine as well as two new components: Animation Simple Player and Animation State Machine.

To get an animation into The Machinery you first export it from your DCC tool as FBX or another suitable file format. Then you import this using File > Import....

An Animation Clip is an asset created from an imported DCC animation asset that adds some additional data. First, you can set a range of the original animation to use for the clip, so you can cut up a long animation into several individual clips. Second, you can specify whether the animation should loop or not as well as its playback speed. You can use a negative playback speed to get a "reverse" animation.

Finally, you can specify a "locomotion" node for the animation. If you do, the delta motion of that node will be extracted from the animation and applied to the entity that animation is played on, instead of to that bone. This lets an animation "drive" the entity and is useful for things like walking and running animations. The locomotion node should typically be the root node of the skeleton. If the root mode is animated and you "don't" specify a locomotion node, the result will be that the root node "walks away" from the animation.

The Animation Simple Player Component is a component that you can add to an entity to play animations on it. The component lets you pick a single animation to play on the entity. This is useful when you want to play simple animations such as doors opening, coins spinning, flags waving, etc. If you want more control over the playback and be able to crossfade and blend between animations you should use an Animation State Machine instead.

Animation state machines

The Animation State Machine Asset represents an Animation State Machine. If you double-click an Animation State Machine Asset in the Asset Browser, an Animation State Machine Editor will open:

Animation State Machine Editor

The Animation State Machine (ASM) introduces a number of concepts: Layers, States, Transitions, Events, Variables, and Blend Sets.

The ASM represents a complex animation setup with multiple animations that can be played on a character by dividing it into States. Each state represents something the character is doing (running, walking, swimming, jumping, etc) and in each state, one particular animation, or a particular blend of animations is being played. The states are the boxes in the State Graph.

The ASM can change state by taking a Transition from one state to another. The transitions are represented by arrows in the graph. When the transition is taken, the animation crossfades over from the animations in one state to the animations in the other state. The properties of the transition specify the crossfade time. Note that even though the crossfade takes some time, the logical state transition is immediate. I.e. as soon as the transition is taken, the state machine will logically be in the new state.

The transitions are triggered by Events. An event is simply a named thing that can be sent to the ASM from gameplay code. For example, the gameplay may send a "jump" event and that triggers the animation to transition to the "jump" state.

Variables are float values that can be set from gameplay code to affect how the animations play. The variables can be used in the states to affect their playback. For example, you may create a variable called run_speed and set the playback Speed of your run state to be run_speed. That way, gameplay can control the speed of the animation.

Note that the Speed of a state can be set to a fixed number, a variable, or a mathematical expression using variables and numbers. (E.g. run_speed * 2.) We have a small expression language that we use to evaluate these expressions.

The ASM supports multiple Layers of state graphs. This works similar to image layering in an application such as Photoshop. Animations in "higher" layers will be played "on top" of animations in the lower layers and hide them.

As an example of how to use layering, you could have a bottom layer that controls the player's basic locomotion (walking, running, etc). Then you could have a second layer on top of that for controlling arm movements. That way, you could play a reload animation on the arms while the legs are still running. Finally, you could have a top layer to play "hurt" animations when the player for example gets hit by a bullet. These hurt animations could then interrupt the reload animations whenever the player got hit.

Blend Sets can be used to control the per-bone opacity of animations playing in higher layers. They simply specify a weight for each bone. In the example above, the animations in the "arm movement" layer would have opacity 1.0 for all arm bones, but 0.0 for all leg bones. That way, they would hide the arm movement from the running animation below, but let the leg movement show through.

The Animation State Machine Editor has a Tree View to the left that lets you browse all the layers, states, transitions, events, variables, and blend sets. The State Graph lets you edit the states and transitions in the current layer. The Properties window lets you set the properties of the currently selected objects and the Preview shows you a preview of what the animation looks like. Note that for the preview to work, you must specify a Preview Entity in the properties of the state machine. This is the entity that will be used to preview the ASM. When you select a state in the State Graph, the preview will update to show that state.

In the Preview window, you also find the Motion Mixer. This allows you to send events to the ASM and change variables to see how the animation reacts.

The ASM currently supports the following animation states:

Regular State

Plays a regular animation.

Random State

Randomly plays an animation out of a set of options.

Empty State

A state that doesn't play any animation at all. Note that this state is only useful in higher layers. You can transition to an empty state to "clear" the animation in the layer and let the animations from the layers below it shine through.

Blend State

Allows you to make a 1D or 2D blend between animations based on the value of variables. This is often used for locomotion. You can use different animations based on the characters running speed and whether the character is turning left or right and position them on a map to blend between them.

Animation Blending

Once you have created an Animation State Machine, you can assign it to a character by giving it an Animation State Machine Component.

For an example of how the animation system works, have a look at the mannequin sample project.

Missing features

Note that the animation system is still under active development. Here are some features that are planned for the near future:

  • Ragdolls.
  • Animation compression.
  • Triggers.
  • More animation states.
    • Offset State.
    • Template State.
    • Expression-based Blend State.
    • Graph-based Blend State.
  • Beat transitions.
  • Constraints.

Animation Compression

We support compressed animations. Compressed animations have the extension .animation. Note that with this, we have three kinds of animation resources:

ResourceDescription
.dcc_assetAnimation imported from a Digital Content Creation (DCC) software, such as Max, Maya, Blender, etc. Note that .dcc_asset is used for all imported content, so it could be animations, textures, models, etc.
.animationA compressed animation. The compressed animation is generated from the .dcc_asset by explicitly compressing it.
.animation_clipSpecifies how an animation should be played: playback speed, whether it plays forward or backward, if it drives the root bone or not, etc.

An Animation Clip references either an uncompressed .dcc_asset or a compressed .animation to access the actual animation data.

To create a compressed animation, right-click a .dcc_asset file that contains an animation and choose Create xxx.animation in the context menu:

A compressed animation.

When you first do this, the animation shows a white skeleton in T-pose and a moving blue skeleton. The blue skeleton is the reference .dcc_animation and the white skeleton is the compressed animation. By comparing the skeletons you can see how big the error in the animation is.

At first, the white skeleton is in T-pose because we haven’t actually generated the compressed data yet. To do that, press the Compress button:

Compressed animation with data.

This will update the Format and Buffer fields and we can see that we have 9.2 KB of compressed data for this animation and that the compression ratio is x 6.66. I.e, the compressed data is 6.66 times smaller than the uncompressed one. The white and the blue skeletons overlap. The compression error is too small to be noticed in this view, we have to really zoom in to see it:

Zoomed in view of one of the fingers.

When you compress an animation like this, The Machinery tries to come up with some good default compression settings. The default settings work in a lot of cases, but they’re not perfect, because The Machinery can’t know how the animation is intended to be viewed in your game.

Are you making a miniature fighting game, and all the models will be viewed from a distant overhead camera? In that case, you can get away with a lot of compression. Or are you animating a gun sight that will be held up really close to the player’s eye? In that case, a small error will be very visible.

To help the engine, you can create an .animation_compression asset. (New Animation Compression in the asset browser.) The Animation Compression asset control the settings for all the animations in the same folder or in its subfolders (unless the subfolders override with a local Animation Compression asset):

Animation Compression settings.

The Animation Compression settings object has two properties:

Max Error specifies the maximum allowed error in the compressed animation. The default value is 0.001 or 1 mm. This means that when we do the compression we allow bones to be off by 1 mm, but not more. The lower you set this value, the less compression you will get.

Skin Size specifies the size we assume for the character’s skin. It defaults to 0.1, or 10 cm. We need the skin size to estimate the effects of rotational errors. For example, if the rotation of a bone is off by 1°, the effect of that in mm depends on how far away from the bone the mesh is.

10 cm is a reasonable approximation for a human character, but notice that there are situations where the skin size can be significantly larger. For example, suppose that a 3 m long staff is attached to the player’s hand bone. In this case, rotational errors in the hand are amplified by the full length of the staff and can lead to really big errors in the position of the staff end. If this gives you trouble, you might want to up the skin size to 3 for animations with the staff.

We don’t support setting a per-bone skin size, because it’s unclear if the effort of specifying per-bone skin sizes is really worth it in terms of the memory savings it can give. (Also, even a per-bone skin size might not be enough to estimate errors perfectly. For example, an animator could have set up a miter joint where the extent of the skin depends on the relative angle of two bones and goes to infinity as the angle approaches zero.)

Note that sometimes animations are exported in other units than meters. In this case, the Skin Size and the Max Error should be specified in the same units that are used in the animation file.

Sound

The Machinery comes with a low-level sound system that can import WAV files into the project and play them back. The sound system can position sounds in 3D space and mix together 2D and 3D sounds for output on a stereo, 5.1, or 7.1 sound system.

You can play a sound by adding a Sound Source Component to an object in the level of by using one of the sound playing nodes in the visual scripting language.

Missing features

The sound system is rudimentary. Here are some features that are planned for the future:

  • Sound streaming
  • Sound compression
  • WASAPI backend
  • React to the user plugging in or unplugging headphones
  • Hermite-interpolated resampling
  • Reverb
  • Directional sound sources
  • Doppler effect
  • Multiple listeners
  • HRTF
  • High-level sound system
    • Random sounds
    • Composite sounds
    • Streaming sounds
    • Compressing sounds

Publishing your game

You publish your game via File -> Publish. The Engine opens the publishing tab.

In there, you have a couple of options:

OptionDescription
Executable NameThe name of the executable, e.g. test.exe
Window TitleThe text which is displayed in the window title.
World EntityThe Entry point of your game.
ResolutionThe default resolution to use when running the published project
FullscreenIf checked the game will launch in Fullscreen
Directory ProjectDecides if the game data is published as binary data or as human readable directory.

Directory vs. none Directory Project

If you check this option, the game is exported as a human-readable project. Otherwise the game data will be compressed and stored in a binary .the_machinery format.

Not recommended for other than debug purposes.

Sculpt Tool

Note: This tool is in a preview state.

With the Sculpt Tool The Machinery supports rapid prototyping when it comes to level white boxing. You make use of the tool by adding a Sculpt Component to an Entity. Using the sculpt component you can quickly sketch out levels or make beautiful blocky art:

A blocky character in a blocky forest setting.

A blocky character in a blocky forest setting.

How to use the Tool

To use the Sculpt Component, first add it to an entity, by right-clicking the entity in the Entity Tree and selecting Add Component. Then, select the newly created Sculpt component in the Entity Tree.

This gives you a new sculpt tool in the toolbar:

Sculpt tool.

Sculpt tool.

With this tool selected, you can drag out prototype boxes on the ground. You can also drag on an existing prototype box to create boxes attached to that box.

The standard Select, Move, Rotate, and Scale tools can be used to move or clone (by shift-dragging) boxes.

You can add physics to your sculpts, by adding a Physics Shape Component, just as you would for any other object.

Note: If you are cooking a physics mesh or convex from your sculpt data, you need to explicitly recook whenever the sculpt data changes.

Here is a video of sculpting in action:

Note: Currently, all the sculpting is done with boxes. We may add additional shape support in the future.

Additional though

In addition to being a useful tool, the Sculpt Component also shows the deep integration you can get with custom plugins in The Machinery. The Sculpt Component is a separate plugin, completely isolated from the rest of The Machinery and if you wanted to, you could write your own plugins to do similar things.

The Truth

The Machinery uses a powerful data model to represent edited assets. This model has built-in support for serialization, streaming, copy/paste, drag-and-drop as well as unlimited undo/redo. It supports an advanced hierarchical prefab model for making derivative object instances and propagating changes. It even has full support for real-time collaboration. Multiple people can work together in the same game project, Google Docs-style. Since all of these features are built into the data model itself, your custom, game-specific data will get them automatically, without you having to write a line of code.

The Data Model

The Machinery stores its data as objects with properties. Each object has a type and the type defines what properties the object has. Available property types are bools, integers, floats, strings, buffers, references, sub-objects and sets of references or sub-objects.

The object/properties model gives us us forward and backward compatibility and allows us to implement operations such as cloning without knowing any details about the data. We can also represent modifications to the data in a uniform way (object, property, old-value, new-value) for undo/redo and collaboration.

The model is memory-based rather than disk-based. I.e. the in-memory representation of the data is considered authoritative. Read/write access to the data is provided by a thread-safe API. If two systems want to cooperate, they do so by talking to the same in-memory model, not by sharing files on disk. Of course, we still need to save data out disk at some point for persistence, but this is just a “backup” of the memory model and we might use different disk formats for different purposes (i.e. a git-friendly representation for collaborative work vs single binary for solo projects).

Since we have a memory-based model which supports cloning and change tracking, copy/paste and undo can be defined in terms of the data model. Real-time collaboration is also supported, by serializing modifications and transmitting them over the network. Since the runtime has equal access to the data model, modifying the data from within a VR session is also possible.

We make a clear distinction between “buffer data” and “object data”. Object data is stuff that can be reasoned about on a per-property level. I.e. if user A changes one property of an object, and user B changes another, we can merge those changes. Buffer data are binary blobs of data that are opaque to the data model. We use it for large pieces of binary data, such as textures, meshes and sound files. Since the data model cannot reason about the content of these blobs it can’t for example merge changes made to the same texture by different users.

Making the distinction between buffer data and object data is important because we pay an overhead for representing data as objects. We only want to pay that overhead when the benefits outweigh the costs. Most of a game’s data (in terms of bytes) is found in things like textures, meshes, audio data, etc and does not really gain anything from being stored in a JSON-like object tree.

In The Truth, references are represented by IDs. Each object has a unique ID and we reference other objects by their IDs. Since references have their own property type in The Truth, it is easy for us to reason about references and find all the dependencies of an object.

Sub-objects in The Truth are references to owned objects. They work just as references, but have special behaviours in some situations. For example, when an object is cloned, all its sub-objects will be cloned too, while its references will not.

For more information checkout the documentation and these blog posts: The Story behind The Truth: Designing a Data Model or this one.

Access values

The truth objects (tm_tt_id_t) are immutable objects unless you explicitly make them writable. Therefore you do not have to be afraid of accidentally changing a value when reading from an object property.

To read from an object property we need access to the correct Truth Instance as well as to an object id. We also need to know what kind of property we want to access. That is why we always want to define our properties in a Header-File. Which allows us and others to find quickly our type definitions. A good practice is to comment on what kind of data type property contains.

Let us assume our object is of type TM_TT_TYPE__RECT:

enum {
    TM_TT_PROP__RECT__X, // float
    TM_TT_PROP__RECT__Y, // float
    TM_TT_PROP__RECT__W, // float
    TM_TT_PROP__RECT__H, // float
};

When we know what we want to access, we call the correct function and access the value. In our example we want to get the width of an object. The width is stored in TM_TT_PROP__RECT__W.

The function we need to call:

void (*get_float)(tm_the_truth_o *tt,const tm_the_truth_object_o *obj, uint32_t property);

With this knowledge we can assemble the following function that logs the width of an object:

void log_with(tm_the_truth_o *tt, tm_tt_id_t my_object){   
	const float width = tm_the_truth_api->get_float(tt,tm_tt_read(tt,my_object),TM_TT_PROP__RECT__W);
    TM_LOG("the width is %f",width);
}

Make the code robust

To ensure we are actually handling the right type we should check this at the beginning of our function. If the type is not correct we should early out and log a warning.

All we need to do is compare the tm_tt_type_t's of our types. Therefore we need to obtain the type id from the object id and from our expected type. From a tm_tt_id_t we can obtain the type by calling tm_tt_type() on them. tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__MY_TYPE); will give us back the object type from a given hash. After that we can do our comparison.

void log_with(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    const tm_tt_type_t expected_type = tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__RECT);

    if (type.u64 != expected_type.u64)
    {
        TM_LOG("The provided type does not mmatch! %p{tm_tt_type_t} != %p{tm_tt_type_t}", &type, &expected_type);
        return;
    }

    const float width = tm_the_truth_api->get_float(tt, tm_tt_read(tt, my_object), TM_TT_PROP__RECT__W);
    TM_LOG("the width is %f", width);
}

Note: Check out the logger documentation for more information on it. log.h

Create an Object

You can create an object of a Truth Type via two steps:

  1. You need to obtain the Type from the type hash. We call the object_type_from_name_hash to obtain the tm_tt_type_t
  2. You need to create an Object from that Type. We call create_object_of_type to create an object tm_tt_id_t . We pass TM_TT_NO_UNDO_SCOPE because we do not need an undo scope for our example.

First, we need to have access to a Truth instance. Otherwise, we could not create an object. In this example, we create a function.

tm_tt_id_t create_my_type_object(tm_the_truth_o *tt)
{
    const tm_tt_type_t my_type = tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__MY_TYPE);
    const tm_tt_id_t my_type_object = tm_the_truth_api->create_object_of_type(tt, my_type, TM_TT_NO_UNDO_SCOPE);
    return my_type_object;
}

Wherever we call this function we can then edit and modify the type and add content to it!

The alternative approach is to use the "Quick Object Creation function".

tm_tt_id_t quick_create_my_type_object(tm_the_truth_o *tt)
{
    return tm_the_truth_api->quick_create_object(tt, TM_TT_NO_UNDO_SCOPE, TM_TT_TYPE_HASH__MY_TYPE, -1);
}

Note: need to pass -1 to tell the function that we are at the end of the creation process. More info here.

What is next?

If you want to learn more about how to create your own custom type, follow the "Custom Truth Type" walkthrough.

Modify an object

To manipulate an object, you need to have its ID (tm_tt_id_t). When you create an object, you should keep its ID around if you intend to edit it later.

Table of Content

In this example, we have a function that gets an object and the Truth instance of that object.

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object){
    //...
}

Important: you can only edit an object that is part of the same instance! Hence your my_object must be created within this instance of the Truth (tt).

1. Make the object writable

To edit an object, we need to make it writable first. In the default state, objects from the Truth are immutable. The Truth API has a function that is called write. When we call it on an object, we make it writable.

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    tm_the_truth_object_o *my_object_w = tm_the_truth_api->write(tt, my_object);
//...
}

2. Write to the object.

We need to know what kind of property we want to edit. That is why we always want to define our properties in a Header-File. A good practice is to comment on what kind of data type property contains.

Let us assume our object is of type TM_TT_TYPE__RECT:

enum {
    TM_TT_PROP__RECT__X, // float
    TM_TT_PROP__RECT__Y, // float
    TM_TT_PROP__RECT__W, // float
    TM_TT_PROP__RECT__H, // float
};

In our example we want to set the width to 100. The width is stored in TM_TT_PROP__RECT__W.

When we know what we want to edit, we call the correct function and change the value.

The function we need to call:

void (*set_float)(tm_the_truth_o *tt, tm_the_truth_object_o *obj, uint32_t property,float value);

Let us bring all of this together:

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    tm_the_truth_object_o *my_object_w = tm_the_truth_api->write(tt, my_object);
    tm_the_truth_api->set_float(tt, my_object_w, TM_TT_PROP__RECT__W, 100);
//..
}

3. Save the change

In the end, we need to commit our change to the system. In this example we do not care about the undo scope. That is why we provide the TM_TT_NO_UNDO_SCOPE define. This means this action is not undoable.

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    tm_the_truth_object_o *my_object_w = tm_the_truth_api->write(tt, my_object);
    tm_the_truth_api->set_float(tt, my_object_w, TM_TT_PROP__RECT__W, 100);
    tm_the_truth_api->commit(tt, my_object_w, TM_TT_NO_UNDO_SCOPE);
}

If we wanted to provide an undo scope we need to create one:

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    tm_the_truth_api->set_float(tt, my_object_w, TM_TT_PROP__RECT__W, 100);
    const tm_tt_undo_scope_t undo_scope = tm_the_truth_api->create_undo_scope(tt, "My Undo Scope");
    tm_the_truth_api->commit(tt, my_object_w, undo_scope);
}

Now this action can be reverted in the Editor.

4. Get a value

Instead of changing the value of width to 100 we can also increment it by 100! All we need to do is get the value first of the Truth Object and add 100 to it. To access a property we need to use the macro tm_tt_read. This will give us an immutable (read only) pointer to the underlying object. This allows us to read the data from it.

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    float wdith = tm_the_truth_api->get_float(tt, tm_tt_read(tt, my_object), TM_TT_PROP__RECT__W);
    wdith += 100;
    tm_the_truth_object_o *my_object_w = tm_the_truth_api->write(tt, my_object);
    tm_the_truth_api->set_float(tt, my_object_w, TM_TT_PROP__RECT__W, wdith);
    const tm_tt_undo_scope_t undo_scope = tm_the_truth_api->create_undo_scope(tt, "My Undo Scope");
    tm_the_truth_api->commit(tt, my_object_w, undo_scope);
}

Note: If we had a lot of read actions we should only call tm_tt_read once and store the result in a const tm_the_truth_object_o* variable and reuse.

5. Make the code robust

To ensure we are actually handling the right type we should check this at the beginning of our function. If the type is not correct we should early out.

All we need to do is compare the tm_tt_type_t's of our types. Therefore we need to obtain the type id from the object id and from our expected type. From a tm_tt_id_t we can obtain the type by calling tm_tt_type() on them. tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__MY_TYPE); will give us back the object type from a given hash. After that we can do our comparison.

void modify_my_object(tm_the_truth_o *tt, tm_tt_id_t my_object)
{
    const tm_tt_type_t type = tm_tt_type(my_object);
    const tm_tt_type_t expected_type = tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__RECT);

    if (type.u64 != expected_type.u64)
        return;

    float wdith = tm_the_truth_api->get_float(tt, tm_tt_read(tt, my_object), TM_TT_PROP__RECT__W);
    wdith += 100;
    tm_the_truth_object_o *my_object_w = tm_the_truth_api->write(tt, my_object);
    tm_the_truth_api->set_float(tt, my_object_w, TM_TT_PROP__RECT__W, wdith);
    const tm_tt_undo_scope_t undo_scope = tm_the_truth_api->create_undo_scope(tt, "My Undo Scope");
    tm_the_truth_api->commit(tt, my_object_w, undo_scope);
}

Common Types

The Truth comes with several useful common types. You can find them in the `the_truth_types. (API Documentation).

MacroDescription
TM_TT_TYPE__BOOL/TM_TT_TYPE_HASH__BOOLThe first property contains the value.
TM_TT_TYPE__UINT32_T/TM_TT_TYPE_HASH__UINT32_TThe first property contains the value.
TM_TT_TYPE__UINT64_T/TM_TT_TYPE_HASH__UINT64_TThe first property contains the value.
TM_TT_TYPE__FLOAT/TM_TT_TYPE_HASH__FLOATThe first property contains the value.
TM_TT_TYPE__DOUBLE /TM_TT_TYPE_HASH__DOUBLEThe first property contains the value.
TM_TT_TYPE__STRING/TM_TT_TYPE_HASH__STRINGThe first property contains the value.
TM_TT_TYPE__VEC2/TM_TT_TYPE_HASH__VEC2The first property contains the x value and the second the y value.
TM_TT_TYPE__VEC3/TM_TT_TYPE_HASH__VEC3The first property contains the x value and the second the y value and the third the z value.
TM_TT_TYPE__VEC4/TM_TT_TYPE_HASH__VEC4The first property contains the x value and the second the y value and the third the z value while the last one contains the w value.
TM_TT_TYPE__POSITION/TM_TT_TYPE_HASH__POSITIONSame as vec4.
TM_TT_TYPE__ROTATION/TM_TT_TYPE_HASH__ROTATIONBased on a vec4. Used to represent the rotation of an object via quaternions.
TM_TT_TYPE__SCALE/TM_TT_TYPE_HASH__SCALESame as vec3.
TM_TT_TYPE__COLOR_RGB/TM_TT_TYPE_HASH__COLOR_RGBRepresents a RGB colour.
TM_TT_TYPE__COLOR_RGBA/TM_TT_TYPE_HASH__COLOR_RGBARepresents a RGBA colour.
TM_TT_TYPE__RECT/TM_TT_TYPE_HASH__RECTThe first property contains the x value and the second the y value and the third the width value while the last one contains the height value.

There is a helper API to handle all of these types in an easy way, to reduce the boilerplate code: tm_the_truth_common_types_api.

Note: There is a list of all Truth Types the Engine comes with available on our API Documentation

Aspects

An “aspect” is an interface (struct of function pointers) identified by a unique identifier. The Truth allows you to associate aspects with object types. This lets you extend The Truth with new functionality. For example, you could add an interface for debug printing an object:

#define TM_TT_ASPECT__DEBUG_PRINT TM_STATIC_HASH("tm_debug_print_aspect_i", 0x39821c78639e0773ULL)

typedef struct tm_debug_print_aspect_i
{
    void (*debug_print)(tm_the_truth_o *tt, tm_tt_id_t object);
} tm_debug_print_aspect_i;

Note: to genereate the TM_STATIC_HASH you need to run hash.exe or tmbuild.exe --gen-hash for more info open the hash.exe guide

You could then use this code to debug print an object o with:

static void example_use_case(tm_the_truth_o *tt, tm_tt_id_t object)
{
    tm_debug_print_aspect_i *dp = tm_the_truth_api->get_aspect(tt, tm_tt_type(object), TM_TT_ASPECT__DEBUG_PRINT);
    if (dp)
        dp->debug_print(tt, object);
}

Note: that plugins can extend the system with completely new aspects.

The best example of how the Engine is using the aspect system is the TM_TT_ASPECT__PROPERTIES which helps us to defines custom UIs for Truth objects.

Create a custom Truth Type

This walkthrough shows you how to create a type for the Truth. The Truth is our centralized data model for editing data in the Engine. For more details on the system itself, click here: The Truth.

You should have basic knowledge about how to write a custom plugin. If not, you might want to check this Guide.

We will cover the following topics:

  • How to define a Type.
  • Type Properties

After this walkthrough you could check out the "Create a custom asset" tutorial!

Table of Content

Define a Type

A Truth-Type in The Machinery consists out of a name (its identifier) and properties.

Note: In theory, you could also define a Type without properties.

To add a Type to the system, you need access to the Truth instance. The Engine may have more than one instance of a Truth.

Example: There is a Project Truth to keep all the project-related settings and an Engine/Application Truth that holds all the application-wide settings.

Generally speaking, you want to define Truth Types at the beginning of the Engine's life cycle. Therefore the designated place is the tm_load_plugin function. The Truth has an interface to register a truth type creation function: tm_the_truth_create_types_i.

This interface expects a function of the signature: void create_truth_types(tm_the_truth_o *tt). Whenever the Engine creates a Truth, it invokes this interface on all loaded plugins, and their types are registered. You do not need to register your Type to the interface if you want to register your Type to a specific Truth.

Note: Mostly this function is called: create_truth_types

Let us define a type. To do that, we need to get the tm_truth_api first:

// beginning of the source file
static struct tm_the_truth_api *tm_the_truth_api;
#include <foundation/api_registry.h>
#include <foundation/the_truth.h>
// ... other code
TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
}

After this, we define our type name once as a constant char define and one hashed version. There are some conventions to keep in mind:

  1. The plain text define should start with: TM_TT_TYPE__.
  2. The hashed define should start with: TM_TT_TYPE_HASH__
  3. The name may or may not start with tm_ but the name plain text and the hashed version need to match!
#define TM_TT_TYPE__MY_TYPE "tm_my_type"
#define TM_TT_TYPE_HASH__MY_TYPE TM_STATIC_HASH("tm_my_type", 0xde0e763ccd72b89aULL)

Tip: Do not forget to run hash.exe. Otherwise, the TM_STATIC_HASH macro will cause an error. You can also run tmbuild --gen-hash

It is good practice to place the types into a header file so others can use these types as well! When that is done we can call the tm_the_truth_api->create_object_type() to create the actual type. It will return a tm_tt_type_t which is the identifier of our type. The tm_tt_id_t will also refer to the type here!

The function expects:

ArgumentDescription
tm_the_truth_o *ttThe Truth instance. This function will add the type to this instance
const char *nameThe name of the type. It will be hashed internally. Therefore the hash value of TM_TT_TYPE__ and TM_TT_TYPE_HASH___ should match! If a type with name already exists, that type is returned. Different types with the same name are not supported!
const tm_the_truth_property_definition_t *propertiesThe definitions of the properties of the type.
uint32_t num_propertiesThe number of properties. Should match properties

The home of this function should be our void create_truth_types(tm_the_truth_o *tt) . We need to add this one to our source file. After this we add the call to create_object_type to it. Remember that we have no properties yet, and our call would look like this:

static void create_truth_types(tm_the_truth_o *tt)
{
    tm_the_truth_api->create_object_type(tt, TM_TT_TYPE__MY_TYPE, 0, 0);
}

The last step is to tell the plugin system that we intend to register our register_truth_type().

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
    tm_add_or_remove_implementation(reg, load, tm_the_truth_create_types_i, create_truth_types);
}

The full source code should look like this:

my_type.h

#pragma once
#include <foundation/api_types.h>

#define TM_TT_TYPE__MY_TYPE "tm_my_type"
#define TM_TT_TYPE_HASH__MY_TYPE TM_STATIC_HASH("tm_my_type", 0xde0e763ccd72b89aULL)

(Tip: Do not forget to run hash.exe)

my_type.c

// beginning of the source file
static struct tm_the_truth_api *tm_the_truth_api;
#include <foundation/api_registry.h>
#include <foundation/the_truth.h>
#include "my_type.h"

static void create_truth_types(tm_the_truth_o *tt)
{
    tm_the_truth_api->create_object_type(tt, TM_TT_TYPE__MY_TYPE, 0, 0);
}

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
    tm_add_or_remove_implementation(reg, load, tm_the_truth_create_types_i, create_truth_types);
}

After all of this you have registered your type and it could be used. This type is just not really useful without properties.

About Properties

In The Truth, an Object-Type is made of one or multiple properties. Properties can represent the basic types:

  • bool, string, float, UINT64, UNIT32, double, `buffer\
  • subobject - An object that lives within this property
  • reference - A reference to another object
  • subobject set - A Set of subobjects
  • reference set - A Set of references.

What is the difference between a reference and a subobject?

To see the difference, consider how clone_object() works in both cases:

  • When you clone an object with references, the clone will reference the same objects as the original, i.e. they now have multiple references to them.
  • When you clone an object with subobjects, all the subobjects will be cloned too. After the clone operation, there is no link between the object's subobjects and the clone's subobjects.

An arbitrary number of objects can reference the same object, but a subobject only has a single owner.

When you destroy an object, any references to that object become NIL references — i.e., they no longer refer to anything.

When you destroy an object that has subobjects, all the subobjects are destroyed with it.

Note: For more information please check: The API Documentation

Adding properties

Let us add some properties to our Type! As you remember, when we created the Type, the function create_object_type() required a pointer to the definition of properties. You can define properties via the tm_the_truth_property_definition_t struct.

typedef struct tm_the_truth_property_definition_t
{
    // Name of the property, e.g. "cast_shadows".
    //
    // This name is used both for serialization and for the UI of editing the property. When
    // displayed in the UI, the name will be automatically capitalized (e.g. "Cast Shadows").
    //
    // The name shouldn't be longer than [[TM_THE_TRUTH_PROPERTY_NAME_LENGTH]] characters.
    const char *name;

    // [[enum tm_the_truth_property_type]] type of the property.
    tm_the_truth_property_type type;

    // [[enum tm_the_truth_editor]] enum defining what editor should be used for editing the property.
    uint32_t editor;

    // Editor specific settings.
    union
    {
        tm_the_truth_editor_enum_t enum_editor;
        tm_the_truth_editor_string_open_path_t string_open_path_editor;
        tm_the_truth_editor_string_save_path_t string_save_path_editor;
    };

    // For properties referring to other objects (references & subobjects), specifies the type of
    // objects that they can refer to. A value of [[TM_TT_TYPE__ANYTHING]] is used for an object
    // that can refer to anything.
    //
    // Note: We currently don't have any system for representing "interfaces" or groups of types.
    // I.e. you can't say "I want this to reference any type that inherits from the GRAPH_NODE_TYPE,
    // but no other types." We may add this in the future.
    tm_strhash_t type_hash;

    // Specifies that the property is allowed to refer to other types than the `type_hash`.
    //
    // !!! NOTE: Note
    //     This flag should not be used going forward. Instead, if a property can refer to multiple
    //     types, you should use a `type_hash` of [[TM_TT_TYPE__ANYTHING]]. It is provided for
    //     compatibility purposes, because some object types have a `type_hash` specified but
    //     store subobjects of other types. We cannot simply change the `type_hash` of those objects
    //     to [[TM_TT_TYPE__ANYTHING]], because there may be saved data that has serialized versions
    //     of those objects that omits the object type (if it is `type_hash`). We can't deserialize
    //     those objects if we don't know the `type_hash` of the type.
    bool allow_other_types;
    TM_PAD(7);

    // For buffer properties, the extension (if any) used to represent the buffer type. This can
    // either be hard-coded in `buffer_extension`, or computed by the `buffer_extension_f()`
    // callback (set the unused option to `NULL`).
    const char *buffer_extension;
    const char *(*buffer_extension_f)(const tm_the_truth_o *tt, tm_tt_id_t object, uint32_t property);

    // Tooltip used to describe the property in more detail. The tooltip text will be displayed in
    // the property editor when the property is hovered over.
    //
    // The tooltip should be registered using [[TM_LOCALIZE_LATER()]]. It will be dynamically
    // localized to the current interface language with [[TM_LOCALIZE_DYNAMIC()]] before being
    // displayed in the UI.
    const char *tooltip;

    // If *true*, this property will be skipped during serialization.
    bool not_serialized;
    TM_PAD(7);

    // If specified, this will be used instead of `name` for the UI.
    const char *ui_name;
} tm_the_truth_property_definition_t;

(API Documentation)

Within our create_truth_types we create an array of type tm_the_truth_property_definition_t. For this example, we define the properties of type bool and string.

// beginning of the source file
static struct tm_the_truth_api *tm_the_truth_api;
#include <foundation/api_registry.h>
// include [macros.h](https://ourmachinery.com/apidoc/foundation/macros.h.html#macros.h) to access TM_ARRAY_COUNT for convinace:
#include <foundation/macros.h>
#include <foundation/the_truth.h>

#include "my_type.h"

static void create_truth_types(tm_the_truth_o *tt)
{
    tm_the_truth_property_definition_t properties[] = {
        {"my_bool", TM_THE_TRUTH_PROPERTY_TYPE_BOOL},
        {"my_string", TM_THE_TRUTH_PROPERTY_TYPE_STRING},
    };
    tm_the_truth_api->create_object_type(tt, TM_TT_TYPE__MY_TYPE, properties, TM_ARRAY_COUNT(properties));
}

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
    tm_add_or_remove_implementation(reg, load, tm_the_truth_create_types_i, create_truth_types);
}

That is all we need to do to define properties for our Type! Also thanks to our automatic "reflection" system you do not have to worry about providing a UI for the type. The Properties View will automatically provide a UI for this type.

What is next?

You can find more in depth and practical tutorials in the tutorial book

Creation Graphs

The creation graph is (as the name implies) a graph based tool for creating assets. It allows developers to define and alter various types of assets using visual scripting. More broadly speaking, the creation graph can be used for arbitrary data processing tasks.

A creation graph thus defines the asset’s pipeline into its final state. For instance, an image asset will have a file that defines the data of the image, but the creation graph asset specifies how that data should be processed. Should it generate mipmaps, should it be compressed, do we need a CPU copy of it, etc.

A more familiar example might be a material shader. In The Machinery this is also defined using a creation graph. This case maps very well to Unity’s shader assets and Unreal’s material assets. In the image below you can see a simple default material with a saturated red base color.

Simple red material

Image loading and material creation are just a few examples of what can be achieved with the creation graph. The table below shows when a creation graph is used compared to the tools one could use in Unity and Unreal.

Asset TypeUnityUnrealThe Machinery
ImagesTextureTextureCreation graphs
MaterialsShaderMaterialCreation graphs
ParticlesParticle EffectCascadeCreation graphs
Post processingShaderMaterialCreation graphs
Procedural materialsProcedural MaterialsMaterialCreation graphs
MeshesMeshStatic MeshDCC Asset +
Creation graphs

Another example for the creation graph is mesh processing. A graph like this will define how the mesh should be draw or traced against. The graph below takes two inputs, a material creation graph and the mesh data from a DCC asset. This data is than imported and passed to the various outputs of our mesh pipeline. In this case those are: a ray tracing instance, a normal draw call, a bounding volume, and a physics shape. Note that not all of these outputs have to be used, rather the code that uses this creation graph might only look for the rendering outputs and ignore the physics shape, whilst some other code might only care about the physics shape output.

Mesh processing

Like the entity graph, the creation graph can executes nodes in sequence from an event. Some examples of this are the Tick, Init, and Compile events which are executed at known intervals. Most of the creation graphs however work with a reverse flow, compiling the graph into a sequence of nodes for a specific output. The two examples presented earlier show this workflow. Some outputs are: Draw Call, Shader Instance, Image, and Physics Shape. Note that these outputs are just blobs of data, an implementation can define more output type in code.

Creation Graphs for Unity Developers

Creation graphs are used for many different assets in The Machinery. When a creation graph is used for a surface shader it most closely relates to Unity’s shader graph. This is what we will focus on first.

Simple surface shader

In the example above the editor’s layout was made to resemble Unity’s shader graph view. When creating a material shader you need to specify a Shader Instance output node. From here we can specify out surface shader, in the example above the Lit node closely resembles Unity’s PBR Master node.

Creation Graphs for Unreal Engine developers

Creation graphs are used for many different assets in The Machinery. When a creation graph is used for a shader it most closely relates to Unreal’s materials (any domain). This is what we will focus on first.

Simple brick material

In the example above the editor closely resembles the material editor from Unreal, this is however not the default layout. You can see the creation graph in the center with its output being a Shader Instance. Adding this allows any consuming code to query the material from this creation graph and it will allow the preview tab to display your material.

Simple rotating particle

The previous example showed a surface or material shader. This example shows a creation graph that fully defines a simple particle. The Shader Instance (material) is now passed to a Draw Call node, with this combination we can now fully render the particle without the need of an explicit mesh. Instead we use the Construct Quad node for a procedural quad mesh. Note that we specify the Instance Count and Num Vertices (for a single quad that is 6).

Node types

Nodes in the creation graph can be subdivided into four types, understanding the difference between these nodes is important when creating new nodes. The diagram below shows how each node can be categorized.

GPU nodes are somewhat special as they will be compiled down into a single shader instead of being interpreted like the CPU part of the creation graph. Note that GPU nodes also have a different background color to distinguish them. GPU nodes will not connect to any CPU node unless their output is a Shader Instance, this is the point where the GPU portion of the graph is compiled and passed to the CPU.

The CPU portion of a creation graph is very similar to the entity graph in terms of layout with one exception. The creation graph often works by querying output nodes from the creation graph and working its way back from there. event nodes on the other hand allow you to follow the same flow as the entity graph, beginning from some event and continuing into other nodes.

In the example above you can see a creation graph that could be passed to a render component because it uses the Draw Call and Bounding Volume output nodes. The parameters to these are: a DCC mesh and a Lit Shader Instance. Note the Variable GPU node that is used to pass the color from the CPU side to the GPU side, this is the only way to connect CPU nodes to GPU nodes. Currently we support the following output nodes, note that multiple of these can be present in a single creation graph.

NameInformation
Image OutputAllows preview, creates asset thumbnail.
Bounding VolumeUsed for culling.
Draw CallGenerally used with the Render Component, allows preview.
Shader InstanceGenerally a material, allows preview.
Physics ShapeGenerally used with a Physics Shape Component.
Ray Trace InstanceUsed to generate acceleration structures and hit shaders.
Entity Spawner - Output TransformsCan be used to query transforms from the Entity Spawner node.

Shader system interaction

A creation graph interacts with the shader system in three main ways:

  • Its GPU nodes are defined using .tmsl shaders.
  • GPU output nodes call special linker functions to evaluate the creation graph.
  • Shader instances in a creation graph are constructed using the shader system.

The last point is a technical detail that doesn’t matter for anyone extending or using the creation graph so it won’t be covered in this guide. Additional information about the creation_graph_node shader block can be found in the Shader System Reference.

Any GPU node that can be used in the creation graph has an associated .tmsl shader file. Most of these can be found here: the_machinery/shaders/nodes/*.tmsl. We also supply a Visual Studio extension for this file format which adds syntax highlighting, this extension will be used in this guide.

This is the shader code for the Sin node. It defines one input (a) and one output (res, which is the same type as a). This shader file will be constructed using the shader system into a single .hlsl function. For more information on how to create basic GPU nodes see Creating custom GPU Nodes.

This is an example of the shader code needed in the creation graph output nodes. When a creation graph node outputs a Shader Instance and has any inputs; it should define these three functions in it’s shader code block so the graph can be evaluated. The tm_graph_read function passes all the stage input variables to the graph (like position, color, uv, etc.). The tm_graph_evaluate function does most of the work. It uses the tm_graph_io_t struct to evaluate the graph by calling the functions generated by the normal nodes. Finally the tm_graph_write function passes all the graph variable to the stage output. It is important to note that whilst the tm_graph_evaluate function is necessary for graph evaluation; the tm_graph_read and tm_graph_write are not, they are helper function. For more information on how to create GPU output nodes see Creating custom GPU Nodes.

Graphics

Modern rendering architecture

The renderer has been designed to take full advantage of modern explicit graphic APIs like Vulkan. You can reason explicitly about advanced setups such as multiple GPUs and GPU execution queues. Similar to the rest of the engine, the built-in rendering pipeline is easy to tweak and extend.

Supported graphics backends

  • Vulkan 1.2
  • Nil

Creation Graphs

The creation graph is (as the name implies) a graph based tool for creating assets. It allows developers to define and alter various types of assets using visual scripting. More broadly speaking, the creation graph can be used for arbitrary data processing tasks.

A creation graph thus defines the asset’s pipeline into its final state. For instance, an image asset will have a file that defines the data of the image, but the creation graph asset specifies how that data should be processed. Should it generate mipmaps, should it be compressed, do we need a CPU copy of it, etc.

A more familiar example might be a material shader. In The Machinery this is also defined using a creation graph. This case maps very well to Unity’s shader assets and Unreal’s material assets. In the image below you can see a simple default material with a saturated red base color.

Simple red material

Image loading and material creation are just a few examples of what can be achieved with the creation graph. The table below shows when a creation graph is used compared to the tools one could use in Unity and Unreal.

Asset TypeUnityUnrealThe Machinery
ImagesTextureTextureCreation graphs
MaterialsShaderMaterialCreation graphs
ParticlesParticle EffectCascadeCreation graphs
Post processingShaderMaterialCreation graphs
Procedural materialsProcedural MaterialsMaterialCreation graphs
MeshesMeshStatic MeshDCC Asset +
Creation graphs

Another example for the creation graph is mesh processing. A graph like this will define how the mesh should be draw or traced against. The graph below takes two inputs, a material creation graph and the mesh data from a DCC asset. This data is than imported and passed to the various outputs of our mesh pipeline. In this case those are: a ray tracing instance, a normal draw call, a bounding volume, and a physics shape. Note that not all of these outputs have to be used, rather the code that uses this creation graph might only look for the rendering outputs and ignore the physics shape, whilst some other code might only care about the physics shape output.

Mesh processing

Like the entity graph, the creation graph can executes nodes in sequence from an event. Some examples of this are the Tick, Init, and Compile events which are executed at known intervals. Most of the creation graphs however work with a reverse flow, compiling the graph into a sequence of nodes for a specific output. The two examples presented earlier show this workflow. Some outputs are: Draw Call, Shader Instance, Image, and Physics Shape. Note that these outputs are just blobs of data, an implementation can define more output type in code.

Shaders

The Creation Graph provides an artist-friendly way to create custom shaders by wiring together nodes into a shader network. Each node in the graph represents a snippet of HLSL code that gets combined by the shader system plugin into full HLSL programs. It can sometimes be nice to work directly with HLSL code for more advanced shaders, either by exposing new helper nodes to the Creation Graph or by directly writing a complete shader program in HLSL. This is typically done by adding new .tmsl files (where tmsl stands for The Machinery Shading Language) that The Machinery loads on boot up.

A .tmsl file is essentially a data-driven JSON front-end for creating and populating a tm_shader_declaration_o structure which is the main building block that the compiler in shader system plugin operates on. While a tm_shader_declaration_o can contain anything needed to compile a complete shader (all needed shader stages, any states and input/output it needs, etc), it is more common that they contain only fragments of and multiple tm_shader_declaration_o are combined into the final shader source that gets compiled into a tm_shader_o that can be used when rendering a draw call (or dispatching a compute job).

Note: You can find all built-in shaders in the folder: ./bin/data/shaders shipped with your engine version. (For source access: ./the_machinery/shaders)

Inserting the creation_graph block in the .tmsl file will get exposed as a node in the Creation Graph. Nodes exposed to the Creation Graph can either be function nodes (see: data/shaders/nodes/) or output nodes (see: data/shaders/output_nodes/). A function node won't compile into anything by itself unless it's connected to an output node responsible for declaring the actual shader stages and evaluating the branches of connected function nodes.

Note: More information about creating creation graph nodes you can find in the Creation Graph Section:

Typically these are function nodes (see data/shaders/nodes) that won't compile into anything without getting connected to an "output" node. We ship with a few built-in output nodes (see data/shaders/output_nodes) responsible for declaring the actual shader stages and glue everything together.

Note: For more details on the Shader Language itself, please check the Shader Reference or the Chapter The Machinery Shading Language.

The whole Shader System is explained in more detail within these posts:

Custom shaders how?

If you intend to write custom shaders you can. All your custom shaders need to be placed under the bin\data\shaders of the engine. They will be automatically compiled (if needed) on boot up of the Editor. For help with how to write a custom shader please follow the The Machinery Shading Language Guide

The Machinery Shader Language

Shaders in The Machinery are defined using The Machinery Shader Language (tmsl). Traditionally shaders (like those written in glsl or hlsl) only contain the shader code itself and some I/O definitions. The Machinery (like other engines) stores not only the shader code, but also the pipeline state in its shader files. Additionally The Machinery allows shaders to define variations and systems that allow for more complex shader generation and combinations. For a complete list of what can be in a tmsl file see the Shader System Reference. For an in depth look at the design goals of these shader files see The Machinery Shader System blog posts.

A shader file can be divided into three distinct sections:

  • Code blocks, these define HLSL code blocks that contain the main shader code.
  • Pipeline state blocks, these define the Pipeline State Objects (PSO) or shader environment required for the code to run.
  • Compilation blocks, these define meta information about how the shader should be compiled. This also allows for multiple variations of shaders, for instance one with multi-sampling enabled and one with multi-sampling disabled.

Let’s have a look at a Hello Triangle shader for The Machinery.

imports: [
    { name: "color" type: "float3" }
]

vertex_shader: {
    import_system_semantics : [ "vertex_id" ]

    code: [[
        const float2 vertices[] = {
            float2(-0.7f, 0.7f),
            float2(0.7f, 0.7f),
            float2(0.0f, -0.7f)
        };

        output.position = float4(vertices[vertex_id], 0.0f, 1.0f);
        return output;
    ]]
}

pixel_shader: {
    code: [[
        output.color = load_color();
        return output;
    ]]
}

compile: {}

In this example we have some shader code, no explicit pipeline state, and an empty compile block. The first thing to note is that tmsl files use a JSON like format. The main sections of code are the vertex_shader and the pixel_shader blocks. Within these are code blocks which specify the HLSL code that needs to run at the relative pipeline stage. In this example we create a screen-space triangle from three constant vertices and give it a color passed on as an input.

If we want to pass anything to a shader we need to define it in the imports block. Anything defined in here will be accessible though load_# or get_# functions. See the Shader System Reference for more information.

We also need to define a compile or system block in order for our shader to be compiled. If neither block is defined then the shader is assumed to be a library type shader which can be included into other shaders.

Note: You can find all built-in shaders in the folder: ./bin/data/shaders/ in the shipped engine (for source code access this is: ./the_machinery/shaders/).

Procedural shaders

Note that shaders don’t have to be written and compiled in this way. You can generate shaders directly from code using the tm_shader_repository_api. You can create a new shader declaration by calling create_shader_declaration(), populate it with your custom code by using the tm_shader_declaration_api, and compile it using create_from_declaration(). Any tmsl file will go through the same pipeline.

Note: Shader are also used to create GPU nodes for the Creation Graph, see Creation Graph: Shader System Interaction for more information.

The Machinery Shader Language Visual Studio Extension

This Visual Studio extension adds The Machinery's .tmsl language support:

  • Syntax highlighting
  • Snippets

image

Installation

Open and download the extension from VS Marketplace and use it in VS Studio.

VS Code will follow at some point.

Extending The Machinery

In The Machinery, everything is a plugin. You can extend, modify or replace existing engine functionality with your plugins.

The Engine explicitly aims to be simple, minimalistic, and easy to understand. All our code is written in plain C, a significantly more straightforward language than modern C++. The entire codebase compiles in less than 30 seconds, and we support hot-reloading of DLLs, allowing for fast iteration cycles. You can modify your plugin code while the editor or the game runs since the plugin system supports hot-reloading. In short, we want to be "hackable." Our APIs are exposed as C interfaces, which means you can easily use them from C, C++, D, or any other language with an FFI for calling into C code.

Guides to follow:

The plugin system

The Machinery is built around a plugin model. All features, even the built-in ones, are provided through plugins. You can extend The Machinery by writing your own plugins.

When The Machinery launches, it loads all the plugins named tm_*.dll in its plugins/ folder. If you write your own plugins, name them so that they start with tm_ and put them in this folder, they will be loaded together with the built-in plugins.

Table of Content

About API's

The Machinery is organized into individual APIs that can be called to perform specific tasks. A plugin is a DLL that exposes one or several of these APIs. In order to implement its functionality, the plugin may in turn rely on APIs exposed by other plugins.

A central object called the API registry is used to keep track of all the APIs. When you want to use an API from another plugin, you ask the API registry for it. Similarly, you expose your APIs to the world by registering them with the API registry.

This may seem a bit abstract at this point, so let’s look at a concrete example, unicode.h which exposes an API for encoding and decoding Unicode strings:

#pragma once

#include "api_types.h"

struct tm_temp_allocator_i;

// API for converting between UTF-8 encoded text and UTF-16 or UTF-32. All strings in The Machinery
// are UTF-8 encoded, but UTF-16 and UTF-32 are sometimes needed to communicate with external APIs.
// For example, Windows uses UTF-16.
//
// !!! TODO: API-REVIEW
//     * Add `tm_str_t codepoint_range(tm_str_t s)`.

struct tm_unicode_api
{
    // UTF-8

    // Returns *true* if `utf8` is a valid UTF-8 string, *false* otherwise.
    bool (*is_valid)(const char *utf8);

    // Fixes the truncation of a UTF-8 encoded string `utf-8` by replacing any split codepoints at
    // the end of the string with `\0` bytes. You can use this after truncating a string to make
    // sure that the resulting string is still a valid UTF-8 string.
    void (*truncate)(char *utf8);

    // UTF-32

    // Encodes the `codepoint` as UTF-8 into `utf8` and returns a pointer to the position where
    // to insert the next codepoint. `utf8` should have room for at least four bytes (the
    // maximum size of a UTF-8 encoded codepoint).
    char *(*utf8_encode)(char *utf8, uint32_t codepoint);

    // Decodes and returns the first codepoint in the UTF-8 string `utf8`. The string pointer is
    // advanced to point to the next codepoint in the string. Will generate an error message
    // if the string is not a UTF-8 string.
    uint32_t (*utf8_decode)(const char **utf8);

    // Returns the number of codepoints in `utf8`.
    uint32_t (*utf8_num_codepoints)(const char *utf8);

    // Decodes the first `n` codepoints in `utf8` to the `codepoints` buffer. If `utf8`
    // contains fewer than `n` codepoints -- decodes as many codepoints there are in `utf8`.
    // Returns the number of decoded codepoints.
    uint32_t (*utf8_decode_n)(uint32_t *codepoints, uint32_t n, const char *utf8);

    // Converts a UTF-8 encoded string to a UTF-32 encoded one, allocated with the supplied
    // temp allocator. Will generate an error message if the string is not a UTF-8 string.
    uint32_t *(*utf8_to_utf32)(const char *utf8, struct tm_temp_allocator_i *ta);

    // As [[utf8_to_utf32()]], but uses an explicit length instead of a zero terminated string. Note
    // that the result string will still be zero terminated.
    uint32_t *(*utf8_to_utf32_n)(const char *utf8, uint32_t n, struct tm_temp_allocator_i *ta);

    // Converts a UTF-32 encoded string to a UTF-8 encoded one, allocated with the specified temp
    // allocator. Generates an error if the data is outside the UTF-8 encoding range.
    char *(*utf32_to_utf8)(const uint32_t *utf32, struct tm_temp_allocator_i *ta);

    // As [[utf32_to_utf8()]], but uses an explicit length instead of a zero terminated string. Note
    // that the result string will still be zero terminated.
    char *(*utf32_to_utf8_n)(const uint32_t *utf32, uint32_t n, struct tm_temp_allocator_i *ta);

    // UTF-16

    // Encodes the codepoint as UTF-16 into `utf16` and returns a pointer to the position where to
    // insert the next codepoint. `utf16` should have at room for at least two `uint16_t` (the
    // maximum size of a UTF-16 encoded codepoint).
    uint16_t *(*utf16_encode)(uint16_t *utf16, uint32_t codepoint);

    // Decodes and returns the first codepoint in the UTF-16 string `utf16`. The string pointer is
    // advanced to point to the next codepoint in the string.
    uint32_t (*utf16_decode)(const uint16_t **utf16);

    // Converts a UTF-8 encoded string to a UTF-16 encoded one, allocated with the supplied temp
    // allocator. Will generate an error message if the data is outside the UTF-8 encoding range.
    uint16_t *(*utf8_to_utf16)(const char *utf8, struct tm_temp_allocator_i *ta);

    // As [[utf8_to_utf16()]] but uses an explicit length instead of a zero terminated string. Note
    // that the result string will still be zero terminated.
    uint16_t *(*utf8_to_utf16_n)(const char *utf8, uint32_t n, struct tm_temp_allocator_i *ta);

    // Converts a UTF-16 encoded string to a UTF-8 encoded one, allocated with the specified
    // temp allocator. Will generate an error message if the string is not a UTF-16 string.
    char *(*utf16_to_utf8)(const uint16_t *utf16, struct tm_temp_allocator_i *ta);

    // As [[utf16_to_utf8()]] but uses an explicit length instead of a zero terminated string. Note
    // that the result string will still be zero terminated.
    char *(*utf16_to_utf8_n)(const uint16_t *utf16, uint32_t n, struct tm_temp_allocator_i *ta);
};

#define tm_unicode_api_version TM_VERSION(1, 0, 0)

// Returns a UTF-8 string representing the codepoint `cp`. The string is stack allocated.
#define tm_codepoint_to_utf8(cp) tm_codepoint_to_utf8_internal(cp, (char[5]){ 0 })

#if defined(TM_LINKS_FOUNDATION)
extern struct tm_unicode_api *tm_unicode_api;
#endif

Let’s go through this.

First, the code includes <api_types.h>. This is a shared header with common type declarations, it includes things like <stdbool.h> and <stdint.h> and also defines a few The Machinery specific types, such as tm_vec3_t.

In The Machinery we have a rule that header files can't include other header files (except for <api_types.h>). This helps keep compile times down, but it also simplifies the structure of the code. When you read a header file you don’t have to follow a long chain of other header files to understand what is happening.

Next follows a block of forward struct declarations (in this case only one).

Next, we have the name of this API defined as a constant tm_unicode_api, followed by the struct tm_unicode_api that defines the functions in the API.

To use this API, you would first use the API registry to query for the API pointer, then using that pointer, call the functions of the API:

static struct tm_unicode_api *tm_unicode_api;
#include <foundation/api_registry.h>
#include <foundation/unicode.h>

static void demo_usage(char *utf8, uint32_t codepoint)
{
    tm_unicode_api->utf8_encode(utf8, codepoint);
    //more code...
}

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_unicode_api = tm_get_api(reg, tm_unicode_api);
}

The different APIs that you can query for and use are documented in their respective header files, and in the apidoc.md.html documentation file (which is just extracted from the headers). Consult these files for information on how to use the various APIs that are available in The Machinery.

In addition to APIs defined in header files, The Machinery also contains some header files with inline functions that you can include directly into your implementation files. For example <math.inl> provides common mathematical operations on vectors and matrices, while <carray.inl> provides a “stretchy-buffer” implementation (i.e. a C version of C++’s std::vector).

About Interfaces

We also add an implementation of the unit test interface to the registry. The API registry has support for both APIs and interfaces. The difference is that APIs only have a single implementation, whereas interfaces can have many implementations. For example, all code that can be unit-tested implements the unit test interface. Unit test programs can query the API registry to find all these implementations and run all the unit tests.

To extend the editor you add implementations to the interfaces used by the editor. For example, you can add implementations of the tm_the_truth_create_types_i in order to create new data types in The Truth, and add implementations of the tm_entity_create_component_i in order to define new entity components. See the sample plugin examples.

It does not matter in which order the plugins are loaded. If you query for a plugin that hasn’t yet been registered, you get a pointer to a nulled struct back. When the plugin is loaded, that struct is filled in with the actual function pointers. As long as you don’t call the functions before the plugin that implements them has been loaded, you are good. (You can test this by checking for NULL pointers in the struct.)

Hot-Reloading

We support hot-reloading of plugins while The Machinery is running. This allows you to work on a plugin and see the changes in real-time without having to shut down and restart the application between each change to your code.

Hot-reloading is enabled by default, but can be disabled with the --no-hot-reload parameter.

When a reload happens, the function pointers in the plugin's API struct are replaced with function pointers to the new code. Since clients hold pointers to this struct, they will use the new function pointers automatically -- they don't have to re-query the system for the API.

Note that hot-reloading is not magical and can break in a lot of situations. For example, if you remove functions in the API or change their parameters, any clients of your API will still try to call them using the old parameter lists, and things will most likely crash. Similarly, if a client has stashed away a function pointer to one of your API functions somewhere, such as in a list of callbacks, there is no way for us to patch that copy and it will continue to call the old code. Also, if you make changes to the layout of live data objects (such as adding or removing struct fields) things will break because we make no attempts to transfer the data to the new struct format.

But adding or removing static functions, or changing the code inside functions should work without problems. We find hot-reloading to be a big time saver even if it doesn't work in all circumstances.

If you want to use global variables in your DLL you should do so using the tm_api_registry_api->static_variable() function in your tm_load_plugin() code. If you just declare a global variable in your .c file, that variable will be allocated in the DLLs memory space and when the DLL is reloaded you will lose all changes to the variable. When you use static_variable(), the variable is allocated on the heap, and its content is preserved when the DLL is reloaded.

If you are using hot-reloading together with a debugger on Windows, be aware that the debugger will lock .pdb files which will prevent you from rebuilding your code. The suggested workflow is something like this:

  • Detach the debugger if it's currently attached.
  • Rebuild your DLL and fix any compiler bugs.
  • When the DLL is built successfully, The Machinery will automatically reload it.
  • If you need to continue debugging, re-attach the debugger.

Write a plugin

This walkthrough shows you how to extend the engine with a custom plugin.

You will learn about:

  • What is needed to write a plugin
  • How to write a plugin

This walk through expects you to have the basic understanding about the plugin system. Otherwise you can read more here.

Table of Content

Where does the engine search for plugins

The Machinery is built around a plugin model. All features, even the built-in ones, are provided through plugins. You can extend The Machinery by writing your own plugins. When The Machinery launches, it loads all the plugins named tm_*.dll in its plugins/ folder. If you write your own plugins, name them so that they start with tm_ and put them in this folder, they will be loaded together with the built-in plugins.

Note: When you create a new plugin via the Engine, the premake file will not copy the plugin into your global plugin folder. The reason behind this is that we do not know if you want to create a plugin asset. This workflow is currently under review.

Important: The plugins created via the Engine expect a binary build version if you are using the source access version you might have to modify the premake file to make it point to the correct version. This workflow is currently under review.

Inspect an existing example to get inspiration

The easiest way to build a plugin is to start with an existing example. There are three places where you can find plugin samples:

  1. The samples folder in the SDK has a number of plugin samples.

  2. The All Sample Projects package in the Download tab has a plugins folder with some small samples.

  3. You can create a new plugin with the menu command File > New Plugin. This will create a new .c file for the plugin together with some helper files for compiling it. (Follow this guide)

The distribution already comes with pre-built .dlls for the sample plugins, such as bin/plugins/tm_pong_tab.dll. You can see this plugin in action by selecting Tab > Pong in the editor to open up its tab:

Pong tab.

What are the build Requirements

To build plugins you need three things:

  1. You need to have Visual Studio 2019 installed including the MS C++ Build Tools on your computer. Note that the Community Edition works fine. (Or clang and the build essentials on Linux)
  2. You need to set the TM_SDK_DIR environment variable to the path of the SDK package that you installed on your computer. When you compile a plugin, it looks for The Machinery headers in the %TM_SDK_DIR%/headers folder.
  3. You need the tmbuild.exe from the SDK package. tmbuild.exe does all the steps needed to compile the plugin. Put it in your PATH or copy it to your plugin folder so that you can run it easily from the command line.

Build the sample plugin

To compile a plugin, simply open a command prompt in the plugin folder and run the tmbuild.exe executable:

sample-projects/plugins/custom_tab> %TM_SDK_DIR%/bin/tmbuild.exe
​~~~ cmd output
Installing 7za.exe...
Installing premake-5.0.0-alpha14-windows...
Building configurations...
Running action 'vs2019'...
Generated custom_tab.sln...
Generated build/custom_tab/custom_tab.vcxproj...
Done (133ms).
Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  custom_tab.c
  custom_tab.vcxproj -> C:\work\themachinery\build\bin\plugins\tm_custom_tab.dll

-----------------------------
tmbuild completed in: 23.471 s

tmbuild.exe will perform the following steps to build your executable:

  1. Create a libs folder and download premake5 into it. (You can set the TM_LIB_DIR environment variable to use a shared libs directory for all your projects.)
  2. Run premake5 to create a Visual Studio project from the premake5.lua script.
  3. Build the Visual Studio project to build the plugin.

Note: You can learn more about tmbuild in its own section.

Programming in C

The Machinery uses C99 as its interface language. I.e., all the header files that you use to communicate with The Machinery are C99 header files, and when you write a plugin you should expose C99 headers for your APIs. The implementation of a plugin can be written in whichever language you like, as long as it exposes and communicates through a C99 header. In particular, you can write the implementation in C++ if you want to. (At Our Machinery, we write the implementations in C99.)

Write your own plugin

To write a plugin you need to implement a tm_load_plugin() function that is called whenever the plugin DLL is loaded/unloaded. In this function, you can interact with various engine interfaces. For example, you can implement tm_unit_test_i to implement unit tests that get run together with the engine unit tests, you can implement tm_the_truth_create_types_i to extend our data model, The Truth, with your own data types or implement tm_entity_create_component_i to extend our entity model with your own component types.

The following guides might help you:

The Engine provides an easy way to create plugins for you via the File -> New Plugins menu. There you can choose default plugin templates. They come with default files:

custom tab folder view

The folder structure for a custom tab called custom_tab.

  • premake5.lua - Your build configuration, on Windows it will generate a .sln file for you.
  • libs.json - Defines the binary dependencies of your projects. tmbuild will automatically download them for you.
  • *.c - Your source file. It contains the sample template code to give you some guidance on what is needed.
  • build.bat / build.sh - quick build files to make building simpler for you.

Note: By default the plugin will not be copied into your Engine's plugins folder. You can modify the premake file or copy it manually in the folder. You can also make use of a Plugin Asset.

Structure of a plugin

Every plugin has tm_load_plugin() as its entry point, in there we register everything we need to register to the Engines Plugin System. It is important that you do not execute heavy code in this function or rely on other plugins, since they might not be loaded yet! This function is just there to perform load and register operations.

Where does my gameplay code live?

Your gameplay lives within the Systems / Engines of the ECS or in the Simulation Entry and they have their own entry points.

How do I update my tool / tab content?

  • The tm_tab_vt defines three functions for your tab to be updated:
    • tm_tab_vt.ui() - Callback for drawing the content of the tab into the specified rect.
    • tm_tab_vt.ui_serial() - Optional. If implemented, called from the main UI job once all parallel UI rendering (fork/join) has finished. This can be used for parts of the UI that needs to run serially, for example because they call out to a non-thread-safe function.
    • tm_tab_vt.hidden_update() - Optional. If the tab wants to do some processing when it is not the selected tab in its tabwell, it can implement this callback. This will be called for all created tabs whose content is currently not visible.

For more information follow the "Write a tab" walkthrough.

Plugin callbacks (Init, Shutdown, Tick)

The plugin system provides also for plugin callbacks. It is recommended to rely on these calls as little as possible. You should not rely on those for your gameplay code!

  • tm_plugin_init_i - Is typically called as early as possible after all plugins have been loaded.

Note: It is not called when a plugin is reloaded.

  • tm_plugin_shutdown_i - Is typically be called as early as possible during the application shutdown sequence

Note: Is not called when a plugin is reloaded.

  • tm_plugin_tick_i - Is typically called as early as possible in the application main loop “tick”.

They are stored in the foundation/plugin_callbacks.h.

How do I deal with static variables?

The use of static variables in DLLs can be problematic, because when the DLL is reloaded, the new instance of the DLL will get a new freshly initialized static variable, losing whatever content the variable had before reload. The tm_api_registry_api provides a way to solve this issue: tm_api_registry_api.static_variable()

By using this function instead of defining it globally, the variable data is saved in permanent memory.

#include <foundation/api_registry.h>

uint64_t *count_ptr;

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    count_ptr = (uint64_t *)reg->static_variable(TM_STATIC_HASH("my_count", 0xa287d4b3ec9c2109ULL), sizeof(uint64_t), __FILE__, __LINE__);
}

void f()
{
    ++*count_ptr;
}

Write your own API

You can also create your own APIs that other plugins can query for. If you create your own APIs, you want to define them in your header file, so that other plugins can #include it and know how to call your APIs.

Note that if you are not defining your own APIs, but just implementing some of the engine's ones, your plugin typically doesn't need a header file.

my_plugin.h:

#include "foundation/api_types.h"

struct my_api
{
    void (*foo)(void);
};

#define my_api_version TM_VERSION(1, 0, 0)

my_plugin.c:

static struct tm_api_registry_api *tm_global_api_registry;
static struct tm_error_api *tm_error_api;
static struct tm_logger_api *tm_logger_api;

#include "my_api.h"

#include "foundation/api_registry.h"
#include "foundation/error.h"
#include "foundation/log.h"
#include "foundation/unit_test.h"

static void foo(void)
{
    // ...
}

static struct my_api *my_api = &(struct my_api){
    .foo = foo,
};

static void my_unit_test_function(tm_unit_test_runner_i *tr, struct tm_allocator_i *a)
{
    // ...
}

static struct tm_unit_test_i *my_unit_test = &(struct tm_unit_test_i){
    .name = "my_api",
    .test = my_unit_test_function,
};

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_global_api_registry = reg;

    tm_error_api = tm_get_api(reg, tm_error_api);
    tm_logger_api = tm_get_api(reg, tm_logger_api);

    tm_set_or_remove_api(reg, load, my_api, my_api);

    tm_add_or_remove_implementation(reg, load, tm_unit_test_i, my_unit_test);
}

When The Machinery loads a plugin DLL, it looks for the tm_load_plugin() function and calls it. If it can't find the function, it prints an error message. We store the API registry pointer in a static variable so that we can use it everywhere in our DLL. We also tm_get_api() some of the API pointers that we will use frequently and store them in static variables so that we don’t have to use the registry to query for them every time we want to use them. Finally, we add our own API to the registry, so others can query for and use it.

Write a Tab

This walkthrough shows you how to add a custom Tab to the Engine.

During this walkthrough, we will cover the following topics:

  • How to create a tab from scratch.
  • Where and how do we register the Tab to the Engine.

You should have basic knowledge about how to write a custom plugin. If not, you might want to check this Guide and the Write a plugin guide. The goal of this walkthrough is to dissect the Tab plugin provided by the Engine.

Where do we start?

In this example, we want to create a new plugin, which contains our Tab. We open the Engine go to file -> New Plugin -> Editor Tab. The file dialog will pop up and ask us where we want to save our file. Pick a location that suits you.

Tip: Maybe store your plugin in a folder next to your game project.

After this, we see that the Engine created some files for us.

folder structure new plugin

Now we need to ensure that we can build our project. In the root folder (The folder with the premake file), we can run tmbuild and see if there is no issue. We will build our projects once and generate the .sln file (on windows).

If there is an issue, we should ensure we have set up the Environment variables correctly and installed all the needed dependencies. For more information, please read this guide.

Now we can open the .c file with our favorite IDE. The file will contain the following content:

static struct tm_api_registry_api *tm_global_api_registry;

static struct tm_draw2d_api *tm_draw2d_api;
static struct tm_ui_api *tm_ui_api;
static struct tm_allocator_api *tm_allocator_api;

#include <foundation/allocator.h>
#include <foundation/api_registry.h>

#include <plugins/ui/docking.h>
#include <plugins/ui/draw2d.h>
#include <plugins/ui/ui.h>
#include <plugins/ui/ui_custom.h>

#include <the_machinery/the_machinery_tab.h>

#include <stdio.h>

#define TM_CUSTOM_TAB_VT_NAME "tm_custom_tab"
#define TM_CUSTOM_TAB_VT_NAME_HASH TM_STATIC_HASH("tm_custom_tab", 0xbc4e3e47fbf1cdc1ULL)

struct tm_tab_o
{
    tm_tab_i tm_tab_i;
    tm_allocator_i allocator;
};

static void tab__ui(tm_tab_o *tab, tm_ui_o *ui, const tm_ui_style_t *uistyle_in, tm_rect_t rect)
{
    tm_ui_buffers_t uib = tm_ui_api->buffers(ui);
    tm_ui_style_t *uistyle = (tm_ui_style_t[]){*uistyle_in};
    tm_draw2d_style_t *style = &(tm_draw2d_style_t){0};
    tm_ui_api->to_draw_style(ui, style, uistyle);

    style->color = (tm_color_srgb_t){.a = 255, .r = 255};
    tm_draw2d_api->fill_rect(uib.vbuffer, *uib.ibuffers, style, rect);
}

static const char *tab__create_menu_name(void)
{
    return "Custom Tab";
}

static const char *tab__title(tm_tab_o *tab, struct tm_ui_o *ui)
{
    return "Custom Tab";
}

static tm_tab_vt *custom_tab_vt;

static tm_tab_i *tab__create(tm_tab_create_context_t *context, tm_ui_o *ui)
{
    tm_allocator_i allocator = tm_allocator_api->create_child(context->allocator, "Custom Tab");
    uint64_t *id = context->id;

    tm_tab_o *tab = tm_alloc(&allocator, sizeof(tm_tab_o));
    *tab = (tm_tab_o){
        .tm_tab_i = {
            .vt = custom_tab_vt,
            .inst = (tm_tab_o *)tab,
            .root_id = *id,
        },
        .allocator = allocator,
    };

    *id += 1000000;
    return &tab->tm_tab_i;
}

static void tab__destroy(tm_tab_o *tab)
{
    tm_allocator_i a = tab->allocator;
    tm_free(&a, tab, sizeof(*tab));
    tm_allocator_api->destroy_child(&a);
}

static tm_tab_vt *custom_tab_vt = &(tm_tab_vt){
    .name = TM_CUSTOM_TAB_VT_NAME,
    .name_hash = TM_CUSTOM_TAB_VT_NAME_HASH,
    .create_menu_name = tab__create_menu_name,
    .create = tab__create,
    .destroy = tab__destroy,
    .title = tab__title,
    .ui = tab__ui};

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_global_api_registry = reg;

    tm_draw2d_api = tm_get_api(reg, tm_draw2d_api);
    tm_ui_api = tm_get_api(reg, tm_ui_api);
    tm_allocator_api = tm_get_api(reg, tm_allocator_api);

    tm_add_or_remove_implementation(reg, load, tm_tab_vt, custom_tab_vt);
}

Code structure

Let us dissect the code structure and discuss all the points of interest.

API and include region

The file begins with all includes and API definitions:

static struct tm_api_registry_api *tm_global_api_registry;

static struct tm_draw2d_api *tm_draw2d_api;
static struct tm_ui_api *tm_ui_api;
static struct tm_allocator_api *tm_allocator_api;

#include <foundation/allocator.h>
#include <foundation/api_registry.h>

#include <plugins/ui/docking.h>
#include <plugins/ui/draw2d.h>
#include <plugins/ui/ui.h>
#include <plugins/ui/ui_custom.h>

#include <the_machinery/the_machinery_tab.h>

#include <stdio.h>

#define TM_CUSTOM_TAB_VT_NAME "tm_custom_tab"
#define TM_CUSTOM_TAB_VT_NAME_HASH TM_STATIC_HASH("tm_custom_tab", 0xbc4e3e47fbf1cdc1ULL)

The code will fill the API definitions with life in the tm_load_plugin function.

The most important aspects here are the two defines on the bottom:

#define TM_CUSTOM_TAB_VT_NAME "tm_custom_tab"
#define TM_CUSTOM_TAB_VT_NAME_HASH TM_STATIC_HASH("tm_custom_tab", 0xbc4e3e47fbf1cdc1ULL)

The first one defines the name of our Tab and the second one represents its hash value. The hash value can be used later on to access, search the Tab in the tm_docking_api.

Note: If you modify the values, please ensure you ran hash.exe again or tmbuild --gen-hash so the hash value is updated!

Define your Data

In the next section, we define the data the Tab can hold. It might be any data you need for the Tab to work and do its job. The tab instance owns the data. It is not shared between Tabs instances. Therefore its lifetime is bound to the current instance.

struct tm_tab_o
{
    tm_tab_i tm_tab_i;
    tm_allocator_i allocator;
};

A tm_tab_i represents a tab object. A tab object is represented as a vtable that defines its function interface and an opaque pointer to the Tab's internal data. This design is used so that the application layer can extend the vtable with its own interface.

Define the actual Tab

Every Tab in The Machinery is based on the tm_tab_vt and registered to the tm_tab_vt in the tm_load_plugin() function.

The default tm_tab_vt offers multiple options and settings we can set for our Tab.

NameDescription
tm_tab_vt.nameName uniquely identifying this tab type.
tm_tab_vt.name_hashA hash of the name.
tm_tab_vt.create_menu_name()Optional. Returns the (localized) name that should be shown for this tab type in menus that allow you to create new tabs. If this function returns NULL, the tab type won't appear in these menus. This can be used for tabs that should only be accessible when certain feature flags are set.
tm_tab_vt.create_menu_category()Optional. Returns the (localized) category that should be shown for this tab type in menus that allow you to create new tabs. If this function returns NULL or is not set, the tab type will appear at the root level of the menu, uncategorized.
tm_tab_vt.create()Creates a new tab of this type and returns a pointer to it. tm_tab_create_context_t is an application defined type containing all the data a tab needs in order to be created. ui s the UI that the tab will be created in.
tm_tab_vt.destroy()Destroys the tab
Object methods
tm_tab_vt.ui()Callback for drawing the content of the tab into the specified rect. The uistyle is the tm_ui_api.default_style() with the clipping rect set to rect.
tm_tab_vt.ui_serial()Optional. If implemented, called from the main UI job once all parallel UI rendering (fork/join) has finished. This can be used for parts of the UI that needs to run serially, for example because they call out to non-thread-safe function.
tm_tab_vt.hidden_update()This function is optional. If the Tab wants to do some processing when it is not the selected Tab in its tabwell, it can implement this callback. This will be called for all created tabs whose content is currently not visible.
tm_tab_vt.title()Returns the localized title to be displayed for the tab. This typically consists of the name of the tab together with the document that is being edited, such as "Scene: Kitchen*"
tm_tab_vt.set_root()Optional. Sets the root object of the tab. If a new Truth is loaded, this is called with set_root(inst, new_tt, 0).
tm_tab_vt.root()Returns the root object and The Truth that is being edited in the tab. This is used, among other things to determine the undo queue that should be used for Undo/Redo operations when the tab has focus
tm_tab_vt.restore_settings()Optional. Allow the tab to restore it's own state to the settings. For example the Asset Browser will use this to save the view size of the assets.
tm_tab_vt.save_settings()Optional. Allow the tab to save it's own state to the settings. For example the Asset Browser will use this to save the view size of the assets.
tm_tab_vt.can_close()Optional. Returns true if the tab can be closed right now and false otherwise. A tab might not be able to close if it's in the middle of an important operation. Tabs that do not implement this method can be closed at any time.
tm_tab_vt.focus_event()documentation
tm_tab_vt.feed_events()Optional. For feeding events to the tab. Useful for feeding events to UIs that are internal to a tab.
tm_tab_vt.process_dropped_os_files()Optional. If set, the tab will receive the path to the files that were dropped from the OS since the previous frame.
tm_tab_vt.toolbars()Optional. Returns a carray of toolbars to be drawn in the tab, allocated using ta. How to add toolbars
tm_tab_vt.need_update()Optional. Allow the tab to decide whether it's UI needs an update. Tabs that have animated components like the pong tab will return always true, while other tab may decide to return true only under certain circumstances. If not provided, the assumed default value will be true, so the tab will be updated every frame. If it returns false the UI will be cached. Therefore any call to .ui wont be called.
tm_tab_vt.hot_reload()Optional. Will be called after any code hot reload has happened.
tm_tab_vt.entity_context()Optional. Should be implemented if tab owns an entity context.
tm_tab_vt.viewer_render_args()Optional. Should be implemented if tab owns an entity context that supports to be rendered outside of it's UI callbacks.
Flags
tm_tab_vt.cant_be_pinnedIf set to true, the tab can't be pinned even though it has a root function.
tm_tab_vt.run_as_jobIf set to true, the tab's UI will run as a background job, parallel to the rest of the UI rendering. Warning: Setting this to true indicates to the docking system that the ui() function is thread-safe. If the function is not actually thread-safe you will see threading errors.
tm_tab_vt.dont_restore_at_startupIf set to true, the tab will be considered volatile, and it won't be restored when the last opened project is automatically opened at startup, even if the user had the tab opened when the project was closed.
tm_tab_vt.dont_restore_root_asset_at_startupIf set to true, the tab will be restored at startup, but the root of the tab won't be set to the one that was set during application shutdown. Basically the project will be restored, but it will be always empty.

In this example, we make use of the following options:

static tm_tab_vt *custom_tab_vt = &(tm_tab_vt){
    .name = TM_CUSTOM_TAB_VT_NAME,
    .name_hash = TM_CUSTOM_TAB_VT_NAME_HASH,
    .create_menu_name = tab__create_menu_name,
    .create = tab__create,
    .destroy = tab__destroy,
    .title = tab__title,
    .ui = tab__ui};

In the cause of the rest of this walkthrough, we will discuss:tab__create_menu_name, tab__create, tab__destroy , tab__title and tab__ui.

Define the metadata functions

As we can see in our definition of the custom_tab_vt object we provide the tm_tab_vt.create_menu_name() and the tm_tab_vt.title(). The create_menu_name is an optional function to allow you to provide a name for the create tab menu. In contrast, the title() function is not optional and is needed. It provides the name of the Tab, which the editor shall show in the tab bar.

static const char *tab__create_menu_name(void)
{
    return "Custom Tab";
}

static const char *tab__title(tm_tab_o *tab, struct tm_ui_o *ui)
{
    return "Custom Tab";
}

Define create and destroy the Tab

As mentioned before, the data of a tab is bound to its lifetime. Therefore you should create the data on create and let go of it on destroy.

The create function provides you the tm_tab_create_context_t access to many essential things, such as an allocator. This allocator is the one you should use directly or create a child allocator.

Note: for more information check tm_tab_create_context_t's documentation.

static tm_tab_vt *custom_tab_vt;

static tm_tab_i *tab__create(tm_tab_create_context_t *context, tm_ui_o *ui)
{
    tm_allocator_i allocator = tm_allocator_api->create_child(context->allocator, "Custom Tab");
    uint64_t *id = context->id;

    tm_tab_o *tab = tm_alloc(&allocator, sizeof(tm_tab_o));
    *tab = (tm_tab_o){
        .tm_tab_i = {
            .vt = custom_tab_vt,
            .inst = (tm_tab_o *)tab,
            .root_id = *id,
        },
        .allocator = allocator,
    };

    *id += 1000000;
    return &tab->tm_tab_i;
}

We use the provided allocator to allocate the Tab struct, and then we initialize it with the data we deem to be needed.

    tm_tab_o *tab = tm_alloc(&allocator, sizeof(tm_tab_o));
    *tab = (tm_tab_o){
        .tm_tab_i = {
            .vt = custom_tab_vt,
            .inst = (tm_tab_o *)tab,
            .root_id = *id,
        },
        .allocator = allocator,
    };

Since we have allocated something, we need to keep track of the used allocator! Hence we have it as a member in our Tab struct.

In the end, we pass a pointer to the Tab interface.

 return &tab->tm_tab_i;

When it comes to free the Tab data, we can just call tm_free() on our Tab:

static void tab__destroy(tm_tab_o *tab)
{
    tm_allocator_i a = tab->allocator;
    tm_free(&a, tab, sizeof(*tab));
    tm_allocator_api->destroy_child(&a);
}

Define the UI update

In the default example, we create a Tab that only updates when the Tab is active and visible. Therefore we do not need the tm_tab_vt.hidden_update() function and can just implement the required one: tm_tab_vt.ui().

The Tab itself shall not be jobifed since run_as_job is not provided (its default value is false). Therefore we know our function itself may contain none thread safe elements.

If we wanted to make our Tab jobifed, we could make use of the tm_tab_vt.hidden_update() function. This function is optional. If the Tab wants to do some processing when it is not the selected Tab in its tabwell, it can implement this callback. This will be called for all created tabs whose content is currently not visible.

Let us digest the current code line by line:

static void tab__ui(tm_tab_o *tab, tm_ui_o *ui, const tm_ui_style_t *uistyle_in, tm_rect_t rect)
{
    tm_ui_buffers_t uib = tm_ui_api->buffers(ui);
    tm_ui_style_t *uistyle = (tm_ui_style_t[]){*uistyle_in};
    tm_draw2d_style_t *style = &(tm_draw2d_style_t){0};
    tm_ui_api->to_draw_style(ui, style, uistyle);

    style->color = (tm_color_srgb_t){.a = 255, .r = 255};
    tm_draw2d_api->fill_rect(uib.vbuffer, *uib.ibuffers, style, rect);
}

The tm_docking_api, which will call our Tab's update, provides us with the essential information:

  • tm_tab_o* tab our tab data to access any data we need
  • tm_ui_o* ui an instance of the UI, needed to call the tm_ui_api
  • const tm_ui_style_t* uistyle_in an instance of the current UI style, can be used to create a local version of it to modify the UI Style for this Tab.
  • tm_rect_t rect the render surface of the Tab.

In the first line of the function body, we create a new instance of the UI Buffers. You may use them to access the underlying buffers for calls to thetm_draw2d_api.Also, this object allows access to the commonly shared metrics and colors.

tm_ui_buffers_t uib = tm_ui_api->buffers(ui);

After this, we define our local copy of the UI Style. Then we create an empty tm_draw2d_style_t instance. We need to create a Style from the UI Style. You need tm_draw2d_style_t* style later for drawing anything with our draw 2d api.

    tm_ui_style_t *uistyle = (tm_ui_style_t[]){*uistyle_in};
    tm_draw2d_style_t *style = &(tm_draw2d_style_t){0};
    tm_ui_api->to_draw_style(ui, style, uistyle);

Now we are set, and we can finally color our tab background to red. You can do this with the tm_draw2d_api.fill_rect() call. Beforehand we need to change our style's color to red and then call the tm_draw2d_api.fill_rect(). We need to pass in the vertex buffer and the index buffer pointer so the function can draw into them.

    style->color = (tm_color_srgb_t){.a = 255, .r = 255};
    tm_draw2d_api->fill_rect(uib.vbuffer, *uib.ibuffers, style, rect);

Note: For more information on the rational behind the UI System please check out this blog post https://ourmachinery.com/post/one-draw-call-ui/

Register the Tab

The last thing before we can compile our project and test it in the Engine is registering the Tab to the Plugin System. As mentioned before, you need to register the Tab to the: tm_tab_vt .

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_global_api_registry = reg;

    tm_draw2d_api = tm_get_api(reg, tm_draw2d_api);
    tm_ui_api = tm_get_api(reg, tm_ui_api);
    tm_allocator_api = tm_get_api(reg, tm_allocator_api);

    tm_add_or_remove_implementation(reg, load, tm_tab_vt, custom_tab_vt);
}

Plugin assets

When you put a plugin in the plugin folder, it will be loaded every time you start The Machinery and used by all projects. This is convenient, but sometimes you want plugins that are project specific, e.g., the gameplay code for a particular game.

Table of Content

The two ways of achieving plugin only assets

There are two ways of doing this.

First, you could create a separate The Machinery executable folder for that specific project. Just make a copy of the The Machinery folder and add any plugins you need. Whenever you want to work on that project, make sure to start that executable instead of the standard one.

In addition to adding project specific plugins, this method also lets you do additional things, such as using different versions of The Machinery for different projects and remove any of the standard plugins that you don't need in your project.

The second method is to store plugins as assets in the project itself. To do this, create a New Plugin in the Asset Browser and set the DLL path of the plugin to your DLL. We call this a Plugin Asset.

The Plugin Assets will be loaded whenever you open the project and unloaded whenever you close the project. Since the plugin is distributed with the project, if you send the project to someone, they will automatically get the plugin too -- they don't have to manually install into their plugin folder. This can be a convenient way of distributing plugins.

WARNING: Security Warning

Since plugin assets can contain arbitrary code and there is no sandboxing, when you run a plugin asset, it will have full access to your machine. Therefore, you should only run plugin assets from trusted sources. When you open a project that contains plugin assets, you will be asked if you want to allow the code to run on your machine or not. You should only click [Allow] if you trust the author of the project.

NOTE: Version Issues

Since The Machinery is still in early adopters mode and doesn't have a stable API, plugins will only work with the specific version they are developed for. If you send a plugin to someone else (for example as a plugin asset in a project), you must make sure that they use the exact same version of The Machinery. Otherwise, the plugin will most likely crash.

How to create a plugin asset

You can create a plugin asset in the Asset Browser. Righ Click -> New -> New Plugin. This will create a plugin asset in your asset browser. On its own this is quite useless. When you select it you can set the DLL Path for your plugin on windows or on linux. The moment you have selected the path to the dll. It will be imported and stored in the asset.

Note: The asset plugin will store the path absolute.

The plugin asset settings look as following:

You would have to repeat the above described workflow every time you change the code of your plugin. This is very annoying, but do not worry hot-reloading comes to rescue!

You can enable hot-reload for plugin assets by checking the Import When Changed checkbox (1) in the plugin properties. If checked, the editor will monitor the plugin's import path for changes and if it detects a file change, it will reimport the plugin.

The Windows & Linux DLL Path (2) can be used to provide the path to the DLLs for the importing the plugin. Plugin Assets need to obey the same rules as normal plugins. Therefore they need to provide the TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load) function. In case this is not possible because the DLL is a helper the helper check box can be called.

Application Hook's

The Machinery allows you to hook your code into specific customization points. Those points happen in different phases and have specific purposes. The biggest difference between the Runner and the Editor is that only the customization points differ in the central update loop.

Table of Content

Application Create

Update

Important side note here tm_plugin_tick_i should not be used for gameplay. To manage your gameplay you should rely on the given gameplay hooks:

  • Entity Component Systems
  • Entity Component Engines
  • Simulation Entry

They are the only recommended way of handling gameplay in the Engine.

Note: Plugin reloads only happen if a plugin has been identified as replaced.

Editor

Runner

Project Hooks

Application Shutdown

Overview

InterfaceDescription
TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)Entry point for all plugins
tm_plugin_init_iIs typically called as early as possible after all plugins have been loaded. Is not called when a plugin is reloaded.
tm_plugin_set_the_truth_iIs called whenever the "main" Truth of the application changes. The "main" Truth is the primary Truth used for editing data in the application. Under API Review
tm_render_pipeline_vt
tm_the_machinery_project_loaded_iIs called when a project is loaded.
tm_plugin_reload_iIs called whenever plugins are reloaded after the reload finishes.
tm_plugin_tick_iIs typically called as early as possible in the application main loop "tick".
A tab that is registered to tm_tab_vt and has a tm_tab_vt.ui() or tm_tab_vt.hidden_update() function.Interface name for the tab vtable. Any part of the UI that can act as a tab should implement this interface.
tm_entity_register_engines_simulation_i_version Is called at the beginning of a simulation (start up phase) and all Systems / Engines are registered to the entity context. tm_entity_system_i.update() or tm_engine_i.update()Used to register a tm_entity_register_engines_i that should run in simulation mode with. More information in the designated chapter. Entity Component System
tm_entity_register_engines_editor_i_versionUsed to register a tm_entity_register_engines_i that should run in editor mode with.
tm_simulation_entry_i tm_simulation_entry_i.start() tm_simulation_entry_i.tick() tm_simulation_entry_i.stop()The Simulation Entry interface tm_simulation_entry_i makes it possible to choose a point of entry for code that should run while the simulation (simulation tab or runner) is active. More information in the designated chapter. Simulation Entry
tm_the_machinery_project_unloaded_iIs called when a project is unloaded.
tm_the_machinery_project_saved_iIs called when a project is saved.
tm_plugin_shutdown_iIs called when the application shutdowns on all plugins that have an interface registered. Is not called when a plugin is reloaded.

Gameplay Coding in The Machinery

In this section, you will learn the basics about Gameplay Coding in The Machinery. There are two primary ways of creating a vivid and active world:

Coding within our Entity Component System

The Machinery uses an Entity Component System; therefore, most of your gameplay code will run via Engines or Systems. To learn more about these, please follow this link.

General code entry points using Simulation Entry Component

The Machinery also offers you a Simulation Entry Component which will, when the parent entity is spawned, set up a system with that is used to run code at start-up and each frame. Read more here.

Simulation Entry (writing gameplay code in C)

This walkthrough will show you how to create a simulation entry and what a simulation entry is.

If you wish to program gameplay using C code, then you need some way for this code to execute. You can either make lots of entity components that do inter-component communication, but if you want a more classic monolithic approach, then you can use a simulation entry.

In order for your code to execute using a simulation entry you need two things. Firstly you need an implementation of the Simulation Entry interface, tm_simulation_entry_i (see simulation_entry.h) and secondly you need a Simulation Entry Component attached to an entity.

Define a tm_simulation_entry_i in a plugin like this:

static tm_simulation_entry_i simulation_entry_i = {
    .id = TM_STATIC_HASH("tm_my_game_simulation_entry", 0x2d5f7dad50097045ULL),
    .display_name = TM_LOCALIZE_LATER("My Game Simulate Entry"),
    .start = start,
    .stop = stop,
    .tick = tick,
    .hot_reload = hot_reload,
};

Where start, stop and tick are functions that are run when the simulation starts, stop and each frame respectively. Make sure that id is a unique identifier.

Note: There is also a plugin template available that does this, see File -> New Plugin -> Simulation Entry with The Machinery Editor.

Note: to generate the TM_STATIC_HASH you need to run hash.exe or tmbuild.exe --gen-hash for more info open the hash.exe guide

When your plugin loads (each plugin has a tm_load_plugin function), make sure to register this implementation of tm_simulation_entry_i on the tm_simulation_entry_i interface name, like so:

tm_add_or_remove_implementation(reg, load, tm_simulation_entry_i, &simulation_entry_i);

When this is done and your plugin is loaded, you can add a Simulation Entry Component to any entity and select your registered implementation. Now, whenever you run a simulation (using Simulate Tab or from a Published build) where this entity is present, your code will run.

The same Simulation Entry interface can be used from multiple Simulation Entry Components and their state will not be shared between them.

Note: For more in-depth examples, we refer to the gameplay samples, they all use Simulation Entry.

What happens under the hood?

When the Simulation Entry Component is loaded within the Simulate Tab or Runner, it will set up an entity system. This system will run your start, stop and tick functions. You may then ask, what is the difference between using a Simulation Entry and just registering a system from your plugin? The answer is the lifetime of the code. If you register a system from your plugin, then that system will run no matter what entity is spawned whereas the Simulation Entry Component will add and remove the system that runs your code when the entity is spawned and despawned.

Example: Source Code

static struct tm_localizer_api *tm_localizer_api;
static struct tm_allocator_api *tm_allocator_api;
// beginning of the source file
#include <foundation/api_registry.h>
#include <foundation/localizer.h>
#include <foundation/allocator.h>

#include <plugins/simulation/simulation_entry.h>

struct tm_simulation_state_o
{
    tm_allocator_i *allocator;
    //..
};

// Starts a new simulation session. Called just before `tick` is called for the first
// time. The return value will later be fed to later calls to `stop` and `update`.
tm_simulation_state_o *start(tm_simulation_start_args_t *args)
{
    tm_simulation_state_o *state = tm_alloc(args->allocator, sizeof(*state));
    *state = (tm_simulation_state_o){
        .allocator = args->allocator,
        //...
    };
    //...
    return state;
}

// Called when the entity containing the Simulation Entry Component is destroyed.
void stop(tm_simulation_state_o *state, struct tm_entity_commands_o *commands)
{
    //...
    tm_allocator_i a = *state->allocator;
    tm_free(&a, state, sizeof(*state));
}

// Called each frame. Implement logic such as gameplay here. See `args` for useful
// stuff like duration of the frame etc.
void tick(tm_simulation_state_o *state, tm_simulation_frame_args_t *args)
{
    //...
}

// Called whenever a code hot reload has occurred. Note that the start, tick and
// stop functions will be updated to any new version automatically, this  callback is for other
// hot reload related tasks such as updating function pointers within the simulation code.
void hot_reload(tm_simulation_state_o *state, struct tm_entity_commands_o *commands)
{
    //...
}

static tm_simulation_entry_i simulation_entry_i = {
    .id = TM_STATIC_HASH("tm_my_game_simulation_entry", 0x2d5f7dad50097045ULL),
    .display_name = TM_LOCALIZE_LATER("My Game Simulate Entry"),
    .start = start,
    .stop = stop,
    .tick = tick,
    .hot_reload = hot_reload,
};

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_localizer_api = tm_get_api(reg, tm_localizer_api);
    tm_allocator_api = tm_get_api(reg, tm_allocator_api);
    tm_add_or_remove_implementation(reg, load, tm_simulation_entry_i, &simulation_entry_i);
}

Entity Graphs

The Entity Graph implements a visual scripting language based on nodes and connections. To use it, right-click on an entity to add a Graph Component and then double click on the Graph Component to open it in the Graph Editor:

Graph editor.

Basic Concepts

How to use the Entity Graph?

You need to add a Graph Component to an Entity of your choice.

After that, you have two ways of opening the Graph:

  • Double Click the Graph Component

  • Click in the Property View on Edit

Now the Graph Editor Opens and you can start adding nodes via:

  • Right Click -> Add Node
  • Press Space

Execution

The Entity Graph is an event-driven Visual Scripting language. This means everything happens after an event is triggered! By default, the Engine comes with the following built-in Events:

NameDescription
Init EventIs called when the component is added to the Entity.
Reload EventIs called when the component is reloaded from the truth.
Tick EventIs called every frame.
Terminate EventIs called before the component is removed from the entity.
Custom EventIs called when the named event
is triggered with either a "Trigger Event"
node or from the outside with "Trigger Remote Event"
Trigger EventTriggers an event.
Trigger Remote EventTriggeres an event on a remote Entity
UI TickIs ticked every frame regardless if the game is paused or not!

Anatomy

There are six types of nodes:

TypeFunction
Event NodeStarting point of the Execution
Query NodeQuery nodes are triggered automatically when their output is requested. Nodes that aren't query nodes need to be triggered. Therefore they are "Pure" and do not modify data!
NodesNormal nodes that have an Event input, hence they might modify the data and produce an output or mutate the graphs state!
SubgraphsGraphs within a Graph! Allow you to organize your graph into smaller units. They can also be reused if saved as Subgraph Prototype.
Input NodeAccepts Input from the Outside world and makes it available to the graph. Mainly used for communication between graphs and subgraphs.
Output NodeAccepts Output and passes it to the Outside world and makes it available to a parent graph. Mainly used for communication between graphs and subgraphs.

Moreover, the Visual Scripting language knows two different types of wires:

  • Event Wires They regulate the execution flow.
  • Data Wires Transport the data from node to node!

Inputs

Graphs can have inputs. They can be used to allow the user of your graph to pass data from the outside (e.g. the Editor) to the graph. This happens via the Input Nodes. In the General Graph settings you can add Inputs from the outside world.

Adding a Public Input

  1. You click on the Settings button which opens the Graphs Settings

image

  1. You expand the Input Accordion and press "Add"

  2. This will add a new Input to your graph! There you have a few options.

To make your node public just check the publicly accessible from another graph or from the Editor check the Public checkbox

If you now select the Graph Component of your Entity you will be able to change the value:

This can be a nice way to customize behaviour of your graph and entity!

  1. Add an Input node to your graph. There you have access to the data.

    The Input Node also allows you to access the settings. Hover over the name of the Input and a Settings option becomes available.

Variables

You can store data within your Graph! The Set / Get Variable nodes are the way to go. They give you access to this function. You can also access variables from distance Entities by using the Set / Get Remote Variable nodes.

Branches and loops

The Language comes with built-in support for branches, the If node. The Language supports multiple bool operators to compare values.

Besides, you have two nodes for loops:

  • The Grid node
  • The For node

The Grid node for example:

Subgraphs

You can organize your code into smaller reusable units and combine nodes as a subgraph! Using subgraphs makes your Graph more user-friendly, and it will look less like spaghetti. You can store your subgraph as a .entity_graph asset in your asset browser and allow it to be reused across your project! Which enables you to have maximal flexibility!

What is next?

In the next chapter you will learn more about Subgraphs and the Debugger! In case you want to provide your own Nodes check out this tutorial Extend the Entity Graph

Subgraphs

Subgraphs are a way to organize your graph better and create smaller units. They make them easier to maintain and easy to follow. In its essence a subgraph is a graph within a graph. They are interfaced via a subgraph node. They can produce Input and Output, such as a normal node could. They can also call and react to normal Events!

Create a subgraph

You create a new subgraph by simply selecting all nodes that shall be part of the subgraph. After that, click on them with the right mouse and select Create Subgraph from the context menu**.** The subgraph will replace the selected nodes. You can change its label in the property view. By simply double click you open the subgraph.

Subgraph Inputs

A subgraph can have inputs and outputs. You can add them to them the same way as for a normal Graph. But you can also just connect the needed wires with the subgraph node, as the following image shows:

Subgraph Prototypes

The Machinery's Prototype system allows you to create, configure, and store an Entity/Creation Graph complete with all its subgraphs, input/output nodes as a reusable Entity / Creation Graph Asset.

Note: Since the Entity Graph and the Creation Graph are conceptually similar the same aspects apply to them both! However this document will only focus on the Entity Graph.

This Asset acts as a template from which you can create new Prototype instances in other Entity Graphs/Creation Graphs. Any edits that you make to the Asset are automatically reflected in the instances of that Graph, allowing you to easily make broad changes across your whole Project without having to repeatedly make the same edit to every copy of the Asset.

Note: This does not mean all Prototype instances are identical. You can override individually and add/remove nodes from them, depending on your need!

Create a subgraph Prototype

You can turn a subgraph into a prototype by simply using the context menu of the subgraph node and selecting Create Subgraph prototype. This will create a Subgraph Prototype Asset (.entity_graph) in your Asset Browser. When you open it you are opening the instanced version. Any change to this version will not be shared across all other versions! Only changes made to the prototype will propagate to all changes! To open a prototype you can use the "Open Prototype" Button.

Debugger

The Entity Graph has a Debugger. You can use this Debugger to inspect the current values or set a breakpoint to see if the Graph behaves the way it should. Besides the graph indicated if a node is executed by a highlighted border!

Note: The Debugger only works if the simulate tab and the graph tab are open at the same time!

You can find the Debugger when you click on the button in the upper toolbar with the bug symbol (1). It will open the Debugger Overlay. Besides the "Bug" button, you can find a dropdown menu (2). This dropdown menu lets you switch between Graph instances quickly. This is useful if the Graph is part of an Entity Prototype or itself a Subgraph prototype!

Debug Overlay

In this overlay, you find three-tab:

  1. Watch Wires; Contains all data wires you are watching.
  2. Breakpoints; This Contains a list of all Breakpoints within this Graph and its subgraph
  3. Instances; A list of all instances of this Graph

Watch Wires

Like in a normal code editor, you can hover over any data wire and observe the values during the execution. If a value changed, it would be red, otherwise white.

This might be cucumber some and difficult for observing multiple wires. This is why you can add them to the watch wire list.

The Watch Wire list will indicate as well if a value has changed. You can also remove them there again and find the node with the find node button.

Keep in mind that this list only works within the current graph instance and its subgraph.

Breakpoints

Unlike watching wires which require no extra step, you cannot just add a breakpoint, and it will break immediately since such behaviour could be annoying. You can add breakpoints at any point in time via Right-Click on a node -> Add Breakpoint.

Note: You can only add breakpoints to all nodes besides Event and Query nodes.

To activate the breakpoints, you need to connect to the Simulation by pressing the Connect Button in the Debug Overlay.

(Alternatively, the Breakpoint Overview will inform you that you need to connect to the Simulation)

The moment you are connected, the Simulation will react appropriately, and your breakpoints will happen.

  1. You can disconnect from the Simulation.
  2. You can continue till the next breakpoint hits.
  3. You can Stepover to the next node.

Entity Component System

This section of the book shall help you understand how to make use of our Entity Component System. Designing games with an ECS can be overwhelming. Especially if you come from a more traditional background: Object Oriented Approaches. This chapter will provide you with the basic understanding of what an ECS is and how you can use it!.

Let us begin with the purpose of the entity system! Its purpose is it to provide a flexible model for objects in a simulation, that allows us to compose complex objects from simpler components in a flexible and performant way. An entity is a game object composed of components. Entities live in a entity context — an isolated world of entities. In the Machinery each of the following tabs has its own Entity Context: Simulate Tab, Scene Tab, Preview Tab. Entities within those tabs only exist within these contexts! Each new instance of the tab has a different context! Entities are composed of components. They are there to hold the needed data while Engines/Systems are there to provide behaviour. Each context (entity context) can have a number of engines or systems registered. (ECS) Engine updates are running on a subset of entities that posses some set of components.

Note: in some entity systems, these are referred to as systems instead, but we choose engine, because it is less ambiguous.

While Systems are just an update with a provided access to the entity context. When we refer to a context in this chapter we mean the entity context.

Table of Content

What is an Entity?

An entity is the fundamental part of the Entity Component System. An entity is a handle to your data. The entity itself does not store any data or behavior. The data is stored in components, which are associated with the Entity. The behavior is defined in Systems and Engines which process those components. Therefore an entity acts as an identifier or key to the data stored in components.

Note: In this example both entities have the same set of components, but they do not own the data they just refer to it!

Entities are managed by the Entity API and exist within an Entity Context. An Entity struct refers to an entity, but is not a real reference. Rather the Entity struct contains an index used to access entity data.

What is an Entity Context?

The Entity Context is the simulation world. It contains all the Entities and Systems/Engines as well as owns all the Component Data. There can be multiple Entity Contexts in the Editor. For example the Simulate tag, Preview Tab have both an Entity Context. When you Register A System/Engine you can decide in which context they shall run. The Default is in all contexts.

Where do Entities live? (Lifecycle)

  • Entities do not live in The Truth. The truth is for assets, not for simulation.
  • Entity data is owned by the entity context and thrown away when the entity context is destroyed.
  • Entities can be spawned from entity assets in The Truth. Multiple entities can be spawned from the same asset.
  • Changes to entity assets can be propagated into a context where those assets are spawned. This is the main way in which we will provide a “preview” of assets in a simulation context.
  • An entity always belongs to a specific entity context and entity IDs are only unique within the entity contexts. Entities can be created and deleted dynamically. When entities are deleted, the existing handles to that entity are no longer valid. Entity IDs act as weak references. If you have an ID you can ask the context whether that entity is still alive or not. tm_entity_api.is_alive()

How is the data stored?

  • An entity is a 64-bit value divided into a 32-bit index and a 32-bit generation.
  • The index points to a slot where entity data is stored.
  • The generation is increased every time we recycle a slot. This allows us to detect stale entity IDs (i.e., weak referencing through is_alive().

What is an Entity type / Archetype?

An Entity Types is a unique combination of component types. The Entity API uses the entity type to group all entities that have the same sets of components.

Note: In this example Entities A-B are of the same entity type while C has a different entity type!

  • An entity type is shared by all entities with a certain component mask.
  • When components are added to or removed from an entity, it’s entity type changes, thus its data must be copied over to the new type.
  • Pointers to component data are thus not permanent.

What are Components?

They are data, that is all they are. Designing them is the most important task you will find yourself doing in an ESC driven game. The reason is that if you change a component you have to update all systems that use it. This data composed together makes up an Entity. It can be changed at runtime, in what ever way required. This data is transformed in Systems/Engines and therefore Systems/Engines provide the Behaviour of our game based on the input/output of other Systems/Engines.

Note: Keep in mind they do not need a Truth Representation. If they do not have one, the Engine cannot display them in the Entity Tree View. This is useful for runtime only components.

  • A component is defined by tm_component_i — it consists of a fixed-size piece of POD data.
  • This data is stored in a huge buffer for each entity type, and indexed by the index.
  • In addition, a component can have a manager.
  • The manager can store additional data for the component that doesn’t fit in the POD data — such as lists, strings, buffers, etc.

You can add callbacks to the component interface which allow you to perform actions on add and remove. The general lifetime of a component is bound to the Entity Context.

It's all about the data

Data is all we have. Data is what we need to transform in order to create a user experience. Data is what we load when we open a document. Data is the graphics on the screen and the pulses from the buttons on your gamepad and the cause of your speakers and headphones producing waves in the air and the method by which you level up and how the bad guy knew where you were to shoot at you and how long the dynamite took to explode and how many rings you dropped when you fell on the spikes and the current velocity of every particle in the beautiful scene that ended the game, that was loaded off the disc and into your life. Any application is nothing without its data. Photoshop without the images is nothing. Word is nothing without the characters. Cubase is worthless without the events. All the applications that have ever been written have been written to output data based on some input data. The form of that data can be extremely complex, or so simple it requires no documentation at all, but all applications produce and need data. (Source)

Best Practice

  • Component Size: Keep them small and atomic. The main reason for this is that it improves caching performance. Besides having a lot of small components allows for more reusability and compostability! Besides if they are atomic units of data, they increase their value to be reused across projects better and can provide more combinations. The biggest disadvantage is that small components make it harder to find them, the larger your project is.
  • Complex component data: Generally speaking you want to avoid storing complex data such as arrays or heap allocated data in a component. It is possible and sometimes not possible to avoid, but it is always good to ask yourself if it is needed.

What is a Component Manager?

  • Can store persistent data from the beginning of the Entity Context till the end

  • Can provide a way to allocate data on adding/removing a component

Game Logic

The behavior is defined in Systems and Engines which process those components. Systems and Engines can be seen as data transformation actions. They take some input (components) and process them to some output (changed component data, different rendering) and a chain of small systems together makes up your game!

What are Engines?

Note: in some entity systems, these are referred to as systems instead, but we choose engine, because it is less ambiguous.

  • An engine is an update that runs for all components matching a certain component mask.

  • Engines registered with the context run automatically on update, in parallel.

  • Parallelization is done automatically, by looking at the components that each engine reads or writes. Before running, an engine waits for the previous engines that wrote to the components that the engine is interested in.

    The following image shows how a time based movement System could look like:

What are Systems?

  • General Update loop that has access to the Entity Context.

  • Can be used for none component specific interactions

  • Can be used for serial interactions that do not interact with the entity system. (Such as Input)

How are Entity Assets translated to ECS Entities?

Since the Truth is an editor concept and our main data model, your scene is stored in the Truth. When you start the simulation your Assets get translated to the ECS via the asset_load() function. In your tm_component_i you can provide this function if you want your component to translate to the ECS world. In there you have access to the Truth, afterwards not anymore. Besides, you can provide some other callbacks for different stages of the translation process.

Important: A Component representation in The Truth may not reflect the runtime ECS representation. This can be used to separate a Truth representation into smaller bits for gameplay programming sake but keep the simplicity for the Front End user.

Example:

You have a Movement Controller Component that can be used via the UI to determine the Entities movement speed. The actual movement system interacts with a Movement Component which keeps track of the actual current speed and can be influenced by other systems while the Movement Controller is only there to keep the fixed const state, but it can be influenced by a Skill Update system or something like this.

Child entities

  • Child entities are entities that are spawned and destroyed together with their parent.

    Note: that we only store child pointers, not parent pointers. Deleting a child entity does not automatically delete it from its parent — it will remain in the parent as a dead pointer.

What are Components?

They are data, that is all they are. Designing them is the most important task you will find yourself doing in an ESC driven game. The reason is that if you change a component you have to update all systems that use it. This data composed together makes up an Entity. It can be changed at runtime, in what ever way required. This data is transformed in Systems/Engines and therefore Systems/Engines provide the Behaviour of our game based on the input/output of other Systems/Engines.

Note: Keep in mind they do not need a Truth Representation. If they do not have one, the Engine cannot display them in the Entity Tree View. This is useful for runtime only components.

  • A component is defined by tm_component_i — it consists of a fixed-size piece of POD data.
  • This data is stored in a huge buffer for each entity type, and indexed by the index.
  • In addition, a component can have a manager.
  • The manager can store additional data for the component that doesn’t fit in the POD data — such as lists, strings, buffers, etc.

You can add callbacks to the component interface which allow you to perform actions on add and remove. The general lifetime of a component is bound to the Entity Context.

It's all about the data

Data is all we have. Data is what we need to transform in order to create a user experience. Data is what we load when we open a document. Data is the graphics on the screen and the pulses from the buttons on your gamepad and the cause of your speakers and headphones producing waves in the air and the method by which you level up and how the bad guy knew where you were to shoot at you and how long the dynamite took to explode and how many rings you dropped when you fell on the spikes and the current velocity of every particle in the beautiful scene that ended the game, that was loaded off the disc and into your life. Any application is nothing without its data. Photoshop without the images is nothing. Word is nothing without the characters. Cubase is worthless without the events. All the applications that have ever been written have been written to output data based on some input data. The form of that data can be extremely complex, or so simple it requires no documentation at all, but all applications produce and need data. (Source)

Best Practice

  • Component Size: Keep them small and atomic. The main reason for this is that it improves caching performance. Besides having a lot of small components allows for more reusability and compostability! Besides if they are atomic units of data, they increase their value to be reused across projects better and can provide more combinations. The biggest disadvantage is that small components make it harder to find them, the larger your project is.
  • Complex component data: Generally speaking you want to avoid storing complex data such as arrays or heap allocated data in a component. It is possible and sometimes not possible to avoid, but it is always good to ask yourself if it is needed.

How can we implement interaction between entities?

There are two problems in an ECS (Entity Component System) regarding the interaction between Entities: The read and the write access.

The truth about the interaction between Entities is that interactions do not genuinely exist. They are hidden beneath the implementation of the underlying relationship. A relationship is then nothing else than the transformation of data.

To choose the right tool for creating those transformations, we need to reason about our code (and what we want to achieve) and ask ourselves the following five questions:

  • On what data do we operate?
  • What is our domain?
  • What is the possible input for our transformation?
  • What is the usage frequency of the data?
  • What are we actually transforming?
  • What could our algorithm look like?
  • How often do we perform our transformation?

For infrequent read access we can easily use the tm_entity_api.get_component() . It allows access to the underlying data directly from a provided entity. It is not recommended to use that for read-access because it is quite slow. You perform random data access. But again, if it is infrequent of the operation and the number of targets (Entities), which are interesting to choose the right tool.

Here, you can use a System better than an Engine, since a System does not run in parallel and provides access to the Entity Context.

The problem

When creating interactions between entities, we mainly face two types of problems:

  1. Read Access: It means we have to read specific properties from a particular entity (object) and react based on this. In terms of games: An Actor needs to query/know some information from another part of the game. For example, within a Quest System: Have all tasks been completed?
  2. Write access: It means we have to write specific properties to a particular entity (object).

The transformation from *Interaction* towards *Relationships*

To start this transformation, we should have a quick look at the first principle of Data-Oriented Design:

Data is not the problem domain. For some, it would seem that data-oriented design is the antithesis of most other programming paradigms because data-oriented design is a technique that does not readily allow the problem domain to enter into the software so readily. It does not recognize the concept of an object in any way, as data is consistent without meaning […] The data-oriented design approach doesn’t build the real-world problem into the code. This could be seen as a failure of the data-oriented approach by veteran object-oriented developers, as many examples of the success of object-oriented design come from being able to bring human concepts to the machine. In this middle ground, a solution can be written in this language that is understandable by both humans and computers. The data-oriented approach gives up some of the human readability by leaving the problem domain in the design document but stops the machine from having to handle human concepts at any level by just that same action — Data Oriented Design Book Chapter 1.2

This principle helps us recognize that interactions do not truly exist. They hide the implementation of the underlying relationship. A relationship is nothing else than a transformation of data. In the case of an ECS, the Entity Manager (In our case, Entity Context) can be seen as a database and the Entity as a Lookup table key that indexes relationships between components.

The systems (or engines) are just here to interpret those relationships and give them meaning. Therefore, a system and engines should only do one job and do this well.

Systems/Engines perform transformations of data. This understanding allows us to create generic systems which are decoupled and easy to reuse, and as such, we should keep the following in mind:

One of the main design goals for Data-Oriented Design-driven applications is to focus on reusability through decoupling whenever possible.

Thus, the Unix philosophy Write programs that do one thing and do it well. Write programs to work together — McIlroy is a good way of expressing what a system/engine should do.

Most ECS's are built with the idea of relationships in mind. When writing systems/engines, we transform data from one state to another to give the data meaning. Therefore systems/engines are defining the purpose of the data relationships. This decoupling provides us with the flexibility we need to design complex software such as video games.

With such a design, we can modify behavior later on without breaking any dependencies.

For example:

You have one movement engine designed for the Player at first. Later on, you want to reuse it for all entities with a movement controller component. It contains the data provided by the Input System, such as which keys have been pressed. Therefore, an AI system can feed this as well for any other Unit (With the Movement Controller component, not the Player). The Movement Engine does not care about where the data comes from or who has it as long as it is present and the other needed component. (E.g. The Physics Mover or Transform)

How do we design Systems?

To implement the before-mentioned relationships, we have to undertake a couple of steps.

These steps are also interesting for programmers who design gameplay systems. Having those fleshed out when they design game mechanics can be good and speed up your work.

We have to ask the following questions:

1. What data transformations are we going to do and on which data?

This question should lead to “what components do we need to create this relationship?” We should always be able to give a reason why we need this data.

2. What is our possible domain? (What kind of inputs do we have?)

When we figure this out, we can make the right decision later. Also, we can reason about our code and how to implement these relationships.

3. How often does the data change?

To determine how often we change the data, we go through component by component and discuss how often we change it. This process is vital to pick the right tool. Knowing those numbers or tendencies is great for reasoning about possible performance bottlenecks and where we could apply optimizations.

4. What are we actually transforming?

Writing down the algorithm (in code or on paper) or the constraints of what we are actually doing with our data is a great solution. To pick the right tool based on the planned algorithm, we need to consider the cost of our algorithm.

What does cost mean? It can mean anything from runtime costs to implementation costs. It is essential first to establish what the proper criteria are. The costs at the end enable us to reason about the code.

To pick the right tool, we need to reason about the costs an algorithm costs us. If we take run time performance as a measurement, it is okay to have a slow algorithm if we do not execute this frequently. If this is not the case, you should consider another solution.

5. How often do we execute the algorithm/transformation?

Based on the information we have already about the data we need for the transformation, it’s pretty easy to determine the execution frequency. The total number of entities/objects is known at this time. (It may be an estimation). Therefore, we can guess how often this might run. Keep in mind that we previously discussed how often we suspect the data to be changed. This leads to transparency, which gives a good idea of the costs of this code.

Keep in mind that the main goal is to keep things simple. A System/Engine should do one job. As the variety of components defines the data type of the Entity. And the combination of Systems/Engines defines the actual game behavior. Therefore you do not need to write diagrams, blueprints, pseudo-code or anything. You may even be able to just write the engine as one goal. It is recommended to do those steps even in your mind before you write your system.

IMPORTANT: When the data changes, the problem changes. Therefore, we have to properly evaluate with the descriptive method the possible outcome and maybe change the implementation.

How to design a Systems & Engines

In the Machinery, you provide the behavior for your gameplay code via Engines and Systems. The difference between Engines and Systems is that Engines provide an explicitly defined subset of components while Systems give you only access to the Entity Context.

Note: Unsure what a System or an Engine is? Please read here

This separation means that Engines are better used for high-frequency operations on many entities. At the same time, Systems are better used for broader operations such as input on a few Entities / Single entities.

Documentation: The difference between engines and systems is that engines are fed component data, whereas systems are not. Thus, systems are useful when the data is stored externally from the components (for example to update a physics simulation), whereas engines are more efficient when the data is stored in the components. (You could use a system to update data in components, but it would be inefficient, because you would have to perform a lot of lookups to access the component data.)

These are a couple of questions you should ask yourself in advance.

  • On what data do we operate?
  • What is our domain?
  • What is the possible input for our transformation?
  • What is the usage frequency of the data?
  • What are we actually transforming?
  • What could our algorithm look like?
  • How often do we perform our transformation?

More details about those questions click here : How entities can interact.

At the end of this, you should be able to answer the following questions:

  • What kind of data am I going to read?
  • What kind of data am I going to write?
  • Should my operation be exclusive? Hence not to be executed in parallel?
  • In which phase does it run?
  • What dependencies do I have?

Those answers are important for the automatic scheduling of the Systems/Engines. Based on all those inputs, the Entity System can determine when and how to schedule what.

Best Practice

  • System/Engine Scope: Systems should be designed to have one job only. This can be difficult at times, especially when designing new features. Therefore it is fine to first create a bigger system and then with time make them smaller. If you find that your engine/system does a lot of things, don't worry. In an ECS things are decoupled from everything else, therefore it is generally pretty easy to split them up into smaller units. This allows you to increase reusability of systems
  • System/Engine Scheduling: Always provide a write list and a list of components your Engine/System is operating on. This is important so the scheduler can do its best! Also do not forget to make use of .before_me, .after_me and .phase more about this in the next chapter!

Example

    const tm_engine_i movement_engine = {
        .ui_name = "movement_engine",
        .hash = TM_STATIC_HASH("movement_engine", 0x336880a23d06646dULL),
        .num_components = 4,
        .components = {keyboard_component, movement_component, transform_component, mover_component},
        .writes = {false, false, true, true},
        .update = movement_update,
        .inst = (tm_engine_o *)ctx,
    };

This movement engine will operate on:

  • keyboard_component
  • movement_component
  • transform_component
  • mover_component

components. The scheduler can now look for those components in other engines and determine based on the .write field how to schedule it efficiently.

In this example the scheduler can schedule any engine that writes to the keyboard and the movement component that the same time as this engine if they do not write to the transform and mover component!

What is next?

More details on writing your own system or engine is explained in the next chapter

Defining a System and Engines

You have to pass the tm_entity_system_i or tm_engine_i instance in your register function.

Table of Content

Ask yourself those questions before you design an Engine / System

The following questions are better explained in the chapter: How entities can interact.

  • On what data do we operate?
  • What is our domain?
  • What is the possible input for our transformation?
  • What is the usage frequency of the data?
  • What are we actually transforming?
  • What could our algorithm look like?
  • How often do we perform our transformation?

and my answers

  • What kind of data am I going to read?
  • What kind of data am I going to write?
  • What kind of data do I want to ignore? (only important for engines)
  • Should my operation be exclusive? Hence not to be executed in parallel?
  • In which phase does it run?
  • What dependencies do I have?

Now it is time to define the dependencies / important items for scheduling.

How do those questions translate?

What kind of data am I going to read? && What kind of data am I going to write?

They translate .write and .components. With those fields, we tell the scheduler what components this system operates. From which components it intends to read from and to which one it writes.

What kind of data do I want to ignore? (only important for engines)

In the tm_engine_i you can provide a way to filter your component. Thus you can decide on which components the engine shall run. The field .excluded is used for this in there you can define which components an entity type shall not have. This means that when the engine is scheduled all entities will be ignored with those components.

For more information see Tagging Entities and Filtering Entities

Should my operation be exclusive? Hence not to be executed in parallel?

If we are sure that our system/engine should not run parallel, we need to tell the scheduler by setting the .exclusive flag to true. It will not run in parallel with any other systems or engines in the entity context. If it is false then the components and writes will be used to determine parallelism.

In which phase does it run?

We can define the .phase to tell the system in which phase we want our operation to run.

What dependencies do I have?

We can define dependencies by saying: .before_me and .after_me. We just pass the string hash of the other engine/system to this, and the scheduler does the rest.

What is next?

In the next chapter we translate this to actual code!

Registering a System or an Engine

To register Systems/Engines, you need to provide a register function to the tm_entity_register_engines_simulation_i interface. This function has the signature:

static void entity_register_engines_i(struct tm_entity_context_o *ctx)

For more information check the tm_entity_register_engines_simulation_i .

Whenever the Machinery creates an Entity Context, it calls this function and registers all your Systems / Engines to this context.

The Entity context is the world in which all your entities exist.

For Engines, you pass an instance of the tm_entity_system_i to the register function.

// example:
static void entity_register_engines_i(struct tm_entity_context_o *ctx)
{

    tm_component_type_t keyboard_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__KEYBOARD_COMPONENT);
    tm_component_type_t movement_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__MOVEMENT_COMPONENT);
    tm_component_type_t transform_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__TRANSFORM_COMPONENT);
    tm_component_type_t mover_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__MOVER_COMPONENT);

    const tm_engine_i movement_engine = {
        .ui_name = "movement_engine",
        .hash = TM_STATIC_HASH("movement_engine", 0x336880a23d06646dULL),
        .num_components = 4,
        .components = {keyboard_component, movement_component, transform_component, mover_component},
        .writes = {false, false, true, true},
        .update = movement_update,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &movement_engine);
}

For Systems, you pass an instance of the tm_engine_i to the register function.

static void register_or_system_engine(struct tm_entity_context_o *ctx)
{
    const tm_entity_system_i winning_system = {
        .ui_name = "winning_system_update",
        .hash = TM_STATIC_HASH("winning_system_update", 0x8f8676e599ca5c7aULL),
        .update = winning_system_update,
        .before_me[0] = TM_STATIC_HASH("maze_generation_system", 0x7f1fcbd9ee85c3cfULL),
        .exclusive = true,
    };
    tm_entity_api->register_system(ctx, &winning_system);
}

In the above example the scheduler will schedule this system after the maze_generation_system system! Since we did not provide any further information in .writes or in .components the scheduler has no other information to work with. In this case it is best to not write anything!

Example load function:

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    // other code ...
    tm_add_or_remove_implementation(reg, load, tm_entity_register_engines_simulation_i, &register_or_system_engine);
}

Register your system or engine to the Editor

You can use the tm_entity_register_engines_simulation_i to register your engine or system to an entity context that runs only in the Editor. This might be good for components that shall only be used in the Editor.

The function signature is the same as the for the other interface!

Register systems & engines outside of the load function

You also can register your System/Engine outside of the load function wherever you have access to the correct Entity Context.

Write a custom component

This walkthrough shows you how to add a custom component to the Engine. During this walkthrough, we will cover the following topics:

  • How to create a component from scratch.
  • Where and how do we register a component.

You should have basic knowledge about how to write a custom plugin. If not, you might want to check this Guide and the Write a plugin guide. The goal of this walkthrough is to dissect the component plugin provided by the Engine.

Table of Content

Where do we start?

In this example, we want to create a new plugin, which contains our component. We open the Engine go to file -> New Plugin -> Entity Component. The file dialog will pop up and ask us where we want to save our file. Pick a location that suits you.

Tip: Maybe store your plugin in a folder next to your game project.

After this, we see that the Engine created some files for us. Now we need to ensure that we can build our project. In the root folder (The folder with the premake file), we run tmbuild, and if there is no issue, we see that it will build our projects once and generate the .sln file (on windows). If there is an issue, we should ensure we have set up the Environment variables correctly and installed all the needed dependencies. For more information, please read this guide.

Now we can open the .c file with our favourite IDE. The file will contain the following content:

static struct tm_entity_api *tm_entity_api;
static struct tm_transform_component_api *tm_transform_component_api;
static struct tm_temp_allocator_api *tm_temp_allocator_api;
static struct tm_the_truth_api *tm_the_truth_api;
static struct tm_localizer_api *tm_localizer_api;

#include <plugins/entity/entity.h>
#include <plugins/entity/transform_component.h>
#include <plugins/the_machinery_shared/component_interfaces/editor_ui_interface.h>

#include <foundation/api_registry.h>
#include <foundation/carray.inl>
#include <foundation/localizer.h>
#include <foundation/math.inl>
#include <foundation/the_truth.h>

#define TM_TT_TYPE__CUSTOM_COMPONENT "tm_custom_component"
#define TM_TT_TYPE_HASH__CUSTOM_COMPONENT TM_STATIC_HASH("tm_custom_component", 0x355309758b21930cULL)

enum
{
    TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY, // float
    TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE, // float
};

struct tm_custom_component_t
{
    float y0;
    float frequency;
    float amplitude;
};

static const char *component__category(void)
{
    return TM_LOCALIZE("Samples");
}

static tm_ci_editor_ui_i *editor_aspect = &(tm_ci_editor_ui_i){
    .category = component__category};

static void truth__create_types(struct tm_the_truth_o *tt)
{
    tm_the_truth_property_definition_t custom_component_properties[] = {
        [TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY] = {"frequency", TM_THE_TRUTH_PROPERTY_TYPE_FLOAT},
        [TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE] = {"amplitude", TM_THE_TRUTH_PROPERTY_TYPE_FLOAT},
    };

    const tm_tt_type_t custom_component_type = tm_the_truth_api->create_object_type(tt, TM_TT_TYPE__CUSTOM_COMPONENT, custom_component_properties, TM_ARRAY_COUNT(custom_component_properties));
    const tm_tt_id_t default_object = tm_the_truth_api->quick_create_object(tt, TM_TT_NO_UNDO_SCOPE, TM_TT_TYPE_HASH__CUSTOM_COMPONENT, TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY, 1.0f, TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE, 1.0f, -1);
    tm_the_truth_api->set_default_object(tt, custom_component_type, default_object);

    tm_the_truth_api->set_aspect(tt, custom_component_type, TM_CI_EDITOR_UI, editor_aspect);
}

static bool component__load_asset(tm_component_manager_o *man, struct tm_entity_commands_o *commands, tm_entity_t e, void *c_vp, const tm_the_truth_o *tt, tm_tt_id_t asset)
{
    struct tm_custom_component_t *c = c_vp;
    const tm_the_truth_object_o *asset_r = tm_tt_read(tt, asset);
    c->y0 = 0;
    c->frequency = tm_the_truth_api->get_float(tt, asset_r, TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY);
    c->amplitude = tm_the_truth_api->get_float(tt, asset_r, TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE);
    return true;
}

static void component__create(struct tm_entity_context_o *ctx)
{
    tm_component_i component = {
        .name = TM_TT_TYPE__CUSTOM_COMPONENT,
        .bytes = sizeof(struct tm_custom_component_t),
        .load_asset = component__load_asset,
    };

    tm_entity_api->register_component(ctx, &component);
}

// Runs on (custom_component, transform_component)
static void engine_update__custom_component(tm_engine_o *inst, tm_engine_update_set_t *data, struct tm_entity_commands_o *commands)
{
    TM_INIT_TEMP_ALLOCATOR(ta);

    tm_entity_t *mod_transform = 0;

    struct tm_entity_context_o *ctx = (struct tm_entity_context_o *)inst;

    double t = 0;
    for (const tm_entity_blackboard_value_t *bb = data->blackboard_start; bb != data->blackboard_end; ++bb)
    {
        if (TM_STRHASH_EQUAL(bb->id, TM_ENTITY_BB__TIME))
            t = bb->double_value;
    }

    for (tm_engine_update_array_t *a = data->arrays; a < data->arrays + data->num_arrays; ++a)
    {
        struct tm_custom_component_t *custom_component = a->components[0];
        tm_transform_component_t *transform = a->components[1];

        for (uint32_t i = 0; i < a->n; ++i)
        {
            if (!custom_component[i].y0)
                custom_component[i].y0 = transform[i].world.pos.y;
            const float y = custom_component[i].y0 + custom_component[i].amplitude * sinf((float)t * custom_component[i].frequency);

            transform[i].world.pos.y = y;
            ++transform[i].version;
            tm_carray_temp_push(mod_transform, a->entities[i], ta);
        }
    }

    tm_entity_api->notify(ctx, data->engine->components[1], mod_transform, (uint32_t)tm_carray_size(mod_transform));

    TM_SHUTDOWN_TEMP_ALLOCATOR(ta);
}

static bool engine_filter__custom_component(tm_engine_o *inst, const tm_component_type_t *components, uint32_t num_components, const tm_component_mask_t *mask)
{
    return tm_entity_mask_has_component(mask, components[0]) && tm_entity_mask_has_component(mask, components[1]);
}

static void component__register_engine(struct tm_entity_context_o *ctx)
{
    const tm_component_type_t custom_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__CUSTOM_COMPONENT);
    const tm_component_type_t transform_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__TRANSFORM_COMPONENT);

    const tm_engine_i custom_component_engine = {
        .ui_name = "Custom Component",
        .hash = TM_STATIC_HASH("CUSTOM_COMPONENT", 0xe093a8316a6c2d29ULL),
        .num_components = 2,
        .components = {custom_component, transform_component},
        .writes = {false, true},
        .update = engine_update__custom_component,
        .filter = engine_filter__custom_component,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &custom_component_engine);
}

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_entity_api = tm_get_api(reg, tm_entity_api);
    tm_transform_component_api = tm_get_api(reg, tm_transform_component_api);
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
    tm_temp_allocator_api = tm_get_api(reg, tm_temp_allocator_api);
    tm_localizer_api = tm_get_api(reg, tm_localizer_api);

    tm_add_or_remove_implementation(reg, load, tm_the_truth_create_types_i, truth__create_types);
    tm_add_or_remove_implementation(reg, load, tm_entity_create_component_i, component__create);
    tm_add_or_remove_implementation(reg, load, tm_entity_register_engines_simulation_i, component__register_engine);
}

Code structure

Let us dissect the code structure and discuss all the points of interest.

API and include region

The file begins with all includes and API definitions:

static struct tm_entity_api *tm_entity_api;
static struct tm_transform_component_api *tm_transform_component_api;
static struct tm_temp_allocator_api *tm_temp_allocator_api;
static struct tm_the_truth_api *tm_the_truth_api;
static struct tm_localizer_api *tm_localizer_api;

#include <plugins/entity/entity.h>
#include <plugins/entity/transform_component.h>
#include <plugins/the_machinery_shared/component_interfaces/editor_ui_interface.h>

#include <foundation/api_registry.h>
#include <foundation/carray.inl>
#include <foundation/localizer.h>
#include <foundation/math.inl>
#include <foundation/the_truth.h>

The code will fill the API definitions with life in the tm_load_plugin function.

Define your Data

The next part contains the Truth Definition of the component and the plain old data struct (POD). In production, we should separate those aspects into a header file!

Note: All components should be plain old data types.

#define TM_TT_TYPE__CUSTOM_COMPONENT "tm_custom_component"
#define TM_TT_TYPE_HASH__CUSTOM_COMPONENT TM_STATIC_HASH("tm_custom_component", 0x355309758b21930cULL)

enum
{
    TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY, // float
    TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE, // float
};

struct tm_custom_component_t
{
    float y0;
    float frequency;
    float amplitude;
};

Add your component to the Truth

After this, we have the region in which we define the category of our component. The Editor will call it to categorize the component into the correct section.

We need to define a tm_ci_editor_ui_i object which uses this function. Later we register this function to the TM_CI_EDITOR_UI aspect of our truth type. If you do not add this aspect later to your Truth Type, the Editor will not know that this Component Type exists, and you can not add it via the Editor, but in C.

Note: More about aspects you can read in the aspects guide.

static const char *component__category(void)
{
    return TM_LOCALIZE("Samples");
}

static tm_ci_editor_ui_i *editor_aspect = &(tm_ci_editor_ui_i){
    .category = component__category};

In this region, we create our component truth type. It is important to remember that the Truth will not reflect the runtime data, just the data you can edit in the Editor. On the other hand, the Entity Context will store your runtime data, the plain old data struct you have defined above. More about how this works later in this section.

Let us take this code apart one more time:

static void truth__create_types(struct tm_the_truth_o *tt)
{
    tm_the_truth_property_definition_t custom_component_properties[] = {
        [TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY] = {"frequency", TM_THE_TRUTH_PROPERTY_TYPE_FLOAT},
        [TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE] = {"amplitude", TM_THE_TRUTH_PROPERTY_TYPE_FLOAT},
    };

    const tm_tt_type_t custom_component_type = tm_the_truth_api->create_object_type(tt, TM_TT_TYPE__CUSTOM_COMPONENT, custom_component_properties, TM_ARRAY_COUNT(custom_component_properties));
    const tm_tt_id_t default_object = tm_the_truth_api->quick_create_object(tt, TM_TT_NO_UNDO_SCOPE, TM_TT_TYPE_HASH__CUSTOM_COMPONENT, TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY, 1.0f, TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE, 1.0f, -1);
    tm_the_truth_api->set_default_object(tt, custom_component_type, default_object);

    tm_the_truth_api->set_aspect(tt, custom_component_type, TM_CI_EDITOR_UI, editor_aspect);
}
  1. We define the component's properties.
  2. We create the actual type in the Truth.
  3. We create an object of our type with quick_create_object and provide a default object to our component. It makes sure that when you add the component to an Entity, you have the expected default values. It is not needed, just a nice thing to have.
  4. Add our TM_CI_EDITOR_UI aspect to the type. It tells the Editor that you can add the component via the Editor. If you do not provide it, the Editor will not suggest this component to you and cannot store it in the Truth. It does not mean you cannot add this component via C.

Define your component

You can register a component to the tm_entity_create_component_i in your plugin load function. This interface expects a function pointer to a create component function of the signature: void tm_entity_create_component_i(struct tm_entity_context_o *ctx).

The Engine will call this function whenever it creates a new Entity Context to populate the context with all the known components. It usually happens at the beginning of the Simulation.

Within this function, you can define your component and register it to the context. The tm_entity_api provides a function tm_entity_api.register_component() which expects the current context and an instance of the tm_component_i. We define one in our function and give it the needed information:

  • A name should be the same as the Truth Type
  • The size of the component struct
  • A load asset function
static void component__create(struct tm_entity_context_o *ctx)
{
    tm_component_i component = {
        .name = TM_TT_TYPE__CUSTOM_COMPONENT,
        .bytes = sizeof(struct tm_custom_component_t),
        .load_asset = component__load_asset,
    };

    tm_entity_api->register_component(ctx, &component);
}

As mentioned before, the Truth does not reflect the runtime data and only holds the data you can edit in the Editor. This is why there needs to be some translation between The Truth and the ECS. This magic is happening in the tm_component_i.load_asset(). This function allows you to translate a tm_tt_id_t asset to the plain old data of the component.

static bool component__load_asset(tm_component_manager_o *man, struct tm_entity_commands_o *commands, tm_entity_t e, void *c_vp, const tm_the_truth_o *tt, tm_tt_id_t asset)
{
    struct tm_custom_component_t *c = c_vp;
    const tm_the_truth_object_o *asset_r = tm_tt_read(tt, asset);
    c->y0 = 0;
    c->frequency = tm_the_truth_api->get_float(tt, asset_r, TM_TT_PROP__CUSTOM_COMPONENT__FREQUENCY);
    c->amplitude = tm_the_truth_api->get_float(tt, asset_r, TM_TT_PROP__CUSTOM_COMPONENT__AMPLITUDE);
    return true;
}

The first step is that we cast the given void* of the component data c_vp to the correct data type. After that, we load the data from the Truth and store it in the component. In the end, we return true because no error occurred.

Define your engine update

In the Machinery, gameplay code is mainly driven by Systems and Engines. They define the behaviour while the components the data describes.

Note: in some entity systems, these are referred to as systems instead, but we choose Engine because it is less ambiguous.

This next section of the code is about defining an Engine.

// Runs on (custom_component, transform_component)
static void engine_update__custom_component(tm_engine_o *inst, tm_engine_update_set_t *data, struct tm_entity_commands_o *commands)
{
    TM_INIT_TEMP_ALLOCATOR(ta);

    tm_entity_t *mod_transform = 0;

    struct tm_entity_context_o *ctx = (struct tm_entity_context_o *)inst;

    double t = 0;
    for (const tm_entity_blackboard_value_t *bb = data->blackboard_start; bb != data->blackboard_end; ++bb)
    {
        if (TM_STRHASH_EQUAL(bb->id, TM_ENTITY_BB__TIME))
            t = bb->double_value;
    }

    for (tm_engine_update_array_t *a = data->arrays; a < data->arrays + data->num_arrays; ++a)
    {
        struct tm_custom_component_t *custom_component = a->components[0];
        tm_transform_component_t *transform = a->components[1];

        for (uint32_t i = 0; i < a->n; ++i)
        {
            if (!custom_component[i].y0)
                custom_component[i].y0 = transform[i].world.pos.y;
            const float y = custom_component[i].y0 + custom_component[i].amplitude * sinf((float)t * custom_component[i].frequency);

            transform[i].world.pos.y = y;
            ++transform[i].version;
            tm_carray_temp_push(mod_transform, a->entities[i], ta);
        }
    }

    tm_entity_api->notify(ctx, data->engine->components[1], mod_transform, (uint32_t)tm_carray_size(mod_transform));

    TM_SHUTDOWN_TEMP_ALLOCATOR(ta);
}

The first thing we do is use a temp allocator for any future allocation that will not leave this function. After that, we cast the tm_engine_o* inst to the tm_entity_context_o* so we have access to the entity context later on.

The next step is to get the time from the Blackboard Values.

    double t = 0;
    for (const tm_entity_blackboard_value_t *bb = data->blackboard_start; bb != data->blackboard_end; ++bb)
    {
        if (TM_STRHASH_EQUAL(bb->id, TM_ENTITY_BB__TIME))
            t = bb->double_value;
    }

The Engine provides a bunch of useful Blackboard values. They are defined in the plugins/entity/entity.h.

  • TM_ENTITY_BB__SIMULATION_SPEED - Speed that the simulation is running at. Defaults to 1.0 for normal speed.
  • TM_ENTITY_BB__DELTA_TIME - Blackboard item representing the simulation delta time of the current frame.
  • TM_ENTITY_BB__TIME - Blackboard item representing the total elapsed time in the Simulation.
  • TM_ENTITY_BB__WALL_DELTA_TIME - Blackboard item representing the wall delta time of the current frame. (Wall delta time is not affected by the Simulation being paused or run in slow motion.)
  • TM_ENTITY_BB__WALL_TIME - Blackboard item representing the total elapsed wall time in the Simulation.
  • TM_ENTITY_BB__CAMERA - Blackboard items for the current camera.
  • TM_ENTITY_BB__EDITOR - Blackboard item that indicates that we are running in Editor mode. This may disable some components and/or simulation engines.
  • TM_ENTITY_BB__SIMULATING_IN_EDITOR - Set to non-zero if the Simulation runs from within the Editor, such as running a game in the simulation tab. It will be zero when we run a game from the Runner. Note the distinction from TM_ENTITY_BB__EDITOR.

The tm_engine_update_set_t gives us access to the needed data, and we can modify our components. The first important information we get are the number of entity types (also known Archetypes). This number is stored in data->num_arrays. Now that we know this information we can iterate over them and access the components per entity type. tm_engine_update_array_t a = data->arrays (Gives us the current entity type's components). a->n is the number of matching components / entities of this entity type.

    for (tm_engine_update_array_t *a = data->arrays; a < data->arrays + data->num_arrays; ++a)
    {
        struct tm_custom_component_t *custom_component = a->components[0];
        tm_transform_component_t *transform = a->components[1];

        for (uint32_t i = 0; i < a->n; ++i)
        {
            if (!custom_component[i].y0)
                custom_component[i].y0 = transform[i].world.pos.y;
            const float y = custom_component[i].y0 + custom_component[i].amplitude * sinf((float)t * custom_component[i].frequency);

            transform[i].world.pos.y = y;
            ++transform[i].version;
            tm_carray_temp_push(mod_transform, a->entities[i], ta);
        }
    }

Note: In case you are not that familiar with C this loop:

 for (tm_engine_update_array_t* a = data->arrays; a < data->arrays + data->num_arrays; ++a) {

is kind of the C equivalent to C++'s for each loop: for(auto a : data->arrays)

As the last step, we add a notifier function call to notify all entities that their components have changed.

    tm_entity_api->notify(ctx, data->engine->components[1], mod_transform, (uint32_t)tm_carray_size(mod_transform));

Register your Engine to the system

You can register a component to the tm_entity_register_engines_simulation_i in your plugin load function. This interface expects a function pointer to a create component function of the signature: void tm_entity_register_engines_i(struct tm_entity_context_o *ctx).

The function itself looks as follows:

static void component__register_engine(struct tm_entity_context_o *ctx)
{
    const tm_component_type_t custom_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__CUSTOM_COMPONENT);
    const tm_component_type_t transform_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__TRANSFORM_COMPONENT);

    const tm_engine_i custom_component_engine = {
        .ui_name = "Custom Component",
        .hash = TM_STATIC_HASH("CUSTOM_COMPONENT", 0xe093a8316a6c2d29ULL),
        .num_components = 2,
        .components = {custom_component, transform_component},
        .writes = {false, true},
        .update = engine_update__custom_component,
        .filter = engine_filter__custom_component,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &custom_component_engine);
}

The first thing we do is to look up the component type. Did we register the type? If not, we will not get the correct type. Here we are using the name we defined beforehand in our component create function.

Then we ask for the transform component next because our Engine shall run on those two components.

After this, we define the actual instance of our engine struct

tm_engine_i.

We provide a .ui_name used in the Profiler to identify our Engine. Moreover, we add a unique string hash identifying this engine/system. This is used for scheduling the engine/system concerning other engines and systems, using thebefore_me and after_me fields.

Then we tell the system how many components the Engine shall operate on and which ones we will modify. This is used for scheduling the engines later one.

At last, we provide the needed update function, which we have discussed earlier, and a filter function.

static bool engine_filter__custom_component(tm_engine_o *inst, const tm_component_type_t *components, uint32_t num_components, const tm_component_mask_t *mask)
{
    return tm_entity_mask_has_component(mask, components[0]) && tm_entity_mask_has_component(mask, components[1]);
}

The filter function will be called on all entity types to determine if the Engine shall run on them or not. To provide this function is optional. If present it specifies a filter function called for each entity type (as

represented by its component mask) to determine if the Engine should run on that entity type. If no tm_engine_i.filter() function is supplied and no excludes[] flags are set, the update will run on entity types that have all the components in the components array. If some excludes[] flags are set, the Engine will run on all entity types that do not have any of the components whose excludes[] flags are set, but have all the other components in the components array.

Note: For more information, check the documentation.

The last thing the register function needs to do is register the Engine to the Entity Context.

 tm_entity_api->register_engine(ctx, &custom_component_engine);

The plugin load function

The most important lines here are the once in which we register our truth types, the component and the engine.

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_entity_api = tm_get_api(reg, tm_entity_api);
    tm_transform_component_api = tm_get_api(reg, tm_transform_component_api);
    tm_the_truth_api = tm_get_api(reg, tm_the_truth_api);
    tm_temp_allocator_api = tm_get_api(reg, tm_temp_allocator_api);
    tm_localizer_api = tm_get_api(reg, tm_localizer_api);

    tm_add_or_remove_implementation(reg, load, tm_the_truth_create_types_i, truth__create_types);
    tm_add_or_remove_implementation(reg, load, tm_entity_create_component_i, component__create);
    tm_add_or_remove_implementation(reg, load, tm_entity_register_engines_simulation_i, component__register_engine);
}

Tagging Entities

The Machinery knows 2 kind of ways to Tag Entities:

  1. using the Tag Component
  2. using a Tag Component to filter the Entity Type

Table of Content

Using the Tag Component

The difference is that the first solution can be used via the tag_component_api and you can add Tags via the Editor to any Entity that has a Tag Component. Later on in your System or Engine you can access the Tagged Entity.

Note: This is not the most performant solution but an easy way for entities which do not exist many times in the world. It is a nice way to identify one or two specific entities for some specific logic.

Adding them via The Editor

You need to select an Entity and Right Click -> Add Component

This will add the Entity Tag Component. When selected you have the chance to add Tags to the Entity by using a simple autocomplete textbox.

Beware: The Engine will create an entity tag folder in your root folder. This is also the place where the Entity Tag API will search for the assets.

Adding and Accessing Tags via C

You can also add tags via the tag_component_api but you need access to the Tag Component Manager. In your System or on Simulate Entry start():

tm_tag_component_manager_o *tag_mgr = (tm_tag_component_manager_o *)tm_entity_api->component_manager(ctx, tag_component);
tm_tag_component_api->add_tag(tag_mgr, my_to_tagged_entity, TM_STATIC_HASH("player", 0xafff68de8a0598dfULL));

You can also receive entities like this:

tm_tag_component_manager_o *tag_mgr = (tm_tag_component_manager_o *)tm_entity_api->component_manager(ctx, tag_component);
tm_entity_t upper_bounding_box = tm_tag_component_api->find_first(tag_mgr, TM_STATIC_HASH("upper_bounding_box", 0x1afc9d34ecb740ecULL));

And than you can read the data from the entity via get_component. This is where you will perform a random look up and this might be slow. Therefore it is mostly recommended to use this for simple interactions where performance is not needed.

Note: Tags do not need to exist in the Asset Browser, therefore you can add any label to the entity. Keep in mind that they will not be created in the Asset Browser!

Tag Components - Entity Type Filter

On the other hand you can define a Tag Component which should not be confused with the previously explained Tag Component. A tag component is a simple typedef of a unit64_t (or something else) or an empty struct in C++ to a component without properties. A tag is a component that does not have any data. The function of this component is to modify the Entity Type / Archetype to group entities together with them.

Example:

You have the following components:

  • Component A
  • Component B

And 2 systems :

  • System A
  • System B

They both shall operate on Component A & B but have different logic based on what the Components represent. To archive this you just add a tag component to each Entity:

#1 Entity:
- Component A
- Component B
- My Tag For System A
#2 Entity:
- Component A
- Component B
- My Tag for System A

In this example System B would not operate on both Entities if we use the .excluded filter to exclude My Tag For System A from the System.

Filtering

To see a real world application of Tag components to filter entity types checkout the next chapter: Filtering

Filtering Entities

The Machinery knows 2 kind of ways to Tag Entities:

  1. using the Tag Component
  2. using a Tag Component to filter the Entity Types

Table of Content

Filtering Entities

In an Engine (tm_engine_i) you can define the .excluded field. This tells the scheduler that this engine shall not run on any entity type that contains these components.

Let us assume we have the following entities:

#1 Entity:
- Component A
- Component B
- Component C
#2 Entity:
- Component A
- Component B
- Component D

Now we have an Engine that shall operate on (Component A,Component B) but we do not want it to operate on entities with Component D we could just check in our update loop:

static void entity_register_engines(struct tm_entity_context_o *ctx)
{

    tm_component_type_t component_a = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__A_COMPONENT);
    tm_component_type_t component_b = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__B_COMPONENT);
    tm_component_type_t component_d = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__D_COMPONENT);

    const tm_engine_i movement_engine = {
        .ui_name = "movement_engine",
        .hash = TM_STATIC_HASH("movement_engine", 0x336880a23d06646dULL),
        .num_components = 2,
        .components = {component_a, component_b},
        .writes = {false, true},
        .excluded = {component_d},
        .num_excluded = 1,
        .update = movement_update,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &movement_engine);
}

or we could define a component mask and use this to filter but both methods are slow. This is because get_component_by_hash or get_component require us to look up internally the entity + the components and search for them. Its aka a random memory access!

To avoid all of this we can just tell the engine to ignore all entity types which contain the component_d via the .excluded field in the tm_engine_i.

    const tm_engine_i movement_engine = {
        .ui_name = "movement_engine",
        .hash = TM_STATIC_HASH("movement_engine", 0x336880a23d06646dULL),
        .num_components = 2,
        .components = {component_a, component_b},
        .writes = {false, true},
        .excluded = {component_d},
        .num_excluded = 1,
        .update = movement_update,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &movement_engine);

Filtering Entities by using Tag Components

Note: You can define a Tag Component which should not be confused with the Tag Component. A tag component is a simple typedef of a unit64_t (or something else) or an empty struct in C++ to a component without properties. The function of this component is it to modify the Entity Type / Archetype to group entities together with them.For more information see the Tagging Entities Chapter.

You have a Movement / Input System which should always work. At some point you do not want an entity to receive any input.

Solution 1

To solve this issue you could remove the Movement Component but that would be annoying because you would loose its state, which might be important.

Better Solution

First you define the component:

#define TM_TT_TYPE__PLAYER_NO_MOVE_TAG_COMPONENT "tm_player_no_move_t"
#define TM_TT_TYE_HASH__PLAYER_NO_MOVE_TAG_COMPONENT TM_STATIC_HASH("tm_player_no_move_t", 0xc58cb6ade683ca88ULL)
static void component__create(struct tm_entity_context_o *ctx)
{
    tm_component_i component = (tm_component_i){
        .name = TM_TT_TYPE__PLAYER_NO_MOVE_TAG_COMPONENT,
        .bytes = sizeof(uint64_t), // since we do not care of its content we can just pick any 8 byte type
    };
    tm_entity_api->register_component(ctx, &component);
}

Then you filter in your update for the Input Engine/ Movement Engine any Entity that has a No Movement Tag:

    tm_component_type_t transform_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__TRANSFORM_COMPONENT);
    tm_component_type_t mover_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__MOVER_COMPONENT);
    tm_component_type_t movement_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYPE_HASH__MOVEMENT_COMPONENT);
    tm_component_type_t no_movement_tag_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYE_HASH__PLAYER_NO_MOVE_TAG_COMPONENT);

    const tm_engine_i movement_engine = {
        .ui_name = "movement_engine",
        .hash = TM_STATIC_HASH("movement_engine", 0x336880a23d06646dULL),
        .num_components = 3,
        .components = {movement_component, transform_component, mover_component},
        .writes = {false, true, true},
        .excluded = {no_movement_tag_component},
        .num_excluded = 1,
        .update = movement_update,
        .inst = (tm_engine_o *)ctx,
    };
    tm_entity_api->register_engine(ctx, &movement_engine);

Whenever another engine/system decides that an entity should not move anymore it just adds a no_movement_tag_component to the entity.

static void my_other_system(tm_engine_o *inst, tm_engine_update_set_t *data, struct tm_entity_commands_o *commands)
{
    struct tm_entity_context_o *ctx = (struct tm_entity_context_o *)inst;
    tm_component_type_t no_movement_tag_component = tm_entity_api->lookup_component_type(ctx, TM_TT_TYE_HASH__PLAYER_NO_MOVE_TAG_COMPONENT);
    // code ..
    for (tm_engine_update_array_t *a = data->arrays; a < data->arrays + data->num_arrays; ++a)
    {
        // code...
        for (uint32_t x = 0; x < a->n; ++x)
        {
            // code...
            if (player_should_not_walk_anymore)
            {
                tm_entity_commands_api->add_component(commands, a->entities[x], no_movement_tag_component);
            }
        }
    }
}

As you can see the Movement Engine will now update all other entities in the game which do not have the No Movement Tag.

Overview of the Entity Context Lifecycle

This page describes the lifecycle of the entity context / the simulation and all its stages.

Update Phases

In your Engine / Systems you an define in which Phase of the Update loop, your Engine / System shall run. This can be managed via the: .before_me, .after_me and .phase fields of your engine or system definition.

Please keep in mind that the Scheduler will order your system based on what kind of components you might modify or not! This is why it is always recommended to say what kind of components your system/engine will operate on and what they will do with them (Write to them or not). Depending on your dependencies the scheduler will decide if your engine/system can run in parallel.

The Engine has default phases:

NameWhen
TM_PHASE__ANIMATIONPhase for animation jobs.
TM_PHASE__PHYSICSPhase for physics jobs.
TM_PHASE__CAMERAPhase for camera jobs.
TM_PHASE__GRAPHPhase for the visual scripting graph update.
TM_PHASE__RENDERPhase for render jobs.

Note: that phases are just string hashes and you can extend the systems with more phases if desired.

Gameplay Entry Point Comparison

In the Machinery you have multiple entry points for your game play code to live in. You can make use of Simulation Entries, Entity Component System: Systems or Engines and also make use of a Entity Graph or custom scripting language component. The question is more when to use which of the tools? The answer to this depends on your game's needs. To summarize it in the Engine you have about four built-in entry points for your game play code which you can use all at the same time and which one to use depends on your use case.

The following table will give a brief overview of the different types and their properties:

TypeParallel ExecutionLifetime based on entityRandom Memory Access by defaultRuns only on a Subset of EntitiesRuns per entityExecution order can be set
Simulation EntryNoYesYesNoNoYes*
ECS SystemMaybeNoYesNoNoYes*
ECS EngineMaybeNoNoYesNoYes*
Entity Graph (Graph Component)NoYesYesNoYesNo

** via .phase or .before_me and .after_me when defining the interface*

Recommendation

Note: These recommendations are no guidelines! You do not have to follow them they are just here to give another more example driven overview of the different types of gameplay entry points.

System vs Engine

Systems

It is recommended to use a System over an Engine when you try to write a complex gameplay system that will handle a few different entity types simultaneously. The number of entities here is important. Since a System uses random memory access through get_component() which may lead to cache misses.

Moreover, a System is a preferred way of updating when the data in the component is just a pointer into some external system. (This is the case, for example, for PhysX components). In the case of PhysX, it is assumed to store its data in a cache-friendly order, which means we do not want to iterate over the entities in the order they are stored in the entity system since this would cause a pointer chasing in the external System. Instead, we just want to send a single update to the external System. It will process the entities in its own (cache-friendly) order.

Another reason to use System over an Engine is that you can use it to execute things on initialization and on the game's shutdown since only Systems have a init() and a shutdown() function, Engines do not.

Engines

It is recommended to use an Engine over a System when you try to write a complex gameplay system that will handle a lot of entities with the same set of components simultaneously. The ECS scheduler will gather all entities with the set of components and enable you to iterate over them in a cache-friendly manner. This allows you to write an engine that can manipulate a lot of entities simultaneously without any loss of performance.

Simulation Entry vs System

It is recommended to use a Simulation Entry when you want to tie the lifetime of the underlying System to an Entity lifetime. This is a very similar concept to the "GameObject" Script concept in Unity. Suppose the entity that hosts the Simulation Entry Component is destroyed. In that case, the Update function will not be ticked anymore, and the System is destroyed. This can be a useful concept for level-specific Gameplay moments. A Simulation Entry also has a start() and stop() function. They are executed when the Simulation Entry is added or removed.

It is not recommended to use a Simulation Entry to handle a large mass of entities. For the same reason as the System is not used for this kind of purpose. The Simulation Entry will not run parallel and have random memory access.

Entity Graph

It is recommended to use a Entity Graph when you want to tie the lifetime to an Entity and if you want to execute unperformant code since the Entity Graph is a Visual Scripting language that is interpreted. It will be naturally slow. The Entity Graph is also good to handle UI/UX elements or for quick prototyping when performance is not important. Keep in mind that an Entity Graph is not executed in parallel and also only access memory via random access.

Collaboration

The editor has built-in support for real-time collaboration, allowing multiple people to work together in the same project. All user actions — importing assets, creating and editing entities, etc, are supported in the collaborative workflow.

If you just want to try out collaboration on your own, you can run the client and the host on the same machine (just start two instances of the_machinery.exe) and connect using the LAN option.

Table of Content

Who's project is edited?

In our collaboration model, one of the users always acts as a host. The host invites others to join her in editing her project. All the changes made by the participants in the session end up in the host’s project and it’s the host’s responsibility to save the project, check in the changes into version control, or do whatever else is needed to make the changes permanent.

WARNING: Please only connect to people you trust. Be aware that Plugin Assets will be sent via a collaboration as well. The Engine will warn you every time a plugin asset is sent.

Host or Join Sessions

You have three options to host or join a collaboration Session:

Host a LAN Server

Host a server on your LAN. The system will choose a free port on your machine for hosting. Other users on the same LAN can join your session by choosing Join LAN Server and selecting your machine.

Host locally

  1. Select "Host LAN Server" from the dropdown
  2. Your Handle, in case of hosting this will be the name the session will have as well as your username.
  3. When you press Host the session starts.

Join a local Server

When you open the collaboration tab the default view is the "Join LAN Server" view. In this view you can join a local server.

The default view of the collab tab is the join local option

  1. Select "Join LAN Server" from the dropdown
  2. You can select a collaboration session. If there is no session available this field is disabled.
  3. Your Handle, the name the other user can see on their side.
  4. If you selected a session you can press this button to join. When you join the Engine will download the host's Project.

Host Internet Server

Host a server that can be accessed over the internet on a specified port.

Host Internet Server

  1. You can select the port of your session. If your router does not support UPnP you might have to port forward your selected port.
  2. Your Handle, the name the other user can see on their side.
  3. If you check the "Use UPnP" checkbox the system will attempt to use UPnP to open the port in your router, so that external users can access your server.
  4. If you selected a session you can press this button to join. When you join the Engine will download the host's Project.

Note: There is no guarantee that UPnP works with your particular router. If Internet hosting is not working for you, you may have to manually forward the hosting port in your router.

Join an internet Server

To connect, an external user would choose Join Internet Server and specify your external IP address.

Join a Internet Server

  1. You need the Online Host's IP Address. In the format: 89.89.89.89:1234.
  2. Your Handle, the name the other user can see on their side.
  3. It will try to connect to the other user.

WARNING: Please only connect to people you trust. Be aware that Plugin Assets will be sent via a collaboration as well. The Engine will warn you every time a plugin asset is sent.

Host a Discord based Session

The Machinery allows you to connect via Discord with your co-workers, team mates or friends. Important for this to work is that both parties: Host and Clients have the following option enabled in their Discord Options: Discord Settings -> Activity Status

Note: Host and client cannot be invisible otherwise invites wont work!

After this setting is enabled you need to add The Machinery as a game so others can see you are playing it. This allows you now to invite your friends via the Engine into your session and vice versa.

Connected

When you are connected to a collaboration session you have this view. In the connected view you can chat with the other participants of the chat. The session will not terminate / disconnect if you close the tab.

QA Pipeline

The Machinery comes with some built-in tools to support you in building games.

Statistic Tab

Allows you to visualize different statistics from different sources.

The Statistic tab consists of a Property View in which you can define your desired method of display and source. You can choose between Table, Line, or no visualization method. As sources, the engine will offer you any of the profiler scopes.

Statistics Overlay

During the simulation in the simulate tab you have the ability to open different statistic overlays.

https://www.dropbox.com/s/cmk3u9lt4d8l3n0/tm_guide_statistics_in_simulate.png?dl=1

Profiler Tab

The profiler tab will display all scopes that have been added to the profiler API. With the tab, you can record for a few moments all scopes and then afterward analyze them.

You can use the profiler API defined in the foundation/profiler.h. in your own projects. After you have loaded the [tm_profiler_api](https://ourmachinery.com/apidoc/foundation/profiler.h.html#structtm_profiler_api) in your plugin load function.

Profiler Macros
TM_PROFILER_BEGIN_FUNC_SCOPE() / TM_PROFILER_END_FUNC_SCOPE()
Starts a profiling scope for the current function. The scope in the profiler will have this name.
TM_PROFILER_BEGIN_LOCAL_SCOPE(tag) / TM_PROFILER_END_LOCAL_SCOPE(tag)
The call to this macro starts a local profiler scope. The scope is tagged with the naked word tag (it gets stringified by the macro). Use a local profiler scope if you need to profile parts of a function.

Example:

void my_function(https://ourmachinery.github.io/themachinery-books/the_machinery_book/*some arguments*/){
   TM_PROFILER_BEGIN_FUNC_SCOPE()
   // .. some code
   TM_PROFILER_END_FUNC_SCOPE()
}

Memory Usage Tab

The Machinery has a built-in Leak detection when one is using the provided allocators. Besides they all will log the used memory in the Memory Usage tab!

The memory tab will display all memory consumed via any allocator. Temporary allocators will be listed as well. Besides the memory from the CPU allocators, you can also inspect device memory used and the memory consumed by your assets.

Logging

The Machinery comes with a built-in Logger system. The Logger System lives in the foundation/log.h and contains the tm_logger_api. This API provides a few connivance macros. We can use them to log our code from anywhere. Besides this it is super easy to create your own logger and add it to the logger API.

Logging cheat sheet

You can log custom types. This is enabled via the tm_sprintf_api. You can log all primitive types like you are used to from C but as well as the engines API types. Just keep in mind the following syntax: %p{<MY_TYPE>} and the fact that you need to provide a pointer to the correct type:

typecall
boolTM_LOG("%p{bool}",&my_value);
tm_vec2_tTM_LOG("%p{tm_vec2_t}",&my_value);
tm_vec3_tTM_LOG("%p{tm_vec2_t}",&my_value);
tm_vec4_tTM_LOG("%p{tm_vec4_t}",&my_value);
tm_mat44_tTM_LOG("%p{tm_mat44_t}",&my_value);
tm_transform_tTM_LOG("%p{tm_transform_t}",&my_value);
tm_rect_tTM_LOG("%p{tm_rect_t}",&my_value);
tm_str_tTM_LOG("%p{tm_str_t}",&my_value);
tm_uuid_tTM_LOG("%p{tm_uuid_t}",&my_value);
tm_color_srgb_tTM_LOG("%p{tm_color_srgb_t}",&my_value);
tm_tt_type_tTM_LOG("%p{tm_tt_type_t}",&my_value);
tm_tt_id_tTM_LOG("%p{tm_tt_id_t}",&my_value);
tm_tt_undo_scope_tTM_LOG("%p{tm_tt_undo_scope_t}",&my_value);
tm_strhash_tTM_LOG("%p{tm_strhash_t}",&my_value);

You can register a support for your own custom type via the tm_sprintf_api.add_printer().

example:

First you define a function with a signature of the type tm_sprintf_printer. int tm_sprintf_printer(char *buf, int count, tm_str_t type, tm_str_t args, const void *data);

static int printer__custom_color_srgb_t(char *buf, int count, tm_str_t type, tm_str_t args, const void *data)
{
    const custom_color_srgb_t *v = data;
    return print(buf, count, "{ .r = %d, .g = %d, .b = %d, .a = %d , .hash = %llu }", v->r, v->g, v->b, v->a, v->hash);
}

After that you register it via the tm_sprintf_api to the add_printer() function.

TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_sprintf_api = reg->get(TM_SPRINTF_API_NAME);
    if(tm_sprintf_api->add_printer){
        tm_sprintf_api->add_printer("custom_color_srgb_t", printer__custom_color_srgb_t);
    }
}

More tm_sprintf_api formatting cheats

FmtValueResult
%I64d(uint64_t)100100
%'d1234512,345
%$d1234512.3 k
%$d10001.0 k
%$.2d25360002.53 M
%$$d25360002.42 Mi
%$$$d25360002.42 M
%_$d25360002.53M
%b36100100
%p{bool}&(bool){true}true
%p{tm_vec3_t}&(tm_vec3_t){ 1, 2, 3 }{ 1, 2, 3 }
%p{tm_vec3_t}0(null)
%p{unknown_type}&(tm_vec3_t){ 1, 2, 3 }%p{unknown_type}
%p{unknown_type:args}&(tm_vec3_t){ 1, 2, 3 }%p{unknown_type:args}
%p{tm_vec3_t&(tm_vec3_t){ 1, 2, 3 }(error)
%p{tm_rect_t}&(tm_rect_t){ 10, 20, 100, 200 }){ 10, 20, 100, 200 }
%p{tm_color_srgb_t}&TM_RGB(0xff7f00){ .r = 255, .g = 127, .b = 0, .a = 255 }

Write a custom logger

If you desire to add your own logger sink to the ecosystem there are a few steps you need to take:

  1. You need to include the foundation/log.h header
  2. You need to define a tm_logger_i in your file
  3. You need add a log function to this interface
    1. If you need some local data (such as an allocator) it might be good to define a .inst as well.
  4. After all of this you can call the tm_logger_api.add_logger() function to register your logger

Example:

#include <foundation/log.h>
// some more code

static void my_log_function(struct tm_logger_o *inst, enum tm_log_type log_type, const char *msg)
{
// do what you feel like doing!
}

tm_logger_i *logger = &(tm_logger_i){
    .log = my_log_function,
};
//.. more code
// This functions gets called at some point and this is the point I would like to register my logger
static void my_custom_api_function(void){
    tm_logger_api->add_logger(logger);
}

Note: This can be a use case for plugin callbacks. More about this see Write a plugin

This walkthrough introduces you to unit-tests.exe and shows you how to use it with The Machinery. You will learn about:

  • How to run tests
  • How to constantly monitor your changes
  • How to write tests in your plugin

This walkthrough expects basic knowledge on how to create a plugin and how a plugin is structured. If you are missing this knowledge, then you can find out more here.

Note: At this point, the testing framework is in the beginning stage. We are extending its capabilities overtime to meet the needs of modern game development QA pipelines.

About unit-tests

You can find the executable alongside tmbuild or the machinery executable in the bin/ folder. If you want to make the unit-tests executable globally accessible, you need to add it to your path environment variable. As you may have noticed, ensuring the quality of your build, tmbuild will run all unit tests of all plugins at the end of its build process. When you add a unit test to your plugin, it is guaranteed that its unit tests run every time you build. This is, of course, only guaranteed if the plugin system can find the plugin and its test.

Note: unit-tests will assume that the plugins live in a folder relative to the executable in the standardized folder plugins. If you need to load a plugin that is not in this folder, you need to provide a valid path via -p/--plugin so that unit-tests can find and run your tests.

How to run tests

To run all unit tests, execute unit-test, and it will run all tests besides the slow execution path tests. To run all unit tests, including the "slow" ones, you run unit-tests.exe -s/--slow-paths

Note: You may have noticed that if you run tmbuild regularly, you are lucky and win in the "lottery" from time to time. This means tmbuild will run all unit tests including the slow ones via unit-tests.

How to constantly monitor your changes

Like the editor, unit-tests supports hot reloading. In a nutshell, whenever plugins are rebuilt unit-tests can detect this and rerun the tests. To run in hot-reload mode startunit-tests with the -r/--hot-reload argument.

When could this be useful? It can be helpful on CI Server where build and test servers are different. The build server's final build step uploads the generated dlls to the test server if everything works fine. The test server is monitoring the filesystem, and whenever the dlls change, unit-tests would rerun all tests, also the slow ones, to ensure that all works. It could save time on the build server so the build times are faster and the developer knows quicker if the build fails. Also, the build server does not need a graphics card to run eventual graphic pipeline-related tests. The test server, on the other hand, could run such tests.

How to write your tests

All that is needed is to write tests is to register them via the TM_UNIT_TEST_INTERFACE_NAME. You can find the interface in the unit_tests.h. TM_UNIT_TEST_INTERFACE_NAME expects a pointer of the type tm_unit_test_i. This interface expects a name and a function pointer to the test entry function.

// Interface for running unit tests. To find all unit test, query the API registry for
// `TM_UNIT_TEST_INTERFACE_NAME` implementations.
typedef struct tm_unit_test_i
{
    // Name of this unit test.
    const char *name;

    // Runs unit tests, using the specified test runner. The supplied allocator can be used for
    // any allocations that the unit test needs to make.
    void (*test)(tm_unit_test_runner_i *tr, struct tm_allocator_i *a);
} tm_unit_test_i;

At this point, we have not tackled the following possible questions:

  • Where and how do we register the interface?
  • What could this interface look like?
  • What does the test itself look like?

Let us walk through those questions:

Where and how do we register the interface? We need to register our tests in the same function as everything else that needs to be executed when a plugin loads: in our tm_load_plugin. It may look like this:

#include <foundation/unit_test.h>
//...
// my amazing plugin
TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_global_api_registry = reg;
    //...
    tm_add_or_remove_implementation(reg, load, TM_UNIT_TEST_INTERFACE_NAME, my_unit_tests);
}

Here we register our test interface to the TM_UNIT_TEST_INTERFACE_NAME.

What could this interface look like? After we have done this, all we need to do is declare our my_unit_tests. It is as easy as it gets:

#include <foundation/unit_test.h>
//...
tm_unit_test_i *entity_unit_test = &(tm_unit_test_i){
    .name = "my_unit_tests",
    .test = test_function,
};

What does the test itself look like? All that's left is to write the test. Let us write this test. In its core all we need to do is write a function of the signature: (tm_unit_test_runner_i *tr, struct tm_allocator_i *a). In its body, we can define our tests.

static void test_function(tm_unit_test_runner_i *test_runner, tm_allocator_i *allocator)
{
    //.. code
}

The test runner variable test_runner is needed to communicate back to the test suite about failures etc. The following macros will help you write tests. They are the heart of the actual tests.

MacroArgumentsDescription
TM_UNIT_TEST(test_runner, assertion)Unit test macro. Tests the assertion using the test runner test_runner.
TM_UNIT_TESTF(test_runner, assertion, format, ...)As TM_UNIT_TEST() but records a formatted string in case of error.
TM_EXPECT_ERROR(test_runner, error)Expect the error message error. If the error message doesn't appear before the next call to record(), or if another error message appears before it, this will be considered a unit test failure.

Note that for TM_EXPECT_ERROR to work properly, you must redirect error messages to go through the test runner, so that it can check that the error message matches what's expected.

It's time for some tests. Let us write some tests for carrays

#include <foundation/unit_test.h>
#include <foundation/carray.inl>
//.. other code
static void test_function(tm_unit_test_runner_i *test_runner, tm_allocator_i *allocator)
{
    /*carray*/ int32_t *a = 0;
    TM_UNIT_TEST(test_runner, tm_carray_size(a) == 0);
    TM_UNIT_TEST(test_runner, tm_carray_capacity(a) == 0);
    TM_UNIT_TEST(test_runner, a == 0);
    tm_carray_push(a, 1, &allocator);

    TM_UNIT_TEST(test_runner, tm_carray_size(a) == 1);
    TM_UNIT_TEST(test_runner, tm_carray_capacity(a) == 16);
    TM_UNIT_TEST(test_runner, a);
    TM_UNIT_TEST(test_runner, a[0] == 1);

    tm_carray_header(a)->size--;

    TM_UNIT_TEST(test_runner, tm_carray_size(a) == 0);
    TM_UNIT_TEST(test_runner, tm_carray_capacity(a) == 16);

    tm_carray_grow(a, 20, &allocator);
    tm_carray_header(a)->size = 20;

    TM_UNIT_TEST(test_runner, tm_carray_size(a) == 20);
    TM_UNIT_TEST(test_runner, tm_carray_capacity(a) == 32);
}

All that's left is to build via tmbuild our plugin and watch the console output if our tests fail. This is how you integrate your tests into the whole build pipeline.

How to write integration tests

This walkthrough shows you how to write integration tests with our integration test framework. You will learn about:

  • How an integration test differs from a unit test.
  • Where to find the integration test framework and how to write a test
  • How to run an integration test.

About integration tests

Integration testing is the phase in testing software in which individual software modules are combined and tested as a group. Integration testing is conducted to evaluate a system's compliance or its interaction as a whole within specified functional requirements. It is generally used after unit testing to ensure that the composition of the software works. It is a potent tool to validate certain bugs that are hard to reproduce only after using software extensively. You can simulate this with integration tests. Besides, it is a very powerful tool validating that a bug fix was successful.

By their very nature, integration tests are slower and more fragile than unit tests, but they can also find issues that are hard to detect with regular unit tests. Each integration test runs in a specific "context", identified by a string hash. The context specifies the "scaffolding" is set up before the unit test runs.

How to write integration tests

Where to find the integration test framework?

The integration test framework can be found in the integration_test.h ,and is part of the foundation library. We need to include this header file, and then we can start writing our tests.

#include <foundation/integration_test.h>

To write a test you need to register it via the TM_INTEGRATION_TEST_INTERFACE_NAME. It expects a pointer of the type tm_integration_test_i. This interface expects a name and a function pointer to the test function (tick). Also, it expects a context. The context is a string hash. For example: TM_INTEGRATION_TEST_CONTEXT__THE_MACHINERY_EDITOR.

// Interface for integration tests.
typedef struct tm_integration_test_i
{
    // Name of the test.
    const char *name;
    // Context that this test will run in. Tests will only be run in contexts that match their
    // `context` setting.
    tm_strhash_t context;
    // Ticks the test. The `tick()` function will be called repeatedly until all it's `wait()` calls
    // have completed.
    void (*tick)(tm_integration_test_runner_i *);
} tm_integration_test_i;

At this point, we have not tackled the following possible questions:

  • Where and how do we register the interface?
  • What could this interface look like?
  • What does the test itself look like?

Let us walk those questions through:

Where and how do we register the interface?

We need to register our tests in the same function as everything else that needs to be executed when a plugin loads: in our tm_load_plugin.

// my amazing plugin
TM_DLL_EXPORT void tm_load_plugin(struct tm_api_registry_api *reg, bool load)
{
    tm_global_api_registry = reg;
    //...
    tm_add_or_remove_implementation(reg, load, TM_INTEGRATION_TEST_INTERFACE_NAME, my_integration_tests);
}

Here we register our test interface to the TM_INTEGRATION_TEST_INTERFACE_NAME.

What could this interface look like?

After we have done this, we need to declare our my_integration_tests.

tm_integration_test_i my_integration_tests = {
    .name = "stress-test",
    //Context that specifies a running The Machinery editor application
    .context = TM_INTEGRATION_TEST_CONTEXT__THE_MACHINERY_EDITOR,
    .tick = my_test_tick,
};

The name field is important because, later on, we need to use this name when we want to run the test. The context makes sure that it runs and boots up the Editor. TM_INTEGRATION_TEST_CONTEXT__THE_MACHINERY_EDITOR is defined in #include <foundation/integration_test.h>. The function my_test_tick gets called and the magic can happen.

What does the test itself look like?

Let us write this test. We need to write a function of the signature: (tm_integration_test_runner_i *). In its body, we can define our tests.

static void my_test_tick(tm_integration_test_runner_i *test_runner)
{
  //.. code
}

The test runner variable test_runner is needed to communicate back to the test suite about failures etc. The following macros will help you write tests. They are the heart of the tests.

MacroArgumentsDescription
TM_WAITtest_runner, secondWaits for the specified time inside an integration test.
TM_WAIT_LOOPtest_runner, second, iSince TM_WAIT() uses the __LINE__ macro to uniquely identify wait points, it doesn't work when called in a loop. In this case you can use TM_WAIT_LOOP() instead. It takes an iteration parameter i that uniquely identifies this iteration of the loop (typically it would just be the iteration index). This together with __LINE__ gives a unique identifier for the wait point.

TM_WAIT_LOOP() WARNING

If you have multiple nested loops, be aware that using just the inner loop index j is not enough to uniquely identify the wait point since it is repeated for each outer loop iteration. Instead, you want to combine the outer and inner indexes.

Lets write some example:

#include <foundation/integration_test.h>
static void my_test_tick(tm_integration_test_runner_i *test_runner)
{
    const float step_time = 0.5f;
    if (TM_WAIT(tr, step_time))
    open(tr, "C:\\work\\sample-projects\\modular-dungeon-kit\\project.the_machinery_dir");
    if (TM_WAIT(tr, step_time))
    save_to_asset_database(tr, "C:\\work\\sample-projects\\modular-dungeon-kit\\modular-dungeon-kit.the_machinery_db");
    // ...
}

How do we run an integration test?

To run your newly created integration test, we need to build the project via tmbuild. Then start The Machinery with the -t/--test [NAME] parameter. It runs the specified integration test.

You can use multiple --test arguments to run multiple tests. This will boot up the engine and run your integration tests.

Example:

./bin/the-machinery.exe --test stress-test

Helper Tools

The Machinery comes with a few tools to make your daily life easier. There are tools for:

  • Generating Static Hash Values: hash
  • Generate Graph Nodes for you:generate-graph-nodes
  • To generate the solution files of The Engine or your plugin: tmbuild
  • Execute your unit tests: unit-test
  • Generate your Localization tables: localize
  • Free your Plugins from unneeded includes: trim-includes.exe

How to use tmbuild

We described tmbuild's core idea in our blog-post One-button source code builds. tmbuild is our custom one-click "build system." and it is quite a powerful tool. It allows you to do the most important tasks when developing with The Machinery: Building your plugin or the whole engine.

You can execute the tool from any terminal such as PowerShell or the VS Code internal Console window.

The key features are:

  • building
  • packaging
  • cleaning the solution/folder
  • downloading all the dependencies
  • running our unit tests

This walkthrough introduces you to tmbuild and shows you how to use and manipulate The Machinery Projects. You will learn about:

  • How to build with it
  • How to build a specific project with it
  • How to package your project

Also, you will learn some more advanced topics such as:

  • How to build/manipulate tmbuild

Table of Content

Installing tmbuild

When you download and unzip The Machinery either via the website or via the download tab you can find tmbuild in the bin folder in the root.

Alternatively, you can build it from source code\utils. We will talk about this later in this walkthrough.

Before we use tmbuild, we need to ensure that we have installed either build-essentials under Linux, XCode on Mac, or Visual Studio 2017 or 2019 (Either the Editor such as the Community Edition or the Build Tools).

Windows Side nodes:

On Windows, it is essential to install the C/C++ Build tools. If you run into the issue that tmbuild cannot find Visual Studios 2019 on Windows, it could be because you installed it on a typical path. No problem, you can just set the environment variable TM_VS2017_DIR or TM_VS2019_DIR to the root C:\Program Files (x86)\Microsoft Visual Studio\2019. The tool will find the right installed version automagically.

Set up our environment variables

Before we can build any project, we need to set up our environment. You need to set the following environment variable: (If this one has not been set the tool will not be able to build)

  • TM_SDK_DIR - This is the path to find the folder headers and the folder lib

If the following variable is not set, the tool will assume that you intend to use the current working directory:

  • TM_LIB_DIR - The folder which determines where to download and install all dependencies (besides the build environments)

How to add environment variables?

Windows

On Windows all you need to do is you need to add the folder where you installed The Machinery to your environment variables. You can do this like this: Start > Edit the system environment variables > environment variables > system variables > click New... > add TM_SDK_DIR or TM_LIB_DIR as the Variable Name and the needed path as the Variable Value. Close and restart the terminal or Visual Studio / Visual Studio Code. As an alternative, you can set an environment variable via PowerShell before you execute tmbuild, which will stay alive till the end of the session: $Env:TM_SDK_DIR="..PATH"

Debian/Ubuntu Linux

You open the terminal or edit with your favorite text editor ~/.bashrc and you add the following lines:

#...
export TM_SDK_DIR=path/to/themachinery/
export TM_LIB_DIR=path/to/themachinery/libs

(e.g. via nano nano ~/.bashrc)

Let us Build a plugin.

All you need to do is: navigate to the root folder of your plugin and run in PowerShell tmbuild.exe.

If you have not added tmbuild.exe to your global PATH, you need to have the right path to where tmbuild is located. user@machine/home/user/tm/plugins/my_plugin/> ./../../bin/tmbuild

This command does all the magic. tmbuild will automatically download all the needed dependencies etc., for you (Either in the location set in TM_LIB_DIR or in the current working directory). You may have noticed tmbuild will always run unit tests at the end of your build process.

Note: tmbuild will only build something when there is a premake5.lua and a libs.json in the current working directory.

Let us build a specific project.

Imagine you have been busy and written a bunch of plugins, and they are all connected and managed via the same Lua file (premake5 file). Now you do not want to check everything all the time. No problem, you can follow these steps: If you run tmbuild --help/-h, you will see many options. One of those options is --project. This one allows you to build a specific project.

tmbuild.exe --project my-project-name

The tool will automatically find the right project and build it. On Windows, you can also provide the relative/absolute path to the project with extension: ``

tmbuild --project /path/to/project.vcxproj

Note: If a project cannot be found it will build all projects.

How how to package a project via tmbuild?

To package a project via tmbuild, all you need to do is use the -p [package name] or --package [package name] command. A package file needs to be of type .json and follow our package scheme, which you can find here.

How to build or manipulate tmbuild from source

You can find the source code of tmbuild in the folder code\utils\tmbuild. In the folder code\utils, you can also find the source code of all the other uses the engine uses.

You can build tmbuild via tmbuild. All you need to do is navigate the code\utils folder and run tmbuild --project tmbuild.

If you do not have access to a build version of tmbuild but to the whole source, you have to follow the following steps:

Windows 10

Make sure you have Visual Studio 19 and the Build Tools installed. Besides, check if you can find msbuild in the terminal. You can install msbuild / vs studio via PowerShell: https://github.com/Microsoft/vssetup.powershell

To check just run:

msbuild

if you cannot find it just add it to your environment path variables: with e.g. C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\

To build from source:

Open a PowerShell instance in The Machinery folder and run the following commands:

# this part can be skipped if you have already downloaded
# all the dependencies and created a highlevel folder to your lib dependencies:
mkdir lib
cd lib
 wget https://ourmachinery.com/lib/bearssl-0.6-r1-win64.zip -OutFile bearssl-0.6-r1-win64.zip
 wget https://ourmachinery.com/lib/premake-5.0.0-alpha14-windows.zip -OutFile premake-5.0.0-alpha14-windows.zip
Expand-Archive -LiteralPath './bearssl-0.6-r1-win64.zip' -DestinationPath "."
Expand-Archive -LiteralPath './premake-5.0.0-alpha14-windows.zip' -DestinationPath "."
cd ..
# continue here if the dependencies are already downloaded
# set TM_LIB_DIR if you have not set it already
$env:TM_LIB_DIR="/path/to/themachinery/lib"
$env:TM_SDK_DIR="/path/to/themachinery/"
# run premake
./../../lib/premake-5.0.0-alpha14-windows/premake5 [vs2019|vs2017]
# navigate to the highlevel folder of the code (here you find the libs.json and the
# premake5.lua
cd code/utils
msbuild.exe "build/tmbuild/tmbuild.vcxproj" /p:Configuration="Debug Win64" /p:Platform=x64

Make sure that you either choose vs2019 or vs2017 not [vs2019|vs2017]

On Debian/Ubuntu

Open a terminal instance and run the following commands:

# If you do not have the huild essentials installed make sure you do:
sudo apt install build-essential clang zip -y
# otherweise continue here:
cd your-folder-of-tm
# this part can be skiped if you have already downloaded
# all the dependencies and created a highlevel folder to your lib dependencies:
mkdir lib
cd ./lib
wget https://ourmachinery.com/lib/bearssl-0.6-r1-linux.zip
wget https://ourmachinery.com/lib/premake-5.0.0-alpha15-linux.zip
unzuip bearssl-0.6-r1-linux.zip .
unzip premake-5.0.0-alpha15-linux.zip .
chmod +x ./premake-5.0.0-alpha15-linux/premake5
cd ..
# continue here if the dependencies are already downloaded
# set TM_LIB_DIR if you have not set it already
export TM_LIB_DIR=/path/to/themachinery/lib
export TM_SDK_DIR=/path/to/themachinery/
# run premake
./../../lib/premake-5.0.0-alpha15-linux/premake5 gmake
# navigate to the highlevel folder of the code (here you find the libs.json and the
# premake5.lua
cd code/utils
# run make:
make tmbuild

How to add tmbuild globally accessible?

Windows On Windows, all you need to do is you need to add the folder/bin to your environment variables. This can be done like this: Start > Edit the system environment variables > environment variables > system variables > search in the list for path > click Edit > click new > add the absolute path to themachinery/bin in there re-login or reboot.

Debian/Ubuntu Linux You open the terminal or edit with your favorite text editor ~/.bashrc, and you add the following lines: export PATH=path/to/themachinery/bin:$PATH (e.g. via nano nano ~/.bashrc)

How to use hash.exe

This walkthrough introduces you to hash.exe and shows you how to use it with The Machinery.

You will learn about:

How to use hash.exe and TM_STATIC_HASH

While working with The Machinery, you will have surely noticed that its systems often expect a hashed version of a string as input. In this document, we will be dealing with static hashes defined with TM_STATIC_HASH to identify, for example, a type in our data model, The Truth. If you are searching documentation on runtime hashes, you’ll want to check out the murmurhash64a.inl files.

In this document, we will be using hash.exe to generate new hash values or update the changed ones. The hash.exe utility checks the entire source code and makes sure that wherever you use TM_STATIC_HASH, the numeric value v matches the actual hash of the string s (if not, the code is updated). If you do not run the executable before you build, you will have compile errors.

Note: tmbuild has the option to run hash.exe before it builds. tmbuild --gen-hash.

Let us look at where this tool is useful and is being used a lot: In The Truth.

For example, When defining Truth Objects or Types, we are using this tool to hash the name of the type statically

//...

#define TM_TT_TYPE__ASSET "tm_asset"

#define TM_TT_TYPE_HASH__ASSET TM_STATIC_HASH("tm_asset", 0xca71127abbb72960ULL)

enum {

 TM_TT_PROP__ASSET__NAME = 0, // string

 TM_TT_PROP__ASSET__DIRECTORY, // reference(ASSET_DIRECTORY)

 TM_TT_PROP__ASSET__UUID_TAGS, // subobject_set(UINT64_T) storing the UUID of the associated tag.

 TM_TT_PROP__ASSET__OBJECT, // subobject(*)

};

//...

In the above example we have the definition of the Truth Type tm_asset. The #define TM_TT_TYPE_HASH__ASSET TM_STATIC_HASH("tm_asset", 0xca71127abbb72960ULL) was generated by the hash utility. Before the tool ran the code looked similar to this: #define TM_TT_TYPE_HASH__ASSET TM_STATIC_HASH("tm_asset")

For example later on we can use it to create an object of this type. If we do not have the corresponding type id tm_tt_type_t we need to ask The Truth:

tm_the_truth_api->object_type_from_name_hash(tt, TM_TT_TYPE_HASH__ASSET);

tmbuild package json Reference

The tmbuild can package a project based on rules set in a .json file. This file needs to adhere to the scheme described here. Every package json file is structured in its core like this:

{
    "name":"package name",
    "steps": []
}

The name will be used to determine the name of the package folder in tm root/build directory. The steps are the key in the system. They will be executed linearly after each other and the next step will only be executed if the previous step was successful. Steps are defined as a normal json object. The following list displays all the functions which can be used plus adequate examples.

Table of Content

Utilities

Logging

Setting ParameterTypeDescription
logstringTakes a string as value and will on execution print this string to the output log

Example

{
    "name":"example-plugin",
    "steps": [
    {
        "log":"Example Plugin"
    }
    ]
}

Change directory

Setting ParameterTypeDescription
chdirstringTakes a string and tries to change directory to this path.

Example

{
    "name":"example-plugin",
    "steps": [
    {
        "log":"change dir",
        "chdir":"utils"
    }
    ]
}

Platforms

Setting ParameterTypeDescription
platformsstring arrayTakes a list of possible platforms this step shall be executed on
{
   "action":"delete-dirs",
   "root":"build/tmbuild/",
   "dirs":[
      "vs2017"
   ],
   "platforms":[
      "windows"
   ]
}

Actions

An action defines what the current step shall do.

Setting ParameterTypeDescription
actionstringTakes a string as value which will determine what kind of action this step shall do

Filesystem operations

build

Will build the current directory unless changed via chdir in release and debug settings. Optionally it has a project field if you only want to build one project. One can also modify the build tool and specify that in an extra field.

Setting ParameterTypeDescription
projectstringTakes the name of the project it shall build
build-toolstringTakes the name of the premake build tool which shall be used.
{
   "log":"Build Lib",
   "action":"build",
   "project":"lib_static",
   "build-tool":"vs2017",
   "platforms":[
      "windows"
   ]
}

copy-files

Will copy files which are specified in a files field. Also the location is needed to where those files should be copied. This location needs to be specified in the fieldto-dir

Setting ParameterTypeDescription
filesstring arrayA list of relative paths to the files the step shall copy
to-dirstringThe destination directory

copy-file-patterns

Will copy files based on file patterns to a set location.

Setting ParameterTypeDescription
from-dirstringSets the location from where files shall be copied
to-dirstringThe destination directory of the action
dir-patternsstring arrayWhat folders shall be copied. Patterns used in gitignore files can be used e.g. *.zip
file-patternsstring arrayWhat files shall be copied. Patterns used in gitignore files can be used e.g. *.zip
{
   "action":"copy-file-patterns",
   "from-dir":"plugins/my-plugin",
   "to-dir":"dest/folder/plugins/my-plugin",
   "dir-patterns":[
      "*"
   ],
   "file-patterns":[
      "*.c",
      "*.h",
      "*.inl"
   ]
}

delete-dirs

Will delete folders which are specified in dirs but taking the root field into account.

Setting ParameterTypeDescription
dirsstring arrayA list of relative paths to the files the step shall delete
rootstringThe root directory

delete-file-patterns

Will delete files based on a pattern.

Setting ParameterTypeDescription
dirstringA folder in the package folder
dir-patternsstring arrayWhat folders shall be deleted. Patterns used in gitignore files can be used e.g. *.zip
file-patternsstring arrayWhat files shall be deleted. Patterns used in gitignore files can be used e.g. *.zip
{
   "action":"delete-file-patterns",
   "dir":"bin/plugins",
   "dir-patterns":[
      "*"
   ],
   "file-patterns":[
      "*.pdb",
      "*.lib",
      "*.exp"
   ]
},

docgen

Will generate documentation based on the provided arguments in the arguments field.

Setting ParameterTypeDescription
argsstringArguments for docgen.exe to generate documentation

localize

Will check if all localizations are present if not fail.

set-sdk-dir

Will set the SDK if not set dir to the current build directory [cwd]/build/[package-name]. Has no other fields.

zip

Will zip the package folder and if the field add-time-stamp is set to true it will add the time stamp to the name of the zipped file.

Setting ParameterTypeDescription
add-time-stampboolIf set to true the zipped file will contain the timestamp

test

Will execute tests defined in the tests field.

Setting ParameterTypeDescription
testsstring arrayNames of tests

clean

Will call the --clean command on tmbuild.

Writing an executable

Writing a stand-alone executable is similar to writing a plugin except that you need to write initialization code for booting up all the systems that you want to use as well as the main loop code that runs the executable.

bin/simple-draw.exe is a sample executable that uses the UI / Draw2D interfaces of The Machinery to implement a simple drawing program:

Simple Draw executable.

The source code of this sample program is available in the samples directory. By playing with and modifying this, you can see how to build your own applications on top of The Machinery.

Note that just as when writing plugins, you need a Visual Studio installation (2019) to build an executable. You need to set TM_SDK_DIR to the location of The Machinery SDK you are using, and you use tmbuild.exe to build the executable.

bin/simple-3d.exe is a sample executable that uses the full 3D API.

Simple 3D executable.

You will find the source code for this too in the samples directory.