Immersive Knowledge Interaction
In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Currently, the player can only interact with knowledge items via a special UI that mirrors the knowledge formalized in MMT as scroll. This 2D UI causes a cognitive dissonance to the 3D world that it talks about. A more immersive solution would fit better (at least for some applications)
For more immersion, we would like to
- a) automatically synchronize certain facts about the world dynamically (e.g. for moving objects)
- b) use in-game triggers to communicate with the logic engine
- c) have in-game interfaces to communicate with the logic engine
- d) allow players to directly assign objects somehow instead of using the corresponding generated icons or build other features that result in more fluent gameplay.