thesis-projects issueshttps://gl.kwarc.info/kwarc/thesis-projects/-/issues2023-01-20T22:27:19Zhttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/52Surveying methods to visualize, narrate, and interact with theorems in educat...2023-01-20T22:27:19ZNavid RouxSurveying methods to visualize, narrate, and interact with theorems in educational math gamesIn the [FrameIT project](https://uframeit.github.io/) we build a framework for educational math games such that mathematical content in these games (e.g., theorems, proofs, deduction of conclusions) is outsourced to a full-blown logical ...In the [FrameIT project](https://uframeit.github.io/) we build a framework for educational math games such that mathematical content in these games (e.g., theorems, proofs, deduction of conclusions) is outsourced to a full-blown logical inference engine (namely, [MMT](https://kwarc.info/systems/mmt/)). This makes it possible to [detangle knowledge management from game design in serious games](https://kwarc.info/people/mkohlhase/papers/cicm20-frameit.pdf).
MMT allows the formalization of complex mathematical theorems in the style "given these prerequisites, those are the consequences." In the FrameIT project, we call theorems formalized in this style *scrolls*. During game play, users can apply such theorems interactively. For example, they can first fill, say, the first prerequisite of a given theorem, then wait (and see the interactive response), and finally fill in the second prerequisite.
So far, we employ a minimalistic visualization, yet one that is already interactive. See Figures 5 and 6 in our workshop paper ["Dynamic User Interfaces via Incremental Knowledge
Management"](https://kwarc.info/teaching/CICM21WS/mathui7.pdf) for a figurative overview and read the paper for all details.
A project or thesis could target enhancing this feature:
1. Requirements Analysis: we could demand theorems be equippable with text descriptions (already done), math formulae (TeX, MathML?), images (SVGs?). How interactive should these elements be?
2. Survey of Possible Implementation Strategies
1. How to tackle rendering? Use HTML or SVG rendering engine in Unity?
2. How to tackle interactivity?
3. Is the interactivity, i.e. data binding, mono- or bidirectional?
3. Specification of FrameIT MMT server <-> Unity frontend communication protocol
4. Implementation
5. Evaluation
A major technical problem may be that rendering of SVG and HTML within Unity is quite limited or only possible via [costly plugins](https://developer.vuplex.com/webview/overview).
Related GitHub issues: https://github.com/UFrameIT/UFrameIT/issues/18, https://github.com/UFrameIT/UFrameIT/issues/70.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/51Frame World 22022-10-21T11:20:21ZRichard MarcusFrame World 2In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world. In the ini...In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world. In the initial prototype game Frame World, the player is tasked to measure the height of a tree with the help of trigonometry.
After a series of projects that have improved game Frame World or explored game variants, we now want to focus our work towards a true successor game **Frame World 2**. Aside from incorporating the lessons learned and technical advancements so far, we define three main pillars that we want to implement for the new game:
#### 1. Playability
While the original game is very much playable, it only consists of a single game level. By now, there have been extensions to that, but to count as a meaningful serious game, more structure is required. In particular, we have the following requirements:
1. a setting that connects different problems and gives the player motivation to solve them
2. a technical progression system that leads the player through the full game
3. a sufficient set of game problems
4. a didactically sound progression through the game that allows players to solve increasingly difficult and complex problems
#### 2. Dynamic Gameplay
One main advantage of video games is that it is possible to visualize and simulate processes that are problematic in the real world because they are, e.g., expensive or dangerous. In the original game this concept is only partly implemented in the sense that abstract mathematical concepts are brought into practical applications. We want to extend this to dynamic processes that involve movement of objects and matter.
On the technical level, this allows us to implement a bigger range of problems in the UFrameIt framework, but more importantly, this is supposed to improve the learning experience as the consequences of the player's actions can be visualized interactively. This would lead to more immersive gameplay and a greater sense of achievement when the problem is solved.
There have been first steps to this in the form of the LoViWo project, where the movement of cogwheels is modeled knowledge based. While this project is isolated from the overall UFrameIt framework so far, it is also based on MMT and should be integrable without greater adaptions. Furthermore, the principles of solving equations to calculate the movement should be adaptable to other dynamic problems. Examples that might fit well to the themes of serious games are simple fluid simulations or trajectory calculations.
An example with less movement but also dynamic interactions could be electrical circuits.
#### 3. Knowledge Based World Design
A common issue in the development of video games is the interaction between different game entities or events. As different parts of the game are developed individually, adding such interactions may be difficult and introduce overhead.
Imagine a merchant that sells items to the player. Depending on the progress and events in the game the available stock may change. In these cases, there needs to be a custom script that updates the stock list accordingly. This is no issue in theory, but in practice these kinds of interactions occur repeatedly throughout games, which makes it very difficult to keep an overview of all dependencies in complex games.
A special case of this are quests and missions. Particularly open world games can easily come with hundreds of them. As simple tasks like collecting a certain amount of items or defeating a monster quickly become repetitive, significant effort is required for the creation of engaging side quests.
If we now look at this from a knowledge based perspective, we can model all kinds of interactions in the same way, just the level of abstraction changes. In UFrameIt, we call elements of information **Facts**. For example, each point the player measures for the determination of the tree height is registered as such a fact. But in the same way, we can also model events that determine the offerings of the merchant as facts. Finally, we interpret the requirements for fulfilling a quest as facts as well. The advantage for the game developer is that with our framework, the different interactions no longer have to be implemented individually but simply stated declaratively for the respective game entities.
In practice, a quest of obtaining a potato could be implemented as follows: a potato is a buyable game entity that is produced by farms in potato land. For merchants to offer potatoes, there needs to be a route between potato land and the current town. Since the old river bridge has collapsed, there no longer is a such a route. Trees are objects that can be chopped down and can be traversed when chopped. Five trees of height greater than the width of the river can form a river bridge. An axe is sharp enough to chop five trees and the player could only acquire one axe. The height of trees can be measured with the correct use of trigonometry.
While this also would be a quest of obtaining an item, the different dependencies are not hardcoded into the quest anymore. More exciting quests can be implemented now by only extending the knowledge entities accordingly (possibly also with scripted events, cutscenes etc.). Last, interactions with the whole game world can be added easily. An NPC could consider the player as man of culture for traveling with a potato and choose to aid the player - thus changing the course of the game forever...Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/49User Guidance in FrameIT (Project/Praktikum)2022-02-11T10:04:06ZRichard MarcusUser Guidance in FrameIT (Project/Praktikum)In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
With the ...In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
With the growing number and complexity of interaction tools, it would be helpful to have some kind of tutorial or instructions for players. The task here will be to investigate and implement different ways of providing this.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/48Knowledge-based simulations of virtual worlds in MMT2021-11-12T10:46:29ZFrederik SchaeferKnowledge-based simulations of virtual worlds in MMTIn the LoViWo for logical virtual worlds, we model the possible component types (cog wheels, pistons, ...) of a virtual world as MMT theories.
The interaction between any two component types is also modeled as an MMT theory using axioms ...In the LoViWo for logical virtual worlds, we model the possible component types (cog wheels, pistons, ...) of a virtual world as MMT theories.
The interaction between any two component types is also modeled as an MMT theory using axioms for the physical laws.
A virtual world is a set of instances of these components and their interactions and is modeled as an MMT theory with corresponding imports/structures.
In this project/thesis you build a solver in MMT that can be used by a game engine like unity to compute updates in the virtual world using the knowledge in the MMT description of the world.
Concretely an event in the virtual world creates an update to some of the constants in the MMT theories representing the world.
Using the new values of the constants, MMT must use the axioms to solve for the values of the other constants and send the new values to the game engine. Your focus will be the MMT side using an interface for receiving/sending updates.
In particular we expect that in most cases the axioms will simplify to systems of linear equations, which can be solved easily.
This is presumably true even if the original axioms are non-linear or not even purely equational because we can plug in concrete values for the updated constants and for constants that are marked as non-changing (positions of the cog wheels as opposed to their angles).https://gl.kwarc.info/kwarc/thesis-projects/-/issues/46Representing Game Levels in MMT (Project or Bachelor Thesis Topic)2021-09-02T12:10:45ZRichard MarcusRepresenting Game Levels in MMT (Project or Bachelor Thesis Topic)In the FrameIT project, we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
We visualize problems or puzzles in the form o...In the FrameIT project, we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
We visualize problems or puzzles in the form of interactive *scrolls*, where players can input their gathered knowledge to obtain more information and, eventually, solve the problem.
We want to have the conditions to solve the game level within MMT!
[WIP, description needs update]
das Konzept von Levels in MMT irgendwie repräsentieren (ähnlich wie wir Scrolls in MMT auch in einer strukturierten Art und Weise repräsentieren)
in Metainformationen sowas repräsentieren wie "the level is solved iff dist(C, A) evaluates to a numeric value"
Insb. könnte mensch sich überlegen, dass es durchaus Levelziele geben wird, die nur Unity überprüfen kann, die MMT-extern sein müssen.
Die könnte mensch trotzdem in MMT repräsentieren a la "level is solved iff [some condition mmt can check internally] and [some condition Unity needs to check]"
Geht in Richtung "flexiformale Repräsentation von Levelzielen in MMT"
For more information, please take a look at the [project issue](https://github.com/UFrameIT/UFrameIT/issues/18) and #1.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/44FrameIT on Android2023-02-03T10:08:16ZRichard MarcusFrameIT on AndroidIn the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
To deploy...In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
To deploy FrameIT games in schools, it is very important to also support Android.
Accordingly, this topic involves adapting current games and features and design new interactions and possibly new games/levels that have synergies with touchscreens.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/43FrameIT in the Real World2021-01-29T13:30:22ZRichard MarcusFrameIT in the Real WorldIn the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
For even ...In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
For even better application to real world problems, we want to add support for augmented reality versions with Mobile AR or devices like the hololens.
Therefore, one of the main tasks will be to capture the geometry of the environment and make it possible to place points and measure information based on this.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/42Immersive Knowledge Interaction2021-12-17T13:13:01ZRichard MarcusImmersive Knowledge InteractionIn the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Currently...In the FrameIT project (#1 and https://uframeit.github.io), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Currently, the player can only interact with knowledge items via a special UI that mirrors the knowledge formalized in MMT as *scroll*. This 2D UI causes a cognitive dissonance to the 3D world that it talks about. A more immersive solution would fit better (at least for some applications)
For more immersion, we would like to
- a) automatically synchronize certain facts about the world dynamically (e.g. for moving objects)
- b) use in-game triggers to communicate with the logic engine
- c) have in-game interfaces to communicate with the logic engine
- d) allow players to directly assign objects somehow instead of using the corresponding generated icons
or build other features that result in more fluent gameplay.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/41Linking Formalized Situations to specific Game Objects/Situations2020-12-20T09:26:44ZRichard MarcusLinking Formalized Situations to specific Game Objects/SituationsIn the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Based on parametric situation theories, ...In the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Based on parametric situation theories, which are instantiated by "pretty small Unity scripts", we want to link the parameters to specific points in the world modeled in Unity.
Our goal here is to create a toolkit for game developers that allows them to integrate the formalized knowledge into the games in the form of modular elements that can linked to game objects.
Work Areas:
- 1. Making the formalized knowledge available to the Unity Editor itself (https://learn.unity.com/tutorial/editor-scripting#), which could enable interactions like this:
- list available fact types so that developers can select which gadgets can create these facts
- list available scrolls so that developers can build levels based on the problems defined in the scrolls
- 2. Scripting the generation of semantic game objects
- we could define certain object attributes on the knowledge side like "base_tree"
- if the developer assigns this attribute to an object in the game it should then be enhanced automatically: place it in a 90 degree angle to the ground, generate snapzones for root and top, configure colliders
- 3. Advanced Toolkit Interactions based on 1 and 2, e.g., if a Fact is connected to a Gadget, we know which Gadget should be used to create a certain fact for a solution
- 4: Building a graphical tool for this. Unity recently acquired [Bolt](https://assetstore.unity.com/packages/tools/visual-scripting/bolt-163802). Maybe it could be integrated into this, maybe we want/need our own editor, needs investigation.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/37FrameIT Multiplayer2023-02-05T00:51:15ZRichard MarcusFrameIT MultiplayerIn the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Many game problems or puzzles could also...In the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Many game problems or puzzles could also be solved through cooperation of multiple players.
To implement this in FrameIT, two aspects need to be investigated and tested:
* The general implementation, which could be a multiplayer server or even local multiplayer.
* The FrameIT-specific implementation. We visualize in the form of interactive *scrolls*, where players can input their gathered knowledge to obtain more information and, eventually, solve the problem.
We require concepts how to manage the knowledge input from multiple players. Some scrolls may need to be available globally while others would benefit from each player having his own.
Additional multiplayer interactions are possible: For example, gathered knowledge or scrolls could be traded between players or players could get points and get a ranking in a scoreboard.https://gl.kwarc.info/kwarc/thesis-projects/-/issues/35FrameIT State Interface2020-12-03T17:45:48ZRichard MarcusFrameIT State InterfaceIn the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Since (formalized) knowledge is generate...In the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Since (formalized) knowledge is generated interactively during runtime, it is difficult to debug with existing MMT utilities.
Accordingly, we need adaptions to make use of these and new concepts to interact with the generated content on the programming language level.
Based on this, we would like to have an interface that helps developers to monitor the state of the underlying knowledge during gameplay.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/34Machine Learning Applications for FrameIT2023-02-03T10:26:41ZRichard MarcusMachine Learning Applications for FrameITIn the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Building on the [Unity machine learning ...In the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Building on the [Unity machine learning integration](https://unity.com/products/machine-learning-agents), we would like to teach an agent to play the games developed with the FrameIT method.
Possible approaches may be:
- reinforcement learning via reward function
- imitation learning
- modularizing tasks based on the MMT knowledge formalization
Note that there is no groundwork for this topic, so a strong background in machine learning is required.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/33FrameIT Virtual Reality Port2023-01-20T22:14:44ZRichard MarcusFrameIT Virtual Reality PortIn the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Our current frameworks aims at supportin...In the FrameIT project (#1), we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
Our current frameworks aims at supporting the development of serious games.
These profit from virtual reality by having intuitive forms of interaction and a more immersive learning experience.
For interaction, we use different *gadgets* and user interfaces, which need to be adapted for use in VR.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/32FrameIT Scroll Rendering2022-02-11T10:05:12ZRichard MarcusFrameIT Scroll RenderingIn the FrameIT project, we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
We visualize problems or puzzles in the form o...In the FrameIT project, we use the Unity game engine in combination with MMT to have access to mathematical knowledge management techniques and interact with the knowledge in the game world.
We visualize problems or puzzles in the form of interactive *scrolls*, where players can input their gathered knowledge to obtain more information and, eventually, solve the problem.
While currently the scroll is visualized as a combination of problem description and a list of input fields, we would prefer a more organic layout and typesetting where you can fill in the blankets directly in the description text.
The biggest technical issue is the rendering of HTML5 and MathML content outside of web browsers, while at the same time allowing interactions between individual elements of the scroll and their counterparts in the game.
Searching for solutions and possibly comparing different ones will be part of this topic.
For more information, please take a look at the [project issue](https://github.com/UFrameIT/UFrameIT/issues/18) and #1.Michael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/3FrameIT for mathematical models.2020-04-27T14:44:15ZMichael Kohlhasemichael.kohlhase@fau.deFrameIT for mathematical models.Mathematical models (e.g. differential equations, ...) are used in many branches of social and natural sciences. There is talk (e.g. by [Karsten Tabelow](http://www.wias-berlin.de/people/tabelow)) of [building collections of math models...Mathematical models (e.g. differential equations, ...) are used in many branches of social and natural sciences. There is talk (e.g. by [Karsten Tabelow](http://www.wias-berlin.de/people/tabelow)) of [building collections of math models as a resource](
http://www.wias-berlin.de/publications/wias-publ/run.jsp?template=abstract&type=Preprint&year=2016&number=2267). I believe that the [FrameIT method](https://www.cicm-conference.org/2016/ceur-ws/W50.pdf) (see also #1) developed at KWARC is exactly the right framework for reprsesenting and applying such models, only that we do not need the game part. The main difference is that we want to explore the real world, not the game world; a main contribution of this thesis might be to design that kind of interaction interface that allows this. Maybe even a "search engine" that suggests math models given a "situation theory".
This application needs to be developed further, and may become a project proposal if successful.
cc: @dmuellerMichael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.dehttps://gl.kwarc.info/kwarc/thesis-projects/-/issues/1FrameIT Game Content (Developing a Serious Game for Maths): B.Sc. or M.Sc thesis2022-10-14T15:55:54ZMichael Kohlhasemichael.kohlhase@fau.deFrameIT Game Content (Developing a Serious Game for Maths): B.Sc. or M.Sc thesisWe have developed a new, logic based framework for serious games (see [this paper](https://www.cicm-conference.org/2016/ceur-ws/W50.pdf)), which combines a logic-based treatment of the underlying knowledge in theory graphs with game el...We have developed a new, logic based framework for serious games (see [this paper](https://www.cicm-conference.org/2016/ceur-ws/W50.pdf)), which combines a logic-based treatment of the underlying knowledge in theory graphs with game elements in unreal engine. The aim is to develop serious games (games where the subjects learn somthing while they are having fun) for mathematics.
The current state is that we have developed a proof-of-concept system that can run exactly one game problem. While this is quite an achievement, we need much more for a "serious game". Some of this should be developed in this thesis (actually, this probably needs multiple thesis): Extensions include
* more problem formalizations in MMT ("shadow scaling method", more trig, triangulation, ... )
* more gadgets (3d models) in unreal engine, e.g. laser pointer for marking points, tape measure for lengths, laser angle finder, level pointer, ... )
* more problem situations (game plans), e.g. tree height on non-horizontal ground.
* automatic scroll generation from sTeX/iMMT formalizations
* integration of Semantic Alliance Technology
* didactically informed storyboards for a simple game (where do problems/scrolls come from, levels, boss-fights at the end, ...)
* argumentation interface for "proving" preconditions of methods.
**There has been a big update, see https://uframeit.github.io/!**
Supervisors @mkohlhase, @dmuellerMichael Kohlhasemichael.kohlhase@fau.deMichael Kohlhasemichael.kohlhase@fau.de