### Purpose While we have a functional AI there are still lots of improvements to be made. Our current set up requires a lot of manual effort and the underlying conceptual map works like a waterfall rather than a web. The last student interview also was incredibly frustrating for the user and the AI didn't work at all despite correct answers. Our goal is to improve the AI so there are less errors and gaps in student understanding is actually uncovered. ### Principles - Robust - We actually want to sell this to customers so to a reasonable degree we'd like the AI to gauge students accurately. Given there is an element of fairness necessary as well across students we'd like to get consistent results. - Thorough - It's very clear from our interviews that gauging understanding is an important part of a teacher's job. Our AI and questions need to strive towards this purpose even if it's just for one topic. ### Brainstorming - What do I already know about this project? - We have a v1 of the AI and we understand some of the challenges around it. We've also learnt a little bit about training our own models and building a concept knowledge map. - What don’t I know that I need to find out? - We don't know the cost or effort necessary for our own models. We also don't know how much data is necessary and how to create synthetic data. - Who can I talk to who might provide insights? - Sundeep might be a good guy to get some insights from. - What can I read or listen to for relevant ideas? - Nothing in particular but it's important to read the right papers and understand how LLMs work in depth. ### Actions - [x] Create concept map and ability for AI to map explanation to concepts ✅ 2024-05-10 - [x] Set up assistants for checking correctness of explanation and creating a response based on user input ✅ 2024-05-17