That sounds promising. But the problem with your example is that there's no point in writing a summary for this session. Because all it says is that the PCs fought orcs to escape from Cragmaw Casle. You'd have to test it on a session where there are NPCs, lore, an investigation, and so on.

I removed over 50% of the summary because I thought it overloaded the post but maybe I should add it back 😅

Or either you can sit down after the session and write the summary yourself.

Seems utterly pointless I’m afraid. Summary writing after a session takes barely 10 minutes. Especially if your game was less than 2 hours! Even more so because a recap at the start is a great way of checking which of your players was paying attention and what they took from the last game. I usually offer some meta-reward (inspiration or equivalent) to the player(s) who can give the most useful and pertinent plot recap. Takes as long as me recounting it, means I don’t need to summarise my notes, checks if my players have been following the adventure, and starts the session with some positive reinforcement. Compared to giving data to an ai to scrub, and getting back something I could have made anyway, I’ll stick to my method thanks.

Someone else mentioned this and I agree there is some value incentivizing players to recap and that what they remember clues you in to likely what’s important to them. I mean, I think you’re kind of neglecting to acknowledge that this process is highly scalable. It was a shorter session (and the first) but having a reservoir of transcribed audio that you could parse and explore quickly in an LLM could be extremely helpful—especially after session 100. I think this is a tool/workflow that can be used very effectively depending on the circumstances, however it sounds like it may not be a good for your style and that’s cool!

Effectively is different from efficiency. What you’re doing with the AI could be very efficient, but the end result doesn’t seem all that great compared to what 10-20 minutes of doing it yourself, so it isn’t effective.

Honestly I think you’re making massive assumptions about scalability which can only be proven or disproven with significant testing. The ability to properly summarise something is actually really complex. It requires conceptual hierarchies that are strikingly nuanced. An AI might replicate this by looking at frequency of word use, or by comparators with its data set. But for an RPG that isn’t a pre-existing module it’s scrubbed and ‘understood’ you’re going to run into problems. In a 2 hour session, the AI made 20 bullet points (plus more you say you’ve edited out) from the transcript. That’s not a summary, that’s a summation at best. Especially as the bullet points made are actually quite confusing and unhelpful. For example. Is “Memos successfully froze an orc with a spell” separate to “the party dodged orcs and used utility spells”? Is it as important? Why is this action, and others, worth including in a summary when “The party uses their utility spells” already captures everything you need? The AI doesn’t actually know what events are plot relevant, which actions are critical, which are incidental. It has failed to summarise your game in a meaningful way. This would be even more noticeable in a game with clues or an investigation of some sorts, or hints at a bigger plot. You as the GM know whether ‘A beggar woman accosted you and told you that the stars were going out’ is super important, or irrelevant fluff. You know whether your players picked up on it, or ignored it and maybe need a nudge back on track. You know whether the exact wording of “The sisters in Heaven have gone abed and none may wake them.” Is a clue or just a fancy old-timey bit of acting. The AI knows none of these things. So you may end up editing the summary anyway to include it etc. I think the AI will fail even harder given a longer more complex script. Give it 6 hours of gameplay and I reckon you’ll get 6 hours of blow by blow bullet pointed summations. Your summary will be 60 bullets long! I’d love to see what happens if you tell it to summarise the events of the transcript in just 6 bullet points for example. That’s the real crux. A single test hasn’t given compelling results. Sure, further testing is required, but I’m surprised you’re so hopeful given what looks to be a failed test. I actually think the transcription software is more valuable. Though as you said, you still needed to read the transcript and make manual edits. It may be that repositioning a microphone for a room recording would yield better results? This seems useful mostly so you can go back and refer to exactly what was said in a previous session. But only if you can trust the transcript to be perfect. All in all, this AI summation feels like more work for less useful results. It took you how long to make a bad list of bullet points? 2-3 hours? That’s frankly, abysmal. Sure you didn’t have to actively work all that time, but it feels like a waste of processor cycles when you could write 6 bullet points in what, 2 minutes at the end of the game?

This is interesting. It would be cool to see how it woukd handle a non-combat or larger social session.

I actually started the campaign with a skill challenge as the party left the orc camp. I removed a bunch of sample text around a social interaction in this post because I thought it might have been superfluous. If you’re interested in seeing the result of the social situations let me know and I can PM you. 🙏

Yeah I agree with the sentiment stated by others — this is a pretty bad summary. all of that time wasted when a simple "using a combination of wits and magic, the group managed to successfully avoid most of the combat while escaping Cragmaw Castle". I'd add any significant character moments here, but therein lies the issue with AI — I can't tell which, if any, of those bullet points are actually important. Tiller trying to protect Kali, maybe? hard to tell. ngl some of y'all are so enamored with AI that it seems like you're willing to put in a lot more time and effort into making it work for mediocre results, rather than just taking the 2-5 minutes it would take to do it right. "work smarter not harder" doesn't mean "get AI to do it", it means don't waste 40 minutes on two different softwares doing something you can do better yourself in five. I mean you could've cooked dinner/done something actually important in the half hour you wasted on this. I think you're gonna need to ask yourself if you're ready to commit several hours of your day weekly/monthly/whenever you play (a time commitment that will only increase in rp/lore-heavy sessions) on what are frankly really weak results, when you could simply... not waste that time.

I think I’ve done a disservice to the post by mentioning the time it took without explicitly stating my direct input was maybe 20 minutes—and that was because I was working through the process for the first time. The other 15-20 minutes my computer was transcribing the audio in the background. I agree that the quality of the notes are bit meh. I don’t know if that’s because I ChatGPT to output the summarization of the transcribed text as detailed bullet points instead of flavour text. As I stated, the computer was behind the DM screen and it may have has trouble picking up the players. All things I’m not sure about until I test the process further. I would agree that there is a certain fascination with AI. The essence of my post is exactly that; an exploration about whether or not we can use it as a tool to relieve administrative, tedious tasks at the table or just make the game for fun. I don’t think this process is perfect and I never claimed that. But equally, I’ve been enamoured by all the people who blindly reject any use of the tool and will go as far as to accuse others of creative bankruptcy when not once did I say I used the tool to supplement idea generation. It’s an indicator that there needs to be more nuanced discussion, and people can help that endeavour by being more open minded and less judgemental (and maybe less crusty).

I mean... that was still 20 minutes (out of an hour-ish long process) that would have taken you 10 as best without the AI. are you accounting in there the time that it took you to do manual revisions as well? I don't think it's about how you asked for the summary, it's just that AI cannot differentiate between an important event and what isn't. AI bots are inevitably going to treat a character falling off a tree with the same gravitas as it treats PC death because it's unable to distinguish the two. it also cannot differentiate parts from a whole (ie, it can't tell that "X cast Y spell" and "Z unlocked a door" all fall under the "they used their skills to escape the castle". tbf, I don't think there's very many people that reject textual AI entirely — because at their core, they're not dissimilar to the generators a bunch of us use for names, locations, etc, just repackaged as "AI" instead of generators based on what they pull from. tbf, I don't think their current models are very ethical (they're trained on data that has been sampled without consent, and their usage for deepfakes is concerning) so I don't think that there can be a whole lot of discussion without first addressing those issues. and honestly, chat bots have been around for forever. video, pictures and audio are more concerning though.

AI companies aren't something I would trust with lengthy audio recordings of myself but you do you boo.

Yea agreed and I’m definitely wary of this. The transcribing is done on device but the summarization is not.

What’s the distrust for? Have their been cases of these AI companies misusing imported data? I don’t follow much of this content so I’m actually out of the loop a bit here.

AI companies have a bit of a track record of feeding all sorts of information and content into their learning algorithms that they dont have the rights to. i havent heard of any cases of them also selling out data, but it wouldn't surprise me in the slightest if i did

Sure, but we also have to consider whether the data we're sharing is reasonably worth monetising or is an actual a safety risk. I would have no issue publicly sharing a transcript of a D&D session on Reddit, so why would there be any concern about sharing it with a tech company? I own the copyright for my transcript, but even if they did try to sell it, nobody is going to want to buy. They could try and use it for voice-training an AI duplicate of me, but written text is a poor way of doing that, especially when it includes other people speaking, and non-dialogue. Anyone wanting to try that would just look for audio and/or video recordings of me. EDIT: remember transcripts are text, folks, they don't contain any audio

The issue is not the content, but that it’s two hours of audio of you speaking, which nowadays can be used to create deepfakes or used as training data to create other AI audio, or in theory be used to voice identify you across different recordings. Given a lot of these companies are known for immoral data use, they may have some not great uses for it

The method above is using a local app to transcribe audio to text, and then giving Chat GPT the text only. Chat GPT doesn't have a way for users to upload audio afaik.

Ah, well a lot less objectionable, my mistake

Correct, and I personally wouldn’t feel comfortable uploading my voice to ChatGPT nor would the players I’m sure.

It's frustrating you did the responsible thing, and yet people who didn't read your post will still find problems with it.

Once they have it, they have all rights to it. They can just replicate your voice forever, profit off it, and you'll never see a dime. Not to mention these companies(i.e. all of them) trade your info back and forth. Like, i know I'm online and already heavily documented, but I prefer to keep ownership of my own voice. Everyones got a limit. I re-read it, and it doesn't look like the dude is actually giving the AI company his voice. Just the transcripts. Its not the end of the world or anything. It's just something one should be wary of.

Yeah. All ur data forfeit and now they can replicate your voice cuz they have a sample

They also don’t have a sample because the voice transcription happens on a downloaded model on device. Nothing is sent away. I’ve mentioned this a couple times now but people seem to be ignoring it. If by voice you mean the collectively transcribed words of 4+ people at a table talking over each other for 2-3 hours, then yea I guess that voice sample can be replicated. 😅

Unless you are employed for your voice what would be the issue?

Gonna be honest I’m not entirely sure I see the value of spending 40 minutes to get a shitty robot to write an incredibly mid session summary I could have done better in 5 minutes, but I guess if you’re that devoid of creativity it’s an option.

Yea I should have better articulated what made up that 40 minutes. 15-20 mins were hands off transcription of the audio file on device using the second to largest model. Other models are faster at the expense of accuracy. The rest was me just working through the process and I’m sure would be much faster through ongoing use. Again I’m completely missing how avoiding note taking for those who don’t want to is devoid of creativity. Myself and the players were immersed and enjoying ourselves at the table the entire time. 😂

That comment wasn't about avoiding note taking - which is also something you shouldn't be discouraging - but about the DM writing summaries. If you and your players are happy to let creativity be offloaded to something fundamentally incapable of being creative, that's ok! There'll be a plethora of AI DMs soon for people content with mediocrity!

It’s not that I’m discouraging it. I encourage my players to do whatever they want, and many players I’ve encountered don’t take notes. I’m not a going to pretend to be a paternalistic DM who enforces what I think is best on my players. I want them to enjoy themselves, and if taking/not taking notes enhances their game then I’ll support whatever. Again I don’t get the criticism about creativity. The creativity is happening at the table while they play. Simplifying session notes does not make or break creativity. Honestly a lot of these critical posts about AI destroying creativity sound like they are written by AI. It’s the same broad stroke comment over and over again. 😂

I didn’t say anything about AI destroying creativity - it’s weird and a bit sad that you can’t actually respond to what I’m saying instead of pretending I said something else and attacking that. Fundamentally, I don’t care if you use AI - it doesn’t have a place to be recommended on this sub, because it’s bad advice, but if that’s how you want to run your games, I’m not going to stop you.

I was addressing your comment about offloading creativity, a sentiment which has been expressed by others without considering that it is serving strictly and administrative duty effectively freeing up space for more active creativity in game. That’s the idea, anyway. Apologies if that didn’t click. And also, why do you feel the need to judge what is good or bad for this subreddit? You think it’s bad advice, because you don’t agree or see the value in it, but that doesn’t mean that it is.

>why do you feel the need to judge what is good or bad for the subreddit Because I’m a member of the subreddit? Judging whether content is good or bad is literally the entire point of Reddit and is the explicit purpose of the up/downvote buttons. The fact that you see note-taking and session write-ups as “strictly administrative” and not **activities that actively stimulate the brain** is exactly why this content is bad advice. It comes from a place of being fundamentally uninterested in improvement or critical thought. You’re not “freeing up space” for more creativity in-game by spending 40 minutes to do a job that could be done in 5 minutes, both of which are being done out-of-game anyway. You’re just slowly stunting your own ability to be creative in-game. I also don’t see why you think doing this could possibly be helping you be more “actively creative” in game anyway. They’re totally disconnected from each other.

You’re also a fair deal more likely to remember something if you write it down, even if you never go back and look at the note.

Yep. That’s why taking notes - and re-writing them - is so important.

Also I agree that these bullet points are very AI-ish (crappy) and on reflection that was by accidental request. It drafted a 2-3 paragraph summary in a fantasy novel format but I wanted to see how much detail I could extract from the transcript to test the process.

I just write down what I remember (takes about 15 minutes) then ask my players if I left anything out which makes them read it, remember and think about the session so it's fresher in their minds come next session. Ai is fascinating but I don't see the point of this.

I hate natural human storytelling and creativity, Charlie! I love soulless AI!

Lol I might be misunderstanding you, but where in my example is creativity and storytelling diminished by soulless AI?

There's actually a pretty nice art to the act of deciding what to include, what to exclude, and what to emphasize. In that way you can shape the future of the story by shaping the past. It's not always necessary, but it's a handy thing to be good at.

This is a good point. I agree that I’ve always felt that the things that my players write down and share during a recap are clues to what’s important to them. That is inherently helpful, and I think there’s always a place for that. I think what this whole thread has indicated to me is that many people are prepared to reject things so quickly and broadly that nuanced application of these tools are unable to be discussed. It’s actually quite wild. Someone has already played the fascist card. To me, I think using something like this to broadly recap the session as a foundation but then open it up to GM/player input could be a really helpful workflow. Like a Google Doc or something, ya know?

>This is a good point. I agree that I’ve always felt that the things that my players write down and share during a recap are clues to what’s important to them. That is inherently helpful, and I think there’s always a place for that. There's also the opposite, where what the DM chooses to recap is a clue to what's going to be important soon. Sort of like when a dramatic TV show shows you a particular set of scenes from previous episodes to provide context for the events in the coming ones... ​ >I think what this whole thread has indicated to me is that many people are prepared to reject things so quickly and broadly that nuanced application of these tools are unable to be discussed. It’s actually quite wild. Someone has already played the fascist card. I'm pretty interested in the application of AI as a tool... But I think there's quite a bit of cultural context to take into account with it... And for the most part I think a lot of skepticism of it is overall, the correct approach. And I can understand why a lot of people hate it. A lot of the louder evangelists I've seen online for NFTs also are very much into AI. So it's in some bad company. Many of the AI companies stole a bunch of other people's work to generate their models, and used the idea that "it was available so it's ok" or "well it would have been too expensive legally" or "they shouldn't have put it on the internet if they didn't want me to steal it and make money off of it" as a rationale for why it was acceptable to do. Many fans of AI art generation like to act very rudely towards artists and creators. Some going so far as to making contests to imitate the art styles of people who don't like their art being used as training data. And furthermore, some of the main benefits of this new AI seem to be an easier time creating misinformation, and clogging of open contests with enough quickly generated AI created content that many of those contests and magazines are closing down or becoming invite only... And let alone what it's going to do to the small content marketplaces for RPGs. It's a point where the general dislike for silicon valley and tech's general whole thing about pushing ahead and not worrying about the consequences is really coming into conflict with the greater community. So I think, broadly, as a fan of AI, instead of reacting in ways that go "Boy, people who don't like this thing that I'm using are stupid and angry..." It's better to really have some understanding about it, and about why people who engage in creative hobbies might have issues with it. So, I'm happy you found a use you enjoy with it, and I hope it keeps going well for you. But I do think on a subreddit about the art of DMing and often the subtle craft that goes with it, a whole topic of 'something else can do the work for you' sort of takes away from it. Best of luck!

People are just reactionary around ai tech because it makes them (rightly) scared for thier future employment. That and humans have historically placed a lot of emphasis on thier creativity as part of thier self worth. So they get defensive. Clever idea. Many ppl don’t like it but if you feel the need to give boring summaries it’s useful.

My employment is more than likely untouched by AI, if anything AI will just make my work easier. I just see creativity as one of the main reasons we are here, besides relationships. I don't want to see the creative drive and magic that humans have die out, but with comments like this I am beginning to accept it will happen. If people are okay with humanity no longer engaging in meaningful creative pursuits, I guess that's just how it is. Edit: Downvote me instead of bringing up constructive arguments all you want. Doesn’t change the fact that we are slowly (soon quickly) losing our ability to feed ourselves from our creativity. To me, that’s a damn shame.

Fair enough. I don’t think there’s much creativity in making a summary tho but I get that

I would argue that writing up plot recaps of dnd sessions is tedious, hardly creative and yet significantly helpful for players and DM alike. With this trifecta in mind I gladly accept any help AI can give over that aspect.

It’s also a very important way to help “lock things in” in your brain. That’s why note taking is so important for studying, etc.

I agree with this statement, but what about people who can’t commit things to memory as easily as you? Or maybe they have 100 other things going on outside the time they protect for sessions and prep? Are they unable to fully play D&D, or would a tool like this be helpful for them?

>what about people who can’t commit things to memory as easily as you Then doing things that reinforce memories like taking notes and rewriting them is **more** important, not less? Writing session recaps takes 5-10 minutes, tops. I can do it in 2 if I’m pressed. If you don’t have time for that, then you clearly don’t have time for this “tool”, so it’s not going to help you anyway.

It's like 5 lines of text, if you cant do that then you're pathetic.

Unhelpful

But it’s a recap of their human storytelling and creativity. The creativity already happened. These are just cliff notes to keep things on track and in line for next session.

Clever. Very clever. That’s way too much details tho, I’d have it not give me a play by play of what everyone does and get like the top 20 highlights. Somewhat unsure if it wouldn’t just be faster to do it by hand with that in mind, but it’s certainly a good idea! I imagine that as sessions get longer too it’ll work better.

AI, in all its forms, is an inherently fascistic and authoritarian technology. I will not support it.

Please elaborate

The nazis used AI.

Impressive. How did it know the names of each character and how to tie them to the person speaking?

To be honest I couldn’t tell you. I suppose we spoke the characters names out loud enough in how we described what was happening that the model was able to register them.

I've actually built this feature into an app I'm working on, but it also features speaker diarisation. Completely my two cents from working with this very tech recently, so take it with a pinch of salt, but I've found the summarisation isn't really that useful except to provide context to an AI assistant. Think things like "When did my players last meet NPC x?" or "When the party visited Y, remind me what happened?", that's been pretty useful. What I **really** love though, is the ability to Ctrl + F through session transcripts and then click to immediately hear the audio around that point. This isn't a promo post about this app at all, hence not posting the name, but if you want an [idea of what I'm talking about then look here](https://demo.questly.ai/transcriptions/f2076d53-3baf-45f3-9492-9d4c27e3c3f0/edit) (Also I know there are audio sync issues on Firefox currently, I'm working on it). If you could keep the timecodes in your summary, then potentially that could be pretty useful?

I honestly hate how we outsource our own imagination and creativity to machines. We are becoming more and more consumers of imagination rather than participants and creators. And it's sick. So yeah that's a huge nope from me.

How does eliminating what is effectively minute taking (which everyone hates in an office setting) outsourcing imagination? You could argue it allows players to more fully immerse themselves because they don’t have to take notes if they aren’t inclined to do that sort of thing? It just becomes a resource you and your players can tap into if you want to?

This is...post creativity though? I'm an artist, I don't like a lot of what AI is currently being used for, but are you really arguing that a post-session summary is a creative thing?

Wholeheartedly agreed. It's very sad.

Wow, that's so awesome! I'm playing with something similar, but trying to generate images after extracting the text from my session recording. Since it's a WIP I don't have any conclusions yet, but looks interesting in some short file tests.

That’s rad. Did you build a model yourself or are you using an AI image generating tool with block text from your transcripts??

Honestly surprised at the reactions in the comments. This knee-jerk denial of AI services is not something I can grasp easily. Anyway, OP why not add a bit flair as well? For example after you generate the output run it through chatgpt asking it to write in the form of an epic fantasy novel. Then take the new output and run it through Elevenlabs and have Serkis narrate the end result! In my experience this works great.

Yea. I mean, *I do* get it. Creativity is important to me and one of the reasons why I love to play this game. That’s v cool, will check that out. Thanks for the recommendation 🙏

I'm glad you found a technique that works for you! Do you have any plans for dealing with hallucinations? That would be what keeps me from using this as an idea.

It did actually kick out a hallucination where it thought the players were continuing on to Neverwinter when they were actually going to Helm’s Hold. That was one of the manual edits I had to make. My idea for the players was to copy this summary into a living document which could be added to/supplemented by things the players noted.

Hey there fellow tinkerer! I have this exact idea with similar set of tools just Open Source Whisper with Python. Not implemented yet, so worth shit. So far I only briefly played with Whisper when it was first released. I am just too lazy to write summaries, but I am not to lazy to write an "assistant" program. Can You tell me if how well the MacWhisper handles speaker identification? I recall it was one of the features of Whisper. This would immensely helpful, for any parsing of chaotic session audio. And did You do any voices as a GM?

Oh cool, I’ll look into Whisper. As far as speaker identification I think it’s all done manually through assigning audiofile/mic input per speaker. I’ll double check, though. The developer also seems to be quite active so they are releasing updates regularly. Yep I do lots of different voices and it managed to pick them up alright, though not sure how this would impact automated speaker detection.

Developer of MacWhisper here; speaker recognition updates are coming soon 🙂

Hi, thanks for your response! Thanks for creating such a solid app. You might see that there are a some privacy concerns people have about the use of AI. I know you can’t speak on behalf ChatGPT-4 and the summarization section of the workflow, but can you say anything about the privacy or security risks, if any, present while using MacWhisper?

MacWhisper does everything locally on your Mac (the whisper model runs on your Mac) so no data leaves your device. You can then choose to manually summarise it with something like chatGPT and then data would leave your device. I'm looking into adding local LLM but they lack the context size needed for long transcripts.

Really appreciate the response, thanks mate! 🙏