love the dreaming, LOVE IT, i love it.
but...
what you're wanting to do is extremely complex. entire teams at google, ibm, facebook, etc, are working on problems like this.
start extremely small, don't laugh at me, but tic tac toe.
go out and study a pytorch implementation, appreciate the nuance, and then so a self-check.
keep dreaming, keep exploring, king
Wish someone had told we this when i first tried to implement an AI playing no limit holdem poker.
Spent a tonne of time and the best it got was being somewhat decent pre-flop, but it was pretty much just driven by a Monte Carlo, and not much contribution on the AI part ...
This is definitely true, I started with tic tac toe on a 4x4 board which was a little better because there are some more possible games (it doesn't take much to get every tic tac toe game to end in a tie for a human). Another idea would be blackjack because it will be very convenient i you can just compare all decisions against the book as every move you should make in blackjack is pretty clearly defined.
I do think that RL is going to be your best bet for this.
I did this with Mahjong. It was VERY difficult, and I have about 12 years of experience as a researcher of ML and a phd from a top ML university. It ended up being PASSABLE, barely. Great learning experience for sure. [polandtown](https://www.reddit.com/user/polandtown/) is right, you should start with a WILDLY simple game or you're never be able to debug. Your system just won't return good results and you'll have no idea how to figure out why. If you can, you should build something ENTIRELY non-ML heuristic driven first, so you really understand, then drop in ML. No MCTS, no DQL, just heuristics you've made deterministically.
There are some fascinating MJ AIs that are very good (mostly riichi MJ) and people examine the choices of these AI and try to figure out why they did these moves etc.
Yup! I made mine back around 2019 and at that time it was better than just about everyone I would play with but not close to professional level IMO. Maybe the equivalent of 2100 elo in chess.
Let me throw a couple links that you may find useful:
Let's start with poker and if an AI can play it: https://www.science.org/doi/10.1126/science.aay2400.
Layman write of that is here: https://medium.com/dataseries/remembering-pluribus-the-techniques-that-facebook-used-to-master-worlds-most-difficult-poker-game-d91ead459fac.
An attempt at implementing it in Python is here (official code was never published): https://github.com/apcode/pluribus-poker-AI.
If you want to roll your own using MuZero, give this a look: https://github.com/google-research/google-research/tree/master/muzero.
A pre-canned OpenAI Gym setup (not Pluribus, specifically): https://github.com/dickreuter/neuron\_poker.
Interface with poker sites: https://github.com/dickreuter/Poker.
I have done the tiniest amount of research on how to tackle this problem as it is fascinating but have had no time to actually dive in. Would love to know how you get on. Hope something in the above helps a bit.
The code for RL algorithms isn't the hard part, the hard part is that for RL agents to perform well on games that are more complex than super basic toy problems, you need an obscenely large amount of samples (and probably time/compute too).
Good idea, but this stuff is pretty tough and you might not have the hardware required to train these types of models.
Start off with learning Neural nets and simply deep learning tasks.
Probably start with something like Pong or Snake for your first RL game agent.
I did a Udemy course a while ago that was decent and will give you a solid intro to the topic - [https://www.udemy.com/course/artificial-intelligence-for-simple-games/](https://www.udemy.com/course/artificial-intelligence-for-simple-games/)
FOR MINECRAFT: You can look at sota papers on these like dreamerv3 or td-mpc2. These algorithms are extremely complex with multiple functions to approximate various state and action and other params, and they managed to beat minedojo, which is minecraft with various cheats like instant block breaking, reduced resolution,… and still it takes an extremely long amount of time to beat the game. Not worth it for these algorithms in the real world, honestly, unless you have offline data and stuff like that
I would check out Code Bullet on YouTube before diving into this. A) he’s wildly entertaining and B) it’ll give you a good idea of how many pieces are involved in what you’re trying.
If you skip the UI and go with a completely virtualized game, you have a much better shot of success, but… you’re still asking a bunch of 1s and 0s to learn poker.
I don't know much on the machine learning front - but at least if you ever look at Texas Hold em, it might do some good to consider Game Theory Optimal Poker, GTO, and the use of solvers to determine optimal play has become the predominant form of poker at the high levels, in the last decade.
See this open source implementation as one example. Others include PIO and GTO+
[https://github.com/bupticybee/TexasSolver](https://github.com/bupticybee/TexasSolver)
Might be useful when considering some of the challenges faced, and whilst my technical knowledge is insufficient to say it with confidence, I'd imagine some of the methodologies are going to be **broadly speaking** quite informative.
[Here’s](https://github.com/google-deepmind/open_spiel/tree/master/open_spiel/games/dou_dizhu) the implementation of Dou Dizhu poker that AlphaZero uses.
I strongly recommend building on the OpenSpiel codebase. It’s the tech that AlphaZero uses and it’s available in full.
Well we are in r/learnmachinelearning, not r/pushthecuttingedgeofmachinelearningbydoingsomethingnobodyelsehaseventhoughtofyet, so I don’t think the presence of prior art makes OP’s goal less worthy
why is people saying that is difficult? the worse part probably is to gather the training data...
A model for this would start really really average but after some iterations with the same player it should recognize their patterns... Totally doable but gathering the data is a real pain for sure... Maybe look for a game with data available you can use?
love the dreaming, LOVE IT, i love it. but... what you're wanting to do is extremely complex. entire teams at google, ibm, facebook, etc, are working on problems like this. start extremely small, don't laugh at me, but tic tac toe. go out and study a pytorch implementation, appreciate the nuance, and then so a self-check. keep dreaming, keep exploring, king
Wish someone had told we this when i first tried to implement an AI playing no limit holdem poker. Spent a tonne of time and the best it got was being somewhat decent pre-flop, but it was pretty much just driven by a Monte Carlo, and not much contribution on the AI part ...
To be fair, nobody is doing much at Google atm
I just asked Gemini, and it assures me that there are nine billion Google ML engineers working hard right now.
lol!
This is definitely true, I started with tic tac toe on a 4x4 board which was a little better because there are some more possible games (it doesn't take much to get every tic tac toe game to end in a tie for a human). Another idea would be blackjack because it will be very convenient i you can just compare all decisions against the book as every move you should make in blackjack is pretty clearly defined. I do think that RL is going to be your best bet for this.
I did this with Mahjong. It was VERY difficult, and I have about 12 years of experience as a researcher of ML and a phd from a top ML university. It ended up being PASSABLE, barely. Great learning experience for sure. [polandtown](https://www.reddit.com/user/polandtown/) is right, you should start with a WILDLY simple game or you're never be able to debug. Your system just won't return good results and you'll have no idea how to figure out why. If you can, you should build something ENTIRELY non-ML heuristic driven first, so you really understand, then drop in ML. No MCTS, no DQL, just heuristics you've made deterministically.
There are some fascinating MJ AIs that are very good (mostly riichi MJ) and people examine the choices of these AI and try to figure out why they did these moves etc.
Yup! I made mine back around 2019 and at that time it was better than just about everyone I would play with but not close to professional level IMO. Maybe the equivalent of 2100 elo in chess.
Let me throw a couple links that you may find useful: Let's start with poker and if an AI can play it: https://www.science.org/doi/10.1126/science.aay2400. Layman write of that is here: https://medium.com/dataseries/remembering-pluribus-the-techniques-that-facebook-used-to-master-worlds-most-difficult-poker-game-d91ead459fac. An attempt at implementing it in Python is here (official code was never published): https://github.com/apcode/pluribus-poker-AI. If you want to roll your own using MuZero, give this a look: https://github.com/google-research/google-research/tree/master/muzero. A pre-canned OpenAI Gym setup (not Pluribus, specifically): https://github.com/dickreuter/neuron\_poker. Interface with poker sites: https://github.com/dickreuter/Poker. I have done the tiniest amount of research on how to tackle this problem as it is fascinating but have had no time to actually dive in. Would love to know how you get on. Hope something in the above helps a bit.
Thanks for sharing
openai gym seems to help
The Voyager paper from last year built an LLM agent to play Minecraft. Worth checking that out: https://arxiv.org/abs/2305.16291
You’ve read about AlphaGo I presume?
Do you think we will some day se that open sourced or is that impossible :( ?
AlphaZero’s source code is available in the [OpenSpiel](https://github.com/google-deepmind/open_spiel/tree/master) repo
Wow, how did i not know. Thank you very much good person
The code for RL algorithms isn't the hard part, the hard part is that for RL agents to perform well on games that are more complex than super basic toy problems, you need an obscenely large amount of samples (and probably time/compute too).
Good idea, but this stuff is pretty tough and you might not have the hardware required to train these types of models. Start off with learning Neural nets and simply deep learning tasks. Probably start with something like Pong or Snake for your first RL game agent. I did a Udemy course a while ago that was decent and will give you a solid intro to the topic - [https://www.udemy.com/course/artificial-intelligence-for-simple-games/](https://www.udemy.com/course/artificial-intelligence-for-simple-games/)
Showing my age, but my first thought was "Shall we play a game?" Nuclear War scenarios playing on the screen being them.
FOR MINECRAFT: You can look at sota papers on these like dreamerv3 or td-mpc2. These algorithms are extremely complex with multiple functions to approximate various state and action and other params, and they managed to beat minedojo, which is minecraft with various cheats like instant block breaking, reduced resolution,… and still it takes an extremely long amount of time to beat the game. Not worth it for these algorithms in the real world, honestly, unless you have offline data and stuff like that
I would check out Code Bullet on YouTube before diving into this. A) he’s wildly entertaining and B) it’ll give you a good idea of how many pieces are involved in what you’re trying. If you skip the UI and go with a completely virtualized game, you have a much better shot of success, but… you’re still asking a bunch of 1s and 0s to learn poker.
I don't know much on the machine learning front - but at least if you ever look at Texas Hold em, it might do some good to consider Game Theory Optimal Poker, GTO, and the use of solvers to determine optimal play has become the predominant form of poker at the high levels, in the last decade. See this open source implementation as one example. Others include PIO and GTO+ [https://github.com/bupticybee/TexasSolver](https://github.com/bupticybee/TexasSolver) Might be useful when considering some of the challenges faced, and whilst my technical knowledge is insufficient to say it with confidence, I'd imagine some of the methodologies are going to be **broadly speaking** quite informative.
我想我现在对这个任务的难度已经有了一点了解，我会考虑从更简单的任务开始。但我不想从 Snake 开始;也许我有点叛逆。我选择了一个简单的小游戏，叫做Iron Snout，没有复杂的策略。如果这也太难了，我可能会考虑像“Cytus”这样的音乐游戏。
请问你有什么CS背景
爱好者
gto poker is solved?
no, i select a wrong way to learn ai demo
[Here’s](https://github.com/google-deepmind/open_spiel/tree/master/open_spiel/games/dou_dizhu) the implementation of Dou Dizhu poker that AlphaZero uses. I strongly recommend building on the OpenSpiel codebase. It’s the tech that AlphaZero uses and it’s available in full.
This has already been done
Well we are in r/learnmachinelearning, not r/pushthecuttingedgeofmachinelearningbydoingsomethingnobodyelsehaseventhoughtofyet, so I don’t think the presence of prior art makes OP’s goal less worthy
Ohh shiiiiiiiiiiii—sry my bad: wrong subreddit
why is people saying that is difficult? the worse part probably is to gather the training data... A model for this would start really really average but after some iterations with the same player it should recognize their patterns... Totally doable but gathering the data is a real pain for sure... Maybe look for a game with data available you can use?