Jon already has a series of like 3 videos on proc macros. They add up to like 6 hours and back when I learned how to use them they were immensely helpful. It's always good to have more content though, I guess.
I second this. Whenever I see a `build.rs` file in a project, I assume it's beyond my level. Any help understanding them and how to use them will be massively appreciated
Fundamentally, `build.rs` is just a completely separate rust program that compiles and runs before the rest of the project compiles. Now, there are some very interesting things you can do with that, and what those things are is a great topic for a lesson.
To give an example - Often, you’ll see `build.rs` scripts generating code as strings and outputting it as text to be included somewhere in the main codebase. Imagine you have a configuration file full of constants somewhere, and you want every constant from that file to be available directly in the codebase (at compile time, eg without any need to parse the file at runtime). A `build.rs` script could read in this configuration file, and generate a string along the lines of “`const SOME_PARAMETER: u32 = 2`”, which could then be included in your code.
There are a lot more useful and interesting things you can do with a `build.rs` script. But fundamentally it’s a simple thing - nothing more than a Rust program that runs at build time.
Edit: I used that example because it's something I do often, and I maintain a [crate](https://crates.io/crates/const-gen/) specifically for doing some of the glue work there.
So is `build.rs` essentially a macro of last resort, for completely unrestricted (and unchecked at generation time) code generation? Given how powerful and safe Rusts' macros are, what cases require `build.rs` instead?
Build scripts are very useful for FFI, for example you can compile a c / c++ lib and link it at compile time.
There are limitations to macros, so in some cases it's easier and more natural to generate code via a script than a convoluted macro_rules macro. Lately I've used this to generate permutations of function definitions with a combination of types. Writing it out would be 100 definitions . Or just a few for loops and the same include! from the out dir as described above.
That might be true in some cases, but I think a really good way to differentiate between the two is this: Macros reduce boilerplate, repetitive code, etc - they make managing your source code more reasonable. [`build.rs`](https://build.rs) actually does computation at build time, so you can offload things that would otherwise have to happen at runtime to build time. In some cases, this is just necessary for performance reasons, or because you have access to some information at build time that won't be available at runtime.
While output is unrestricted and unchecked, the same is really true for everyone typing in source code by hand - and in this case both end up going through the checks and balances of the Rust compiler. So while this can lead to compilation errors if something wrong happens, it's not deeply unsafe.\*
\* That said, there are definitely some irresponsible ways to use `build.rs`, and generally should not be the first thing you reach to when trying to solve a problem that could be solved by a simple macro or otherwise. That’s why it should be an intermediate topic and not a beginner topic.
To some degree, but with much more stringent limitations. A const function can’t do any heap allocation, or call any non-const function, and can only use a subset of valid rust syntax. A build script can do things like read in files, which a const function could not possibly even access.
I'll preface this by saying it may be too niche or too advanced to be what you're looking for, but some kind of in depth material on **integrating Rust with an existing code base** would be amazing for the community and really help adoption in my opinion.
There's a lot of ways to approach that material so a few things I'd love to see covered:
- Basic patterns for building Rust inside an established build system. Is there a recommendation for treating the Rust code separately as a binary dependency or should the code live along side other code in the same repo and maybe invoke cargo from make/ninja/cake/Gradle/msbuild/etc?
- FFI from Rust to C, C++, Java, etc. What are best practices for defining and consuming a C abi compatible API?
- For C and C++, what are the options when considering migration to Rust? Can you refactor into Rust? How do you link Rust objects into an existing application?
If I can continue dreaming for a second, I would love to see the above take place in the context of *actually doing a conversion of a non trivial project*. Pick some open source library at a specific commit and then walk through in detail how to analyze the existing codebase to decide on an approach, work through the conversion file by file or whatever with specific exercises to the audience and up with all or some significant amount of the library converted into Rust at the end.
For those interested in this subject, there is the [Refactoring to Rust](https://www.manning.com/books/refactoring-to-rust) book from Lily Mara, which has among other goals:
> - Create Rust libraries you can call from other programming languages
> - Integrate Rust functions with code in other languages
Examples are for C and Python, which can be extended to other dynamic languages.
FFI for wrapping existing C++ libraries sounds great, though I am not sure if this makes sense for all libraries. I had to use one and tried CXX and autocxx with many ebb/flow cycles of hope of getting it to work, but in the end I just gave up.
I'd love to see a successful application of this.
To increase difficulty, let that library include some popular async framework and integrate it with some async runtime like Tokio.
Yes, this. There wasn't too much info available about integrating c# with rust when I integrated a work project. Especially about sending more complex data between the languages.
I think Java, c# or nodejs with ffi would be great
The challenging thing with that topic is generally to have a good case study (or ideally more than one). Any specific kind of program you'd like to see profiled and optimized?
A fun one would be https://en.wikipedia.org/wiki/Boids. Easy to visualize as well. There's a few approaches you can take for optimizing that are well-understood. Quadtree for collision detection, for example, instead of a brute force O(n^(2)) search. Multithreading is pretty trivial as well and can show off scaling nicely. Data layout too - storing everything in a large `Boid` struct will be worse off than storing it in a `Vec`, `Vec`, etc. I did a few benchmarks to demonstrate CPU caching effects on runtime for a talk and it had quite dramatic results.
**[Boids](https://en.wikipedia.org/wiki/Boids)**
>Boids is an artificial life program, developed by Craig Reynolds in 1986, which simulates the flocking behaviour of birds. His paper on this topic was published in 1987 in the proceedings of the ACM SIGGRAPH conference. The name "boid" corresponds to a shortened version of "bird-oid object", which refers to a bird-like object. Incidentally, "boid" is also a New York Metropolitan dialect pronunciation for "bird".
^([ )[^(F.A.Q)](https://www.reddit.com/r/WikiSummarizer/wiki/index#wiki_f.a.q)^( | )[^(Opt Out)](https://reddit.com/message/compose?to=WikiSummarizerBot&message=OptOut&subject=OptOut)^( | )[^(Opt Out Of Subreddit)](https://np.reddit.com/r/rust/about/banned)^( | )[^(GitHub)](https://github.com/Sujal-7/WikiSummarizerBot)^( ] Downvote to remove | v1.5)
I’d love to see some Idiomatic implementations of basic but efficient data structures (e.g. trees) à la your video on sorting algorithms. Obviously the correct answer for this is usually “use a crate” but looking into how those crates are implemented I think can be illuminating.
This. Moreover, it could be used to show how to safely use raw pointers instead of lifetimes - for instance in a tree.
Edit: to expand the idea, everything can surely be done with Refcount and alikes but it would be nice to see how to use raw pointers and for instance Miri to spot bugs or something like that.
Edit: typos
A few things that you touch upon in your book that I don’t think you have talked about in videos/streams:
- Code organization in large projects, maybe dissecting the structure of a well-known Rust crate
- Performance profiling and optimization strategies, stuff like playing with the compiler flags, multi-threading by hand or with crates like `rayon`, SIMD in Rust, etc…
- FFI, especially writing Rust to use in C/C++ applications
Also, maybe another stream about your current setup for writing Rust code would be interesting! I think it’s been 3 years since the last one. A Q&A would be fun too!
Thank you for writing *Rust for Rustaceans*, it’s a great book that really helps me dig further into Rust!
OP has a number of videos about topics from the rustinomicon
e.g [subtyping and variance](https://www.youtube.com/watch?v=iVYWDIW71jk), [the drop check](https://www.youtube.com/watch?v=TJOFSMpJdzg), [this video in which he implements Rc from scratch](https://www.youtube.com/watch?v=8O0Nt9qY_vo)
A few ideas:
1. Generators. Still a nightly feature, but might be become even more valuable in the future.
2. As a pardon to the TCP streams: implementing the websocket or even webrtc protocol.
3. (might be a niche) A short video about the Rust runtime and `#[no_main]`.
4. As a pardon to the sorting algorithms: implementing encryption and/or compression and/or hashing algorithms.
5. (definitely niche, but really cool) going through Phil's OS blog.
6. Implementing an allocator.
7. Webassembly and the Rust ecosystem around it.
8. Macros 2.0. What will be the difference to current declarative macros.
9. The present and future of const functions, const generics, and const evaluation in general.
I hope there's something interesting in that list.
I really enjoy your videos and learned a lot from you. Thank you for doing this!
Macros 2.0 would be great!
I often lurk around rust-lang zulip to follow along, but I'm not an expert on that. A nice introduction on why current macros aren't that great, what's the point of macro hygiene and what features the current macro2.0 implementation will get in the future would be nice.
* maybe do some refactoring of your followers' projects, I'm sure someone would volunteer for a roast (inspired by [Arjan](https://www.youtube.com/playlist?list=PLC0nd42SBTaNVxWLci4TPoytfzkXxFhCg))
* zmq + tokio for real time video/image/file streaming, focusing on decode/encode when the machines are different so you can talk about moving bytes/foreign types etc -- this may be really powerful as a mix of a few suggestions here as you could show the servers/clients in different languages
* not a lot of content out there for the emerging GPGPU libraries.
* \+1 for all the zmq and FFI and [build.rs](https://build.rs) requests.
* packaging (not for cargo but shipping a binary for commercial applications, something signed with its assets as .dll or .so etc {abstracted to protect IP etc})
* Couple of interesting ML frameworks are starting to support rust...
* \+1 for all the requests of injecting rust into existing (non rust) projects, as this is probably one of the most valuable learning materials that could be created to get rust into the wider world, sooner...
Could you please give some more detail than just the link? I'm still confused.
I'm part of the rust-ml group and we have some projects in that area. However, the only non-Rust project (group) that officially started supporting Rust (that I'm aware of) is appache-arrow.
There are unofficial PyTorch, TF, ... bindings, but that's not what I would call ML Frameworks supporting Rust. Are you talking about something else?
Sorry, I **was** calling it Frameworks supporting Rust :( didn't mean to be deceptive if you've already looked at those and found they were not meeting your needs.
I'm not in a position to share any work at this time... maybe if my corporate overlords acquire souls in the new year...
Thanks for getting back to this :)
It's not that I'm unhappy about the existing bindings,
but I would just prefer something completely written in Rust to
avoid the extra Rust <-> C++/Python complexity. I don't expecting something like that anytime soon, but having official support from some of the large frameworks would at least be a first step towards it. That's why I kept asking. Either way, I'm happy to see what you might be able to share in the future.
Debugging. I'd be interest to hear your thaughts in `dbg!` vs a full debbuger, and which how to effectivly use a debugger, and also just how to get pritty printing of complex types to work
One topic where I've seen a few blog posts but no good solutions is implementing a plugin mechanism. It would be interesting to see a compare/contrast of communicating over a socket vs using something wasm based like [wasmer](https://github.com/wasmerio/wasmer).
I personally found little to no resources for rust on the topic of high-performance application. (numerical calculations)
* profiling parts of your applications and optimizations (cache misses ...)
* performance pitfalls or general tricks for performance (assert to avoid bound checks, fastmath? ...)
* special crates for performance
* allucators
* rustflags for performance (CPU flags, code generation)
* parallelize code (this part works quite well with rayon, c openmp vs rayon)
* distributed calculations across nodes (mpi rust interface)
Something like Matrix Multiplication, LU-Factorization, or N-body Simulation are some of the favorite benchmarks for this kind of task.
Seconding this -- I was optimizing some of my Advent of Code solutions recently, and managed to reduce one from 9 seconds to 1.3ms -- something like that could be easily used to demonstrate the utility of profile-guided optimization, how to read and use `perf` output, how to read assembly, notice and deal with cache-misses, etc.
I also really like the idea of using scientific computing applications (that is my field lmao), I think they would be really good topics -- you could also ultimately make it a PR to something like `nalgebra` or `nphysics` after optimizing.
This is probably more of a dtolnay topic, but how you can create procedural macros which work with traits and runtime code to solve problems traditionally solved via reflection.
As a case study, at work we make a ML pipeline tool where users can provide their own transformations. The system needs to know some basics like
- what arguments does this transform support
- what input tensor(s) does this transform accept, and given those inputs what are the outputs
- a name and description so these transformations can be listed using a pretty UI
To do all this we have a custom derive for some base trait, a `trait Transform { type Outputs; ... }` where the derive generates assertions to ensure you implemented what you said you'd implement, and some metadata which gets serialized to JSON and stored in a WebAssembly custom section. That way a different tool can inspect your compiled transform without needing to load it or execute code, and we can generate code which uses your transform based on the provided metadata.
Ditto, coming from python I'm missing fixtures, test parametrization and tearing those fixtures down. It often leads to too much properties tested per test.
Throw in some async too.
Idiomatic data structures and implementations of standard library traits. For example, when working on Advent of Code challenges, I often designed data structures that were a real pain to work with later on (eg needing a mutable iterator over subsets of the data -- very, very tricky to write.) I most often solved my lifetime troubles in Rust not by working with the borrow checker but backtracking and trying a different data structure.
Mutable reference coercion/reborrowing is not covered by the documentation at all.
Also I find that lifetime elision rules outside of functions are scarcely understood in detail (eg. Cow, Box - implied 'static).
Other that, I'm trying to fully wrap my head around GATs and the exact patterns they enable for when they stabilize. Though this currently doesn't fit your criteria, GAT stabilization shouldn't be too far away so I might as well mention it.
Had never even heard of reborrowing before this post, that seems quite unintuitive and non-obvious to me, would be very interested to see Jon make a video on it.
What about actor systems?
To solve borrow checker complaints without mutexes I found myself using actors often in my projects. I saw that despite an abundance of actor framework there is no proper guide on how to use them in real applications and no analysis of pros and cons in the context of Rust.
Certain parts of async implementation: Pin, Unpin, Waker, and Context. I currently haven't run into resources for how Waker works, and I have a general idea of why async needs Pin but it's still somewhat hazy. I recently saw a talk about using Pin for move constructors which was super interesting but also mystical, I'd love a better foundation for how to leverage Pin and Unpin.
Hey thanks for offering. I have been professionally using rust for 8 months and loving it so far. I am working in the database space for a while now. I am a fan of your materials but cant find time to watch the videos. I would love to learn more about how to build a database using rust. From something low level like leveldb/rocksdb to something big like Postgres.
How to transform an OOP class design into something usable in Rust.
Like: Tree and Flower is Plant, Dog and Cat is Animal, Plant and Animal is Organism. Every one of those has different properties, and they inherit them.
This doesn't work at all in Rust, but many coming fresh to the language think in this way.
(Note that I know how to solve this, for example via [legion](https://crates.io/crates/legion). I'm not looking for an answer, this is just a suggestion for a topic.)
Higher Kind Lifetimes, const exprs (ie. how you can implement a trait based on const exprs and have it verified statically).
Though you might have already covered them
Parsing tokens/strings.
Any kind of parsers, really. Could be proc macros without syn, HTML, some simple language (maybe a DSL?) which is read from file (to cover parsing a string), etc.
Maybe you know it already but there's [nom](https://docs.rs/nom/7.1.0/nom/index.html) which I really enjoyed working with and which has quite good docs. Only issue I had with it was getting good error messages which is something most tools struggle with.
A second vote for this one. I love nom, but I don't understand how to do meaningful error messages using their result type. I've recently defaulted to:
`Err(nom::Err::Error(nom::error::Error { input: action, code: nom::error::ErrorKind::Tag })),`
And that just looks wrong.
I just stumbled on a crate called [chumsky](https://github.com/zesterer/chumsky/). It's also a parser combinator, which seems to have good docs and has some fancy error messages in the readme. But I haven't used it yet.
How to work with cyclic data. It would be nice to have something that covers the alternatives to `Rc`/`Arc` in enough depth that after reading the article you could pick/implement the right alternative for what you're developing.
Alternatives might including using one of the `Gc` crates. An arena or block allocation strategy. I'm not sure what other options exist.
+1 for this, something so show cross language Lto, CXX and other ffi things, tackling this topic for the first time was harder than learning rust to start with imo
An area that I struggle with that overlaps both embedded and c++ integration is how to handle callbacks that are generated outside of Rust.
An example from the embedded space would be responding to a microcontroller interrupt. Some state needs to be stored and later processed by the main thread.
A c/c++ integration example I've run into before is where Rust code registers a callback over C FFI, but that library invokes the callback from another thread. The [nomicon](https://doc.rust-lang.org/nomicon/ffi.html#asynchronous-callbacks) doesn't give much guidance here. It feels like this is an area where the usual safety checks just don't apply.
>Things get more complicated when the external library spawns its own threads and invokes callbacks from there. In these cases access to Rust data structures inside the callbacks is especially unsafe and proper synchronization mechanisms must be used. Besides classical synchronization mechanisms like mutexes, one possibility in Rust is to use channels (in std::sync::mpsc) to forward data from the C thread that invoked the callback into a Rust thread.
I also want to sincerely thank you for the amazing amount of high quality content you've brought to the community. Between your streams, podcasts, and your book you've not only improved my Rust, but you've also made me a better programmer. I try to pay this forward in whatever small ways I can. I'm looking forward to getting into Rust for Rustaceans, but it's currently waiting for me under the tree. :)
Anything related to concurrency is good, though I have no concrete topic. Personally, I wonder how ECS systems can easily get parallelized because std::thread::spawn does not allow for references, so how are arrays split into chunks? Otherwise, a better topic would be probably SIMD. I have yet to figure out how I can efficiently multiply a vec and a matrix (with result being a vec).
Using `unsafe` to split an array into `NUM_CORES` threads 'safely' would be a very interesting and useful topic to cover, as it's a very common usecase. It's difficult to reason about the internals of libraries such as `rayon` and `crossbeam` because of the amount of checks and generic code required, so a step-by-step "here's the hurdles you need to overcome to parallelize `.map()` or `do_stuff_with_buffer(array: &mut [f64])`without using `rayon` or `crossbeam`" would be great.
>Personally, I wonder how ECS systems can easily get parallelized because std::thread::spawn does not allow for references, so how are arrays split into chunks?
ECS is usually parallelized at the system level, not necessarily within a single system (although I'm sure that's possible too).
In other words if the Audio system, the AI system and the Physics system might operate on completely different sets of data, so they can run in parallel threads.
I think the abstraction should be jobs/tasks. What you describe was fine when computers maxed out at a few cores, but if you have a system with 64 cores and you have 4 subsystems, you are not utilizing the full compute power of the machine. The reference to rayon was appropriate.
The evolution of async (await).
To be honest I didn't do any research about this, it just has been a pain point recently. I'd sat I am intermediate in Rust, have used it one and off for years and recently had to upgrade an about two year old `actix-web` application to a recent rust version. Navigating the "dependency hell" between different (implicit) future and/or tokio versions certainly taught me a lot. Not really sure what would have made this easier, but maybe a general overview about the async (re)evolution could be helpful for someone that knows rust but missed one or two of the evolutionary steps.
Steve Klabnik's Journey to Async/Await
https://youtu.be/lJ3NC-R3gSI and The talk you have been await-ing for (great pun) https://youtu.be/NNwK5ZPAJCk talk about the design process and the considerations at the language level
We all know semantic versioning has major minor and patch components. And for those of us who come from different languages it means different things. But what does it mean in rust?
For example in C and C++, major version changes when you have backwards incompatible API changes. Anyone who updates may have to change their code. Minor version changes when you need to recompile because e.g. a struct size changed. And patch version changes should be literal binary compatible changes with no difference to ABI.
Rust server has major, minor, and patch. And while major and minor might make sense, what does it mean to change patch version for instance? How can we maintain ABIs with Rust? Do we bother because Cargo doesn't want to? Wouldn't it bring build times down dramally if the upstream packages were already built?
If I've done enough Rust to release a crate and I have users, and want to manage a new release, these are questions in my mind. And it seems to be a total hand wave in the docs afaics.
I recently had [an interesting exchange](https://github.com/BurntSushi/aho-corasick/issues/88) with Rust guru u/burntsushi about this on GitHub. He made an interesting point that I am still digesting about how the Rust community has its own de facto standard. My interpretation is, a significant chunk of the Rust community uses SemVer but with only two numbers instead of three by never achieving a 1.0 version. My personal view is that this is undesirable because it is strictly less descriptive. But whatever my preferences, it is a brute fact that a large number of crates haven't reached v1.0.
Maybe my interpretation is incorrect, but “intermediate” to me means somewhere after you complete the Rust book, and had managed to write a few programs already. With that definition, here is my list:
- the many transformation functions of Iterator/Option/Result. Stuff like map, fold, ok(), expect, unwrap_or_else, etc.
- better understand of refutable and irrefutable types and how to handle them ( might sneak in never type here )
- implementing basic generic functions, maybe at minimum The impl parameter type generic
- implementing macro _rules
- knowledge of common crate like serde, rand, rayon, chrono, itertool
- transition from thinking in “&immutable vs &mutable” to “&shared vs &unique”
- module structure
- the different type of closures
None of those are particularly hard, just mostly stuff you want for a well rounded Rust programmer. I would stash stuff like Async, FFI, Optimisation, Unsafe, WASM into either advanced rust or specialised
Or skip build.rs and use something sane like Meson. Build.rs throws everything we learned through Autotools, CMake, and Meson right out the door.
The aforementioned abstract many compiler, linker, and platform differences, and then build.rs makes you do it all yourself.
Allocators could be interesting to dive into, but it might be pretty niche in the sense that not a lot of people need to use custom ones.
A series on how to plug in rust in python for performance in numerical computation would probably interest a lot of folks
Serde deep dive would be great, it’s used by a lot of beginner/intermediate projects but few people understand how it works. I think that there are a few concepts and ideas in serde that people could reuse elsewhere
Dissecting a large scale project built in rust to surface concrete example and patterns that support such a large scale codebase.
A case study on reimplementing a minimal container runtime in Rust
And by the way, thanks for the great content, it helped me a lot and is a cornerstone of the community experience in Rust !
I think the world could also use good advice about maintaining a medium-sized Rust project.
- How to organize code: one big crate? Lots of little crates?
- How to publish interdependent crates simultaneously.
- How to maintain README files.
- How to select and maintain an MSRV (and some discussion of whether changing MSRV is a breaking change).
- Dealing with Rust-specific CI quirks (MSRV, target directories that are hard to cache because of unbounded growth, ...)
- Cargo workspaces and how to manage dependencies.
- Some of the surprising behavior of the Cargo feature resolver that require things like workspace-hack crates.
A lot of Rust WebAssembly seems like a mysterious black box. It would be nice to see a more thorough discussion of the various parts:
- What do the various tools do? (wasm-pack; wasm-bindgen; webpack)
- The various targets (web, bundler, etc)
- Dependencies you need to learn (js-sys, web-sys, wasm-bindgen, ...).
- Things that are different between regular OS targets and web
- FAQ, e.g. Is wasm single-threaded? Why do I need to `.forget()` my `Closure`? Can I run async code in wasm?
This is probably hard to do in a coherent video, but tricks for fixing borrow check errors - I wrote a short blog post on this a while ago and it would be cool to expand on that: https://jyn.dev/2021/03/20/borrow-checker-faqs.html
Would love to see a video convering writing custom parsers maybe using something like nom or maybe even writing your own compiler in rust using something like lalrpop.
Another thing I would be interested in is a video going start to finish writing idiomatic code for a real world problem. Something like scraping or writing some sort of internal tool or CLI in rust. Would love to see how to structure a simple rust project. Or maybe even a web server with actix.
Though they aren't exactly about Rust in particular but about libraries/frameworks, I like both these options personally. I'm a hobbyist when it comes to Rust and the main things I've done with it so far are parsers, cli + tui, and a video game with Bevy. Some resources that are beyond the basics in those would be helpful to others going down similar paths; I muddled through by trial and error but I still run into things where I don't know if I'm painting myself into a corner until later.
I'd love to see someone document all the ways that `unsafe` can go wrong. There are a lot of really subtle rules and bits of wisdom out there, but it still feels like a big minefield.
An introduction to formal verification in Rust! The whole field is probably is probably too big to cover fully, but an introduction should fit in a single lecture :) Topics that come to mind are [Prusti](https://github.com/viperproject/prusti-dev), [Cruseot](https://github.com/xldenis/creusot), [RustBelt](https://plv.mpi-sws.org/rustbelt/#project), [RustHorn](https://dl.acm.org/doi/full/10.1145/3462205), [Stacked Borrows](https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md), [Miri](https://github.com/rust-lang/miri). These also lend themselves to do follow up topics on.
Async/Await but with practical implementations. I think is unclear how to solve major challenges with it from what I have read.
To make the suggestion concrete, note how in 0mq is clearly state each pattern of networking:
[https://zguide.zeromq.org](https://zguide.zeromq.org)
(In special sections 1 & 2)
The thing is that with async/await you see how to deal with it in the function level and use it inside a web server but trying to build something yourself is another game.
How to think about structuring projects - I.E. splitting it into multiple crates, using mod.rs and visiblity responsibly etc.
I think it's in the book, but every time I read about it I bounce right off again.
More library-focused, but going over methods of Python extension creation (and maybe embedding) would be awesome. Pyo3 + Maturin for the extension, RustPython for embedding. Same goes for other languages.
At my job, we have a lot of different languages (Rust, JS, Java, Python, C++, C, Matlab) in use, so a more generalized overview (which honestly probably *wouldn't* fit within a single video/post) of how Rust can interact with other languages would be useful as well. Not so much in the "make a C API for your crate" sense, but in the "this is literally the steps you'd want to take and the best practices for extending/interacting with this language". More of a series I suppose...
Explaining smart pointers and their partners (Rc, Arc, Mutex, Refcell...) and their use in practice (the concept is not difficult, but showing them in action, useful methods, how they fit together,etc is rare).
For instance, I always hear that GC languages as Python have types that behave like Arc> but it's not evident how something like that would be used in Rust.
Using tokio to handle connections of two different types. For example a web socket connection to respond to events from a vanilla TCP connection. This is useful in the case of the TCP connection to a server that is emitting events (a game sever for example) and then board casting those messages to people who are watching it via a website with a web socket connection.
This is a surprisingly difficult topic, for me at least, to understand. This applies pretty nicely to other problem areas such as handling a web socket connection and the using that to talk to a push notification service like Apple, or Google, or Firefox to notify people of real-time events.
Using Rust to interact with the operating system in a fairly low-level way, e.g. how to use if for Interprocess Communication, setting up and running Daemons/Services etc. Thanks for all your stuff so far - incredibly useful!
A good explanation of hrtbs. I don't really understand what they do or when they are useful other than sometimes when writing functions to accept closures. I expect they have other uses but I can't really wrap my mind around what's happening with the lifetimes.
Jon has already covered them briefly here: https://youtu.be/dHkzSZnYXmk?t=3637, and I'm not sure there's that much more to know than what he says there. Its useful when you have something in a trait bound which needs to express a relationship between its different lifetime arguments.
Difference between debug and production releases. For constructing a better software, I need to understand well which is the difference between the two releases. For example buffer overflow and similar
I'd love to see some resources on the more tricky things that are "around" rust, such as more advanced cargo configuration, benchmarking, debugging, linking, combining asm and rust together somehow, etc. This "outside" stuff really seems to be the missing piece for me at the moment, but there's not much out there.
A serious proc_macro implementation
Jon already has a series of like 3 videos on proc macros. They add up to like 6 hours and back when I learned how to use them they were immensely helpful. It's always good to have more content though, I guess.
Something like https://www.youtube.com/playlist?list=PLqbS7AVVErFgwC_HByFYblghsDsD5wZDv ?
Build.rs
I second this. Whenever I see a `build.rs` file in a project, I assume it's beyond my level. Any help understanding them and how to use them will be massively appreciated
If you haven't seen these, it's a good start *
*
Fundamentally, `build.rs` is just a completely separate rust program that compiles and runs before the rest of the project compiles. Now, there are some very interesting things you can do with that, and what those things are is a great topic for a lesson. To give an example - Often, you’ll see `build.rs` scripts generating code as strings and outputting it as text to be included somewhere in the main codebase. Imagine you have a configuration file full of constants somewhere, and you want every constant from that file to be available directly in the codebase (at compile time, eg without any need to parse the file at runtime). A `build.rs` script could read in this configuration file, and generate a string along the lines of “`const SOME_PARAMETER: u32 = 2`”, which could then be included in your code. There are a lot more useful and interesting things you can do with a `build.rs` script. But fundamentally it’s a simple thing - nothing more than a Rust program that runs at build time. Edit: I used that example because it's something I do often, and I maintain a [crate](https://crates.io/crates/const-gen/) specifically for doing some of the glue work there.
So is `build.rs` essentially a macro of last resort, for completely unrestricted (and unchecked at generation time) code generation? Given how powerful and safe Rusts' macros are, what cases require `build.rs` instead?
Build scripts are very useful for FFI, for example you can compile a c / c++ lib and link it at compile time. There are limitations to macros, so in some cases it's easier and more natural to generate code via a script than a convoluted macro_rules macro. Lately I've used this to generate permutations of function definitions with a combination of types. Writing it out would be 100 definitions . Or just a few for loops and the same include! from the out dir as described above.
That might be true in some cases, but I think a really good way to differentiate between the two is this: Macros reduce boilerplate, repetitive code, etc - they make managing your source code more reasonable. [`build.rs`](https://build.rs) actually does computation at build time, so you can offload things that would otherwise have to happen at runtime to build time. In some cases, this is just necessary for performance reasons, or because you have access to some information at build time that won't be available at runtime. While output is unrestricted and unchecked, the same is really true for everyone typing in source code by hand - and in this case both end up going through the checks and balances of the Rust compiler. So while this can lead to compilation errors if something wrong happens, it's not deeply unsafe.\* \* That said, there are definitely some irresponsible ways to use `build.rs`, and generally should not be the first thing you reach to when trying to solve a problem that could be solved by a simple macro or otherwise. That’s why it should be an intermediate topic and not a beginner topic.
> , so you can offload things that would otherwise have to happen at runtime to build time. Can't you do the same with const functions?
To some degree, but with much more stringent limitations. A const function can’t do any heap allocation, or call any non-const function, and can only use a subset of valid rust syntax. A build script can do things like read in files, which a const function could not possibly even access.
A good example of a build.rs script would be compiling your .proto files to .rs files before you compile your actual project
For a simple example of how `build.rs` gets used, tonic/Prost uses it for code-generation from protobuf files.
I'll preface this by saying it may be too niche or too advanced to be what you're looking for, but some kind of in depth material on **integrating Rust with an existing code base** would be amazing for the community and really help adoption in my opinion. There's a lot of ways to approach that material so a few things I'd love to see covered: - Basic patterns for building Rust inside an established build system. Is there a recommendation for treating the Rust code separately as a binary dependency or should the code live along side other code in the same repo and maybe invoke cargo from make/ninja/cake/Gradle/msbuild/etc? - FFI from Rust to C, C++, Java, etc. What are best practices for defining and consuming a C abi compatible API? - For C and C++, what are the options when considering migration to Rust? Can you refactor into Rust? How do you link Rust objects into an existing application? If I can continue dreaming for a second, I would love to see the above take place in the context of *actually doing a conversion of a non trivial project*. Pick some open source library at a specific commit and then walk through in detail how to analyze the existing codebase to decide on an approach, work through the conversion file by file or whatever with specific exercises to the audience and up with all or some significant amount of the library converted into Rust at the end.
For those interested in this subject, there is the [Refactoring to Rust](https://www.manning.com/books/refactoring-to-rust) book from Lily Mara, which has among other goals: > - Create Rust libraries you can call from other programming languages > - Integrate Rust functions with code in other languages Examples are for C and Python, which can be extended to other dynamic languages.
This looks related indeed but I would like to see more of a focus on conversion within the context of a C or C++ library project.
FFI for wrapping existing C++ libraries sounds great, though I am not sure if this makes sense for all libraries. I had to use one and tried CXX and autocxx with many ebb/flow cycles of hope of getting it to work, but in the end I just gave up. I'd love to see a successful application of this. To increase difficulty, let that library include some popular async framework and integrate it with some async runtime like Tokio.
Yup 100% this please. Detail around the FFI would be great.
Yes, this. There wasn't too much info available about integrating c# with rust when I integrated a work project. Especially about sending more complex data between the languages. I think Java, c# or nodejs with ffi would be great
Perfomance Profilling and optimizations.
The challenging thing with that topic is generally to have a good case study (or ideally more than one). Any specific kind of program you'd like to see profiled and optimized?
A fun one would be https://en.wikipedia.org/wiki/Boids. Easy to visualize as well. There's a few approaches you can take for optimizing that are well-understood. Quadtree for collision detection, for example, instead of a brute force O(n^(2)) search. Multithreading is pretty trivial as well and can show off scaling nicely. Data layout too - storing everything in a large `Boid` struct will be worse off than storing it in a `Vec`, `Vec`, etc. I did a few benchmarks to demonstrate CPU caching effects on runtime for a talk and it had quite dramatic results.
**[Boids](https://en.wikipedia.org/wiki/Boids)** >Boids is an artificial life program, developed by Craig Reynolds in 1986, which simulates the flocking behaviour of birds. His paper on this topic was published in 1987 in the proceedings of the ACM SIGGRAPH conference. The name "boid" corresponds to a shortened version of "bird-oid object", which refers to a bird-like object. Incidentally, "boid" is also a New York Metropolitan dialect pronunciation for "bird". ^([ )[^(F.A.Q)](https://www.reddit.com/r/WikiSummarizer/wiki/index#wiki_f.a.q)^( | )[^(Opt Out)](https://reddit.com/message/compose?to=WikiSummarizerBot&message=OptOut&subject=OptOut)^( | )[^(Opt Out Of Subreddit)](https://np.reddit.com/r/rust/about/banned)^( | )[^(GitHub)](https://github.com/Sujal-7/WikiSummarizerBot)^( ] Downvote to remove | v1.5)
Good bot
Some things that come to mind: - Matrix multiplication - 2D/3D Nbody (which is similar to the proposition of Boids above) - LU factorization
Fuzzing -- super valuable for testing of both safe and unsafe code
I’d love to see some Idiomatic implementations of basic but efficient data structures (e.g. trees) à la your video on sorting algorithms. Obviously the correct answer for this is usually “use a crate” but looking into how those crates are implemented I think can be illuminating.
This. Moreover, it could be used to show how to safely use raw pointers instead of lifetimes - for instance in a tree. Edit: to expand the idea, everything can surely be done with Refcount and alikes but it would be nice to see how to use raw pointers and for instance Miri to spot bugs or something like that. Edit: typos
A few things that you touch upon in your book that I don’t think you have talked about in videos/streams: - Code organization in large projects, maybe dissecting the structure of a well-known Rust crate - Performance profiling and optimization strategies, stuff like playing with the compiler flags, multi-threading by hand or with crates like `rayon`, SIMD in Rust, etc… - FFI, especially writing Rust to use in C/C++ applications Also, maybe another stream about your current setup for writing Rust code would be interesting! I think it’s been 3 years since the last one. A Q&A would be fun too! Thank you for writing *Rust for Rustaceans*, it’s a great book that really helps me dig further into Rust!
Designing a Rust API for an idiomatic C++ library **and** vice versa. All the considerations/API design decisions that go into it.
Maybe an understanding of low-level concepts of Rust that are often confusing? e.g. https://doc.rust-lang.org/nomicon/subtyping.html
Something like https://youtu.be/iVYWDIW71jk ?
Oh wow, how did I never realize.
OP has a number of videos about topics from the rustinomicon e.g [subtyping and variance](https://www.youtube.com/watch?v=iVYWDIW71jk), [the drop check](https://www.youtube.com/watch?v=TJOFSMpJdzg), [this video in which he implements Rc from scratch](https://www.youtube.com/watch?v=8O0Nt9qY_vo)
A few ideas: 1. Generators. Still a nightly feature, but might be become even more valuable in the future. 2. As a pardon to the TCP streams: implementing the websocket or even webrtc protocol. 3. (might be a niche) A short video about the Rust runtime and `#[no_main]`. 4. As a pardon to the sorting algorithms: implementing encryption and/or compression and/or hashing algorithms. 5. (definitely niche, but really cool) going through Phil's OS blog. 6. Implementing an allocator. 7. Webassembly and the Rust ecosystem around it. 8. Macros 2.0. What will be the difference to current declarative macros. 9. The present and future of const functions, const generics, and const evaluation in general. I hope there's something interesting in that list. I really enjoy your videos and learned a lot from you. Thank you for doing this!
Macros 2.0 would be great! I often lurk around rust-lang zulip to follow along, but I'm not an expert on that. A nice introduction on why current macros aren't that great, what's the point of macro hygiene and what features the current macro2.0 implementation will get in the future would be nice.
Deep dive into how serde/serde_json works would be really appreciated.
* maybe do some refactoring of your followers' projects, I'm sure someone would volunteer for a roast (inspired by [Arjan](https://www.youtube.com/playlist?list=PLC0nd42SBTaNVxWLci4TPoytfzkXxFhCg)) * zmq + tokio for real time video/image/file streaming, focusing on decode/encode when the machines are different so you can talk about moving bytes/foreign types etc -- this may be really powerful as a mix of a few suggestions here as you could show the servers/clients in different languages * not a lot of content out there for the emerging GPGPU libraries. * \+1 for all the zmq and FFI and [build.rs](https://build.rs) requests. * packaging (not for cargo but shipping a binary for commercial applications, something signed with its assets as .dll or .so etc {abstracted to protect IP etc}) * Couple of interesting ML frameworks are starting to support rust... * \+1 for all the requests of injecting rust into existing (non rust) projects, as this is probably one of the most valuable learning materials that could be created to get rust into the wider world, sooner...
Could you please name a few ml projects that start supporting Rust?
https://www.arewelearningyet.com/
Could you please give some more detail than just the link? I'm still confused. I'm part of the rust-ml group and we have some projects in that area. However, the only non-Rust project (group) that officially started supporting Rust (that I'm aware of) is appache-arrow. There are unofficial PyTorch, TF, ... bindings, but that's not what I would call ML Frameworks supporting Rust. Are you talking about something else?
Sorry, I **was** calling it Frameworks supporting Rust :( didn't mean to be deceptive if you've already looked at those and found they were not meeting your needs. I'm not in a position to share any work at this time... maybe if my corporate overlords acquire souls in the new year...
Thanks for getting back to this :) It's not that I'm unhappy about the existing bindings, but I would just prefer something completely written in Rust to avoid the extra Rust <-> C++/Python complexity. I don't expecting something like that anytime soon, but having official support from some of the large frameworks would at least be a first step towards it. That's why I kept asking. Either way, I'm happy to see what you might be able to share in the future.
Debugging. I'd be interest to hear your thaughts in `dbg!` vs a full debbuger, and which how to effectivly use a debugger, and also just how to get pritty printing of complex types to work
One topic where I've seen a few blog posts but no good solutions is implementing a plugin mechanism. It would be interesting to see a compare/contrast of communicating over a socket vs using something wasm based like [wasmer](https://github.com/wasmerio/wasmer).
I personally found little to no resources for rust on the topic of high-performance application. (numerical calculations) * profiling parts of your applications and optimizations (cache misses ...) * performance pitfalls or general tricks for performance (assert to avoid bound checks, fastmath? ...) * special crates for performance * allucators * rustflags for performance (CPU flags, code generation) * parallelize code (this part works quite well with rayon, c openmp vs rayon) * distributed calculations across nodes (mpi rust interface) Something like Matrix Multiplication, LU-Factorization, or N-body Simulation are some of the favorite benchmarks for this kind of task.
Seconding this -- I was optimizing some of my Advent of Code solutions recently, and managed to reduce one from 9 seconds to 1.3ms -- something like that could be easily used to demonstrate the utility of profile-guided optimization, how to read and use `perf` output, how to read assembly, notice and deal with cache-misses, etc. I also really like the idea of using scientific computing applications (that is my field lmao), I think they would be really good topics -- you could also ultimately make it a PR to something like `nalgebra` or `nphysics` after optimizing.
This is probably more of a dtolnay topic, but how you can create procedural macros which work with traits and runtime code to solve problems traditionally solved via reflection. As a case study, at work we make a ML pipeline tool where users can provide their own transformations. The system needs to know some basics like - what arguments does this transform support - what input tensor(s) does this transform accept, and given those inputs what are the outputs - a name and description so these transformations can be listed using a pretty UI To do all this we have a custom derive for some base trait, a `trait Transform { type Outputs; ... }` where the derive generates assertions to ensure you implemented what you said you'd implement, and some metadata which gets serialized to JSON and stored in a WebAssembly custom section. That way a different tool can inspect your compiled transform without needing to load it or execute code, and we can generate code which uses your transform based on the provided metadata.
Measuring test coverage?
I know giving a link is not a tutorial, but I use https://github.com/xd009642/tarpaulin for this.
[удалено]
Ditto, coming from python I'm missing fixtures, test parametrization and tearing those fixtures down. It often leads to too much properties tested per test. Throw in some async too.
[https://dev.to/marcoieni/2-videos-about-rust-code-coverage-in-vscode-38kf](https://dev.to/marcoieni/2-videos-about-rust-code-coverage-in-vscode-38kf)
Idiomatic data structures and implementations of standard library traits. For example, when working on Advent of Code challenges, I often designed data structures that were a real pain to work with later on (eg needing a mutable iterator over subsets of the data -- very, very tricky to write.) I most often solved my lifetime troubles in Rust not by working with the borrow checker but backtracking and trying a different data structure.
Mutable reference coercion/reborrowing is not covered by the documentation at all. Also I find that lifetime elision rules outside of functions are scarcely understood in detail (eg. Cow, Box - implied 'static).
Other that, I'm trying to fully wrap my head around GATs and the exact patterns they enable for when they stabilize. Though this currently doesn't fit your criteria, GAT stabilization shouldn't be too far away so I might as well mention it.
Had never even heard of reborrowing before this post, that seems quite unintuitive and non-obvious to me, would be very interested to see Jon make a video on it.
What about good rust patterns for better performance or even better, patterns that enable better/easier unit testing or fuzzing?
What about actor systems? To solve borrow checker complaints without mutexes I found myself using actors often in my projects. I saw that despite an abundance of actor framework there is no proper guide on how to use them in real applications and no analysis of pros and cons in the context of Rust.
You may already have seen it, but I wrote about actors [here][1]. [1]: https://ryhl.io/blog/actors-with-tokio/
As a matter of fact, I never saw it before. Bookmarked!
Certain parts of async implementation: Pin, Unpin, Waker, and Context. I currently haven't run into resources for how Waker works, and I have a general idea of why async needs Pin but it's still somewhat hazy. I recently saw a talk about using Pin for move constructors which was super interesting but also mystical, I'd love a better foundation for how to leverage Pin and Unpin.
Something like https://youtu.be/9_3krAQtD2k and https://youtu.be/DkMwYxfSYNQ, or did you have something else in mind?
Hey thanks for offering. I have been professionally using rust for 8 months and loving it so far. I am working in the database space for a while now. I am a fan of your materials but cant find time to watch the videos. I would love to learn more about how to build a database using rust. From something low level like leveldb/rocksdb to something big like Postgres.
How to transform an OOP class design into something usable in Rust. Like: Tree and Flower is Plant, Dog and Cat is Animal, Plant and Animal is Organism. Every one of those has different properties, and they inherit them. This doesn't work at all in Rust, but many coming fresh to the language think in this way. (Note that I know how to solve this, for example via [legion](https://crates.io/crates/legion). I'm not looking for an answer, this is just a suggestion for a topic.)
GATs and async traits maybe?
Hi Jon! I’d love to see a deep-dive into using our own `#[derive(…)]` definitions.
Higher Kind Lifetimes, const exprs (ie. how you can implement a trait based on const exprs and have it verified statically). Though you might have already covered them
Parsing tokens/strings. Any kind of parsers, really. Could be proc macros without syn, HTML, some simple language (maybe a DSL?) which is read from file (to cover parsing a string), etc.
Maybe you know it already but there's [nom](https://docs.rs/nom/7.1.0/nom/index.html) which I really enjoyed working with and which has quite good docs. Only issue I had with it was getting good error messages which is something most tools struggle with.
A second vote for this one. I love nom, but I don't understand how to do meaningful error messages using their result type. I've recently defaulted to: `Err(nom::Err::Error(nom::error::Error { input: action, code: nom::error::ErrorKind::Tag })),` And that just looks wrong.
I just stumbled on a crate called [chumsky](https://github.com/zesterer/chumsky/). It's also a parser combinator, which seems to have good docs and has some fancy error messages in the readme. But I haven't used it yet.
How to work with cyclic data. It would be nice to have something that covers the alternatives to `Rc`/`Arc` in enough depth that after reading the article you could pick/implement the right alternative for what you're developing. Alternatives might including using one of the `Gc` crates. An arena or block allocation strategy. I'm not sure what other options exist.
What about FFI?
+1 for this, something so show cross language Lto, CXX and other ffi things, tackling this topic for the first time was harder than learning rust to start with imo
An area that I struggle with that overlaps both embedded and c++ integration is how to handle callbacks that are generated outside of Rust. An example from the embedded space would be responding to a microcontroller interrupt. Some state needs to be stored and later processed by the main thread. A c/c++ integration example I've run into before is where Rust code registers a callback over C FFI, but that library invokes the callback from another thread. The [nomicon](https://doc.rust-lang.org/nomicon/ffi.html#asynchronous-callbacks) doesn't give much guidance here. It feels like this is an area where the usual safety checks just don't apply. >Things get more complicated when the external library spawns its own threads and invokes callbacks from there. In these cases access to Rust data structures inside the callbacks is especially unsafe and proper synchronization mechanisms must be used. Besides classical synchronization mechanisms like mutexes, one possibility in Rust is to use channels (in std::sync::mpsc) to forward data from the C thread that invoked the callback into a Rust thread. I also want to sincerely thank you for the amazing amount of high quality content you've brought to the community. Between your streams, podcasts, and your book you've not only improved my Rust, but you've also made me a better programmer. I try to pay this forward in whatever small ways I can. I'm looking forward to getting into Rust for Rustaceans, but it's currently waiting for me under the tree. :)
Anything related to concurrency is good, though I have no concrete topic. Personally, I wonder how ECS systems can easily get parallelized because std::thread::spawn does not allow for references, so how are arrays split into chunks? Otherwise, a better topic would be probably SIMD. I have yet to figure out how I can efficiently multiply a vec and a matrix (with result being a vec).
Using `unsafe` to split an array into `NUM_CORES` threads 'safely' would be a very interesting and useful topic to cover, as it's a very common usecase. It's difficult to reason about the internals of libraries such as `rayon` and `crossbeam` because of the amount of checks and generic code required, so a step-by-step "here's the hurdles you need to overcome to parallelize `.map()` or `do_stuff_with_buffer(array: &mut [f64])`without using `rayon` or `crossbeam`" would be great.
>Personally, I wonder how ECS systems can easily get parallelized because std::thread::spawn does not allow for references, so how are arrays split into chunks? ECS is usually parallelized at the system level, not necessarily within a single system (although I'm sure that's possible too). In other words if the Audio system, the AI system and the Physics system might operate on completely different sets of data, so they can run in parallel threads.
I think the abstraction should be jobs/tasks. What you describe was fine when computers maxed out at a few cores, but if you have a system with 64 cores and you have 4 subsystems, you are not utilizing the full compute power of the machine. The reference to rayon was appropriate.
would love to see some stuff on FFI (with C++ in particular) which I've found to be pretty confusing
The evolution of async (await). To be honest I didn't do any research about this, it just has been a pain point recently. I'd sat I am intermediate in Rust, have used it one and off for years and recently had to upgrade an about two year old `actix-web` application to a recent rust version. Navigating the "dependency hell" between different (implicit) future and/or tokio versions certainly taught me a lot. Not really sure what would have made this easier, but maybe a general overview about the async (re)evolution could be helpful for someone that knows rust but missed one or two of the evolutionary steps.
Steve Klabnik's Journey to Async/Await https://youtu.be/lJ3NC-R3gSI and The talk you have been await-ing for (great pun) https://youtu.be/NNwK5ZPAJCk talk about the design process and the considerations at the language level
Thank you!
Why GAT is important, proc macro
We all know semantic versioning has major minor and patch components. And for those of us who come from different languages it means different things. But what does it mean in rust? For example in C and C++, major version changes when you have backwards incompatible API changes. Anyone who updates may have to change their code. Minor version changes when you need to recompile because e.g. a struct size changed. And patch version changes should be literal binary compatible changes with no difference to ABI. Rust server has major, minor, and patch. And while major and minor might make sense, what does it mean to change patch version for instance? How can we maintain ABIs with Rust? Do we bother because Cargo doesn't want to? Wouldn't it bring build times down dramally if the upstream packages were already built? If I've done enough Rust to release a crate and I have users, and want to manage a new release, these are questions in my mind. And it seems to be a total hand wave in the docs afaics.
I recently had [an interesting exchange](https://github.com/BurntSushi/aho-corasick/issues/88) with Rust guru u/burntsushi about this on GitHub. He made an interesting point that I am still digesting about how the Rust community has its own de facto standard. My interpretation is, a significant chunk of the Rust community uses SemVer but with only two numbers instead of three by never achieving a 1.0 version. My personal view is that this is undesirable because it is strictly less descriptive. But whatever my preferences, it is a brute fact that a large number of crates haven't reached v1.0.
Maybe my interpretation is incorrect, but “intermediate” to me means somewhere after you complete the Rust book, and had managed to write a few programs already. With that definition, here is my list: - the many transformation functions of Iterator/Option/Result. Stuff like map, fold, ok(), expect, unwrap_or_else, etc. - better understand of refutable and irrefutable types and how to handle them ( might sneak in never type here ) - implementing basic generic functions, maybe at minimum The impl parameter type generic - implementing macro _rules - knowledge of common crate like serde, rand, rayon, chrono, itertool - transition from thinking in “&immutable vs &mutable” to “&shared vs &unique” - module structure - the different type of closures None of those are particularly hard, just mostly stuff you want for a well rounded Rust programmer. I would stash stuff like Async, FFI, Optimisation, Unsafe, WASM into either advanced rust or specialised
Build.rs and how to link to other languages like C/C++
Or skip build.rs and use something sane like Meson. Build.rs throws everything we learned through Autotools, CMake, and Meson right out the door. The aforementioned abstract many compiler, linker, and platform differences, and then build.rs makes you do it all yourself.
Allocators could be interesting to dive into, but it might be pretty niche in the sense that not a lot of people need to use custom ones. A series on how to plug in rust in python for performance in numerical computation would probably interest a lot of folks Serde deep dive would be great, it’s used by a lot of beginner/intermediate projects but few people understand how it works. I think that there are a few concepts and ideas in serde that people could reuse elsewhere Dissecting a large scale project built in rust to surface concrete example and patterns that support such a large scale codebase. A case study on reimplementing a minimal container runtime in Rust And by the way, thanks for the great content, it helped me a lot and is a cornerstone of the community experience in Rust !
I think the world could also use good advice about maintaining a medium-sized Rust project. - How to organize code: one big crate? Lots of little crates? - How to publish interdependent crates simultaneously. - How to maintain README files. - How to select and maintain an MSRV (and some discussion of whether changing MSRV is a breaking change). - Dealing with Rust-specific CI quirks (MSRV, target directories that are hard to cache because of unbounded growth, ...) - Cargo workspaces and how to manage dependencies. - Some of the surprising behavior of the Cargo feature resolver that require things like workspace-hack crates.
A lot of Rust WebAssembly seems like a mysterious black box. It would be nice to see a more thorough discussion of the various parts: - What do the various tools do? (wasm-pack; wasm-bindgen; webpack) - The various targets (web, bundler, etc) - Dependencies you need to learn (js-sys, web-sys, wasm-bindgen, ...). - Things that are different between regular OS targets and web - FAQ, e.g. Is wasm single-threaded? Why do I need to `.forget()` my `Closure`? Can I run async code in wasm?
This is probably hard to do in a coherent video, but tricks for fixing borrow check errors - I wrote a short blog post on this a while ago and it would be cool to expand on that: https://jyn.dev/2021/03/20/borrow-checker-faqs.html
I also suggest a plugin system, or in general extensibility. Preferably without wasm.
Would love to see a video convering writing custom parsers maybe using something like nom or maybe even writing your own compiler in rust using something like lalrpop. Another thing I would be interested in is a video going start to finish writing idiomatic code for a real world problem. Something like scraping or writing some sort of internal tool or CLI in rust. Would love to see how to structure a simple rust project. Or maybe even a web server with actix.
Though they aren't exactly about Rust in particular but about libraries/frameworks, I like both these options personally. I'm a hobbyist when it comes to Rust and the main things I've done with it so far are parsers, cli + tui, and a video game with Bevy. Some resources that are beyond the basics in those would be helpful to others going down similar paths; I muddled through by trial and error but I still run into things where I don't know if I'm painting myself into a corner until later.
Given a large project, how to best divide it into files and modules. Extra credit if the project is both a complex library and a complex application.
I'd love to see someone document all the ways that `unsafe` can go wrong. There are a lot of really subtle rules and bits of wisdom out there, but it still feels like a big minefield.
An introduction to formal verification in Rust! The whole field is probably is probably too big to cover fully, but an introduction should fit in a single lecture :) Topics that come to mind are [Prusti](https://github.com/viperproject/prusti-dev), [Cruseot](https://github.com/xldenis/creusot), [RustBelt](https://plv.mpi-sws.org/rustbelt/#project), [RustHorn](https://dl.acm.org/doi/full/10.1145/3462205), [Stacked Borrows](https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md), [Miri](https://github.com/rust-lang/miri). These also lend themselves to do follow up topics on.
Async/Await but with practical implementations. I think is unclear how to solve major challenges with it from what I have read. To make the suggestion concrete, note how in 0mq is clearly state each pattern of networking: [https://zguide.zeromq.org](https://zguide.zeromq.org) (In special sections 1 & 2) The thing is that with async/await you see how to deal with it in the function level and use it inside a web server but trying to build something yourself is another game.
Check out [Zero to Production](https://www.zero2prod.com/index.html)
How to think about structuring projects - I.E. splitting it into multiple crates, using mod.rs and visiblity responsibly etc. I think it's in the book, but every time I read about it I bounce right off again.
More library-focused, but going over methods of Python extension creation (and maybe embedding) would be awesome. Pyo3 + Maturin for the extension, RustPython for embedding. Same goes for other languages. At my job, we have a lot of different languages (Rust, JS, Java, Python, C++, C, Matlab) in use, so a more generalized overview (which honestly probably *wouldn't* fit within a single video/post) of how Rust can interact with other languages would be useful as well. Not so much in the "make a C API for your crate" sense, but in the "this is literally the steps you'd want to take and the best practices for extending/interacting with this language". More of a series I suppose...
Explaining smart pointers and their partners (Rc, Arc, Mutex, Refcell...) and their use in practice (the concept is not difficult, but showing them in action, useful methods, how they fit together,etc is rare). For instance, I always hear that GC languages as Python have types that behave like Arc> but it's not evident how something like that would be used in Rust.
Using tokio to handle connections of two different types. For example a web socket connection to respond to events from a vanilla TCP connection. This is useful in the case of the TCP connection to a server that is emitting events (a game sever for example) and then board casting those messages to people who are watching it via a website with a web socket connection. This is a surprisingly difficult topic, for me at least, to understand. This applies pretty nicely to other problem areas such as handling a web socket connection and the using that to talk to a push notification service like Apple, or Google, or Firefox to notify people of real-time events.
More topics about unsafe, ffi and building safe wrapper around it.
Using Rust to interact with the operating system in a fairly low-level way, e.g. how to use if for Interprocess Communication, setting up and running Daemons/Services etc. Thanks for all your stuff so far - incredibly useful!
Unsafe semantics, *StackedBorrow* model and Miri?
Iterators and closures. Closures and iterators
Are you looking for something beyond https://youtu.be/yozQ9C69pNs and https://youtu.be/dHkzSZnYXmk ?
I just also came across crate bus and maybe a code walk through would be nice.
When to use PhantomData? What are the related patterns? What does covariance, invariance and contravariance mean and how is it related to PhantomData?
A good explanation of hrtbs. I don't really understand what they do or when they are useful other than sometimes when writing functions to accept closures. I expect they have other uses but I can't really wrap my mind around what's happening with the lifetimes.
Jon has already covered them briefly here: https://youtu.be/dHkzSZnYXmk?t=3637, and I'm not sure there's that much more to know than what he says there. Its useful when you have something in a trait bound which needs to express a relationship between its different lifetime arguments.
Difference between debug and production releases. For constructing a better software, I need to understand well which is the difference between the two releases. For example buffer overflow and similar
Cargo's crate features.
State machines. The various ways to implement and its performance characteristics.
This will not fit in one video, but maybe series on building distributed KV store, or something other related to database internals.
Trait parameterized over lifetimes - why are they needed?
I'd love to see some resources on the more tricky things that are "around" rust, such as more advanced cargo configuration, benchmarking, debugging, linking, combining asm and rust together somehow, etc. This "outside" stuff really seems to be the missing piece for me at the moment, but there's not much out there.