Rust macro for matching on struct fields
updated: 2023-08-29
This was just a Rust macro_rules! macro example before. I've since appended a proc_macro_derive example.
OpenGL code has been mostly done, but I moved on to something else for now: Dwarf Fortress RAWs. There are around 220 text files for configuring Dwarf Fortress. There's a lot there, it's the soul of DF, and it's in the public domain. Fuck yah!
I've been working on a parser for the raw files, and it finally forced me to learn Rust macros. Here's a quick post where I'm going to step thru a macro I just made line by line, because the internet needs more Rust macro examples. Even if it doesn't, too late.
First, the problem I wanted to solve. I have a giant struct that looks like this:
set_token! {
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Default, PartialEq)]
pub struct material_tokens {
pub IMPLIES_ANIMAL_KILL: bool,
pub ALCOHOL_PLANT: bool,
pub ALCOHOL_CREATURE: bool,
pub CHEESE_PLANT: bool,
pub CHEESE_CREATURE: bool,
pub POWDER_MISC_PLANT: bool,
pub POWDER_MISC_CREATURE: bool,
pub STOCKPILE_GLOB: bool,
pub STOCKPILE_GLOB_PASTE: bool,
pub STOCKPILE_GLOB_PRESSED: bool,
pub STOCKPILE_PLANT_GROWTH: bool,
pub LIQUID_MISC_PLANT: bool,
pub LIQUID_MISC_CREATURE: bool,
pub LIQUID_MISC_OTHER: bool,
pub STRUCTURAL_PLANT_MAT: bool,
pub SEED_MAT: bool,
pub BONE: bool,
pub WOOD: bool,
pub THREAD_PLANT: bool,
pub TOOTH: bool,
pub HORN: bool,
pub HAIR: bool,
pub PEARL: bool,
pub SHELL: bool,
pub LEATHER: bool,
pub SILK: bool,
pub SOAP: bool,
pub GENERATES_MIASMA: bool,
pub MEAT: bool,
pub ROTS: bool,
pub BLOOD_MAP_DESCRIPTOR: bool,
pub ICHOR_MAP_DESCRIPTOR: bool,
pub GOO_MAP_DESCRIPTOR: bool,
pub SLIME_MAP_DESCRIPTOR: bool,
pub PUS_MAP_DESCRIPTOR: bool,
pub SWEAT_MAP_DESCRIPTOR: bool,
pub TEARS_MAP_DESCRIPTOR: bool,
pub SPIT_MAP_DESCRIPTOR: bool,
pub EVAPORATES: bool,
pub ENTERS_BLOOD: bool,
pub EDIBLE_VERMIN: bool,
pub EDIBLE_RAW: bool,
pub EDIBLE_COOKED: bool,
pub DO_NOT_CLEAN_GLOB: bool,
pub NO_STONE_STOCKPILE: bool,
pub ITEMS_METAL: bool,
pub ITEMS_BARRED: bool,
pub ITEMS_SCALED: bool,
pub ITEMS_LEATHER: bool,
pub ITEMS_SOFT: bool,
pub ITEMS_HARD: bool,
pub IS_STONE: bool,
pub UNDIGGABLE: bool,
pub DISPLAY_UNGLAZED: bool,
pub YARN: bool,
pub STOCKPILE_THREAD_METAL: bool,
pub IS_METAL: bool,
pub IS_GLASS: bool,
pub CRYSTAL_GLASSABLE: bool,
pub ITEMS_WEAPON: bool,
pub ITEMS_WEAPON_RANGED: bool,
pub ITEMS_ANVIL: bool,
pub ITEMS_AMMO: bool,
pub ITEMS_DIGGER: bool,
pub ITEMS_ARMOR: bool,
pub ITEMS_DELICATE: bool,
pub ITEMS_SIEGE_ENGINE: bool,
pub ITEMS_QUERN: bool,
}
}
I'm going to have other big ass structs, too. I want to add a function to that struct and others that will let me easily set any of those flags. It'll save me from writing many lines of code.
Instead of doing something like this:
let mut obj = material_template::default();
match key {
"IMPLIES_ANIMAL_KILL" => {
obj.MATERIAL_TOKENS.IMPLIES_ANIMAL_KILL = true;
}
"ALCOHOL_PLANT" => {
obj.MATERIAL_TOKENS.ALCOHOL_PLANT = true;
}
// etc.
}
I would much rather do something like the following, and handle all those cases with a single call:
let mut obj = material_template::default();
let success = obj.MATERIAL_TOKENS.set_token(key);
Before I go through the macro I'd like to give credit for some of the inspiration to the bitflags crate. It would've taken me longer without using that code as an example. I also used The Little Book of Rust Macros.
As you can see from the struct code above, the macro wraps around the entire struct. That's because I need access to each field and also need to know where the struct ends (so I can put the impl there).
Here's my macro:
// modify struct to add set_token() for easily setting tokens/errors
#[macro_export]
macro_rules! set_token {
($(#[$outer:meta])*
$sv:vis struct $name:ident {
$($fv:vis $field_name:ident: $field_type:ty,)*
}) => {
$(#[$outer])*
$sv struct $name {
$($fv $field_name: $field_type,)*
}
impl $name {
pub fn set_token(&mut self, token: &str) -> bool {
match token {
$(stringify!($field_name) => { self.$field_name = true; true })*
_ => { false }
}
}
}
}
}
A macro is split into two parts: ($matcher) => {$expansion}. There are 5 lines above with the $(...)* pattern: 2 in the matcher and 3 in the expansion section. They will each match or expand 0 or more times.
The only lines which can repeat in the expansion above are the ones that follow that pattern.
- First line of matcher:
($(#[$outer:meta])*matches any attributes.$outermatchescfg_attr(feature = "serde", derive(Serialize, Deserialize))andderive(Default)from the struct above. - On the next line
$sv:vis struct $name:ident {,$svmatchespuband$namematchesmaterial_tokens. - Last line of matcher:
$($fv:vis $field_name:ident: $field_type:ty,)*matches each field.$fvfor first match ispub,$field_nameisIMPLIES_ANIMAL_KILL, and$field_typeisbool - First line of expansion:
$(#[$outer])*expands back to the attributes from above, unchanged. $sv struct $name {expands topub struct material_tokens {$($fv $field_name: $field_type,)*duplicates each field. First iteration expands topub IMPLIES_ANIMAL_KILL: bool,impl $name {expands toimpl material_tokens {$(stringify!($field_name) => { self.$field_name = true; true })*expands to the fields matched above. stringify! macro creates a&str. First iteration expands to"IMPLIES_ANIMAL_KILL" => { self.IMPLIES_ANIMAL_KILL = true; true }
Here's what a snippet of the expanded struct looks like:
/// Recursive expansion of set_token! macro
// ========================================
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Default, PartialEq)]
pub struct material_tokens {
pub IMPLIES_ANIMAL_KILL: bool,
pub ALCOHOL_PLANT: bool,
// snip
pub ITEMS_QUERN: bool,
}
impl material_tokens {
pub fn set_token(&mut self, token: &str) -> bool {
match token {
"IMPLIES_ANIMAL_KILL" => {
self.IMPLIES_ANIMAL_KILL = true;
true
}
"ALCOHOL_PLANT" => {
self.ALCOHOL_PLANT = true;
true
}
// snip
"ITEMS_QUERN" => {
self.ITEMS_QUERN = true;
true
}
_ => false,
}
}
}
I use Visual Studio Code for development, and the rust-analyzer plugin has a pretty decent macro expander. It produced the output above. To use it right-click in the IDE and select Command Palette... or press Ctrl+Shift+P, start typing in macro and you should see a match for rust-analyzer: Expand macro recursively.
You can set a hotkey for it right there, too.
After my first test here's the relevant output from my parser:
INFO [dfraw_parse] material_template_ = ../vanilla\material_template_default.txt
INFO [dfraw_parse::material_template] parsing started after line: 3
ERROR [example] errors found: 14
00001 line number: 448, unknown MATERIAL_TOKEN: 'CARTILAGE'
00002 line number: 539, unknown MATERIAL_TOKEN: 'FEATHER'
00003 line number: 584, unknown MATERIAL_TOKEN: 'SCALE'
00004 line number: 772, unknown MATERIAL_TOKEN: 'NERVOUS_TISSUE'
00005 line number: 1475, unknown MATERIAL_TOKEN: 'HOOF'
00006 line number: 1983, unknown MATERIAL_TOKEN: 'CHITIN'
00007 line number: 2484, unknown MATERIAL_TOKEN: 'SYNDROME'
00008 line number: 2488, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
00009 line number: 2489, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
00010 line number: 2490, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'
00011 line number: 3014, unknown MATERIAL_TOKEN: 'SYNDROME'
00012 line number: 3018, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
00013 line number: 3019, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
00014 line number: 3020, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'
Looks like I missed a few materials, and now I need to parse syndromes. I finally feel like I'm writing nearly proper Rust.
added: 2023-08-29
Same macro as above, but proc_macro_derive type instead
I used a macro_rules! above because it was the first macro type I learned. However, I recently needed to make a custom derive macro, so I converted this macro at the same time.
I followed the guide here for the initial setup. It involves creating a separate crate and using dependencies like the syn and quote crates to help.
I'm not going to repeat a lot of stuff here, here's the macro above as a proc_macro_derive:
use proc_macro::TokenStream;
use quote::quote;
use syn::{Data, DeriveInput, Fields, Type};
// set bool fields on struct by string, return true on success
#[proc_macro_derive(SetToken)]
pub fn set_token_derive(input: TokenStream) -> TokenStream {
let ast = syn::parse(input).unwrap();
impl_set_token(&ast)
}
fn impl_set_token(ast: &DeriveInput) -> TokenStream {
let name = &ast.ident;
let mut matches = quote!();
match &ast.data {
Data::Struct(ref data) => {
match &data.fields {
Fields::Named(fields) => {
for f in &fields.named {
match &f.ty {
Type::Path(p) => {
match p.path.segments.first() {
Some(s) => {
match s.ident.to_string().as_str() {
"bool" => {
match &f.ident {
Some(i) => {
let s = i.to_string();
matches.extend(quote!{
#s => {
self.#i = true;
true
}
});
}
_ => {}
}
}
_ => {}
}
}
_=> {}
}
}
_ => {}
}
}
}
_ => {
}
}
}
_ => {
panic!("struct only!");
}
};
quote! {
impl SetToken for #name {
fn set_token(&mut self, token: &str) -> bool {
match token {
#matches
_ => {
false
}
}
}
}
}.into()
}
SetToken trait:
pub trait SetToken {
fn set_token(&mut self, token: &str) -> bool;
}
Holy shit, I thought the first macro was painful to read.
The differences between this macro and the previous one:
- This macro only generates matches for
boolstruct fields. - Used with
#[derive(SetToken)]attribute instead of using!set_tokenmacro