updated: 2023-08-29

This was just a Rust macro_rules! macro example before. I've since appended a proc_macro_derive example.

OpenGL code has been mostly done, but I moved on to something else for now: Dwarf Fortress RAWs. There are around 220 text files for configuring Dwarf Fortress. There's a lot there, it's the soul of DF, and it's in the public domain. Fuck yah!

I've been working on a parser for the raw files, and it finally forced me to learn Rust macros. Here's a quick post where I'm going to step thru a macro I just made line by line, because the internet needs more Rust macro examples. Even if it doesn't, too late.

First, the problem I wanted to solve. I have a giant struct that looks like this:

set_token! {
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Default, PartialEq)]
pub struct material_tokens {
    pub IMPLIES_ANIMAL_KILL: bool,
    pub ALCOHOL_PLANT: bool,
    pub ALCOHOL_CREATURE: bool,
    pub CHEESE_PLANT: bool,
    pub CHEESE_CREATURE: bool,
    pub POWDER_MISC_PLANT: bool,
    pub POWDER_MISC_CREATURE: bool,
    pub STOCKPILE_GLOB: bool,
    pub STOCKPILE_GLOB_PASTE: bool,
    pub STOCKPILE_GLOB_PRESSED: bool,
    pub STOCKPILE_PLANT_GROWTH: bool,
    pub LIQUID_MISC_PLANT: bool,
    pub LIQUID_MISC_CREATURE: bool,
    pub LIQUID_MISC_OTHER: bool,
    pub STRUCTURAL_PLANT_MAT: bool,
    pub SEED_MAT: bool,
    pub BONE: bool,
    pub WOOD: bool,
    pub THREAD_PLANT: bool,
    pub TOOTH: bool,
    pub HORN: bool,
    pub HAIR: bool,
    pub PEARL: bool,
    pub SHELL: bool,
    pub LEATHER: bool,
    pub SILK: bool,
    pub SOAP: bool,
    pub GENERATES_MIASMA: bool,
    pub MEAT: bool,
    pub ROTS: bool,
    pub BLOOD_MAP_DESCRIPTOR: bool,
    pub ICHOR_MAP_DESCRIPTOR: bool,
    pub GOO_MAP_DESCRIPTOR: bool,
    pub SLIME_MAP_DESCRIPTOR: bool,
    pub PUS_MAP_DESCRIPTOR: bool,
    pub SWEAT_MAP_DESCRIPTOR: bool,
    pub TEARS_MAP_DESCRIPTOR: bool,
    pub SPIT_MAP_DESCRIPTOR: bool,
    pub EVAPORATES: bool,
    pub ENTERS_BLOOD: bool,
    pub EDIBLE_VERMIN: bool,
    pub EDIBLE_RAW: bool,
    pub EDIBLE_COOKED: bool,
    pub DO_NOT_CLEAN_GLOB: bool,
    pub NO_STONE_STOCKPILE: bool,
    pub ITEMS_METAL: bool,
    pub ITEMS_BARRED: bool,
    pub ITEMS_SCALED: bool,
    pub ITEMS_LEATHER: bool,
    pub ITEMS_SOFT: bool,
    pub ITEMS_HARD: bool,
    pub IS_STONE: bool,
    pub UNDIGGABLE: bool,
    pub DISPLAY_UNGLAZED: bool,
    pub YARN: bool,
    pub STOCKPILE_THREAD_METAL: bool,
    pub IS_METAL: bool,
    pub IS_GLASS: bool,
    pub CRYSTAL_GLASSABLE: bool,
    pub ITEMS_WEAPON: bool,
    pub ITEMS_WEAPON_RANGED: bool,
    pub ITEMS_ANVIL: bool,
    pub ITEMS_AMMO: bool,
    pub ITEMS_DIGGER: bool,
    pub ITEMS_ARMOR: bool,
    pub ITEMS_DELICATE: bool,
    pub ITEMS_SIEGE_ENGINE: bool,
    pub ITEMS_QUERN: bool,
}
}

I'm going to have other big ass structs, too. I want to add a function to that struct and others that will let me easily set any of those flags. It'll save me from writing many lines of code.

Instead of doing something like this:

let mut obj = material_template::default();
match key {
	"IMPLIES_ANIMAL_KILL" => { 
		obj.MATERIAL_TOKENS.IMPLIES_ANIMAL_KILL = true; 
	}
	"ALCOHOL_PLANT" => { 
		obj.MATERIAL_TOKENS.ALCOHOL_PLANT = true; 
	}
	// etc.
}

I would much rather do something like the following, and handle all those cases with a single call:

let mut obj = material_template::default();
let success = obj.MATERIAL_TOKENS.set_token(key);

Before I go through the macro I'd like to give credit for some of the inspiration to the bitflags crate. It would've taken me longer without using that code as an example. I also used The Little Book of Rust Macros. As you can see from the struct code above, the macro wraps around the entire struct. That's because I need access to each field and also need to know where the struct ends (so I can put the impl there).

Here's my macro:

// modify struct to add set_token() for easily setting tokens/errors
#[macro_export]
macro_rules! set_token {
    ($(#[$outer:meta])*
    $sv:vis struct $name:ident {
        $($fv:vis $field_name:ident: $field_type:ty,)*
    }) => {
        $(#[$outer])*
        $sv struct $name {
            $($fv $field_name: $field_type,)*
        }
        impl $name {
            pub fn set_token(&mut self, token: &str) -> bool {
                match token {
                    $(stringify!($field_name) => { self.$field_name = true; true })*
                    _ => { false }
                }
            }
        }
    }
}

A macro is split into two parts: ($matcher) => {$expansion}. There are 5 lines above with the $(...)* pattern: 2 in the matcher and 3 in the expansion section. They will each match or expand 0 or more times. The only lines which can repeat in the expansion above are the ones that follow that pattern.

• First line of matcher: ($(#[$outer:meta])* matches any attributes. $outer matches cfg_attr(feature = "serde", derive(Serialize, Deserialize)) and derive(Default) from the struct above.
• On the next line $sv:vis struct $name:ident {, $sv matches pub and $name matches material_tokens.
• Last line of matcher: $($fv:vis $field_name:ident: $field_type:ty,)* matches each field. $fv for first match is pub, $field_name is IMPLIES_ANIMAL_KILL, and $field_type is bool
• First line of expansion: $(#[$outer])* expands back to the attributes from above, unchanged.
• $sv struct $name { expands to pub struct material_tokens {
• $($fv $field_name: $field_type,)* duplicates each field. First iteration expands to pub IMPLIES_ANIMAL_KILL: bool,
• impl $name { expands to impl material_tokens {
• $(stringify!($field_name) => { self.$field_name = true; true })* expands to the fields matched above. stringify! macro creates a &str. First iteration expands to "IMPLIES_ANIMAL_KILL" => { self.IMPLIES_ANIMAL_KILL = true; true }

Here's what a snippet of the expanded struct looks like:

/// Recursive expansion of set_token! macro
// ========================================

#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
#[derive(Default, PartialEq)]
pub struct material_tokens {
    pub IMPLIES_ANIMAL_KILL: bool,
    pub ALCOHOL_PLANT: bool,
    // snip
    pub ITEMS_QUERN: bool,
}
impl material_tokens {
    pub fn set_token(&mut self, token: &str) -> bool {
        match token {
            "IMPLIES_ANIMAL_KILL" => {
                self.IMPLIES_ANIMAL_KILL = true;
                true
            }
            "ALCOHOL_PLANT" => {
                self.ALCOHOL_PLANT = true;
                true
            }
            // snip
            "ITEMS_QUERN" => {
                self.ITEMS_QUERN = true;
                true
            }
            _ => false,
        }
    }
}

I use Visual Studio Code for development, and the rust-analyzer plugin has a pretty decent macro expander. It produced the output above. To use it right-click in the IDE and select Command Palette... or press Ctrl+Shift+P, start typing in macro and you should see a match for rust-analyzer: Expand macro recursively. You can set a hotkey for it right there, too.

After my first test here's the relevant output from my parser:

INFO  [dfraw_parse] material_template_ = ../vanilla\material_template_default.txt
INFO  [dfraw_parse::material_template] parsing started after line: 3
ERROR [example] errors found: 14
00001 line number: 448, unknown MATERIAL_TOKEN: 'CARTILAGE'
00002 line number: 539, unknown MATERIAL_TOKEN: 'FEATHER'
00003 line number: 584, unknown MATERIAL_TOKEN: 'SCALE'
00004 line number: 772, unknown MATERIAL_TOKEN: 'NERVOUS_TISSUE'
00005 line number: 1475, unknown MATERIAL_TOKEN: 'HOOF'
00006 line number: 1983, unknown MATERIAL_TOKEN: 'CHITIN'
00007 line number: 2484, unknown MATERIAL_TOKEN: 'SYNDROME'
00008 line number: 2488, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
00009 line number: 2489, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
00010 line number: 2490, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'
00011 line number: 3014, unknown MATERIAL_TOKEN: 'SYNDROME'
00012 line number: 3018, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
00013 line number: 3019, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
00014 line number: 3020, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'

Looks like I missed a few materials, and now I need to parse syndromes. I finally feel like I'm writing nearly proper Rust.

added: 2023-08-29

Same macro as above, but proc_macro_derive type instead

I used a macro_rules! above because it was the first macro type I learned. However, I recently needed to make a custom derive macro, so I converted this macro at the same time.

I followed the guide here for the initial setup. It involves creating a separate crate and using dependencies like the syn and quote crates to help.

I'm not going to repeat a lot of stuff here, here's the macro above as a proc_macro_derive:

use proc_macro::TokenStream;
use quote::quote;
use syn::{Data, DeriveInput, Fields, Type};

// set bool fields on struct by string, return true on success
#[proc_macro_derive(SetToken)]
pub fn set_token_derive(input: TokenStream) -> TokenStream {
    let ast = syn::parse(input).unwrap();
    impl_set_token(&ast)
}

fn impl_set_token(ast: &DeriveInput) -> TokenStream {
    let name = &ast.ident;
    let mut matches = quote!();
    match &ast.data {
        Data::Struct(ref data) => {
            match &data.fields {
                Fields::Named(fields) => {
                    for f in &fields.named {
                        match &f.ty {
                            Type::Path(p) => {
                                match p.path.segments.first() {
                                    Some(s) => {
                                        match s.ident.to_string().as_str() {
                                            "bool" => {
                                                match &f.ident {
                                                    Some(i) => {
                                                        let s = i.to_string();
                                                        matches.extend(quote!{
                                                                #s => { 
                                                                    self.#i = true; 
                                                                    true
                                                                }
                                                        });
                                                    }
                                                    _ => {}
                                                }
                                            }
                                            _ => {}
                                        }
                                    }
                                    _=> {}
                                }
                            }
                            _ => {}
                        }
                    }
                }
                _ => {
                }
            }
        }
        _ => {
            panic!("struct only!");
        }
    };
    quote! {
        impl SetToken for #name {
            fn set_token(&mut self, token: &str) -> bool {
                match token {
                    #matches
                    _ => {  
                        false
                    }
                }
            }
        }
    }.into()
}

SetToken trait:

pub trait SetToken {
    fn set_token(&mut self, token: &str) -> bool;
}

Holy shit, I thought the first macro was painful to read.

The differences between this macro and the previous one:

• This macro only generates matches for bool struct fields.
• Used with #[derive(SetToken)] attribute instead of using !set_token macro

I've made some progress learning OpenGL. For tile locations on screen, I calculate all the vertex positions per tile (2 triangles per tile, 6 vertices per triangle = 12 vertices) and store them in a vertex buffer object. I add the 0.0 Z coordinate in the shader, so that doesn't need to be stored, at least.

Here's what the geometry looks like for a 1920x1200 display configured for 32x32 pixel tiles:

Screenshot is from NVIDIA Nsight Graphics, a pretty awesome tool for a beginner like me. RenderDoc is great, too, but Nsight Graphics was able to immediately tell me what was wrong with my render pipeline. (Remember to bind the vertex array every draw call)

Here's the code for calculating the tile vertices:

let mut tile = 0.0;
loop {
    let verts = get_tex_vertices(512.0, 512.0, 32.0, 32.0, tile);        
    //upper right        
    vertices.push(xf + 32.0);        
    vertices.push(yf + 32.0);        
    //upper left        
    vertices.push(xf);        
    vertices.push(yf + 32.0);        
    //lower left        
    vertices.push(xf);        
    vertices.push(yf);        
    //upper right        
    vertices.push(xf + 32.0);        
    vertices.push(yf + 32.0);        
    //lower left        
    vertices.push(xf);        
    vertices.push(yf);        
    //lower right        
    vertices.push(xf + 32.0);        
    vertices.push(yf);        

    uvs.extend_from_slice(&verts);        
    tile += 1.0;        
    if tile > 255.0 {        
        tile = 0.0;        
    }        
    xf += 32.0;        
    if xf > 1920.0 {        
        xf = 0.0;        
        yf += 32.0;        
    }        
    if yf > 1200.0 {        
        break;        
    }        
}                

Screen resolution is temporarily hard coded so that I could check the geometry without running fullscreen. Generally, the vertex shader wants vertices in normalized device coordinates which range from -1.0,-1.0 to 1.0,1.0. Right now I'm normalizing the coordinates in the shader like so:

#version 330
layout (location = 0) in vec2 vert;
layout (location = 1) in vec2 _uv;
out vec2 uv;
void main()
{
    uv = _uv;
    gl_Position = vec4((vert.x / 1920.0 * 2.0) - 1.0, (vert.y / 1200.0 * 2.0) - 1.0, 0.0, 1.0);
};

But when I rewrite my shaders in a moment I'll just handle it on the CPU since I only need to calculate those vertices once.

Next I had to figure out how to access individual sprites in a sprite sheet (also called a texture atlas). I'm targeting 32x32 tile size, so I looked for an example to do testing with. I found a cool tileset for Dwarf Fortress by Vettlingr.

Here's the tileset:

Note: I'm not planning on using these graphics officially, it's just for demonstration purposes.

The image is 512x512 pixels. Each tile is 32x32 pixels, which means there are 16x16 (or 256) tiles total. In the code above I call get_tex_vertices() to generate the texture coordinates for each tile. Textures use yet another coordinate system that ranges from 0.0,0.0 to 1.0,1.0.

Here's the code for calculating texture coordinates based on texture size, sprite size, and sprite index:

// tile index = 0 from left to right, top to bottom.
fn get_tex_vertices(        
    sheet_w: f32,        
    sheet_h: f32,        
    tile_w: f32,        
    tile_h: f32,        
    tile_index: f32,        
) -> [f32; 12] {        
    let mut vertices: [f32; 12] = [0.0; 12];        
    let cols = sheet_w / tile_w;        
    let x = tile_index % cols;        
    let y = (tile_index / cols).floor();        
    //invert y by default        
    //lower right        
    vertices[0] = (x * tile_w + tile_w) / sheet_w;        
    vertices[1] = (y * tile_h) / sheet_h;        
    //lower left        
    vertices[2] = (x * tile_w) / sheet_w;        
    vertices[3] = (y * tile_h) / sheet_h;        
    //upper left        
    vertices[4] = (x * tile_w) / sheet_w;        
    vertices[5] = (y * tile_h + tile_h) / sheet_h;        
    //lower right        
    vertices[6] = (x * tile_w + tile_w) / sheet_w;        
    vertices[7] = (y * tile_h) / sheet_h;        
    //upper left        
    vertices[8] = (x * tile_w) / sheet_w;        
    vertices[9] = (y * tile_h + tile_h) / sheet_h;        
    //upper right        
    vertices[10] = (x * tile_w + tile_w) / sheet_w;        
    vertices[11] = (y * tile_h + tile_h) / sheet_h;        
    vertices        
}        

This code automatically inverts the y coordinates. That's because there are more coordinate systems fighting each other here. Unless configured differently, drawing in OpenGL at the origin (0,0) usually means the lower left corner of the screen. Many image formats have their 0,0 at the upper left of the screen. Instead of rotating lots of image data around in memory, I'll rotate the texture coordinates instead.

Here's the vertex geometry of the sprite sheet visualized:

I put together a few transforms for texture coordinates. This lets me pre-calculate flipped, rotated, mirrored, etc. texture coordinates for animation later on.

Here are a few examples:

fn transform_uvs(
    uvs: &mut [f32; 12],
    sheet_w: f32,
    sheet_h: f32,
    tile_w: f32,
    tile_h: f32,
    tile_index: f32,
    transform: Transforms,
) {
    let cols = sheet_w / tile_w;
    if transform == Transforms::FlipVertical {
        let y = (tile_index / cols).floor();
        let y_low = (y * tile_h) / sheet_h;
        let y_high = (y * tile_h + tile_h) / sheet_h;
        let mut flip = |index: usize| {
            if uvs[index] == y_low {
                uvs[index] = y_high;
            } else {
                uvs[index] = y_low;
            }
        };
        // odd vertices = y coordinates
        for i in (1..=11).step_by(2) {
            flip(i);
        }
    }
    if transform == Transforms::FlipHorizontal {
        let x = tile_index % cols;
        let x_low = (x * tile_w) / sheet_w;
        let x_high = (x * tile_w + tile_w) / sheet_w;
        let mut flip = |index: usize| {
            if uvs[index] == x_low {
                uvs[index] = x_high;
            } else {
                uvs[index] = x_low;
            }
        };
        //even vertices = x coordinates
        for i in (0..=10).step_by(2) {
            flip(i);
        }
    }
    if transform == Transforms::Rotate90Left {
        let x = tile_index % cols;
        let y = (tile_index / cols).floor();
        uvs[0] = (x * tile_w + tile_w) / sheet_w;
        uvs[1] = (y * tile_h + tile_h) / sheet_h;
        uvs[2] = (x * tile_w + tile_w) / sheet_w;
        uvs[3] = (y * tile_h) / sheet_h;
        uvs[4] = (x * tile_w) / sheet_w;
        uvs[5] = (y * tile_h) / sheet_h;
        uvs[6] = (x * tile_w + tile_w) / sheet_w;
        uvs[7] = (y * tile_h + tile_h) / sheet_h;
        uvs[8] = (x * tile_w) / sheet_w;
        uvs[9] = (y * tile_h) / sheet_h;
        uvs[10] = (x * tile_w) / sheet_w;
        uvs[11] = (y * tile_h + tile_h) / sheet_h;
    }
    if transform == Transforms::Rotate90LeftHMirror {
        let x = tile_index % cols;
        let y = (tile_index / cols).floor();
        uvs[0] = (x * tile_w + tile_w) / sheet_w;
        uvs[1] = (y * tile_h) / sheet_h;
        uvs[2] = (x * tile_w + tile_w) / sheet_w;
        uvs[3] = (y * tile_h + tile_h) / sheet_h;
        uvs[4] = (x * tile_w) / sheet_w;
        uvs[5] = (y * tile_h + tile_h) / sheet_h;
        uvs[6] = (x * tile_w + tile_w) / sheet_w;
        uvs[7] = (y * tile_h) / sheet_h;
        uvs[8] = (x * tile_w) / sheet_w;
        uvs[9] = (y * tile_h + tile_h) / sheet_h;
        uvs[10] = (x * tile_w) / sheet_w;
        uvs[11] = (y * tile_h) / sheet_h;
    }
} 

This math is specific to the coordinates I generated before. It's also specific for my vertex buffer layout. It's just there to give an idea on how to approach the problem for other people trying to learn how to 2d graphics in 2023 =)

Next post I'm going to finish the shaders, allowing me to update tile indexes on demand and render many layers. I'm also going to add in palette support. My near term goal is to import the Dwarf Fortress raws and basically duplicate the rendering system.

Here's one last picture showing my title screen with the Vettlingr tiles in the background: