Attempting to crack NIST curve seeds

2023-10-09T00:00:00+00:00

Nothing game

A few

The challenge I put together

Tt's

code:

change change MUT_LEN change STRING_VARIATIONS

use </span>use </span> </span>extern </span>use </span>use </span>use std::thread; </span> </span> </span> </span> </span> </span> </span>            { </span> </span>            } </span> </span> </span> </span>            } </span>        } </span> </span> </span>        } </span>    } </span>    found </span>} </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>    } </span>} </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>} </span> </span> </span>        .write(true) </span>        .append(true) </span>        .create(true) </span>        .open(path) </span>        .unwrap(); </span> </span> </span> </span> </span> </span>} </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>    ]; </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>                } </span> </span>            } </span> </span> </span> </span> </span>            } </span> </span> </span>        } </span>    } </span> </span> </span>} </span> </span> </span> </span> </span> </span>    } </span>    result </span>} </span> </span> </span> </span> </span> </span> </span> </span>        } </span>    } </span>    result </span>} </span> </span> </span> </span> </span> </span> </span> </span>        } </span> </span> </span> </span>    } </span> </span> </span>} </span> </span> </span> </span> </span> </span> </span>    } </span> </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span>        } </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>        } </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span>                } </span>            } </span> </span> </span> </span> </span> </span>                } </span>            } </span>        } </span>    }); </span>} </span> </span></code></pre>

Rust
2023-09-09T00:00:00+00:00 related this time, just a little distraction. days ago I came across the page here</a>. is to figure out the phrase that Jerry Solinas used to help derive some NIST curve seeds. a quick little attempt in Rust. I didn't find shit, but the code is here</a> if anyone wants to try their luck. not the fastest thing in the world (55 million/sec vs hashcat's 150 million/sec on my machine), but it calculates the hashes a lot faster than I can come up with sane guesses, so there's that. MAX_THREADS to match your hardware. if you modify number of string mutations. when you change the string templates (that's the whole point, right?) style="background-color:#151515;color:#e8e8d3;" class="language-rust ">use std::{fs::{self}, io::{BufReader, BufRead}, sync::atomic::{AtomicBool, Ordering, AtomicU64}, time, env}; std::io::prelude::; std::fs::OpenOptions; #[macro_use(concat_string)] crate concat_string; sha1::{Sha1, Digest, digest::{generic_array::GenericArray, typenum::U20}}; arrayvec::ArrayString; style="color:#888888;">// max string length, adjust as needed style="color:#8fbfdc;">const LEN: usize = 64; style="color:#888888;">// largest number to append to strings style="color:#8fbfdc;">const MAX_COUNT: i16 = 3333; style="color:#888888;">// adjust as needed, 3 minimum because i'm lazy style="color:#8fbfdc;">const MAX_THREADS: usize = 17; style="color:#8fbfdc;">const ABOOL: AtomicBool = AtomicBool::new(false); style="color:#8fbfdc;">const AZERO: AtomicU64 = AtomicU64::new(0); style="color:#8fbfdc;">static mut FINISHED: [AtomicBool; MAX_THREADS - 1] = [ABOOL; MAX_THREADS - 1]; style="color:#8fbfdc;">static mut TOTAL: [AtomicU64; MAX_THREADS - 1] = [AZERO; MAX_THREADS - 1]; #[inline] style="color:#8fbfdc;">fn check_result(input: &GenericArray<u8,U20>, nist: &[[u8;20];NIST_LEN]) -> bool { let mut found = false; for h in nist { for x in 0..20 { if h[x] == input[x] found = true; else { found = false; break; if found { break; style="color:#888888;">// this has to match max number of strings generated in mutate_string() below style="color:#8fbfdc;">const MUT_LEN: usize = 24; #[inline] style="color:#8fbfdc;">fn mutate_string(input: &str, counter: i16, output: &mut [ArrayString<LEN>;MUT_LEN STRING_VARIATIONS], index: usize) { // example: Jerry and Satan deserve a raise666. if counter != -1 { // ascii numbers let count = counter.to_string(); let tmp = concat_string!(input, count); output[index * MUT_LEN + 0].push_str(&tmp); output[index * MUT_LEN + 1].push_str(&tmp); output[index * MUT_LEN + 1].make_ascii_lowercase(); output[index * MUT_LEN + 2].push_str(&tmp); output[index * MUT_LEN + 2].make_ascii_uppercase(); // i don't like this one let tmp = concat_string!(input, count, "."); output[index * MUT_LEN + 3].push_str(&tmp); output[index * MUT_LEN + 4].push_str(&tmp); output[index * MUT_LEN + 4].make_ascii_lowercase(); output[index * MUT_LEN + 5].push_str(&tmp); output[index * MUT_LEN + 5].make_ascii_uppercase(); let tmp = concat_string!(input, ".", count); output[index * MUT_LEN + 6].push_str(&tmp); output[index * MUT_LEN + 7].push_str(&tmp); output[index * MUT_LEN + 7].make_ascii_lowercase(); output[index * MUT_LEN + 8].push_str(&tmp); output[index * MUT_LEN + 8].make_ascii_uppercase(); let tmp = concat_string!(input, "!", count); output[index * MUT_LEN + 9].push_str(&tmp); output[index * MUT_LEN + 10].push_str(&tmp); output[index * MUT_LEN + 10].make_ascii_lowercase(); output[index * MUT_LEN + 11].push_str(&tmp); output[index * MUT_LEN + 11].make_ascii_uppercase(); // binary numbers let bin = counter.to_le_bytes(); let sbin = unsafe { core::str::from_utf8_unchecked(&bin) }; let tmp = concat_string!(input, sbin); output[index * MUT_LEN + 12].push_str(&tmp); output[index * MUT_LEN + 13].push_str(&tmp); output[index * MUT_LEN + 13].make_ascii_lowercase(); output[index * MUT_LEN + 14].push_str(&tmp); output[index * MUT_LEN + 14].make_ascii_uppercase(); // still don't like this one let tmp = concat_string!(input, sbin, "."); output[index * MUT_LEN + 15].push_str(&tmp); output[index * MUT_LEN + 16].push_str(&tmp); output[index * MUT_LEN + 16].make_ascii_lowercase(); output[index * MUT_LEN + 17].push_str(&tmp); output[index * MUT_LEN + 17].make_ascii_uppercase(); let tmp = concat_string!(input, "!", sbin); output[index * MUT_LEN + 18].push_str(&tmp); output[index * MUT_LEN + 19].push_str(&tmp); output[index * MUT_LEN + 19].make_ascii_lowercase(); output[index * MUT_LEN + 20].push_str(&tmp); output[index * MUT_LEN + 20].make_ascii_uppercase(); let tmp = concat_string!(input, ".", sbin); output[index * MUT_LEN + 21].push_str(&tmp); output[index * MUT_LEN + 22].push_str(&tmp); output[index * MUT_LEN + 22].make_ascii_lowercase(); output[index * MUT_LEN + 23].push_str(&tmp); output[index * MUT_LEN + 23].make_ascii_uppercase(); // -1 means don't append a counter // example: Jerry and Satan deserve a raise. else { output[index * MUT_LEN + 0].push_str(&input); output[index * MUT_LEN + 1].push_str(&input); output[index * MUT_LEN + 1].make_ascii_lowercase(); output[index * MUT_LEN + 2].push_str(&input); output[index * MUT_LEN + 2].make_ascii_uppercase(); let tmp = concat_string!(input, "."); output[index * MUT_LEN + 3].push_str(&tmp); output[index * MUT_LEN + 4].push_str(&tmp); output[index * MUT_LEN + 4].make_ascii_lowercase(); output[index * MUT_LEN + 5].push_str(&tmp); output[index * MUT_LEN + 5].make_ascii_uppercase(); let tmp = concat_string!(input, "!"); output[index * MUT_LEN + 6].push_str(&tmp); output[index * MUT_LEN + 7].push_str(&tmp); output[index * MUT_LEN + 7].make_ascii_lowercase(); output[index * MUT_LEN + 8].push_str(&tmp); output[index * MUT_LEN + 8].make_ascii_uppercase(); style="color:#888888;">// has to match the number of string variations in get_strings() below style="color:#8fbfdc;">const STRING_VARIATIONS: usize = 16; #[inline] style="color:#8fbfdc;">fn get_strings(counter: i16, name: &str, mut output: &mut [ArrayString<LEN>;MUT_LEN * STRING_VARIATIONS]) { let mut index = 0; let s = concat_string!("Jerry and ", name, " deserve a raise"); // 1 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!(name, " and Jerry deserve a raise"); // 2 mutate_string(&s, counter, &mut output, index); index += 1; let s = "Jerry deserves a raise"; // 3 mutate_string(&s, counter, &mut output, index); index += 1; let s = "Jerry deserves a break"; // 4 mutate_string(&s, counter, &mut output, index); index += 1; let s = "Jerry needs a coffee"; // 5 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Jerry and ", name, " deserve raises"); // 6 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!(name, " and Jerry deserve raises"); // 7 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Jerry and ", name, " deserve promotions"); // 8 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!(name, " and Jerry deserve promotions"); // 9 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Jerry and ", name, " deserve a promotion"); // 10 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!(name, " and Jerry deserve a promotion"); // 11 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Give Jerry and ", name, " a raise"); // 12 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Give ", name, " and Jerry a raise"); // 13 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Give ", name, " and Jerry raises"); // 14 mutate_string(&s, counter, &mut output, index); index += 1; let s = concat_string!("Give Jerry and ", name, " raises"); // 15 mutate_string(&s, counter, &mut output, index); index += 1; let s = "Give Jerry a raise"; // 16 mutate_string(&s, counter, &mut output, index); #[inline] style="color:#8fbfdc;">fn log_found(msg: &[u8], path: &str) { let mut f = OpenOptions::new() // prints fucked up UTF-8 to console, saves actual bytes to log // might have to view log file with a hex editor println!("FOUND: '{}'", String::from_utf8_lossy(msg)); f.write(msg).expect("error writing file..."); f.write("\n".as_bytes()).expect("error writing file..."); style="color:#888888;">// update this if length of nist array below is modified style="color:#8fbfdc;">const NIST_LEN: usize = 5; #[inline] style="color:#8fbfdc;">fn worker(id: usize, names: &[String], start: i16, end: i16, log: &str) { let nist: [[u8;20];NIST_LEN] = [ //[1,125,225,238,87,121,14,202,36,38,70,93,194,152,145,244,32,50,251,70], // Jerry and Satan deserve a raise'0x05''0x0D'. //[38,232,14,202,25,250,218,187,145,59,48,16,185,82,241,75,189,87,53,117], // Jerry and Satan deserve a raise.3333 [48,69,174,111,200,66,47,100,237,87,149,40,211,129,32,234,225,33,150,213], // # NIST P-192, ANSI prime192v1 [189,113,52,71,153,213,199,252,220,69,181,159,163,185,171,143,106,148,139,197], // # NIST P-224 [196,157,54,8,134,231,4,147,106,102,120,225,19,157,38,183,129,159,126,144], // # NIST P-256, ANSI prime256v1 [163,53,146,106,163,25,162,122,29,0,137,106,103,115,164,130,122,205,172,115], // # NIST P-384 [208,158,136,0,41,28,184,83,150,204,103,23,57,50,132,170,160,218,100,186], // # NIST P-521 let mut output: [ArrayString<LEN>;MUT_LEN * STRING_VARIATIONS] = [ArrayString::<LEN>::new();MUT_LEN * STRING_VARIATIONS]; let mut digest; let mut total = 0; let mut update_counter = 0; for n in names { for x in start..=end { get_strings(x, n, &mut output); for y in 0..output.len() { digest = Sha1::digest(output[y].as_bytes()); if check_result(&digest, &nist) { log_found(output[y].as_bytes(), log); output[y].clear(); // update infos every 100,000 if update_counter > 100_000 { unsafe { TOTAL[id].store(total, Ordering::Relaxed) }; update_counter = 0; total += (MUT_LEN * STRING_VARIATIONS) as u64; update_counter += MUT_LEN * STRING_VARIATIONS; unsafe { TOTAL[id].store(total, Ordering::Release) }; unsafe { FINISHED[id].store(true, Ordering::Release) }; #[inline] style="color:#8fbfdc;">fn get_total() -> u64 { let mut result = 0; for x in 0..MAX_THREADS - 1 { unsafe { result += TOTAL[x].load(Ordering::Relaxed); } #[inline] style="color:#8fbfdc;">fn threads_done() -> bool { let mut result = false; for x in 0..MAX_THREADS - 1 { unsafe { result = FINISHED[x].load(Ordering::Acquire); } if !result { break; style="color:#8fbfdc;">fn watcher(total_hashes: u64) { let mut total; let mut last_total: u64 = 0; loop { thread::sleep(time::Duration::from_secs(1)); if threads_done() { break; total = get_total(); println!("{} hashes per second, {:.0}% done", total - last_total, (total as f32 / total_hashes as f32) * 100.0); last_total = total; total = get_total(); println!("Finished.\n{} total hashes checked.", total); style="color:#8fbfdc;">fn main() { let args: Vec<_> = env::args().collect(); if args.len() != 2 && args.len() != 3 { println!("Usage: {} wordlist logfile(optional)", args[0]); println!("Matches are printed to console and saved to output.log by default"); println!("Example: {} names.txt", args[0]); return; let mut log = "output.log"; if args.len() == 3 { log = &args[2]; let file = fs::File::open(&args[1]).unwrap(); let reader = BufReader::new(file); let mut names: Vec<String> = vec![]; for line in reader.lines() { if let Ok(l) = line { names.push(l); } else if let Err(e) = line { println!("error reading wordlist: {}", e.to_string()); println!("{} names loaded from wordlist", names.len()); println!{"{} threads to be spun up...", MAX_THREADS}; let mut start: i16 = -1; // assign each thread a number range // (thread id, start number, end number) let mut thread_info: Vec<(usize, i16, i16)> = vec![]; for x in 0..MAX_THREADS { let mut c = MAX_COUNT / (MAX_THREADS - 1) as i16; if x == MAX_THREADS - 2 { c = MAX_COUNT - start; thread_info.push((x, start, start + c)); start += c + 1; let total_hashes = MUT_LEN as u64 * STRING_VARIATIONS as u64 * (MAX_COUNT + 2) as u64 * names.len() as u64; thread::scope(|s| { for x in thread_info.iter() { if x.0 == MAX_THREADS - 1 { let builder = thread::Builder::new(); let res = builder.spawn_scoped(s, || watcher(total_hashes) ); if res.is_err() { println!("error spinning up watcher: {}", res.err().unwrap()); else { let builder = thread::Builder::new(); let res = builder.spawn_scoped(s, || worker(x.0, &names, x.1, x.2, log) ); if res.is_err() { println!("error spinning up worker: {}", res.err().unwrap()); crate for setting struct fields by string


I just published a tiny little Rust crate.  It's the third iteration of my macro for setting struct fields.</p>

The crate is called set_field</a>.</p>
Example usage:</p>
use set_field::SetField;
</span>
</span>#[</span>derive</span>(SetField)]
</span>struct </span>Foo </span>{
</span>	</span>a</span>: </span>i32</span>,
</span>	</span>b</span>: Option<</span>bool</span>>,
</span>	</span>c</span>: </span>i32</span>,
</span>}
</span>fn </span>test</span>() {
</span>	</span>let mut</span> t = Foo { a: </span>777</span>, b: None, c: </span>0 </span>};
</span>	</span>// return true on success:
</span>	assert_eq!(t.set_field(</span>"</span>a</span>"</span>, </span>888</span>), true);
</span>	</span>// return true on success:
</span>	assert_eq!(t.set_field(</span>"</span>b</span>"</span>, Some(true)), true);
</span>	assert_eq!(t.a, </span>888</span>);
</span>	assert_eq!(t.b, Some(true));
</span>	</span>// return false on nonexistent field:
</span>	assert_eq!(t.set_field(</span>"</span>d</span>"</span>, </span>0</span>), false);
</span>	</span>// return false on wrong type:
</span>	assert_eq!(t.set_field(</span>"</span>b</span>"</span>, </span>0</span>), false);
</span>	</span>// won't compile:
</span>	</span>// assert_eq!(t.set_field("a", 0.0), false);
</span>}
</span></code></pre>

set_field returns true on success</li>
set_field returns false if field doesn't exist</li>
set_field returns false if you attempt to set a field to the wrong type</li>
</ul>
The SetField macro expands Foo into this: </p>
struct </span>Foo </span>{
</span>	</span>a</span>: </span>i32</span>,
</span>	</span>b</span>: Option<</span>bool</span>>,
</span>	</span>c</span>: </span>i32</span>,
</span>}
</span>impl </span>SetField<</span>i32</span>> for </span>Foo </span>{
</span>    </span>fn </span>set_field</span>(&</span>mut </span>self</span>, </span>field</span>: &</span>str</span>, </span>value</span>: </span>i32</span>) -> </span>bool </span>{
</span>        </span>match</span> field {
</span>            </span>"</span>a</span>" </span>=> {
</span>                </span>self</span>.a = value;
</span>                true
</span>            }
</span>            </span>"</span>c</span>" </span>=> {
</span>                </span>self</span>.c = value;
</span>                true
</span>            }
</span>            _ => false,
</span>        }
</span>    }
</span>}
</span>impl </span>SetField<Option<</span>bool</span>>> for </span>Foo </span>{
</span>    </span>fn </span>set_field</span>(&</span>mut </span>self</span>, </span>field</span>: &</span>str</span>, </span>value</span>: Option<</span>bool</span>>) -> </span>bool </span>{
</span>        </span>match</span> field {
</span>            </span>"</span>b</span>" </span>=> {
</span>                </span>self</span>.b = value;
</span>                true
</span>            }
</span>            _ => false,
</span>        }
</span>    }
</span>}
</span></code></pre>
dependencies:</p>

set_field_macro</a></li>
syn</a></li>
quote</a></li>
</ul>



Rust macro for matching on struct fields
2023-08-20T00:00:00+00:00
updated: 2023-08-29</strong></p>
This was just a Rust macro_rules! macro example before.</strong>
I've since appended a proc_macro_derive example.</strong></u></p>
OpenGL code has been mostly done, but I moved on to something else for now: Dwarf Fortress RAWs</a>.
There are around 220 text files for configuring Dwarf Fortress.  There's a lot there, it's the soul of DF, and it's in the public domain. Fuck yah!</p>
I've been working on a parser for the raw files, and it finally forced me to learn Rust macros</a>.
Here's a quick post where I'm going to step thru a macro I just made line by line, because the internet needs more Rust macro examples.  Even if it doesn't, too late.</p>
First, the problem I wanted to solve.  I have a giant struct that looks like this:</p>
set_token! {
</span>#[</span>cfg_attr</span>(feature = </span>"</span>serde</span>"</span>, </span>derive</span>(Serialize, Deserialize))]
</span>#[</span>derive</span>(Default, PartialEq)]
</span>pub struct </span>material_tokens </span>{
</span>    </span>pub </span>IMPLIES_ANIMAL_KILL</span>: </span>bool</span>,
</span>    </span>pub </span>ALCOHOL_PLANT</span>: </span>bool</span>,
</span>    </span>pub </span>ALCOHOL_CREATURE</span>: </span>bool</span>,
</span>    </span>pub </span>CHEESE_PLANT</span>: </span>bool</span>,
</span>    </span>pub </span>CHEESE_CREATURE</span>: </span>bool</span>,
</span>    </span>pub </span>POWDER_MISC_PLANT</span>: </span>bool</span>,
</span>    </span>pub </span>POWDER_MISC_CREATURE</span>: </span>bool</span>,
</span>    </span>pub </span>STOCKPILE_GLOB</span>: </span>bool</span>,
</span>    </span>pub </span>STOCKPILE_GLOB_PASTE</span>: </span>bool</span>,
</span>    </span>pub </span>STOCKPILE_GLOB_PRESSED</span>: </span>bool</span>,
</span>    </span>pub </span>STOCKPILE_PLANT_GROWTH</span>: </span>bool</span>,
</span>    </span>pub </span>LIQUID_MISC_PLANT</span>: </span>bool</span>,
</span>    </span>pub </span>LIQUID_MISC_CREATURE</span>: </span>bool</span>,
</span>    </span>pub </span>LIQUID_MISC_OTHER</span>: </span>bool</span>,
</span>    </span>pub </span>STRUCTURAL_PLANT_MAT</span>: </span>bool</span>,
</span>    </span>pub </span>SEED_MAT</span>: </span>bool</span>,
</span>    </span>pub </span>BONE</span>: </span>bool</span>,
</span>    </span>pub </span>WOOD</span>: </span>bool</span>,
</span>    </span>pub </span>THREAD_PLANT</span>: </span>bool</span>,
</span>    </span>pub </span>TOOTH</span>: </span>bool</span>,
</span>    </span>pub </span>HORN</span>: </span>bool</span>,
</span>    </span>pub </span>HAIR</span>: </span>bool</span>,
</span>    </span>pub </span>PEARL</span>: </span>bool</span>,
</span>    </span>pub </span>SHELL</span>: </span>bool</span>,
</span>    </span>pub </span>LEATHER</span>: </span>bool</span>,
</span>    </span>pub </span>SILK</span>: </span>bool</span>,
</span>    </span>pub </span>SOAP</span>: </span>bool</span>,
</span>    </span>pub </span>GENERATES_MIASMA</span>: </span>bool</span>,
</span>    </span>pub </span>MEAT</span>: </span>bool</span>,
</span>    </span>pub </span>ROTS</span>: </span>bool</span>,
</span>    </span>pub </span>BLOOD_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>ICHOR_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>GOO_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>SLIME_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>PUS_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>SWEAT_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>TEARS_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>SPIT_MAP_DESCRIPTOR</span>: </span>bool</span>,
</span>    </span>pub </span>EVAPORATES</span>: </span>bool</span>,
</span>    </span>pub </span>ENTERS_BLOOD</span>: </span>bool</span>,
</span>    </span>pub </span>EDIBLE_VERMIN</span>: </span>bool</span>,
</span>    </span>pub </span>EDIBLE_RAW</span>: </span>bool</span>,
</span>    </span>pub </span>EDIBLE_COOKED</span>: </span>bool</span>,
</span>    </span>pub </span>DO_NOT_CLEAN_GLOB</span>: </span>bool</span>,
</span>    </span>pub </span>NO_STONE_STOCKPILE</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_METAL</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_BARRED</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_SCALED</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_LEATHER</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_SOFT</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_HARD</span>: </span>bool</span>,
</span>    </span>pub </span>IS_STONE</span>: </span>bool</span>,
</span>    </span>pub </span>UNDIGGABLE</span>: </span>bool</span>,
</span>    </span>pub </span>DISPLAY_UNGLAZED</span>: </span>bool</span>,
</span>    </span>pub </span>YARN</span>: </span>bool</span>,
</span>    </span>pub </span>STOCKPILE_THREAD_METAL</span>: </span>bool</span>,
</span>    </span>pub </span>IS_METAL</span>: </span>bool</span>,
</span>    </span>pub </span>IS_GLASS</span>: </span>bool</span>,
</span>    </span>pub </span>CRYSTAL_GLASSABLE</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_WEAPON</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_WEAPON_RANGED</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_ANVIL</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_AMMO</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_DIGGER</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_ARMOR</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_DELICATE</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_SIEGE_ENGINE</span>: </span>bool</span>,
</span>    </span>pub </span>ITEMS_QUERN</span>: </span>bool</span>,
</span>}
</span>}
</span></code></pre>
I'm going to have other big ass structs, too.  I want to add a function to that struct and others that will let me easily set any of those flags.
It'll save me from writing many lines of code.</p>
Instead of doing something like this:</p>
let mut</span> obj = material_template::default();
</span>match</span> key {
</span>	</span>"</span>IMPLIES_ANIMAL_KILL</span>" </span>=> { 
</span>		obj.</span>MATERIAL_TOKENS</span>.</span>IMPLIES_ANIMAL_KILL </span>= true; 
</span>	}
</span>	</span>"</span>ALCOHOL_PLANT</span>" </span>=> { 
</span>		obj.</span>MATERIAL_TOKENS</span>.</span>ALCOHOL_PLANT </span>= true; 
</span>	}
</span>	</span>// etc.
</span>}
</span></code></pre>
I would much rather do something like the following, and handle all those cases with a single call:</p>
let mut</span> obj = material_template::default();
</span>let</span> success = obj.</span>MATERIAL_TOKENS</span>.set_token(key);
</span>
</span></code></pre>
Before I go through the macro I'd like to give credit for some of the inspiration to the bitflags</a> crate.  It would've taken me longer without using that code as an example.  I also used The Little Book of Rust Macros</a>.
As you can see from the struct code above, the macro wraps around the entire struct.  That's because I need access to each field and also need to know where the struct ends (so I can put the impl</code> there).</p>
Here's my macro:</p>
// modify struct to add set_token() for easily setting tokens/errors
</span>#[</span>macro_export</span>]
</span>macro_rules! </span>set_token </span>{
</span>    ($(#[</span>$outer</span>:</span>meta</span>])*
</span>    $sv:vis struct </span>$name</span>:</span>ident</span> {
</span>        $($fv:vis </span>$field_name</span>:</span>ident</span>: </span>$field_type</span>:</span>ty</span>,)*
</span>    }) => {
</span>        $(#[</span>$outer</span>])*
</span>        </span>$sv </span>struct </span>$name</span> {
</span>            $(</span>$fv $field_name</span>: </span>$field_type</span>,)*
</span>        }
</span>        </span>impl $name </span>{
</span>            </span>pub</span> fn set_token(&mut self, </span>token</span>: &</span>str</span>) -> </span>bool</span> {
</span>                match token {
</span>                    $(stringify!(</span>$field_name</span>) => { self.</span>$field_name</span> = true; true })*
</span>                    _ => { false }
</span>                }
</span>            }
</span>        }
</span>    }
</span>}
</span></code></pre>
A macro is split into two parts: ($matcher) => {$expansion}</code>.  There are 5 lines above with the $(...)*</code> pattern: 2 in the matcher and 3 in the expansion section.  They will each match or expand 0 or more times.
The only</u> lines which can repeat in the expansion above are the ones that follow that pattern.</p>

First line of matcher: ($(#[$outer:meta])*</code> matches any attributes.
$outer</code> matches cfg_attr(feature = "serde", derive(Serialize, Deserialize))</code> and derive(Default)</code> from the struct above.</li>
On the next line $sv:vis struct $name:ident {</code>,
$sv</code> matches pub</code> and $name</code> matches material_tokens</code>.</li>
Last line of matcher: $($fv:vis $field_name:ident: $field_type:ty,)*</code> matches each field.
$fv</code> for first match is pub</code>, $field_name</code> is IMPLIES_ANIMAL_KILL</code>, and $field_type</code> is bool</code></li>
First line of expansion: $(#[$outer])*</code> expands back to the attributes from above, unchanged.</li>
$sv struct $name {</code> expands to pub struct material_tokens {</code></li>
$($fv $field_name: $field_type,)*</code> duplicates each field.
First iteration expands to pub IMPLIES_ANIMAL_KILL: bool,</code></li>
impl $name {</code> expands to impl material_tokens {</code></li>
$(stringify!($field_name) => { self.$field_name = true; true })*</code> expands to the fields matched above.
stringify!</a> macro creates a &str</code>.
First iteration expands to "IMPLIES_ANIMAL_KILL" => { self.IMPLIES_ANIMAL_KILL = true; true }</code></li>
</ul>
Here's what a snippet of the expanded struct looks like:</p>
/// Recursive expansion of set_token! macro
</span>// ========================================
</span>
</span>#[</span>cfg_attr</span>(feature = </span>"</span>serde</span>"</span>, </span>derive</span>(Serialize, Deserialize))]
</span>#[</span>derive</span>(Default, PartialEq)]
</span>pub struct </span>material_tokens </span>{
</span>    </span>pub </span>IMPLIES_ANIMAL_KILL</span>: </span>bool</span>,
</span>    </span>pub </span>ALCOHOL_PLANT</span>: </span>bool</span>,
</span>    </span>// snip
</span>    </span>pub </span>ITEMS_QUERN</span>: </span>bool</span>,
</span>}
</span>impl </span>material_tokens </span>{
</span>    </span>pub fn </span>set_token</span>(&</span>mut </span>self</span>, </span>token</span>: &</span>str</span>) -> </span>bool </span>{
</span>        </span>match</span> token {
</span>            </span>"</span>IMPLIES_ANIMAL_KILL</span>" </span>=> {
</span>                </span>self</span>.</span>IMPLIES_ANIMAL_KILL </span>= true;
</span>                true
</span>            }
</span>            </span>"</span>ALCOHOL_PLANT</span>" </span>=> {
</span>                </span>self</span>.</span>ALCOHOL_PLANT </span>= true;
</span>                true
</span>            }
</span>            </span>// snip
</span>            </span>"</span>ITEMS_QUERN</span>" </span>=> {
</span>                </span>self</span>.</span>ITEMS_QUERN </span>= true;
</span>                true
</span>            }
</span>            _ => false,
</span>        }
</span>    }
</span>}
</span></code></pre>
I use Visual Studio Code for development, and the rust-analyzer plugin has a pretty decent macro expander.  It produced the output above. To use it right-click in the IDE and select Command Palette...</code> or press Ctrl+Shift+P</code>, start typing in macro</code> and you should see a match for rust-analyzer: Expand macro recursively</code>.
You can set a hotkey for it right there, too.</p>
After my first test here's the relevant output from my parser:</p>
INFO  [dfraw_parse] material_template_ = ../vanilla\material_template_default.txt
</span>INFO  [dfraw_parse::material_template] parsing started after line: 3
</span>ERROR [example] errors found: 14
</span>00001 line number: 448, unknown MATERIAL_TOKEN: 'CARTILAGE'
</span>00002 line number: 539, unknown MATERIAL_TOKEN: 'FEATHER'
</span>00003 line number: 584, unknown MATERIAL_TOKEN: 'SCALE'
</span>00004 line number: 772, unknown MATERIAL_TOKEN: 'NERVOUS_TISSUE'
</span>00005 line number: 1475, unknown MATERIAL_TOKEN: 'HOOF'
</span>00006 line number: 1983, unknown MATERIAL_TOKEN: 'CHITIN'
</span>00007 line number: 2484, unknown MATERIAL_TOKEN: 'SYNDROME'
</span>00008 line number: 2488, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
</span>00009 line number: 2489, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
</span>00010 line number: 2490, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'
</span>00011 line number: 3014, unknown MATERIAL_TOKEN: 'SYNDROME'
</span>00012 line number: 3018, unknown MATERIAL_TOKEN: 'SYN_INGESTED'
</span>00013 line number: 3019, unknown MATERIAL_TOKEN: 'SYN_INJECTED'
</span>00014 line number: 3020, unknown MATERIAL_TOKEN: 'SYN_NO_HOSPITAL'
</span></code></pre>
Looks like I missed a few materials, and now I need to parse syndromes.  I finally feel like I'm writing nearly proper Rust.</p>
added: 2023-08-29</strong></p>
Same macro as above, but proc_macro_derive type instead</h1>
I used a macro_rules! above because it was the first macro type I learned.
However, I recently needed to make a custom derive macro, so I converted this macro at the same time.</p>
I followed the guide here</a> for the initial setup.
It involves creating a separate crate and using dependencies like the syn</a> and quote</a> crates to help.</p>
I'm not going to repeat a lot of stuff here, here's the macro above as a proc_macro_derive:</p>
use proc_macro::TokenStream;
</span>use quote::quote;
</span>use syn::{Data, DeriveInput, Fields, Type};
</span>
</span>// set bool fields on struct by string, return true on success
</span>#[</span>proc_macro_derive</span>(SetToken)]
</span>pub fn </span>set_token_derive</span>(</span>input</span>: TokenStream) -> TokenStream {
</span>    </span>let</span> ast = syn::parse(input).unwrap();
</span>    impl_set_token(&ast)
</span>}
</span>
</span>fn </span>impl_set_token</span>(</span>ast</span>: &DeriveInput) -> TokenStream {
</span>    </span>let</span> name = &ast.ident;
</span>    </span>let mut</span> matches = quote!();
</span>    </span>match </span>&ast.data {
</span>        Data::Struct(</span>ref</span> data) => {
</span>            </span>match </span>&data.fields {
</span>                Fields::Named(fields) => {
</span>                    </span>for</span> f in &fields.named {
</span>                        </span>match </span>&f.ty {
</span>                            Type::Path(p) => {
</span>                                </span>match</span> p.path.segments.first() {
</span>                                    Some(s) => {
</span>                                        </span>match</span> s.ident.to_string().as_str() {
</span>                                            </span>"</span>bool</span>" </span>=> {
</span>                                                </span>match </span>&f.ident {
</span>                                                    Some(i) => {
</span>                                                        </span>let</span> s = i.to_string();
</span>                                                        matches.extend(quote!{
</span>                                                                #s => { 
</span>                                                                    </span>self</span>.#i = true; 
</span>                                                                    true
</span>                                                                }
</span>                                                        });
</span>                                                    }
</span>                                                    _ => {}
</span>                                                }
</span>                                            }
</span>                                            _ => {}
</span>                                        }
</span>                                    }
</span>                                    _=> {}
</span>                                }
</span>                            }
</span>                            _ => {}
</span>                        }
</span>                    }
</span>                }
</span>                _ => {
</span>                }
</span>            }
</span>        }
</span>        _ => {
</span>            panic!(</span>"</span>struct only!</span>"</span>);
</span>        }
</span>    };
</span>    quote! {
</span>        </span>impl </span>SetToken for #</span>name </span>{
</span>            </span>fn </span>set_token</span>(&</span>mut </span>self</span>, </span>token</span>: &</span>str</span>) -> </span>bool </span>{
</span>                </span>match</span> token {
</span>                    #matches
</span>                    _ => {  
</span>                        false
</span>                    }
</span>                }
</span>            }
</span>        }
</span>    }.into()
</span>}
</span></code></pre>
SetToken trait:</p>
pub trait </span>SetToken </span>{
</span>    </span>fn </span>set_token</span>(&</span>mut </span>self</span>, </span>token</span>: &</span>str</span>) -> </span>bool</span>;
</span>}
</span></code></pre>
Holy shit, I thought the first macro was painful to read.</p>
The differences between this macro and the previous one:</p>

This macro only generates matches for bool</code> struct fields.</li>
Used with #[derive(SetToken)]</code> attribute instead of using !set_token</code> macro</li>
</ul>


Rendering Sprite Sheets in Rust
2023-07-19T00:00:00+00:00
I've made some progress learning OpenGL.  For tile locations on screen, I calculate all the vertex positions per tile (2 triangles per tile, 6 vertices per triangle = 12 vertices) and store them in a vertex buffer object</a>.  I add the 0.0 Z coordinate in the shader, so that doesn't need to be stored, at least.</p>
Here's what the geometry looks like for a 1920x1200 display configured for 32x32 pixel tiles:</p>
</a></p>
Screenshot is from NVIDIA Nsight Graphics</a>, a pretty awesome tool for a beginner like me.  RenderDoc</a> is great, too, but Nsight Graphics was able to immediately tell me what was wrong with my render pipeline.  (Remember to bind the vertex array every draw call)</p>
Here's the code for calculating the tile vertices:</p>
let mut</span> tile = </span>0.0</span>;
</span>loop </span>{
</span>    </span>let</span> verts = get_tex_vertices(</span>512.0</span>, </span>512.0</span>, </span>32.0</span>, </span>32.0</span>, tile);        
</span>    </span>//upper right        
</span>    vertices.push(xf + </span>32.0</span>);        
</span>    vertices.push(yf + </span>32.0</span>);        
</span>    </span>//upper left        
</span>    vertices.push(xf);        
</span>    vertices.push(yf + </span>32.0</span>);        
</span>    </span>//lower left        
</span>    vertices.push(xf);        
</span>    vertices.push(yf);        
</span>    </span>//upper right        
</span>    vertices.push(xf + </span>32.0</span>);        
</span>    vertices.push(yf + </span>32.0</span>);        
</span>    </span>//lower left        
</span>    vertices.push(xf);        
</span>    vertices.push(yf);        
</span>    </span>//lower right        
</span>    vertices.push(xf + </span>32.0</span>);        
</span>    vertices.push(yf);        
</span>
</span>    uvs.extend_from_slice(&verts);        
</span>    tile += </span>1.0</span>;        
</span>    </span>if</span> tile > </span>255.0 </span>{        
</span>        tile = </span>0.0</span>;        
</span>    }        
</span>    xf += </span>32.0</span>;        
</span>    </span>if</span> xf > </span>1920.0 </span>{        
</span>        xf = </span>0.0</span>;        
</span>        yf += </span>32.0</span>;        
</span>    }        
</span>    </span>if</span> yf > </span>1200.0 </span>{        
</span>        </span>break</span>;        
</span>    }        
</span>}                
</span></code></pre>
Screen resolution is temporarily hard coded so that I could check the geometry without running fullscreen.  Generally, the vertex shader wants vertices in normalized device coordinates</a> which range from -1.0,-1.0 to 1.0,1.0.  Right now I'm normalizing the coordinates in the shader like so:</p>
#version </span>330
</span>layout (location = </span>0</span>) in vec2 vert;
</span>layout (location = </span>1</span>) in vec2 _uv;
</span>out vec2 uv;
</span>void main()
</span>{
</span>    uv = _uv;
</span>    gl_Position = vec4((vert.x / </span>1920.0 </span>* </span>2.0</span>) - </span>1.0</span>, (vert.y / </span>1200.0 </span>* </span>2.0</span>) - </span>1.0</span>, </span>0.0</span>, </span>1.0</span>);
</span>};
</span></code></pre>
But when I rewrite my shaders in a moment I'll just handle it on the CPU since I only need to calculate those vertices once.</p>
Next I had to figure out how to access individual sprites in a sprite sheet (also called a texture atlas</a>).  I'm targeting 32x32 tile size, so I looked for an example to do testing with.  I found a cool tileset for Dwarf Fortress by Vettlingr</a>.</p>
Here's the tileset:</p>
</a></p>
Note: I'm not planning on using these graphics officially, it's just for demonstration purposes.</p>
The image is 512x512 pixels.  Each tile is 32x32 pixels, which means there are 16x16 (or 256) tiles total. In the code above I call get_tex_vertices() to generate the texture coordinates for each tile.  Textures use yet another coordinate system</a> that ranges from 0.0,0.0 to 1.0,1.0.</p>
Here's the code for calculating texture coordinates based on texture size, sprite size, and sprite index:</p>
// tile index = 0 from left to right, top to bottom.
</span>fn </span>get_tex_vertices</span>(        
</span>    </span>sheet_w</span>: </span>f32</span>,        
</span>    </span>sheet_h</span>: </span>f32</span>,        
</span>    </span>tile_w</span>: </span>f32</span>,        
</span>    </span>tile_h</span>: </span>f32</span>,        
</span>    </span>tile_index</span>: </span>f32</span>,        
</span>) -> [</span>f32</span>; </span>12</span>] {        
</span>    </span>let mut</span> vertices: [</span>f32</span>; </span>12</span>] = [</span>0.0</span>; </span>12</span>];        
</span>    </span>let</span> cols = sheet_w / tile_w;        
</span>    </span>let</span> x = tile_index % cols;        
</span>    </span>let</span> y = (tile_index / cols).floor();        
</span>    </span>//invert y by default        
</span>    </span>//lower right        
</span>    vertices[</span>0</span>] = (x * tile_w + tile_w) / sheet_w;        
</span>    vertices[</span>1</span>] = (y * tile_h) / sheet_h;        
</span>    </span>//lower left        
</span>    vertices[</span>2</span>] = (x * tile_w) / sheet_w;        
</span>    vertices[</span>3</span>] = (y * tile_h) / sheet_h;        
</span>    </span>//upper left        
</span>    vertices[</span>4</span>] = (x * tile_w) / sheet_w;        
</span>    vertices[</span>5</span>] = (y * tile_h + tile_h) / sheet_h;        
</span>    </span>//lower right        
</span>    vertices[</span>6</span>] = (x * tile_w + tile_w) / sheet_w;        
</span>    vertices[</span>7</span>] = (y * tile_h) / sheet_h;        
</span>    </span>//upper left        
</span>    vertices[</span>8</span>] = (x * tile_w) / sheet_w;        
</span>    vertices[</span>9</span>] = (y * tile_h + tile_h) / sheet_h;        
</span>    </span>//upper right        
</span>    vertices[</span>10</span>] = (x * tile_w + tile_w) / sheet_w;        
</span>    vertices[</span>11</span>] = (y * tile_h + tile_h) / sheet_h;        
</span>    vertices        
</span>}        
</span></code></pre>
This code automatically inverts the y coordinates.  That's because there are more coordinate systems fighting each other here.  Unless configured differently, drawing in OpenGL at the origin (0,0) usually means the lower</em> left corner of the screen.  Many image formats have their 0,0 at the upper</em> left of the screen.  Instead of rotating lots of image data around in memory, I'll rotate the texture coordinates instead.</p>
Here's the vertex geometry of the sprite sheet visualized:
</a></p>
I put together a few transforms for texture coordinates.  This lets me pre-calculate flipped, rotated, mirrored, etc. texture coordinates for animation later on.</p>
Here are a few examples:</p>
fn </span>transform_uvs</span>(
</span>    </span>uvs</span>: &</span>mut</span> [</span>f32</span>; 12],
</span>    </span>sheet_w</span>: </span>f32</span>,
</span>    </span>sheet_h</span>: </span>f32</span>,
</span>    </span>tile_w</span>: </span>f32</span>,
</span>    </span>tile_h</span>: </span>f32</span>,
</span>    </span>tile_index</span>: </span>f32</span>,
</span>    </span>transform</span>: Transforms,
</span>) {
</span>    </span>let</span> cols = sheet_w / tile_w;
</span>    </span>if</span> transform == Transforms::FlipVertical {
</span>        </span>let</span> y = (tile_index / cols).floor();
</span>        </span>let</span> y_low = (y * tile_h) / sheet_h;
</span>        </span>let</span> y_high = (y * tile_h + tile_h) / sheet_h;
</span>        </span>let mut </span>flip </span>= |</span>index</span>: </span>usize</span>| {
</span>            </span>if</span> uvs[index] == y_low {
</span>                uvs[index] = y_high;
</span>            } </span>else </span>{
</span>                uvs[index] = y_low;
</span>            }
</span>        };
</span>        </span>// odd vertices = y coordinates
</span>        </span>for</span> i in (</span>1</span>..=</span>11</span>).step_by(</span>2</span>) {
</span>            flip(i);
</span>        }
</span>    }
</span>    </span>if</span> transform == Transforms::FlipHorizontal {
</span>        </span>let</span> x = tile_index % cols;
</span>        </span>let</span> x_low = (x * tile_w) / sheet_w;
</span>        </span>let</span> x_high = (x * tile_w + tile_w) / sheet_w;
</span>        </span>let mut </span>flip </span>= |</span>index</span>: </span>usize</span>| {
</span>            </span>if</span> uvs[index] == x_low {
</span>                uvs[index] = x_high;
</span>            } </span>else </span>{
</span>                uvs[index] = x_low;
</span>            }
</span>        };
</span>        </span>//even vertices = x coordinates
</span>        </span>for</span> i in (</span>0</span>..=</span>10</span>).step_by(</span>2</span>) {
</span>            flip(i);
</span>        }
</span>    }
</span>    </span>if</span> transform == Transforms::Rotate90Left {
</span>        </span>let</span> x = tile_index % cols;
</span>        </span>let</span> y = (tile_index / cols).floor();
</span>        uvs[</span>0</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>1</span>] = (y * tile_h + tile_h) / sheet_h;
</span>        uvs[</span>2</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>3</span>] = (y * tile_h) / sheet_h;
</span>        uvs[</span>4</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>5</span>] = (y * tile_h) / sheet_h;
</span>        uvs[</span>6</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>7</span>] = (y * tile_h + tile_h) / sheet_h;
</span>        uvs[</span>8</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>9</span>] = (y * tile_h) / sheet_h;
</span>        uvs[</span>10</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>11</span>] = (y * tile_h + tile_h) / sheet_h;
</span>    }
</span>    </span>if</span> transform == Transforms::Rotate90LeftHMirror {
</span>        </span>let</span> x = tile_index % cols;
</span>        </span>let</span> y = (tile_index / cols).floor();
</span>        uvs[</span>0</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>1</span>] = (y * tile_h) / sheet_h;
</span>        uvs[</span>2</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>3</span>] = (y * tile_h + tile_h) / sheet_h;
</span>        uvs[</span>4</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>5</span>] = (y * tile_h + tile_h) / sheet_h;
</span>        uvs[</span>6</span>] = (x * tile_w + tile_w) / sheet_w;
</span>        uvs[</span>7</span>] = (y * tile_h) / sheet_h;
</span>        uvs[</span>8</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>9</span>] = (y * tile_h + tile_h) / sheet_h;
</span>        uvs[</span>10</span>] = (x * tile_w) / sheet_w;
</span>        uvs[</span>11</span>] = (y * tile_h) / sheet_h;
</span>    }
</span>} 
</span></code></pre>
This math is specific to the coordinates I generated before.  It's also specific for my vertex buffer layout.  It's just there to give an idea on how to approach the problem for other people trying to learn how to 2d graphics in 2023 =)</p>
Next post I'm going to finish the shaders, allowing me to update tile indexes on demand and render many layers.  I'm also going to add in palette support.  My near term goal is to import the Dwarf Fortress raws and basically duplicate the rendering system.</p>
Here's one last picture showing my title screen with the Vettlingr tiles in the background:</p>
</a></p>


Cross-platform GUI demo
2023-07-08T00:00:00+00:00
Lately I've been building my 2D tile rendering engine.  The current plan is for square tiles and each tile can be composed of 8 or more layers.  Also, I'll do something similar to how the current Dwarf Fortress does it:  I'll render a few Z levels below with increasing opacity.  I'll add animation too, but overall it's going to be as simple and straightforward as possible so that I can jump into the actual game logic ASAP.</p>
My plan was</em> to use array textures</a>, but since I'm targeting OpenGL 3.3, I'm only guaranteed GL_MAX_ARRAY_TEXTURE_LAYERS of 256 or more, so I guess I'll stick to using a good old texture atlas</a>.</p>
This is the first time I've done any graphics programming, so it's a bit of a learning curve for me.  In the meantime I've made a little recording of my title screen mock-up.</p>
YouTube video is located here</a>.</p>
That's just a simple little proof of concept that uses egui</a>, glow</a>, and winit</a> for cross-platform compatibility.</p>
The background fragment shader is ported from this ShaderToy submission</a> to GLSL</a>.</p>
Background vertex shader:</p>
const </span>VERTEX_SHADER_SOURCE</span>: &</span>str </span>= </span>r</span>#"</span>#version 130
</span>const vec2 coords[] = vec2[3] (
</span>    vec2(-1., -1.),
</span>    vec2(3., -1.),
</span>    vec2(-1., 3.)
</span>);
</span>void main()
</span>{
</span>    gl_Position = vec4(coords[gl_VertexID], 0., 1.);
</span>}
</span>"#</span>;
</span></code></pre>
The vertex shader source code is from this page</a> about full screen triangle optimization, ported to GLSL.  </p>


Terrain erosion part III
2023-06-22T00:00:00+00:00
Last part of my terrain erosion series.  Here are some example images showing a couple before and afters of my hydraulic erosion simulation.
Tested on a 8192x8192 game map, one pixel represents one tile.  These images show a small fraction of the smallest possible game map.</p>
Before:</p>
</a></p>
After:</p>
</a>
(light blue to black colors represent water, grey to white represent mountains and snow)</p>
Here are some more, but zoomed in further:</p>
Before:</p>
</a></p>
After:</p>
</a></p>
On my 8-core dev machine the erosion sim carved over 16,000 separate paths through the terrain in about 30 seconds.  There are already quite a few variables in place to allow tuning the process.  The image above roughly equates to medium density, medium intensity, and with maximum coverage (no map sections left out).  I can currently limit areas based on average elevation, and further along my worldgen process I'll take into account precipitation levels for particular biomes.</p>
Next up I'm going to start on OpenGL graphics and UI code so that I can plug in the beginnings of my worldgen code.</p>


I'm officially a citizen of The Republic of Slowjamastan!
2023-06-20T00:00:00+00:00
Today I am proud to announce that I am officially a citizen of the totally real nation of Slowjamastan</a>.
My Slowjamastani name is Urist McMiner.  Though my native country of Happyhardcoristan has traditionally had fraught relations with Slowjamastan, in the interests of peace, love, unity, and respect I dutifully submitted my application for citizenship.</p>
Proof of my citizenship is recorded on the republic's website</a>:</p>
</p>
=)</p>


Simple Coordinate Subdivision in Rust
2023-06-20T00:00:00+00:00
Just a quick little post about a simple topic.  I want to subdivide my square game map into X by X squares for various reasons...some current reasons, and some future reasons.</p>
I want to more efficiently narrow down parts of the map to change for my erosion simulation.  It's also something close to what I'll need in the future because I plan on eventually dealing with my map in 256x256 tile chunks.  That's a topic for the future though.</p>
Here's the quick and dirty code:</p>
pub struct </span>Section </span>{
</span>    </span>pub </span>from</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>pub </span>to</span>: (</span>usize</span>, </span>usize</span>),
</span>}
</span>
</span>impl </span>Section </span>{
</span>    </span>pub fn </span>with_width</span>(</span>width</span>: </span>usize</span>) -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            from: (</span>0</span>, </span>0</span>),
</span>            to: (width - </span>1</span>, width - </span>1</span>),
</span>        }
</span>    }
</span>    </span>pub fn </span>subdivide</span>(&</span>self</span>, </span>divisor</span>: </span>usize</span>) -> Vec<Section> {
</span>        </span>let mut</span> retval: Vec<Section> = vec![];
</span>        </span>let</span> width = (</span>self</span>.to.</span>0 </span>- </span>self</span>.from.</span>0</span>) + </span>1</span>;
</span>        </span>if</span> divisor < </span>2 </span>|| width < </span>2 </span>{
</span>            </span>return</span> retval;
</span>        }
</span>        </span>let</span> step = width / divisor;
</span>        </span>let mut</span> x1 = </span>0</span>;
</span>        </span>let mut</span> y1 = </span>0</span>;
</span>        </span>let mut</span> x2 = step - </span>1</span>;
</span>        </span>let mut</span> y2 = step - </span>1</span>;
</span>        </span>while</span> y2 < width {
</span>            retval.push(Section {
</span>                from: (</span>self</span>.from.</span>0 </span>+ x1, </span>self</span>.from.</span>1 </span>+ y1),
</span>                to: (</span>self</span>.from.</span>0 </span>+ x2, </span>self</span>.from.</span>1 </span>+ y2),
</span>            });
</span>            x1 += step;
</span>            x2 = x1 + step - </span>1</span>;
</span>            </span>if</span> x1 > width - </span>1 </span>{
</span>                x1 = </span>0</span>;
</span>                x2 = step - </span>1</span>;
</span>                y1 += step;
</span>                y2 = y1 + step - </span>1</span>;
</span>            }
</span>        }
</span>        retval
</span>    }
</span>}
</span></code></pre>
One caveat: it's only perfect if width modulo divisor == 0.  This is all I need though, so it's all I wrote.  However, even if you use it on a width that doesn't evenly divide, all the coordinates will still be valid, but a bit of the area will be left out. (at least (width % divisor) * width?)</p>
With the following test code:</p>
let</span> s = Section::with_width(</span>99</span>);
</span>let</span> s2 = s.subdivide(</span>11</span>);
</span>let</span> s3 = s2[</span>0</span>].subdivide(</span>3</span>);
</span></code></pre>
When I render each square in s2 with alternating colors it looks like this (11x11 squares, 9x9 pixels each):</p>
</p>
When I render s3 it looks like this (the first of the previous squares further subdivided into 3x3 squares, 3x3 pixels each):</p>
</p>
(the grey on grey grid background is just my standard background for my test images, and these images are scaled up 4x)</p>
That's pretty much it.  For the Section struct, from = the upper left corner of the square, to = the lower right corner of the square.  There's no concept of pixels, tiles, etc. in this code...it's just coordinates I can make squares with.</p>
Cheers!</p>


A Tale of Two Bresenhams
2023-06-16T00:00:00+00:00
...or How To Draw Thick Bresenham Lines in Rust</h3>
I decided last post to construct rivers basically using a bunch of circles.  One reason for that is because it would make it easy to have the middle of my rivers be deeper than the edges.  Another reason is because properly drawing thick lines has been frustrating me a bit lately and I was eager to just move on.  They were both pretty bad reasons.   Bad because the zillion circle idea turned out to be really slow (partly because I kept a ring buffer of previous tiles so I could avoid where my circles overlapped), and also bad because where's the fun in being lazy and not figuring something out?</p>
I tried lots of algorithms for thick lines.  Lots.  Some of them just didn't work, possibly my fault.  Some of them were just plain wrong and the line ends weren't perpendicular.  I suck at math, so it took me a bit to wrap my head around the problem.  At first, I'm thinking to myself:  A thick line is just a rectangle...all I need are a couple perpendicular lines to set the corners to, and fill in all the pixels one line at a time.  How hard can it be?</p>
Every friggin' site on the internet says that to draw a perpendicular line, just invert the slope.  Problem solved!  (I mean, except for zero and infinity, but that's easy because at that point we're dealing with vertical or horizontal lines, and even I</em> can add to X or add to Y to get a perpendicular point).  But alas, the internet lies.  Just blindly inverting a slope will give you a perpendicular slope only some</em> of the time.</p>
I finally found a promising link here: https://github.com/ArminJo/Arduino-BlueDisplay/blob/master/src/LocalGUI/ThickLine.hpp</a>.
At first I recoiled in horror at the camelCase variable names, the same exact</em> way I hated Rust's snake_case names at first.  Rust got this one right though.  I ended up making 3 different versions, all tailored for my specific use, which is why I'm not making (or contributing to) a crate.</p>
get_thick_line_unchecked() will make a proper fat line, even if it overlaps canvas boundaries (which are still only square). I call it unchecked because if any parts of the line go out of bounds, there will be duplicate points.  get_thick_line() will also make a proper fat line, account for out of bounds, but also remove duplicates.  Option #3 is the scary one, and I'll write about it in a bit.  Both of the unchecked versions will produce perfect lines with zero</em> duplicates if no parts of the line</del> rectangle go out of bounds.  Here's the code for the first version:</p>
#[</span>inline</span>]
</span>// ported from https://github.com/ArminJo/Arduino-BlueDisplay/blob/master/src/LocalGUI/ThickLine.hpp
</span>pub fn </span>get_thick_line_unchecked</span>(
</span>    </span>from</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>to</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>line_width</span>: </span>usize</span>,
</span>    </span>thick_mode</span>: ThicknessMode,
</span>    </span>map_width</span>: </span>usize</span>,
</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> line: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> x1 = from.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> x2 = to.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> y1 = from.</span>1 </span>as </span>isize</span>;
</span>    </span>let mut</span> y2 = to.</span>1 </span>as </span>isize</span>;
</span>    </span>let mut</span> dx: </span>isize</span>;
</span>    </span>let mut</span> dy: </span>isize</span>;
</span>    </span>let</span> dx2: </span>isize</span>;
</span>    </span>let</span> dy2: </span>isize</span>;
</span>    </span>let mut</span> err: </span>isize</span>;
</span>    </span>let mut</span> step_x: </span>isize</span>;
</span>    </span>let mut</span> step_y: </span>isize</span>;
</span>    </span>let</span> max_width = (map_width - </span>1</span>) as </span>isize</span>;
</span>    </span>if</span> x1 > max_width {
</span>        x1 = max_width;
</span>    }
</span>    </span>if</span> x2 > max_width {
</span>        x2 = max_width;
</span>    }
</span>    </span>if</span> y1 > max_width {
</span>        y1 = max_width;
</span>    }
</span>    </span>if</span> y2 > max_width {
</span>        y2 = max_width;
</span>    }
</span>    </span>if</span> line_width <= </span>1 </span>{
</span>        </span>return </span>get_line_overlap((x1, y1), (x2, y2), </span>LINE_OVERLAP_NONE</span>, map_width);
</span>    }
</span>    dy = x2 - x1;
</span>    dx = y2 - y1;
</span>    </span>let mut</span> swap = true;
</span>    </span>if</span> dx < </span>0 </span>{
</span>        dx = -dx;
</span>        step_x = -</span>1</span>;
</span>        swap = !swap;
</span>    } </span>else </span>{
</span>        step_x = </span>1</span>;
</span>    }
</span>    </span>if</span> dy < </span>0 </span>{
</span>        dy = -dy;
</span>        step_y = -</span>1</span>;
</span>        swap = !swap;
</span>    } </span>else </span>{
</span>        step_y = </span>1</span>;
</span>    }
</span>    dx2 = dx << </span>1</span>;
</span>    dy2 = dy << </span>1</span>;
</span>    </span>let mut</span> overlap: </span>usize</span>;
</span>    </span>let mut</span> draw_start_adjust_count = line_width / </span>2</span>;
</span>    </span>if</span> thick_mode == ThicknessMode::LineThicknessDrawCounterclockwise {
</span>        draw_start_adjust_count = line_width - </span>1</span>;
</span>    } </span>else if</span> thick_mode == ThicknessMode::LineThicknessDrawClockwise {
</span>        draw_start_adjust_count = </span>0</span>;
</span>    }
</span>    </span>if</span> dx >= dy {
</span>        </span>if</span> swap {
</span>            draw_start_adjust_count = (line_width - </span>1</span>) - draw_start_adjust_count;
</span>            step_y = -step_y;
</span>        } </span>else </span>{
</span>            step_x = -step_x;
</span>        }
</span>        err = dy2 - dx;
</span>        </span>for </span>_ in (</span>0</span>..draw_start_adjust_count).rev() {
</span>            x1 -= step_x;
</span>            x2 -= step_x;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                y1 -= step_y;
</span>                y2 -= step_y;
</span>                err -= dx2;
</span>            }
</span>            err += dy2;
</span>        }
</span>        line.append(&</span>mut </span>get_line_overlap(
</span>            (x1, y1),
</span>            (x2, y2),
</span>            </span>LINE_OVERLAP_NONE</span>,
</span>            map_width,
</span>        ));
</span>        err = dy2 - dx;
</span>        </span>for </span>_ in (</span>1</span>..line_width).rev() {
</span>            x1 += step_x;
</span>            x2 += step_x;
</span>            overlap = </span>LINE_OVERLAP_NONE</span>;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                y1 += step_y;
</span>                y2 += step_y;
</span>                err -= dx2;
</span>                overlap = </span>LINE_OVERLAP_MAJOR</span>;
</span>            }
</span>            err += dy2;
</span>            line.append(&</span>mut </span>get_line_overlap(
</span>                (x1, y1),
</span>                (x2, y2),
</span>                overlap,
</span>                map_width,
</span>            ))
</span>        }
</span>    } </span>else </span>{
</span>        </span>if</span> swap {
</span>            step_x = -step_x;
</span>        } </span>else </span>{
</span>            draw_start_adjust_count = (line_width - </span>1</span>) - draw_start_adjust_count;
</span>            step_y = -step_y;
</span>        }
</span>        err = dx2 - dy;
</span>        </span>for </span>_ in (</span>0</span>..draw_start_adjust_count).rev() {
</span>            y1 -= step_y;
</span>            y2 -= step_y;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                x1 -= step_x;
</span>                x2 -= step_x;
</span>                err -= dy2;
</span>            }
</span>            err += dx2;
</span>        }
</span>        line.append(&</span>mut </span>get_line_overlap(
</span>            (x1, y1),
</span>            (x2, y2),
</span>            </span>LINE_OVERLAP_NONE</span>,
</span>            map_width,
</span>        ));
</span>        err = dx2 - dy;
</span>        </span>for </span>_ in (</span>1</span>..line_width).rev() {
</span>            y1 += step_y;
</span>            y2 += step_y;
</span>            overlap = </span>LINE_OVERLAP_NONE</span>;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                x1 += step_x;
</span>                x2 += step_x;
</span>                err -= dy2;
</span>                overlap = </span>LINE_OVERLAP_MAJOR</span>;
</span>            }
</span>            err += dx2;
</span>            line.append(&</span>mut </span>get_line_overlap(
</span>                (x1, y1),
</span>                (x2, y2),
</span>                overlap,
</span>                map_width,
</span>            ));
</span>        }
</span>    }
</span>    </span>// these are 2.5x slower than using get_thick_line() to remove dupes:
</span>    </span>//line.sort();
</span>    </span>//line.dedup();
</span>    line
</span>}
</span></code></pre>
and....</p>
const </span>LINE_OVERLAP_NONE</span>: </span>usize </span>= </span>0</span>; </span>// No line overlap, like in standard Bresenham
</span>const </span>LINE_OVERLAP_MAJOR</span>: </span>usize </span>= </span>0x01</span>; </span>// Overlap - first go major then minor direction. Pixel is drawn as extension after actual line
</span>const </span>LINE_OVERLAP_MINOR</span>: </span>usize </span>= </span>0x02</span>; </span>// Overlap - first go minor then major direction. Pixel is drawn as extension before next line
</span>const </span>LINE_OVERLAP_BOTH</span>: </span>usize </span>= </span>0x03</span>; </span>// Overlap - both
</span>
</span>#[</span>derive</span>(PartialEq)]
</span>pub enum </span>ThicknessMode </span>{
</span>    LineThicknessMiddle = </span>0</span>,
</span>    LineThicknessDrawClockwise = </span>1</span>,
</span>    LineThicknessDrawCounterclockwise = </span>2</span>,
</span>}
</span>
</span>#[</span>inline</span>]
</span>// ported from https://github.com/ArminJo/Arduino-BlueDisplay/blob/master/src/LocalGUI/ThickLine.hpp
</span>fn </span>get_line_overlap</span>(
</span>    </span>from</span>: (</span>isize</span>, </span>isize</span>),
</span>    </span>to</span>: (</span>isize</span>, </span>isize</span>),
</span>    </span>overlap</span>: </span>usize</span>,
</span>    </span>map_width</span>: </span>usize</span>,
</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> x1 = from.</span>0</span>;
</span>    </span>let</span> x2 = to.</span>0</span>;
</span>    </span>let mut</span> y1 = from.</span>1</span>;
</span>    </span>let</span> y2 = to.</span>1</span>;
</span>    </span>let mut</span> line: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> dx: </span>isize</span>;
</span>    </span>let mut</span> dy: </span>isize</span>;
</span>    </span>let</span> dx2: </span>isize</span>;
</span>    </span>let</span> dy2: </span>isize</span>;
</span>    </span>let mut</span> err: </span>isize</span>;
</span>    </span>let</span> step_x: </span>isize</span>;
</span>    </span>let</span> step_y: </span>isize</span>;
</span>    </span>let</span> max_width = (map_width - </span>1</span>) as </span>isize</span>;
</span>    dx = x2 - x1;
</span>    dy = y2 - y1;
</span>    </span>if</span> dx < </span>0 </span>{
</span>        dx = -dx;
</span>        step_x = -</span>1</span>;
</span>    } </span>else </span>{
</span>        step_x = </span>1</span>;
</span>    }
</span>    </span>if</span> dy < </span>0 </span>{
</span>        dy = -dy;
</span>        step_y = -</span>1</span>;
</span>    } </span>else </span>{
</span>        step_y = </span>1</span>;
</span>    }
</span>    dx2 = dx << </span>1</span>;
</span>    dy2 = dy << </span>1</span>;
</span>    line.push((
</span>        x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>        y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>    ));
</span>    </span>if</span> dx > dy {
</span>        err = dy2 - dx;
</span>        </span>while</span> x1 != x2 {
</span>            x1 += step_x;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MAJOR </span>!= </span>0 </span>{
</span>                    line.push((
</span>                        x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                        y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                    ));
</span>                }
</span>                y1 += step_y;
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MINOR </span>!= </span>0 </span>{
</span>                    line.push((
</span>                        (x1 - step_x).clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                        y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                    ));
</span>                }
</span>                err -= dx2;
</span>            }
</span>            err += dy2;
</span>            line.push((
</span>                x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>            ));
</span>        }
</span>    } </span>else </span>{
</span>        err = dx2 - dy;
</span>        </span>while</span> y1 != y2 {
</span>            y1 += step_y;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MAJOR </span>!= </span>0 </span>{
</span>                    line.push((
</span>                        x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                        y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                    ));
</span>                }
</span>                x1 += step_x;
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MINOR </span>!= </span>0 </span>{
</span>                    line.push((
</span>                        x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                        (y1 - step_y).clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                    ));
</span>                }
</span>                err -= dy2;
</span>            }
</span>            err += dx2;
</span>            line.push((
</span>                x1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>                y1.clamp(</span>0</span>, max_width) as </span>usize</span>,
</span>            ));
</span>        }
</span>    }
</span>    line
</span>}
</span></code></pre>
Okay, that's the first version.  It works, but possibly includes duplicate points.  If you're actually trying to rasterize a rectangle pixel by pixel in 2023, then dupes might not matter.</p>
Here's the guaranteed dupe free version, with an option for an unsafe unchecked version:</p>
#[</span>inline</span>]
</span>// ported from https://github.com/ArminJo/Arduino-BlueDisplay/blob/master/src/LocalGUI/ThickLine.hpp
</span>// i strongly discourage use of the unsafe_unchecked version unless you're 100% sure of what you're doing
</span>pub fn </span>get_thick_line</span>(
</span>    </span>from</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>to</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>line_width</span>: </span>usize</span>,
</span>    </span>thick_mode</span>: ThicknessMode,
</span>    </span>map_width</span>: </span>usize</span>,
</span>    </span>unsafe_unchecked</span>: </span>bool</span>,
</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> line: Vec<(</span>isize</span>, </span>isize</span>)> = vec![];
</span>    </span>let mut</span> x1 = from.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> x2 = to.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> y1 = from.</span>1 </span>as </span>isize</span>;
</span>    </span>let mut</span> y2 = to.</span>1 </span>as </span>isize</span>;
</span>    </span>let mut</span> dx: </span>isize</span>;
</span>    </span>let mut</span> dy: </span>isize</span>;
</span>    </span>let</span> dx2: </span>isize</span>;
</span>    </span>let</span> dy2: </span>isize</span>;
</span>    </span>let mut</span> err: </span>isize</span>;
</span>    </span>let mut</span> step_x: </span>isize</span>;
</span>    </span>let mut</span> step_y: </span>isize</span>;
</span>    </span>let</span> max_width = (map_width - </span>1</span>) as </span>isize</span>;
</span>    </span>if</span> x1 > max_width {
</span>        x1 = max_width;
</span>    }
</span>    </span>if</span> x2 > max_width {
</span>        x2 = max_width;
</span>    }
</span>    </span>if</span> y1 > max_width {
</span>        y1 = max_width;
</span>    }
</span>    </span>if</span> y2 > max_width {
</span>        y2 = max_width;
</span>    }
</span>    </span>fn </span>convert</span>(</span>input</span>: &Vec<(</span>isize</span>, </span>isize</span>)>, </span>max_width</span>: </span>isize</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>        </span>let mut</span> retval: Vec<(</span>usize</span>, </span>usize</span>)> = Vec::with_capacity(input.len());
</span>        </span>for</span> x in </span>0</span>..input.len() {
</span>            </span>if</span> input[x].</span>0 </span>>= </span>0
</span>                && input[x].</span>0 </span><= max_width
</span>                && input[x].</span>1 </span>>= </span>0
</span>                && input[x].</span>1 </span><= max_width
</span>            {
</span>                retval.push((input[x].</span>0 </span>as </span>usize</span>, input[x].</span>1 </span>as </span>usize</span>));
</span>            }
</span>        }
</span>        retval
</span>    }
</span>    </span>unsafe fn </span>fast_convert</span>(</span>input</span>: Vec<(</span>isize</span>, </span>isize</span>)>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>        </span>let mut</span> v = std::mem::ManuallyDrop::new(input);
</span>        Vec::from_raw_parts(v.as_mut_ptr() as </span>*mut </span>(</span>usize</span>, </span>usize</span>), v.len(), v.capacity())
</span>    }
</span>    </span>if</span> line_width <= </span>1 </span>{
</span>        line = get_line_overlap2((x1, y1), (x2, y2), </span>LINE_OVERLAP_NONE</span>);
</span>        </span>return </span>convert(&line, max_width);
</span>    }
</span>    dy = x2 - x1;
</span>    dx = y2 - y1;
</span>    </span>let mut</span> swap = true;
</span>    </span>if</span> dx < </span>0 </span>{
</span>        dx = -dx;
</span>        step_x = -</span>1</span>;
</span>        swap = !swap;
</span>    } </span>else </span>{
</span>        step_x = </span>1</span>;
</span>    }
</span>    </span>if</span> dy < </span>0 </span>{
</span>        dy = -dy;
</span>        step_y = -</span>1</span>;
</span>        swap = !swap;
</span>    } </span>else </span>{
</span>        step_y = </span>1</span>;
</span>    }
</span>    dx2 = dx << </span>1</span>;
</span>    dy2 = dy << </span>1</span>;
</span>    </span>let mut</span> overlap: </span>usize</span>;
</span>    </span>let mut</span> draw_start_adjust_count = line_width / </span>2</span>;
</span>    </span>if</span> thick_mode == ThicknessMode::LineThicknessDrawCounterclockwise {
</span>        draw_start_adjust_count = line_width - </span>1</span>;
</span>    } </span>else if</span> thick_mode == ThicknessMode::LineThicknessDrawClockwise {
</span>        draw_start_adjust_count = </span>0</span>;
</span>    }
</span>    </span>if</span> dx >= dy {
</span>        </span>if</span> swap {
</span>            draw_start_adjust_count = (line_width - </span>1</span>) - draw_start_adjust_count;
</span>            step_y = -step_y;
</span>        } </span>else </span>{
</span>            step_x = -step_x;
</span>        }
</span>        err = dy2 - dx;
</span>        </span>for </span>_ in (</span>0</span>..draw_start_adjust_count).rev() {
</span>            x1 -= step_x;
</span>            x2 -= step_x;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                y1 -= step_y;
</span>                y2 -= step_y;
</span>                err -= dx2;
</span>            }
</span>            err += dy2;
</span>        }
</span>        line.append(&</span>mut </span>get_line_overlap2(
</span>            (x1, y1),
</span>            (x2, y2),
</span>            </span>LINE_OVERLAP_NONE</span>,
</span>        ));
</span>        err = dy2 - dx;
</span>        </span>for </span>_ in (</span>1</span>..line_width).rev() {
</span>            x1 += step_x;
</span>            x2 += step_x;
</span>            overlap = </span>LINE_OVERLAP_NONE</span>;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                y1 += step_y;
</span>                y2 += step_y;
</span>                err -= dx2;
</span>                overlap = </span>LINE_OVERLAP_MAJOR</span>;
</span>            }
</span>            err += dy2;
</span>            line.append(&</span>mut </span>get_line_overlap2((x1, y1), (x2, y2), overlap));
</span>        }
</span>    } </span>else </span>{
</span>        </span>if</span> swap {
</span>            step_x = -step_x;
</span>        } </span>else </span>{
</span>            draw_start_adjust_count = (line_width - </span>1</span>) - draw_start_adjust_count;
</span>            step_y = -step_y;
</span>        }
</span>        err = dx2 - dy;
</span>        </span>for </span>_ in (</span>0</span>..draw_start_adjust_count).rev() {
</span>            y1 -= step_y;
</span>            y2 -= step_y;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                x1 -= step_x;
</span>                x2 -= step_x;
</span>                err -= dy2;
</span>            }
</span>            err += dx2;
</span>        }
</span>        line.append(&</span>mut </span>get_line_overlap2(
</span>            (x1, y1),
</span>            (x2, y2),
</span>            </span>LINE_OVERLAP_NONE</span>,
</span>        ));
</span>        err = dx2 - dy;
</span>        </span>for </span>_ in (</span>1</span>..line_width).rev() {
</span>            y1 += step_y;
</span>            y2 += step_y;
</span>            overlap = </span>LINE_OVERLAP_NONE</span>;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                x1 += step_x;
</span>                x2 += step_x;
</span>                err -= dy2;
</span>                overlap = </span>LINE_OVERLAP_MAJOR</span>;
</span>            }
</span>            err += dx2;
</span>            line.append(&</span>mut </span>get_line_overlap2((x1, y1), (x2, y2), overlap));
</span>        }
</span>    }
</span>    </span>if</span> unsafe_unchecked {
</span>        </span>unsafe </span>{ fast_convert(line) }
</span>    } </span>else </span>{
</span>        convert(&line, max_width)
</span>    }
</span>}
</span></code></pre>
and...</p>
#[</span>inline</span>]
</span>// ported from https://github.com/ArminJo/Arduino-BlueDisplay/blob/master/src/LocalGUI/ThickLine.hpp
</span>fn </span>get_line_overlap2</span>(
</span>    </span>from</span>: (</span>isize</span>, </span>isize</span>),
</span>    </span>to</span>: (</span>isize</span>, </span>isize</span>),
</span>    </span>overlap</span>: </span>usize</span>,
</span>) -> Vec<(</span>isize</span>, </span>isize</span>)> {
</span>    </span>let mut</span> x1 = from.</span>0</span>;
</span>    </span>let</span> x2 = to.</span>0</span>;
</span>    </span>let mut</span> y1 = from.</span>1</span>;
</span>    </span>let</span> y2 = to.</span>1</span>;
</span>    </span>let mut</span> line: Vec<(</span>isize</span>, </span>isize</span>)> = vec![];
</span>    </span>let mut</span> dx: </span>isize</span>;
</span>    </span>let mut</span> dy: </span>isize</span>;
</span>    </span>let</span> dx2: </span>isize</span>;
</span>    </span>let</span> dy2: </span>isize</span>;
</span>    </span>let mut</span> err: </span>isize</span>;
</span>    </span>let</span> step_x: </span>isize</span>;
</span>    </span>let</span> step_y: </span>isize</span>;
</span>    dx = x2 - x1;
</span>    dy = y2 - y1;
</span>    </span>if</span> dx < </span>0 </span>{
</span>        dx = -dx;
</span>        step_x = -</span>1</span>;
</span>    } </span>else </span>{
</span>        step_x = </span>1</span>;
</span>    }
</span>    </span>if</span> dy < </span>0 </span>{
</span>        dy = -dy;
</span>        step_y = -</span>1</span>;
</span>    } </span>else </span>{
</span>        step_y = </span>1</span>;
</span>    }
</span>    dx2 = dx << </span>1</span>;
</span>    dy2 = dy << </span>1</span>;
</span>    line.push((x1, y1));
</span>    </span>if</span> dx > dy {
</span>        err = dy2 - dx;
</span>        </span>while</span> x1 != x2 {
</span>            x1 += step_x;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MAJOR </span>!= </span>0 </span>{
</span>                    line.push((x1, y1));
</span>                }
</span>                y1 += step_y;
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MINOR </span>!= </span>0 </span>{
</span>                    line.push((x1 - step_x, y1));
</span>                }
</span>                err -= dx2;
</span>            }
</span>            err += dy2;
</span>            line.push((x1, y1));
</span>        }
</span>    } </span>else </span>{
</span>        err = dx2 - dy;
</span>        </span>while</span> y1 != y2 {
</span>            y1 += step_y;
</span>            </span>if</span> err >= </span>0 </span>{
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MAJOR </span>!= </span>0 </span>{
</span>                    line.push((x1, y1));
</span>                }
</span>                x1 += step_x;
</span>                </span>if</span> overlap & </span>LINE_OVERLAP_MINOR </span>!= </span>0 </span>{
</span>                    line.push((x1, y1 - step_y));
</span>                }
</span>                err -= dy2;
</span>            }
</span>            err += dx2;
</span>            line.push((x1, y1));
</span>        }
</span>    }
</span>    line
</span>}
</span></code></pre>
Above I talked about scary Option #3.  That's calling get_thick_line() with unsafe_unchecked = true.  The unsafe version just directly casts an (isize, isize) vec into a (usize, usize) vec.  If you are 100% damn sure that the entirety of your thick line lies within the canvas boundaries, you'll get a thick line with no duplicates and no crashing.</p>
If, however, you cast negative signed coordinates into unsigned ones, and actually attempt to use them without checking them...Bad Things Will Happen™ (and if you have overflow checks enabled, the app will crash before you can even attempt to crash the app in a different way)</p>
Here are some example images produced using the code, showing proper handling of out of bounds coordinates:</p>
</p>
</p>
</p>
</p>
</p>
</p>
I'll have a crude little benchmark further below.  First though, a little tidbit about a problem I had testing the code originally.  Thick lines in the middle of the canvas were like the ones above, perfectly fine.  But as parts of the line got further out of bounds, the line would take on a trapezoid shape, or sometimes sort of "bounce" off the boundary of the canvas.</p>
Here's an example of the trapezoid anomaly (notice the starting points in yellow are off-center as well):</p>
</p>
I eventually tracked it down to the beginning of the drawLineOverlap function here</a>.
So I snipped out my version of this code:</p>
    </span>if </span>(aXStart >= LOCAL_DISPLAY_WIDTH) {
</span>        aXStart = LOCAL_DISPLAY_WIDTH - </span>1</span>;
</span>    }
</span>
</span>    </span>if </span>(aXEnd >= LOCAL_DISPLAY_WIDTH) {
</span>        aXEnd = LOCAL_DISPLAY_WIDTH - </span>1</span>;
</span>    }
</span>
</span>    </span>if </span>(aYStart >= LOCAL_DISPLAY_HEIGHT) {
</span>        aYStart = LOCAL_DISPLAY_HEIGHT - </span>1</span>;
</span>    }
</span>
</span>    </span>if </span>(aYEnd >= LOCAL_DISPLAY_HEIGHT) {
</span>        aYEnd = LOCAL_DISPLAY_HEIGHT - </span>1</span>;
</span>    }
</span></code></pre>
...and the issue disappeared.  I'm not sure if it's actually a bug in the implementation I copied, or if it was my fault in how I ported the code.  All I know is that it's fixed now in my code =)</p>
Crude benchmark code:</p>
fn </span>test_fat_lines</span>(</span>tests</span>: </span>usize</span>, </span>map_width</span>: </span>usize</span>, </span>line_width</span>: </span>usize</span>) {
</span>    println!(</span>"</span>----------------------- [ test begin ] -----------------------</span>"</span>);
</span>    </span>let mut</span> rng = nanorand::tls_rng();
</span>    </span>let mut</span> points1: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> points2: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> line: Vec<(</span>usize</span>, </span>usize</span>)>;
</span>    </span>let mut</span> len: </span>usize </span>= </span>0</span>;
</span>    </span>for </span>_ in </span>0</span>..tests {
</span>        points1.push((rng.generate_range(</span>0</span>..map_width), rng.generate_range(</span>0</span>..map_width)));
</span>        points2.push((rng.generate_range(</span>0</span>..map_width), rng.generate_range(</span>0</span>..map_width)));
</span>    }
</span>
</span>    </span>let</span> start = Instant::now();
</span>    </span>for</span> x in </span>0</span>..tests {
</span>        line = utils::get_thick_line_unchecked(
</span>            points1[x],
</span>            points2[x],
</span>            line_width,
</span>            utils::ThicknessMode::LineThicknessMiddle,
</span>            map_width,
</span>        );
</span>        len += line.len();
</span>    }
</span>    </span>let</span> duration = start.elapsed();
</span>    println!(
</span>        </span>"</span>{}</span> iterations for get_thick_line_unchecked() took: </span>{:?}</span>"</span>,
</span>        tests, duration
</span>    );
</span>    println!(
</span>        </span>"</span>total pixels = </span>{}</span>, line width = </span>{}</span>, canvas size = </span>{}</span>x</span>{}</span>\n</span>"</span>,
</span>        len, line_width, map_width, map_width
</span>    );
</span>
</span>
</span>    len = </span>0</span>;
</span>    </span>let</span> start = Instant::now();
</span>    </span>for</span> x in </span>0</span>..tests {
</span>        line = utils::get_thick_line(
</span>            points1[x],
</span>            points2[x],
</span>            line_width,
</span>            utils::ThicknessMode::LineThicknessMiddle,
</span>            map_width,
</span>            false,
</span>        );
</span>        len += line.len();
</span>    }
</span>    </span>let</span> duration = start.elapsed();
</span>    println!(
</span>        </span>"</span>{}</span> iterations for get_thick_line() took: </span>{:?}</span>"</span>,
</span>        tests, duration
</span>    );
</span>    println!(
</span>        </span>"</span>total pixels = </span>{}</span>, line width = </span>{}</span>, canvas size = </span>{}</span>x</span>{}</span>\n</span>"</span>,
</span>        len, line_width, map_width, map_width
</span>    );
</span>
</span>    len = </span>0</span>;
</span>    </span>let</span> start = Instant::now();
</span>    </span>for</span> x in </span>0</span>..tests {
</span>        line = utils::get_thick_line(
</span>            points1[x],
</span>            points2[x],
</span>            line_width,
</span>            utils::ThicknessMode::LineThicknessMiddle,
</span>            map_width,
</span>            true,
</span>        );
</span>        len += line.len();
</span>    }
</span>    </span>let</span> duration = start.elapsed();
</span>    println!(
</span>        </span>"</span>{}</span> iterations for get_thick_line(unsafe unchecked) took: </span>{:?}</span>"</span>,
</span>        tests, duration
</span>    );
</span>    println!(
</span>        </span>"</span>total pixels = </span>{}</span>, line width = </span>{}</span>, canvas size = </span>{}</span>x</span>{}</span>\n</span>"</span>,
</span>        len, line_width, map_width, map_width
</span>    );
</span>}
</span></code></pre>
And the results of a few tests with different parameters:</p>
----------------------- [ test begin ] -----------------------
</span>5000 iterations for get_thick_line_unchecked() took: 16.8303775s
</span>total pixels = 1470765496, line width = 500, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line() took: 22.3055861s
</span>total pixels = 1348434370, line width = 500, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line(unsafe unchecked) took: 16.1727516s
</span>total pixels = 1470765496, line width = 500, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>5000 iterations for get_thick_line_unchecked() took: 5.7484512s
</span>total pixels = 589019150, line width = 200, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line() took: 7.6234731s
</span>total pixels = 578641650, line width = 200, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line(unsafe unchecked) took: 5.1384081s
</span>total pixels = 589019150, line width = 200, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>10000 iterations for get_thick_line_unchecked() took: 1.8566417s
</span>total pixels = 294929612, line width = 50, canvas size = 1000x1000
</span>
</span>10000 iterations for get_thick_line() took: 2.2578805s
</span>total pixels = 294517820, line width = 50, canvas size = 1000x1000
</span>
</span>10000 iterations for get_thick_line(unsafe unchecked) took: 1.6420243s
</span>total pixels = 294929612, line width = 50, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>50000 iterations for get_thick_line_unchecked() took: 1.5864814s
</span>total pixels = 288522185, line width = 10, canvas size = 1000x1000
</span>
</span>50000 iterations for get_thick_line() took: 1.854512s
</span>total pixels = 288501000, line width = 10, canvas size = 1000x1000
</span>
</span>50000 iterations for get_thick_line(unsafe unchecked) took: 1.4808243s
</span>total pixels = 288522185, line width = 10, canvas size = 1000x1000
</span></code></pre>
The unsafe code is always fastest of course, but it's not that far behind the version with potential duplicates.  In the pixel counts you can see how many pixels were removed by the version of the function which removes dupes. get_thick_line(unsafe unchecked) will ONLY</em> have duplicates if it has been abused (like in my above tests).</p>
That's all for today, code for this post and the previous one can be found here: https://github.com/0xDEADFED5/dwarf_dreams/tree/main/thick_bresenham</a>.</p>
Kudos and respect to ArminJo</a>, without their code I'd probably still be scratching my head.</p>
quick edit:</p>
I ran the tests again, this time with coordinates firmly within the canvas, but with the same parameters as the previous tests.  As expected, each function call returned the same number of pixels, but with differing performance.  Here are the results:</p>
----------------------- [ test begin ] -----------------------
</span>5000 iterations for get_thick_line_unchecked() took: 22.2698107s
</span>total pixels = 2500000000, line width = 500, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line() took: 30.3468048s
</span>total pixels = 2500000000, line width = 500, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line(unsafe unchecked) took: 19.8017243s
</span>total pixels = 2500000000, line width = 500, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>5000 iterations for get_thick_line_unchecked() took: 8.2815981s
</span>total pixels = 1000000000, line width = 200, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line() took: 11.9877238s
</span>total pixels = 1000000000, line width = 200, canvas size = 1000x1000
</span>
</span>5000 iterations for get_thick_line(unsafe unchecked) took: 7.7751189s
</span>total pixels = 1000000000, line width = 200, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>10000 iterations for get_thick_line_unchecked() took: 3.6693278s
</span>total pixels = 500000000, line width = 50, canvas size = 1000x1000
</span>
</span>10000 iterations for get_thick_line() took: 6.2115697s
</span>total pixels = 500000000, line width = 50, canvas size = 1000x1000
</span>
</span>10000 iterations for get_thick_line(unsafe unchecked) took: 3.3808899s
</span>total pixels = 500000000, line width = 50, canvas size = 1000x1000
</span>
</span>----------------------- [ test begin ] -----------------------
</span>50000 iterations for get_thick_line_unchecked() took: 1.749504s
</span>total pixels = 500000000, line width = 10, canvas size = 1000x1000
</span>
</span>50000 iterations for get_thick_line() took: 2.2008861s
</span>total pixels = 500000000, line width = 10, canvas size = 1000x1000
</span>
</span>50000 iterations for get_thick_line(unsafe unchecked) took: 1.4769065s
</span>total pixels = 500000000, line width = 10, canvas size = 1000x1000
</span></code></pre>


Bresenham, splines, and nipples, oh my!
2023-06-11T00:00:00+00:00
Site update: added Yandex and Twitter login support to the comment system.</p>
Well, I've been testing lots of different hydraulic erosion simulation code on my noise based terrain.  I started off with a pretty complex sim I put together that modeled water sources, water quantity, water velocity, amount of sediment carried in the water, soil deposition rates, etc.  I tested this on 8000x8000 maps of f32</a> data.</p>
Even though my code is fully threaded (current machine: 8 core/16 thread), my simulations were slow.  Incredibly slow.  Like possibly approaching the heat death of the universe for my simulation to end slow.  After several iterations of simplifying the simulation I had a realization.  I don't need some crazy full map erosion process that takes forever.  I mostly want mountains that don't suck and realistic looking rivers.  After I get that nailed I can get to work implementing terrain for specific biomes.</p>
My new simplified river plan is this, for each river:</p>

Starting point is a mountain peak which receives precipitation</li>
Chart a path downhill</li>
Widen river as conditions are met</li>
End the river at an existing body of water, a depression to form a lake, or the edge of map</li>
Also, erosion only at this time</li>
When water physics is added in I'll add evaporation and take another look at sediment deposits</li>
A modest speed increase: I was sharing one map between 16 threads and using reader/writer locking semantics.  I didn't measure contention, but I suspected a fair amount.  So now I create 16 copies of the map, give each thread their own version to work on, and merge the maps back together at the end.  I got around a 10% performance boost.  RAM is meant to be used right? ....right?</li>
</ul>
I'm not going to traverse every single tile/pixel downhill.  First off, it's friggin' slow.  Secondly, it produces some weird shaped rivers.  That can be fixed by running a gazillion iterations, but that's one of the things I'm trying to avoid.  So instead I'm going to search for the lowest point in a set radius, draw a line to it, and keep doing that until I have a river of attached line segments.</p>
First I need to be able to draw a line from x1, y1 to x2, y2, preferably as quickly as possible.  Luckily for me, some guy named Jack Elton Bresenham</a> came up with a fast algorithm called Bresenham's line algorithm</a> back in 1962.</p>
This</a> page has some cool examples of Bresenham's algorithm in C. (for some reason uBlock hates this page now, but as of this blog post date that page is safe and ad-free)</p>
Here's my Rust implementation based on the code from that page:</p>
// adapted from http://members.chello.at/%7Eeasyfilter/bresenham.html
</span>#[</span>inline</span>]
</span>pub fn </span>get_line_unchecked</span>(</span>from</span>: (</span>usize</span>, </span>usize</span>), </span>to</span>: (</span>usize</span>, </span>usize</span>)) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> line: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> ix0 = from.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> iy0 = from.</span>1 </span>as </span>isize</span>;
</span>    </span>let</span> ix1 = to.</span>0 </span>as </span>isize</span>;
</span>    </span>let</span> iy1 = to.</span>1 </span>as </span>isize</span>;
</span>    </span>let</span> dx: </span>isize </span>= ix1.abs_diff(ix0) as </span>isize</span>;
</span>    </span>let mut</span> sx: </span>isize </span>= </span>1</span>;
</span>    </span>if</span> ix0 > ix1 {
</span>        sx = -</span>1</span>;
</span>    }
</span>    </span>let</span> dy: </span>isize </span>= -(iy1.abs_diff(iy0) as </span>isize</span>);
</span>    </span>let mut</span> sy: </span>isize </span>= </span>1</span>;
</span>    </span>if</span> iy0 > iy1 {
</span>        sy = -</span>1
</span>    }
</span>    </span>let mut</span> err = dx + dy;
</span>    </span>let mut</span> e2: </span>isize </span>= </span>0</span>;
</span>    </span>loop </span>{
</span>        line.push((ix0 as </span>usize</span>, iy0 as </span>usize</span>));
</span>        </span>if</span> ix0 == ix1 && iy0 == iy1 {
</span>            </span>break</span>;
</span>        }
</span>        e2 = </span>2 </span>* err;
</span>        </span>if</span> e2 >= dy {
</span>            err += dy;
</span>            ix0 += sx;
</span>        }
</span>        </span>if</span> e2 <= dx {
</span>            err += dx;
</span>            iy0 += sy;
</span>        }
</span>    }
</span>    line
</span>}
</span></code></pre>
I called it get_line_unchecked because it's naive.  I intend to draw lines on a specific canvas/map size within only the positive coordinate plane.  This function doesn't know that.  It just gives me a vector of points from one point to the other.  Since I intend to use each of these points as a direct memory index into my map I want to make sure I return valid points.  If a line extends off my map and I try to use that as a map index I'm going to experience 1 of 2 possible errors:  Best case scenario, I index the wrong location on the map and modify something where I didn't intend to.  Worst case scenario is I attempt to index beyond my maximum map size, which results in an array out of bounds error, or memory access exception, or whatever happens in Rust...and then the application crashes.  Also, this function uses signed integers (to support the entire coordinate plane), so if I get a negative coordinate and cast it to an unsigned integer for my map index one of those 2 possible errors I just wrote about is going to happen.  It's definitely more likely to be the crashy kind of error though.</p>
So I updated the code a bit.  I still use that previous code when I know it's safe and I want extra speed.  Otherwise I use this one:</p>
// adapted from http://members.chello.at/%7Eeasyfilter/bresenham.html
</span>/// if remove_out_of_bounds = false, points outside of map will be placed on edge of map
</span>/// if remove_out_of_bounds = true, points outside of map will be ignored
</span>#[</span>inline</span>]
</span>pub fn </span>get_line</span>(
</span>    </span>from</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>to</span>: (</span>usize</span>, </span>usize</span>),
</span>    </span>map_width</span>: </span>usize</span>,
</span>    </span>remove_out_of_bounds</span>: </span>bool</span>,
</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> line: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> ix0 = from.</span>0 </span>as </span>isize</span>;
</span>    </span>let mut</span> iy0 = from.</span>1 </span>as </span>isize</span>;
</span>    </span>let</span> ix1 = to.</span>0 </span>as </span>isize</span>;
</span>    </span>let</span> iy1 = to.</span>1 </span>as </span>isize</span>;
</span>    </span>let</span> dx: </span>isize </span>= ix1.abs_diff(ix0) as </span>isize</span>;
</span>    </span>let mut</span> sx: </span>isize </span>= </span>1</span>;
</span>    </span>let</span> imap_width = map_width as </span>isize</span>;
</span>    </span>let mut</span> clamped: </span>bool </span>= false;
</span>    </span>let mut </span>try_clamp </span>= |</span>coord</span>: </span>isize</span>| -> </span>usize </span>{
</span>        </span>if</span> coord > imap_width - </span>1 </span>{
</span>            clamped = true;
</span>            map_width - </span>1
</span>        } </span>else if</span> coord < </span>0 </span>{
</span>            clamped = true;
</span>            </span>0
</span>        } </span>else </span>{
</span>            coord as </span>usize
</span>        }
</span>    };
</span>
</span>    </span>if</span> ix0 > ix1 {
</span>        sx = -</span>1</span>;
</span>    }
</span>    </span>let</span> dy: </span>isize </span>= -(iy1.abs_diff(iy0) as </span>isize</span>);
</span>    </span>let mut</span> sy: </span>isize </span>= </span>1</span>;
</span>    </span>if</span> iy0 > iy1 {
</span>        sy = -</span>1
</span>    }
</span>    </span>let mut</span> err = dx + dy;
</span>    </span>let mut</span> e2: </span>isize </span>= </span>0</span>;
</span>    </span>let mut</span> skip_point: </span>bool</span>;
</span>    </span>loop </span>{
</span>        skip_point = false;
</span>        </span>if</span> remove_out_of_bounds
</span>            && (ix0 > imap_width - </span>1 </span>|| ix0 < </span>0 </span>|| iy0 > imap_width - </span>1 </span>|| iy0 < </span>0</span>)
</span>        {
</span>            skip_point = true;
</span>        }
</span>        </span>if </span>!skip_point {
</span>            line.push((try_clamp(ix0), try_clamp(iy0)))
</span>        };
</span>        </span>if</span> ix0 == ix1 && iy0 == iy1 {
</span>            </span>break</span>;
</span>        }
</span>        e2 = </span>2 </span>* err;
</span>        </span>if</span> e2 >= dy {
</span>            err += dy;
</span>            ix0 += sx;
</span>        }
</span>        </span>if</span> e2 <= dx {
</span>            err += dx;
</span>            iy0 += sy;
</span>        }
</span>    }
</span>    </span>if </span>!remove_out_of_bounds && clamped {
</span>        line.sort();
</span>        line.dedup();
</span>    }
</span>    line
</span>}
</span></code></pre>
My points are usize tuples because for better or worse, Rust forces slice indexes to be usize... and I use my points as indexes.  I only have a width parameter because my game maps are square.  I use the try_clamp closure so that I can keep track if any points have extended beyond the map.  I keep track because if I'm not removing out of bounds map points and placing them on the map edge I could end up with duplicate points.  The whole reason I want these points is so I can manipulate those parts of the map.  I don't want to perform those manipulations multiple times at the same location on accident.  Therefore, if any point locations have been clamped, I know that I need to sort my vector and remove possible duplicates.</p>
Here are some totally random lines representing a sort of rudimentary river:</p>
</p>
Okay, next thing I need to do is figure out how to select a circle of pixels/tiles.  For every point along my "river", depending on how wide the river is, I want to select a circle of tiles, and erode earth from that circle, eroding less earth the further out from the center of the circle that I go.  I sort of picture it like dragging an ice cream cone shape along the path of the river, carving out the earth using that shape.</p>
On that page I shared earlier</a> there's a version of Bresenham's algorithm for circles</a>.  Here's my Rust version for a filled circle:</p>
#[</span>inline</span>]
</span>//adapted from http://members.chello.at/%7Eeasyfilter/bresenham.html
</span>pub fn </span>get_full_circle</span>(</span>xm</span>: </span>usize</span>, </span>ym</span>: </span>usize</span>, </span>mut </span>r</span>: </span>isize</span>, </span>width</span>: </span>usize</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let</span> xm = xm as </span>isize</span>;
</span>    </span>let</span> ym = ym as </span>isize</span>;
</span>    </span>//let mut r = r as isize;
</span>    </span>let</span> width2 = width as </span>isize</span>;
</span>
</span>    </span>let mut</span> x = -r;
</span>    </span>let mut</span> y = </span>0</span>;
</span>    </span>let mut</span> err: </span>isize </span>= </span>2 </span>- </span>2 </span>* r;
</span>    </span>let mut</span> empty_circle: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> full_circle: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> clamped: </span>bool </span>= false;
</span>    </span>// need to know if clamp has ocurred, as this can create duplicate coordinates
</span>    </span>let mut </span>try_clamp </span>= |</span>coord</span>: </span>isize</span>| -> </span>usize </span>{
</span>        </span>if</span> coord < </span>0 </span>{
</span>            clamped = true;
</span>            </span>0
</span>        } </span>else if</span> coord >= width2 {
</span>            clamped = true;
</span>            width - </span>1
</span>        } </span>else </span>{
</span>            coord as </span>usize
</span>        }
</span>    };
</span>    </span>while</span> x < </span>0 </span>{
</span>        empty_circle.push((try_clamp(xm - x), try_clamp(ym + y)));
</span>        empty_circle.push((try_clamp(xm - y), try_clamp(ym - x)));
</span>        empty_circle.push((try_clamp(xm + x), try_clamp(ym - y)));
</span>        empty_circle.push((try_clamp(xm + y), try_clamp(ym + x)));
</span>        r = err;
</span>        </span>if</span> r <= y {
</span>            y += </span>1</span>;
</span>            err += y * </span>2 </span>+ </span>1</span>;
</span>        }
</span>        </span>if</span> r > x || err > y {
</span>            x += </span>1</span>;
</span>            err += x * </span>2 </span>+ </span>1</span>;
</span>        }
</span>    }
</span>    </span>if</span> empty_circle.len() == </span>0 </span>{
</span>        </span>return</span> empty_circle;
</span>    }
</span>    </span>// sort by X axis. this allows easy dupe removal and quickly getting coords inside the circle
</span>    empty_circle.sort();
</span>    </span>if</span> clamped {
</span>        empty_circle.dedup();
</span>    }
</span>    </span>let mut</span> lowest = </span>usize</span>::</span>MAX</span>;
</span>    </span>let mut</span> highest = </span>0</span>;
</span>    </span>let mut</span> current = empty_circle.first().unwrap().</span>0</span>;
</span>    </span>//find lowest Y and highest Y for each X coordinate, and grab all the coords between (inclusive)
</span>    </span>for</span> p in empty_circle {
</span>        </span>if</span> p.</span>0 </span>!= current {
</span>            </span>for</span> y in lowest..=highest {
</span>                full_circle.push((current, y));
</span>            }
</span>            lowest = </span>usize</span>::</span>MAX</span>;
</span>            highest = </span>0</span>;
</span>            current = p.</span>0</span>;
</span>        }
</span>        </span>if</span> p.</span>1 </span>< lowest {
</span>            lowest = p.</span>1</span>;
</span>        }
</span>        </span>if</span> p.</span>1 </span>> highest {
</span>            highest = p.</span>1</span>;
</span>        }
</span>    }
</span>    </span>for</span> y in lowest..=highest {
</span>        full_circle.push((current, y));
</span>    }
</span>    full_circle
</span>}
</span></code></pre>
This code looks quite a bit different because I want a filled circle, not just a circle outline.  Basically for each horizontal (X) step across the circle diameter I make, I fill in the whole range of vertical (Y) pixels. You could do it the other way around too, of course.  I think it could be optimized a bit, but it works.  Oh yeah, and like before I've got my clamping code in there in case I try to get a filled circle too close to the map edge.</p>
Here's an example:</p>
</p>
I'm going to be drawing lots of circles though, and I wasn't sure if that was the fastest way to get a filled circle.  So I wrote a more naive version.  Basically, if I want a filled circle of radius R, first get the tiles/pixels in a square 2R X 2R centered where I want the circle.  Then calculate the distance of each pixel from the center.  If it's less than or equal to the circle radius I want, then it's inside the circle.</p>
Here's my code for that version:</p>
#[</span>inline</span>]
</span>pub fn </span>get_full_circle_naive</span>(
</span>    </span>xm</span>: </span>usize</span>,
</span>    </span>ym</span>: </span>usize</span>,
</span>    </span>r</span>: </span>f32</span>,
</span>    </span>width</span>: </span>usize</span>,
</span>    </span>with_nipples</span>: </span>bool</span>,
</span>    </span>ignore_center</span>: </span>bool</span>,
</span>) -> Vec<(</span>usize</span>, </span>usize</span>)> {
</span>    </span>let mut</span> full_circle: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> nipples: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let</span> startx: </span>usize</span>;
</span>    </span>let</span> starty: </span>usize</span>;
</span>    </span>let</span> endx: </span>usize</span>;
</span>    </span>let</span> endy: </span>usize</span>;
</span>    </span>let</span> r2 = r as </span>usize</span>;
</span>    </span>if</span> xm >= r2 {
</span>        startx = xm - r2;
</span>    } </span>else </span>{
</span>        startx = </span>0</span>;
</span>    }
</span>    </span>if</span> ym >= r2 {
</span>        starty = ym - r2;
</span>    } </span>else </span>{
</span>        starty = </span>0</span>;
</span>    }
</span>    </span>if</span> width > xm + r2 {
</span>        endx = xm + r2
</span>    } </span>else </span>{
</span>        endx = width - </span>1</span>;
</span>    }
</span>    </span>if</span> width > ym + r2 {
</span>        endy = ym + r2;
</span>    } </span>else </span>{
</span>        endy = width - </span>1</span>;
</span>    }
</span>    </span>let mut</span> dist: </span>f32</span>;
</span>    </span>for</span> x in startx..=endx {
</span>        </span>for</span> y in starty..=endy {
</span>            </span>if</span> ignore_center && x == xm && y == ym {
</span>                </span>continue</span>;
</span>            }
</span>            dist = distance((x, y), (xm, ym));
</span>            </span>if</span> dist < r {
</span>                full_circle.push((x, y));
</span>            } </span>else if</span> with_nipples && dist == r {
</span>                nipples.push((x, y));
</span>            }
</span>        }
</span>    }
</span>    </span>if</span> with_nipples {
</span>        full_circle.append(&</span>mut</span> nipples);
</span>    }
</span>    full_circle
</span>}
</span></code></pre>
Oh right, and my terribly complex distance function:</p>
#[</span>inline</span>(always)]
</span>pub fn </span>distance</span>(</span>pos1</span>: (</span>usize</span>, </span>usize</span>), </span>pos2</span>: (</span>usize</span>, </span>usize</span>)) -> </span>f32 </span>{
</span>    </span>let</span> x1 = pos1.</span>0 </span>as </span>f32</span>;
</span>    </span>let</span> x2 = pos2.</span>0 </span>as </span>f32</span>;
</span>    </span>let</span> y1 = pos1.</span>1 </span>as </span>f32</span>;
</span>    </span>let</span> y2 = pos2.</span>1 </span>as </span>f32</span>;
</span>    ((x1 - x2).powi(</span>2</span>) + (y1 - y2).powi(</span>2</span>)).sqrt()
</span>}
</span></code></pre>
Here's what it looks like rendered:</p>
</p>
Well, that's what it looked like with the first version I made.  I call it the nipple version because well...I mean look at it.  I usually like nipples, but not those ones.  So I modified the code a bit, version 2 is what you see above.  And here's an example of calling the function + with_nipples = false:</p>
</p>
Much better.  The Bresenham one still looks the best IMO though.  What about performance?</p>
Here's my janky test code (using nanorand</a> crate).  I didn't look at the assembly code that was emitted, but I tried to stop the compiler from cheating:</p>
fn </span>test_circles</span>(</span>test_num</span>: </span>usize</span>, </span>r</span>: </span>isize</span>) {
</span>    debug_assert!(r > </span>0</span>);
</span>    </span>let</span> rf = r as </span>f32</span>;
</span>    </span>let mut</span> points: Vec<(</span>usize</span>, </span>usize</span>)> = vec![];
</span>    </span>let mut</span> rng = nanorand::tls_rng();
</span>    </span>for </span>_ in </span>0</span>..test_num {
</span>        points.push((rng.generate_range(</span>0</span>..</span>10_000</span>), rng.generate_range(</span>0</span>..</span>10_000</span>)));
</span>    }
</span>    </span>let mut</span> circle: Vec<(</span>usize</span>, </span>usize</span>)>;
</span>    </span>let mut</span> len: </span>usize </span>= </span>0</span>;
</span>    </span>let</span> start = Instant::now();
</span>    </span>for</span> x in </span>0</span>..test_num {
</span>        circle = utils::get_full_circle(points[x].</span>0</span>, points[x].</span>1</span>, r, </span>11000</span>);
</span>        len += circle.len();
</span>    }
</span>    </span>let</span> duration = start.elapsed();
</span>    println!(</span>"</span>len = </span>{}</span>"</span>, len);
</span>    println!(
</span>        </span>"</span>{}</span> iterations for utils::get_full_circle() took: </span>{:?}</span>"</span>,
</span>        test_num, duration
</span>    );
</span>
</span>    len = </span>0</span>;
</span>    </span>let</span> start = Instant::now();
</span>    </span>for</span> x in </span>0</span>..test_num {
</span>        circle = utils::get_full_circle_naive(points[x].</span>0</span>, points[x].</span>1</span>, rf, </span>11000</span>, true, false);
</span>        len += circle.len();
</span>    }
</span>    </span>let</span> duration = start.elapsed();
</span>    println!(</span>"</span>len = </span>{}</span>"</span>, len);
</span>    println!(
</span>        </span>"</span>{}</span> iterations for utils::get_full_circle_naive() took: </span>{:?}</span>"</span>,
</span>        test_num, duration
</span>    );
</span>}
</span></code></pre>
I tested with a large number of smallish circles, and a smaller number of larger circles, results were not terribly different.
In debug mode with opt-level = 0, the naive filled circle code is just over 4x slower!  Ouch.  I tried integer math that I ported from C for sqrt and distance calculations, and it was freakishly slower yet.
In release mode with opt-level = 3, the bresenham filled circle code is 28% faster with MSVC toolchain, and it's about 10% faster with GNU toolchain.  Interestingly, the integer code is slower with GNU, and the floating point code is faster with GNU, explaining the narrowed performance difference.</p>
This brings me back to the first picture I posted above.  My sharp angled, not very river-like line.
This thing:</p>
</p>
I'm no riverologist, but I don't think rivers usually have those sharp angles.  So now I need to smooth em out.  I did a bit of searching and it looks like I want some cubic splines</a>.  I used the first crate I found, cubic_spline</a> and after running those same points through it, here's my result:</p>
</p>
Much better.  Of course, after I finished up that rendering code, I found another crate called splines</a>.  I haven't tested it yet, but I will, because I'm curious about any differences.  Next post I'll hopefully have some mountainy mountains and rivery rivers to show off.</p>
In other news, rust-analyzer started doing this thing where it would reindex everything</em> every new line of code I typed. Sucked a lot.  I trashed everything Rust related on the machine and started out clean, now my coding sanity is restored.  Well, as much as it could be anyhow.</p>


Terrain generation
2023-03-28T00:00:00+00:00
I've started work on generating terrain.  I've settled on world sizes of 64,000x64,000, 48,000x48,000, and 32,000x32,000 tiles.  For speed and memory efficiency I'm going to generate terrain data at 1/4 that detail and then scale it up.  That means I'll be generating between 8,000^2 and 16,000^2 terrain data points.</p>
My plan is to generate some nice looking noise, and use that as a height map...pretty standard for game design.  I've tested out a number of algorithms:  Different flavors of Perlin</a>, Simplex</a>, OpenSimplex</a>, and fractional Brownian motion</a>.</p>
Regarding libraries, at first I did a lot of experimentation with noise-rs</a>.  It's really nice and quite flexible.  When I wanted to scale up the size of the noise maps produced it got quite slow, however.  So I transitioned to simdnoise</a>.  Noise-rs has a really useful utility for rendering noise maps here</a>, so I refactored it to work with bytes instead of floating points, which works better for my current workflow.  Any color renders below were produced using a customized version of color_gradient.rs from noise-rs.</p>
I settled on fractional Brownian motion(fBm) because I like how it looks and how flexible it is.  Here's a scaled down version of 8,000^2 fBm rendered in greyscale:</p>
</p>
Just looks like a cloud, right?  Well, if we imagine that black is the lowest elevation, white is the highest elevation, and render the varying "heights" of each pixel according to a handful of colors to represent water, land, grass, mountains, snow, etc. (10 base colors + blending)...we get a result like this:</p>
</p>
That's with water level set at 88 Z levels out of 256.  It's kinda cool.  Sort of get a shape of the continent with bodies of water.  The mountains kinda suck though.  They're just clouds pasted on top of the land.</p>
Various noise functions can be tweaked to produce ridge patterns.  For instance here's one that I scaled down from 8,000^2, rendered in greyscale:</p>
</p>
Here's what it looks like if it's rendered in color as before, with water level at 88:</p>
</p>
Yikes.  It's pretty ugly, but the mountain ranges are more realistic with ridges that look decent.  What if I blended the two noise maps together, and then rendered the result? Simdnoise returns a Vec<f32></code> for generated noise maps.</p>
Here's my code for blending two maps (dest is one of the noise maps being blended, it will be overwritten):</p>
fn </span>blend</span>(</span>dest</span>: &</span>mut </span>Vec<</span>f32</span>>, </span>other</span>: &Vec<</span>f32</span>>, </span>amount</span>: </span>f32</span>) {
</span>    </span>for</span> x in </span>0</span>..dest.len() {
</span>        </span>if</span> other[x] > dest[x] {
</span>            dest[x] = (other[x]-dest[x]).mul_add(amount, dest[x]);
</span>        }
</span>        </span>else </span>{
</span>            dest[x] = (dest[x]-other[x]).mul_add(amount, other[x]);
</span>        }
</span>    }
</span>}
</span></code></pre>
Pretty basic, but here's the results for 0.5 blend from 48 water level to 128 water level in 20 Z level increments:</p>
</p>
</p>
</p>
</p>
</p>
I'm pretty happy with the shape of the worlds I'm generating now.  But the mountains still need a little love, and I have to figure out where to place rivers.  So next post I'll be playing around with simulating hydraulic erosion.</p>


Fun With Data Structures
2023-03-22T00:00:00+00:00
Okay, time for my first post.  This is my first project in Rust.  I'm an amateur coder, so don't expect anything too mind-blowing here.</p>
I played around with terrain generation a bit, but I got ahead of myself.  Before I can start to really plan out the scope of this thing I need to see what my limits are regarding RAM and disk space usage.  Dwarf Fortress' largest worlds are I believe somewhere around 196608x196608x256? tiles.  That's an impressive number.  If I were to store just a single byte for each of those tiles it would add up to over 9.2 TiB of data.</p>
So obviously some kind of sparse data structure is going to be helpful here. B-trees</a> seem like a good fit, and the Rust stdlib has something called BTreeMap</a>.  With a B-tree I can have a huge blank canvas, but only save the x,y,z coordinates of discovered or modified tiles.</p>
After thinking a bit, I decided to limit map size to 65536x65536x256 tiles. It's plenty huge to get started with, and my coordinates of (u16,u16,u8) will take up less memory.</p>
I put together a little test.  I wanted to test how fast I could insert 2,000,000 triplets of (u16,u16,u8) into a B-tree, write it to disk, read it back from disk, and compare each value to make sure that I serialized and deserialized all the data correctly.  For serialization I'll be using bincode</a>.</p>
Results for flat b-tree as XYZ: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 598.6441ms, maximum time = 659.3505ms, mean time = 616.20734ms, file size = 11.44 MiB
</span></code></pre>
Well that doesn't seem terrible. 2 million inserts, serialize/deserialize, and 2 million compares on a B-tree in 663ms...and bincode even worked some magic to make the file nice and compact.  I wonder though, would reordering the coordinates to z,x,y speed it up since I only have 256 Z levels, but 64,000 width/height?</p>
I think I'll extend this test in the most ugly way possible and find out.</p>
Results for flat b-tree as XYZ: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 607.8915ms, maximum time = 656.4575ms, mean time = 619.5944ms, file size = 11.44 MiB
</span>
</span>Results for flat b-tree as ZXY: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 867.2752ms, maximum time = 900.4827ms, mean time = 881.3094ms, file size = 11.44 MiB
</span></code></pre>
Well that's no good, it's slower.  I can't help but think there must be a more efficient way to store these coordinates.  If I nest the B-trees I can index individual B-trees with just u8.</p>
My first attempt looks like this:</p>
pub struct </span>DeeplyNestedBTree</span><T> {
</span>    </span>buf</span>: BTreeMap<</span>u8</span>, Branch4<T>>,
</span>}
</span>
</span>struct </span>Branch</span><T> {
</span>    </span>pub </span>buf</span>: BTreeMap<</span>u8</span>, T>
</span>}
</span>
</span>struct </span>Branch2</span><T> {
</span>    </span>pub </span>buf</span>: BTreeMap<</span>u8</span>, Branch<T>>
</span>}
</span>
</span>struct </span>Branch3</span><T> {
</span>    </span>pub </span>buf</span>: BTreeMap<</span>u8</span>, Branch2<T>>
</span>}
</span>
</span>struct </span>Branch4</span><T> {
</span>    </span>pub </span>buf</span>: BTreeMap<</span>u8</span>, Branch3<T>>
</span>}
</span></code></pre>
It didn't go well.  For every get and insert it has to traverse all</em> the B-trees.  Here are the test results:</p>
Results for flat b-tree as XYZ: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 586.4466ms, maximum time = 639.7647ms, mean time = 602.92022ms, file size = 11.44 MiB
</span>
</span>Results for flat b-tree as ZXY: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 850.5513ms, maximum time = 863.2367ms, mean time = 858.85032ms, file size = 11.44 MiB
</span>
</span>Results for deeply nested btree: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 1.5296632s, maximum time = 1.5497243s, mean time = 1.5375108s, file size = 22.08 MiB
</span></code></pre>
It's not only slower, but takes up a lot more memory. I had another idea.  I could transform (u16,u16,u8) coordinates into a group of shallower nested B-trees.  Here's what I came up with:</p>
pub struct </span>NestedBTree</span><T> {
</span>    </span>x65535y65535</span>: BTreeMap<(</span>u8</span>, </span>u8</span>), BTreeMap<(</span>u8</span>, </span>u8</span>, </span>u8</span>), T>>,
</span>    </span>x65535y255</span>: BTreeMap<(</span>u8</span>, </span>u8</span>), BTreeMap<(</span>u8</span>, </span>u8</span>, </span>u8</span>), T>>,
</span>    </span>x255y65535</span>: BTreeMap<(</span>u8</span>, </span>u8</span>), BTreeMap<(</span>u8</span>, </span>u8</span>, </span>u8</span>), T>>,
</span>    </span>x255y255</span>: BTreeMap<(</span>u8</span>, </span>u8</span>), BTreeMap<(</span>u8</span>, </span>u8</span>, </span>u8</span>), T>>,
</span>}
</span>
</span>impl</span><T> </span>NestedBTree</span><T> {
</span>    </span>pub fn </span>new</span>() -> NestedBTree<T> {
</span>        </span>Self </span>{
</span>            x255y255: BTreeMap::new(),
</span>            x255y65535: BTreeMap::new(),
</span>            x65535y255: BTreeMap::new(),
</span>            x65535y65535: BTreeMap::new(),
</span>        }
</span>    }
</span>    #[</span>inline</span>(always)]
</span>    </span>pub fn </span>get </span>(&</span>self</span>, </span>x</span>:</span>u16</span>, </span>y</span>:</span>u16</span>, </span>z</span>: </span>u8</span>) -> Result<&T, &</span>'static str</span>> {
</span>        </span>if</span> x > </span>255 </span>&& y > </span>255 </span>{
</span>            </span>let</span> child = </span>match </span>self</span>.x65535y65535.get(&((x/</span>256</span>) as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>let</span> child = </span>match</span> child.get(&((x%</span>256</span>) as </span>u8</span>, (y%</span>256</span>) as </span>u8</span>, z)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>return </span>Ok(&child);
</span>        }
</span>        </span>else if</span> x > </span>255 </span>{
</span>            </span>let</span> child = </span>match </span>self</span>.x65535y255.get(&((x/</span>256</span>) as </span>u8</span>, y as </span>u8</span>)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>let</span> child = </span>match</span> child.get(&((x%</span>256</span>) as </span>u8</span>, y as </span>u8</span>, z)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>return </span>Ok(&child);
</span>        }
</span>        </span>else if</span> y > </span>255 </span>{
</span>            </span>let</span> child = </span>match </span>self</span>.x255y65535.get(&(x as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>let</span> child = </span>match</span> child.get(&(x as </span>u8</span>, (y%</span>256</span>) as </span>u8</span>, z)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>return </span>Ok(&child);
</span>        }
</span>        </span>else </span>{
</span>            </span>let</span> child = </span>match </span>self</span>.x255y255.get(&(x as </span>u8</span>, y as </span>u8</span>)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>let</span> child = </span>match</span> child.get(&(x as </span>u8</span>, y as </span>u8</span>, z)) {
</span>                None => </span>return </span>Err(</span>"</span>Not found</span>"</span>),
</span>                Some(s) => s,
</span>            };
</span>            </span>return </span>Ok(&child);
</span>        }
</span>    }
</span>    #[</span>inline</span>(always)]
</span>    </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>x</span>:</span>u16</span>, </span>y</span>:</span>u16</span>, </span>z</span>: </span>u8</span>, </span>value</span>: T) {
</span>        </span>if</span> x > </span>255 </span>&& y > </span>255 </span>{
</span>            </span>if </span>!</span>self</span>.x65535y65535.contains_key(&((x/</span>256</span>) as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)){
</span>                </span>self</span>.x65535y65535.insert(((x/</span>256</span>) as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>), BTreeMap::new());
</span>            }
</span>            </span>let</span> child = </span>self</span>.x65535y65535.get_mut(&((x/</span>256</span>) as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)).unwrap();
</span>            child.insert(((x%</span>256</span>) as </span>u8</span>, (y%</span>256</span>) as </span>u8</span>, z), value);
</span>        }
</span>        </span>else if</span> x > </span>255 </span>{
</span>            </span>if </span>!</span>self</span>.x65535y255.contains_key(&((x/</span>256</span>) as </span>u8</span>, y as </span>u8</span>)){
</span>                </span>self</span>.x65535y255.insert(((x/</span>256</span>) as </span>u8</span>, y as </span>u8</span>), BTreeMap::new());
</span>            }
</span>            </span>let</span> child = </span>self</span>.x65535y255.get_mut(&((x/</span>256</span>) as </span>u8</span>, y as </span>u8</span>)).unwrap();
</span>            child.insert(((x%</span>256</span>) as </span>u8</span>, y as </span>u8</span>, z), value);
</span>        }
</span>        </span>else if</span> y > </span>255 </span>{
</span>            </span>if </span>!</span>self</span>.x255y65535.contains_key(&(x as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)){
</span>                </span>self</span>.x255y65535.insert((x as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>), BTreeMap::new());
</span>            }
</span>            </span>let</span> child = </span>self</span>.x255y65535.get_mut(&(x as </span>u8</span>, (y/</span>256</span>) as </span>u8</span>)).unwrap();
</span>            child.insert((x as </span>u8</span>, (y%</span>256</span>) as </span>u8</span>, z), value);
</span>        }
</span>        </span>else </span>{
</span>            </span>if </span>!</span>self</span>.x255y255.contains_key(&(x as </span>u8</span>, y as </span>u8</span>)){
</span>                </span>self</span>.x255y255.insert((x as </span>u8</span>, y as </span>u8</span>), BTreeMap::new());
</span>            }
</span>            </span>let</span> child = </span>self</span>.x255y255.get_mut(&(x as </span>u8</span>, y as </span>u8</span>)).unwrap();
</span>            child.insert((x as </span>u8</span>, y as </span>u8</span>, z), value);
</span>        }
</span>    }
</span>}
</span></code></pre>
And the results are much better (it's the last test).  Around 25% faster and a good bit smaller memory footprint too:</p>
Results for flat b-tree as XYZ: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 590.74ms, maximum time = 673.3517ms, mean time = 615.34436ms, file size = 11.44 MiB
</span>
</span>Results for flat b-tree as ZXY: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 857.0916ms, maximum time = 877.4087ms, mean time = 865.27806ms, file size = 11.44 MiB
</span>
</span>Results for deeply nested btree: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 1.5320715s, maximum time = 1.5865624s, mean time = 1.54960012s, file size = 22.08 MiB
</span>
</span>Results for nested btree: # of tests: 5, iterations = 2000000, width = 64000, serialize=true
</span>minimum time = 432.0617ms, maximum time = 450.3567ms, mean time = 440.51756ms, file size = 7.78 MiB
</span></code></pre>
I want to do one more test.  I want to see what 100,000,000 bytes inserted into the different B-trees looks like:</p>
x,y,z insert finished...23.9162098s elapsed
</span>x,y,z write finished...2.5539319s elapsed
</span>x,y,z read finished...9.7052168s elapsed
</span>x,y,z compare finished...33.5086576s elapsed
</span>Data is the same = true
</span>total elapsed = 71.030024s, iterations = 100000000, dimenions = 64000 X 64000 X 256, serialize/deserialize = true
</span>
</span>z,x,y insert finished...30.5735892s elapsed
</span>z,x,y write finished...3.8273196s elapsed
</span>z,x,y read finished...11.3962669s elapsed
</span>z,x,y compare finished...46.5156209s elapsed
</span>Data is the same = true
</span>total elapsed = 94.0144163s, iterations = 100000000, dimenions = 256 X 64000 X 64000, serialize/deserialize = true
</span>
</span>deeply nested insert finished...40.1103969s elapsed
</span>deeply nested write finished...10.5372181s elapsed
</span>deeply nested read finished...14.5991097s elapsed
</span>deeply nested compare finished...48.1683471s elapsed
</span>Data is the same = true
</span>total elapsed = 127.0032545s, iterations = 100000000, dimenions = 64000 X 64000 X 256, serialize/deserialize = true
</span>
</span>nested insert finished...16.0633079s elapsed
</span>nested write finished...1.4942538s elapsed
</span>nested read finished...10.3389582s elapsed
</span>nested compare finished...17.84099s elapsed
</span>Data is the same = true
</span>total elapsed = 48.4802249s, iterations = 100000000, dimenions = 64000 X 64000 X 256, serialize/deserialize = true
</span>
</span>Results for flat b-tree as XYZ: # of tests: 1, iterations = 100000000, width = 64000, serialize=true
</span>minimum time = 71.030024s, maximum time = 71.030024s, mean time = 71.030024s, file size = 572.20 MiB
</span>
</span>Results for flat b-tree as ZXY: # of tests: 1, iterations = 100000000, width = 64000, serialize=true
</span>minimum time = 94.0144163s, maximum time = 94.0144163s, mean time = 94.0144163s, file size = 572.20 MiB
</span>
</span>Results for deeply nested btree: # of tests: 1, iterations = 100000000, width = 64000, serialize=true
</span>minimum time = 127.0032545s, maximum time = 127.0032545s, mean time = 127.0032545s, file size = 1053.44 MiB
</span>
</span>Results for nested btree: # of tests: 1, iterations = 100000000, width = 64000, serialize=true
</span>minimum time = 48.4802249s, maximum time = 48.4802249s, mean time = 48.4802249s, file size = 382.73 MiB
</span></code></pre>
Not bad.  It's a good bit faster on all metrics, except for deserializing, which was just a tad slower than the flat B-tree.  Memory footprint is also much improved.  Now I can move on to terrain generation.</p>
Source code from this post is available at btree_test</a>.</p>


[ insert title here ]
2023-03-21T00:00:00+00:00
This blog is a record of my attempt at making a Dwarf Fortress clone in Rust, as the very first thing I've ever coded in Rust</a>.</p>
I'm creating my clone because I want it to be faster (on modern hardware) and have different bugs.</p>

Contact me at admin@dwarfdreams.com</li>
</ul>


Welcome
2023-03-21T00:00:00+00:00
Welcome to DwarfDreams.com!</h3>
This site will chronicle my work on a game I'm making: a clone of a game called Dwarf Fortress by Bay 12 Games</a>.</p>
Here are my plans in no particular order:</p>

Coded in Rust</li>
Fully threaded, will use all available CPU cores</li>
Custom OpenGL 3.3 tilemap engine</li>
Windows/Mac/Linux compatibility (hence the OpenGL)</li>
Animated characters (gratefully accepting art submissions!)</li>
No concept of 'embark area', world can be freely explored</li>
Implement all systems of Dwarf Fortress</li>
Support importation of DF raws</li>
Fix bugs/annoyances in DF that I personally want to fix</li>
Add some DFHack features like blueprints</li>
Simplify language system to facilitate future translations</li>
</ul>
Longer term goals:</p>

Adventure mode</li>
Lua scripting for mods</li>
My own added features</li>
</ul>

Do six-fingered dwarves dream of electric ale?</p>