Writing a Cache

I spent a large chunk of the day implementing a cache using TaffyDB.

The cache processes every page (.md) file, and puts their combined configuration, content, as well as other metadata into a database that is then passed throughout the program.

This serves two purposes:

1. Centralize configuration processing

Before, all of the project level configuration was processed in one file, then directory level configuration, then file level configuration at the individual build step. Now with the cache being created as a preprocessing step, every other part of the program now has access to the full configuration.

2. Feeds

The power of using a database (with queries!) is now…I can query it.

This is mainly used for things like blog indexes, where now I can run:

cache.db({ tags: ["web"] })

to retrieve all the pages with the web tag applied.

Currently, it’s used only to create category pages, and possibly a giant list of projects. In the future, it can be used to create RSS feeds, or other interesting lists of projects, and also be used for search.

Downsides

It’s slow. This took the program from a ~500ms build time to > 1sec, because it has to build the cache up.

But hopefully with incremental building, it’s not too much a hassle to work with on a day to day basis.