Berkeley DB Reference Guide: Checkpoints

Berkeley DB Reference Guide:
Berkeley DB Transactional Data Store Applications

Checkpoints

The second component of the infrastructure is performing checkpoints of the log files. Performing checkpoints is necessary for two reasons.

First, you may be able to remove Berkeley DB log files from your database environment after a checkpoint. Change records are written into the log files when databases are modified, but the actual changes to the database are not necessarily written to disk. When a checkpoint is performed, changes to the database are written into the backing database file. Once the database pages are written, log files can be archived and removed from the database environment because they will never be needed for anything other than catastrophic failure. (Log files which are involved in active transactions may not be removed, and there must always be at least one log file in the database environment.)

The second reason to perform checkpoints is because checkpoint frequency is inversely proportional to the amount of time it takes to run database recovery after a system or application failure. This is because recovery after failure has to redo or undo changes only since the last checkpoint, as changes before the checkpoint have all been flushed to the databases.

Berkeley DB provides a separate utility, db_checkpoint, which can be used to perform checkpoints. Alternatively, applications can write their own checkpoint utility using the underlying DB_ENV->txn_checkpoint function. The following code fragment checkpoints the database environment every 60 seconds:

int
main(int argc, char *argv)
{
	extern int optind;
	DB *db_cats, *db_color, *db_fruit;
	DB_ENV *dbenv;
	pthread_t ptid;
	int ch;

	while ((ch = getopt(argc, argv, "")) != EOF)
		switch (ch) {
		case '?':
		default:
			usage();
		}
	argc -= optind;
	argv += optind;

	env_dir_create();
	env_open(&dbenv);

	/* Start a checkpoint thread. */
	if ((errno = pthread_create(
	    &ptid, NULL, checkpoint_thread, (void *)dbenv)) != 0) {
		fprintf(stderr,
		    "txnapp: failed spawning checkpoint thread: %s\n",
		    strerror(errno));
		exit (1);
	}

	/* Open database: Key is fruit class; Data is specific type. */
	db_open(dbenv, &db_fruit, "fruit", 0);

	/* Open database: Key is a color; Data is an integer. */
	db_open(dbenv, &db_color, "color", 0);

	/*
	 * Open database:
	 *	Key is a name; Data is: company name, cat breeds.
	 */
	db_open(dbenv, &db_cats, "cats", 1);

	add_fruit(dbenv, db_fruit, "apple", "yellow delicious");

	add_color(dbenv, db_color, "blue", 0);
	add_color(dbenv, db_color, "blue", 3);

	add_cat(dbenv, db_cats,
		"Amy Adams",
		"Oracle",
		"abyssinian",
		"bengal",
		"chartreaux",
		NULL);

	return (0);
}

void *
checkpoint_thread(void *arg)
{
	DB_ENV *dbenv;
	int ret;

	dbenv = arg;
	dbenv->errx(dbenv, "Checkpoint thread: %lu", (u_long)pthread_self());

	/* Checkpoint once a minute. */
	for (;; sleep(60))
		if ((ret = dbenv->txn_checkpoint(dbenv, 0, 0, 0)) != 0) {
			dbenv->err(dbenv, ret, "checkpoint thread");
			exit (1);
		}

	/* NOTREACHED */
}

Because checkpoints can be quite expensive, choosing how often to perform a checkpoint is a common tuning parameter for Berkeley DB applications.