Berkeley DB Reference Guide: Logical record numbers

Berkeley DB Reference Guide:
Access Methods

Logical record numbers

The Berkeley DB Btree, Queue and Recno access methods can operate on logical record numbers. Record numbers are 1-based, not 0-based, that is, the first record in a database is record number 1.

In all cases for the Queue and Recno access methods, and when calling the Btree access method using the DB->get and DBcursor->get methods with the DB_SET_RECNO flag specified, the data field of the key DBT must be a pointer to a memory location of type db_recno_t, as typedef'd in the standard Berkeley DB include file. The size field of the key DBT should be the size of that type (for example, "sizeof(db_recno_t)" in the C programming language). The db_recno_t type is a 32-bit unsigned type, which limits the number of logical records in a Queue or Recno database, and the maximum logical record which may be directly retrieved from a Btree database, to 4,294,967,295.

Record numbers in Recno databases can be configured to run in either mutable or fixed mode: mutable, where logical record numbers change as records are deleted or inserted, and fixed, where record numbers never change regardless of the database operation. Record numbers in Queue databases are always fixed, and never change regardless of the database operation. Record numbers in Btree databases are always mutable, and as records are deleted or inserted, the logical record number for other records in the database can change. See Logically renumbering records for more information.

When appending new data items into Queue databases, record numbers wrap around. When the tail of the queue reaches the maximum record number, the next record appended will be given record number 1. If the head of the queue ever catches up to the tail of the queue, Berkeley DB will return the system error EFBIG. Record numbers do not wrap around when appending new data items into Recno databases.

Configuring Btree databases to support record numbers can severely limit the throughput of applications with multiple concurrent threads writing the database, because locations used to store record counts often become hot spots that many different threads all need to update. In the case of a Btree supporting duplicate data items, the logical record number refers to a key and all of its data items, as duplicate data items are not individually numbered.

The following is an example function that reads records from standard input and stores them into a Recno database. The function then uses a cursor to step through the database and display the stored records.

int
recno_build(dbp)
	DB *dbp;
{
	DBC *dbcp;
	DBT key, data;
	db_recno_t recno;
	u_int32_t len;
	int ret;
	char buf[1024];

	/* Insert records into the database. */
	memset(&key, 0, sizeof(DBT));
	memset(&data, 0, sizeof(DBT));
	for (recno = 1;; ++recno) {
		printf("record #%lu> ", (u_long)recno);
		fflush(stdout);
		if (fgets(buf, sizeof(buf), stdin) == NULL)
			break;
		if ((len = strlen(buf)) <= 1)
			continue;

		key.data = &recno;
		key.size = sizeof(recno);
		data.data = buf;
		data.size = len - 1;

		switch (ret = dbp->put(dbp, NULL, &key, &data, 0)) {
		case 0:
			break;
		default:
			dbp->err(dbp, ret, "DB->put");
			break;
		}
	}
	printf("\n");

	/* Acquire a cursor for the database. */
	if ((ret = dbp->cursor(dbp, NULL, &dbcp, 0)) != 0) {
		dbp->err(dbp, ret, "DB->cursor");
		return (1);
	}

	/* Re-initialize the key/data pair. */
	memset(&key, 0, sizeof(key));
	memset(&data, 0, sizeof(data));

	/* Walk through the database and print out the key/data pairs. */
	while ((ret = dbcp->c_get(dbcp, &key, &data, DB_NEXT)) == 0)
		printf("%lu : %.*s\n",
		    *(u_long *)key.data, (int)data.size, (char *)data.data);
	if (ret != DB_NOTFOUND)
		dbp->err(dbp, ret, "DBcursor->get");

	/* Close the cursor. */
	if ((ret = dbcp->c_close(dbcp)) != 0) {
		dbp->err(dbp, ret, "DBcursor->close");
		return (1);
	}
	return (0);
}