Berkeley DB Reference Guide: Duplicate data items

Berkeley DB Reference Guide:
Access Methods

Duplicate data items

The Btree and Hash access methods support the creation of multiple data items for a single key item. By default, multiple data items are not permitted, and each database store operation will overwrite any previous data item for that key. To configure Berkeley DB for duplicate data items, call the DB->set_flags method with the DB_DUP flag. Only one copy of the key will be stored for each set of duplicate data items. If the Btree access method comparison routine returns that two keys compare equally, it is undefined which of the two keys will be stored and returned from future database operations.

By default, Berkeley DB stores duplicates in the order in which they were added, that is, each new duplicate data item will be stored after any already existing data items. This default behavior can be overridden by using the DBcursor->put method and one of the DB_AFTER, DB_BEFORE DB_KEYFIRST or DB_KEYLAST flags. Alternatively, Berkeley DB may be configured to sort duplicate data items.

When stepping through the database sequentially, duplicate data items will be returned individually, as a key/data pair, where the key item only changes after the last duplicate data item has been returned. For this reason, duplicate data items cannot be accessed using the DB->get method, as it always returns the first of the duplicate data items. Duplicate data items should be retrieved using a Berkeley DB cursor interface such as the DBcursor->get method.

There is a flag that permits applications to request the following data item only if it is a duplicate data item of the current entry, see DB_NEXT_DUP for more information. There is a flag that permits applications to request the following data item only if it is not a duplicate data item of the current entry, see DB_NEXT_NODUP and DB_PREV_NODUP for more information.

It is also possible to maintain duplicate records in sorted order. Sorting duplicates will significantly increase performance when searching them and performing equality joins, common operations when using secondary indices. To configure Berkeley DB to sort duplicate data items, the application must call the DB->set_flags method with the DB_DUPSORT flag (in addition to the DB_DUP flag). In addition, a custom comparison function may be specified using the DB->set_dup_compare method. If the DB_DUPSORT flag is given, but no comparison routine is specified, then Berkeley DB defaults to the same lexicographical sorting used for Btree keys, with shorter items collating before longer items.

If the duplicate data items are unsorted, applications may store identical duplicate data items, or, for those that just like the way it sounds, duplicate duplicates.

In this release it is an error to attempt to store identical duplicate data items when duplicates are being stored in a sorted order. This restriction is expected to be lifted in a future release. There is a flag that permits applications to disallow storing duplicate data items when the database has been configured for sorted duplicates, see DB_NODUPDATA for more information. Applications not wanting to permit duplicate duplicates in databases configured for sorted duplicates should begin using the DB_NODUPDATA flag immediately.

For further information on how searching and insertion behaves in the presence of duplicates (sorted or not), see the DB->get, DB->put, DBcursor->get and DBcursor->put documentation.