The Dxstore Database System is a database management system for Linux, FreeBSD, and other operating systems. In the tradition of dbm/ndbm, it is distributed as an embeddable C or C++ library and shares with them the model of a database as a collection of key/value pairs. Dxstore is a completely new design and implementation, however, and has greatly extended functionality. It concentrates on providing very fast yet flexible access to a huge number of data items and vast amounts of data. Some features are listed below.

[Dxstore was initially called ``Dx'', but the name was changed because it wasn't particularly distinct.]

Dxstore is open source software. Please refer to LICENSE for details.

As part of this project, a database of free database systems is being compiled.

• README

• dxstore-0.2.tgz (183KB, includes the documentation and README)

• Documentation:
  • dx.html / dxstore.html.gz (266KB / 43KB)
  • dxstore.pdf.gz / dxstore.pdf (162KB / 505KB)
  • dxstoredoc.tgz (HTML + PDF + INFO) (239KB)

Development for the next release (0.3 Alpha) is concentrating on:

• flexible support for an unlimited number of duplicate keys and
• addition of a second access method to provide efficient indexed and sequential access to items.

Here are some of Dxstore's most significant features (excerpted from the manual):

• Immense databases striped over a huge number of file systems can be created (RAID level 0). When configured to use 64 bits internally, you will almost certainly reach disk space capacity limits or operating system limits before exceeding Dxstore's capacity; specifically, Dxstore can address over 2^64 data items, each one of those data items can be on the order of 2^64 bytes in size, and the data can be spread over 2^32 file systems. This is far greater than the commonly-found terabyte limits.

• A Dxstore database looks like any another file on the system, so the usual commands can be used to control access to the database, rename or copy it, and so on. Any file system that supports the handful of system calls that Dxstore requires can be used. No special hardware or dedicated disk partitions are required and disk space for a database does not need to be preallocated. Installation is simple and there is practically no database administration.

• There's no complicated query language to learn or use. A data item is stored and accessed using a simple key.

• Flexible ways of storing and retrieving data are provided. New data can be inserted anywhere in a stored value. Any contiguous region of a stored value can be retrieved.

• In some databases, the size of a stored value must be known at the time the data is given to the database and the data must all reside in contiguous memory. With Dxstore, data can be streamed into and out of the database; the application does not need to tell Dxstore how much data there will be and the data does not need to be in memory all at once.

• Sophisticated data editing functionality is provided. A region of a stored value can be overwritten or replaced with new data. If a stored value represents a table, for example, a single row can be replaced.

• Keys can be segregated into keyspaces, allowing small databases to be combined into a single physical database.

• Frame memory provides a rock-solid way of preventing memory leakage and allows dynamically allocated memory to truly be freed and released to the operating system. As a bonus, persistent dynamically-allocated memory functionality is provided. The frame memory functions are largely orthogonal to the database functions; they can be used with little or no knowledge of how to use the database.

• Descriptive error codes are made available to the user.

• High performance is paramount. Databases running out of main memory, rather than the file system, can be used. This makes it suitable for environments with a small disk drive or no drive at all.

• Dxstore has a small memory footprint.

• A Dxstore database can be configured to be portable so that it can be moved to a platform with a different processor architecture.

The design of Dxstore concentrates on providing very fast yet flexible access to a huge number of data items and vast amounts of data. Its philosophy generally favours performance over storage requirements. While Dxstore is not cavalier in its use of disk storage, disks are cheap and getting cheaper. Time is always expensive.

No query language is provided, per se; a data item is named by an arbitrary binary string. Dxstore is intended to be used by programmers as a core component of a larger system, acting as the data storage and retrieval engine, or in applications where support for complicated queries need not be provided. It can be integrated into scripting languages, such as Perl, Tcl, and PHP, used by a full-blown database, object store, or network-attached storage (NAS) or storage-area network (SAN) facility. Dxstore could even be used to implement a user-space file system.

Please contact me at: brachmanATdss.ca