Howto-BAM(7.3ce) BitKeeper User's Manual Howto-BAM(7.3ce)
NAME
bk Howto-BAM - configuring the Binary Asset Management (BAM) subsystem
OVERVIEW
Binary Asset Management, or BAM, is a storage management system for
versioning larger binary assets, such as tool chains, libraries, pho-
tos, music, videos, etc.
The BAM system is a hybrid, combining the best of the traditional cen-
tralized models with advantages of the distributed model developed by
BitKeeper. When BAM is enabled, and a BAM server is set, then BAM
files in BitKeeper repositories are passed by reference rather than
copied. BAM files appear to be the same as regular BitKeeper files but
they contain no data, only a pointer which names the data. The data
typically resides in the BAM server and is fetched on demand as needed.
What this means to a user is that a 20GB repository full of BAM files
can be cloned in a few seconds and if the user needs to work on only a
small portion of the data, only that data needs to be fetched from the
BAM server.
CONFIGURATION
In order to have larger binaries be stored in BAM format, the configu-
ration file (BitKeeper/etc/config) will need to have BAM enabled:
BAM: on
In order to fetch BAM data on demand a server must be configured:
$ bk bam server bk://MyBigBox/MyRepo
Note that the BAM server is per repository construct that is inherited
on clone, much like the repository level is inherited on clone.
CONVERTING REPOS
If you have a repository that contains larger binaries (minimum checked
out size of 64KB in revision 1.1), then you may do a conversion to BAM.
The conversion should be done in a clone because the conversion
changes the identity of the repository, i.e., the converted repository
will no longer be able to synchronize with the unconverted reposito-
ries. Note that while the conversion does change the package identity,
it is idempotent so other repositories may also do the conversion and
they will end up with the same package identity and be able to synchro-
nize.
The conversion process will add a "BAM:on" variable to your config
file. If you wish to minimize the number of those changesets, turn
that variable on in your main tree and tell your team to pull that
changeset before converting.
The conversion process tends to be longer in projects with more change-
sets and files (for example, the MySQL 5.2 tree with 51,000 changesets
and 14,000 files took 5 minutes on a 2Ghz Opteron running Linux).
The conversion process is simple:
bk clone my_repo my_repo.BAM
cd my_repo.BAM
bk bam convert
USAGE
Once the repository is converted, usage is as it was in the past, you
clone, pull, push, etc., as normal. In order to check out BAM files
the BAM server listed in the config file must be accessible. At the
end of a "bk -U get" command you may see lines like:
Fetching 18 BAM files from bk://MyBigBox/MyRepo...
BitKeeper/BAM/62/62af0d9c.d1 (5.41M of 6.29M)
followed by the normal check out messages for the associated files.
CHECKOUT
There is a BAM_checkout configuration variable which is specific to BAM
files. If this variable is not set then BAM files use the checkout
configuration variable. For large collections of large files parti-
tioned by directories, the BitKeeper support team suggests BAM_check-
out:last as the best setting. That setting will not pull BAM data into
a clone until a user requests an edit or a checkout. After that, the
file will remain in whatever mode it was, even across deltas.
SEE ALSO
bk Howto
bk bam
bk newroot
CATEGORY
Overview
BitKeeper Inc 1E1 Howto-BAM(7.3ce)
Overview