bigmemory-package {bigmemory}R Documentation

bigmemory: massive matrices in (possibly shared) memory.

Description

bigmemory implements massive matricies in C++ and supports their basic manipulation and exploration. Access to and manipulation of a big.matrix object is exposed in R by an S4 class whose interface is simlar to an R matrix.

Details

Package: bigmemory
Type: Package
Version: 3.10
Date: 2009-07-15
License: LGPL-3

Multi-gigabyte data sets challenge and frustrate R users even on well-equipped hardware. C/C++ and Fortran programming can be helpful, but are cumbersome for interactive data analysis and lack the flexibility and power of R's rich statistical programming environment. The package bigmemory bridges this gap, implementing massive matrices and supporting their basic manipulation and exploration. It is ideal for problems involving the analysis in R of manageable subsets of the data, or when an analysis is conducted mostly in C++. The data structures may be allocated to shared memory with transparent read and write locking, allowing separate processes on the same computer to share access to a single copy of the data set. The data structures may also be file-backed, allowing users to more easily manage and analyze data sets larger than available RAM. These features of bigmemory open the door for powerful and memory-efficient parallel analyses and data mining of massive data sets.

Note that options(bigmemory.typecast.warning) is available and can be set to avoid annoying warnings that might occur if, for example you assign R objects (typically type double) to char, short, or integer big.matrix objects.

This package is still actively developed. Please send us an email letting us know you are trying the package, and we'll keep you abreast on updates.

NOTE: The 3.X and earlier versions support a limited number of columns (due to mutex limitations), roughly 50,000 on a typical Linux system. This restriction will be removed in versions 4.0 and beyond. There were row limitations (due to a bug that has now been fixed) in versions 3.8 and earlier of roughly 1 billion, but this has been fixed in versions 3.82 and later. We apologize for the inconvenience, and appreciate and and all feedback. - Jay and Mike

Author(s)

John W. Emerson and Michael J. Kane

Maintainer: Jay Emerson <john.emerson@yale.edu>

References

See http://www.stat.yale.edu/~jay/bigmemory.

See Also

For example, big.matrix, mwhich, colmean, biglm

Examples

# Our examples are all trivial in size, rather than burning huge amounts
# of memory simply to demonstrate the package functionality.

x <- big.matrix(5, 2, type="integer", init=0)
colnames(x)=c("alpha", "beta")
x
x[,]
x[,1] <- 1:5
x[,]
mean(x)
colmean(x)
summary(x)

[Package bigmemory version 3.10 Index]