Eigen and huge dense 2D arrays

I am using 2D Eigen::Arrays for a project, and I like to keep using them in the case of huge 2D arrays.

Eigen::Array

For avoiding memory issues, I thought to use memory mapped files to manage (read/modify/write) these arrays, but I cannot find working examples.

The closest example that I have found is this based on boost::interprocess, but it uses shared-memory (while I'd prefer to have persistent storage).

boost::interprocess

The lack of examples makes me worry if there is a better, main-stream alternative solution to my problem. Is this the case? A minimal example would be very handy.

EDIT:

This is a minimal example explaining my use case in the comments:

#include <Eigen/Dense>


int main()
{
// Order of magnitude of the required arrays
Eigen::Index rows = 50000;
Eigen::Index cols = 40000;

{
// Array creation (this is where the memory mapped file should be created)
Eigen::ArrayXXf arr1 = Eigen::ArrayXXf::Zero( rows, cols );

// Some operations on the array
for(Eigen::Index i = 0; i < rows; ++i)
{
for(Eigen::Index j = 0; j < cols; ++j)
{
arr1( i, j ) = float(i * j);
}
}

// The array goes out of scope, but the data are persistently stored in the file
}

{
// This should actually use the data stored in the file
Eigen::ArrayXXf arr2 = Eigen::ArrayXXf::Zero( rows, cols );

// Manipulation of the array data
for(Eigen::Index i = 0; i < rows; ++i)
{
for(Eigen::Index j = 0; j < cols; ++j)
{
arr2( i, j ) += 1.0f;
}
}

// The array goes out of scope, but the data are persistently stored in the file
}

}


This question has not received enough attention.

A simple working example would be enough.

ArrayXXf

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.