Moving Cubes

by · Published · Updated

This is a hack, so proceed at your own risk!  I haven’t tested this or used it in a production environment, so things may break/crash/catch on fire.

Say that you have built a really nice dashboard, deployed it to production, and everyone loves it.  This dashboard is based on a cube that takes an hour to load.  Now you’ve made modifications to the dashboard and cube in Dev and you’ve just migrated it to Prod.  However, the cube instance is invalid because the data doesn’t match effectively taking your beautiful dashboard offline for an hour while the cube refreshes.  If only there was a way to move the cube’s data from Dev to Prod …

Intelligent Cube File Storage
When an Intelligent Cube is loaded, it is simultaneously stored on disk and in memory.  This is so that the IServer can have the option of unloading old cubes if it runs out of memory or reloading cubes quickly if the server crashes or restarts.

The location that files are stored is configured in your project settings.  From there, they are in a folder that contains the server definition name (not necessarily the server name you connect to, but whatever was defined at the time of configuration!) and some random letters and numbers.  Inside this folder are two critical components of the cube, the .cube file and the _info.cube file.  The first is the actual data and the second is metadata for the cube.  There are other .idx files, but it appears that you don’t need them (see the opening two sentences about warnings).  These files share the same ID in the name and that ID is generated at the time the cube is loaded.  If you delete and reload the cube, it will change, but if you simply unload and reload the cube, it won’t change.

Exploiting the File Storage
First, the requirements:

  1. You have to have the same project in both locations, ie, a project that you can migrate between.
  2. The cube has to already exist in both places and be loaded in both places.  There doesn’t seem to be any way to get the IServer to load a cube cold from storage, only hot (I’ll explain that in a second).
Step 1 – Open the Cube Monitor, right click on the cube you want to move, and choose Quick View.  One of the fields is File Name which gives you the full location and file name of the cube.  Locate the .cube and _info.cube files from source (ie, Dev) and target (ie, Prod).  They will have different IDs.
Step 2 – Copy the files from source to target, but rename the source file to match the file name of the target.  Effectively, you’re overwriting the target files with the source files (and renaming).
Step 3 – In the Cube Monitor in the target (ie, Prod), right click on the cube and choose Unload from memory quickly followed by Load in memory.  It’s done like magic!
Wait, What?
So, basically what happens is when you unload a cube from memory, it doesn’t write anything to disk.  When you load it into memory, it then reads those files.  This means that it’s safe for us to swap out the files underneath the system without affecting what’s in memory.  In fact, you can delete those files and the IServer won’t notice until it restarts.  Once we’ve swapped the files, it’s them a simple matter of telling the IServer to reload them via the Unload/Reload commands.  After this is all finished, you only had about a 1 second outage instead of 1 hour, or however long it takes to load your cube.
Fail Over Usage
Even if you don’t need this hack for migrations, it’s actually really useful for fail overs.  Say that you have a disaster recovery site with a passive IServer node that you’ll turn on in the event your main data center goes down.  If you store your cubes on a SAN that is accessible/redundant to the recovery site, then a simple script could allow you to instantly load every cube in your system from disk instead of needing to reload them from the database.
Conclusion
This hack probably also works for report caches, but I don’t personally use those anymore.  I don’t see why it wouldn’t, since cache storage works the same way as cubes.
Even if you don’t find this particularly useful (I suspect I’ll use it very rarely, but handy when I do), it’s still something kind of neat!

Tags:

You may also like...

Follow:

Categories

About Me


Bryan Brandow
viz hacker, blogger, data enthusiast