Tuning disk access in instantOLAP

instantOLAP uses an extremely fast internal database system to store its dimensions, stores, caches and other information on the harddisk. Though the performance is good enough for the very most situations, you can even boost it by changing some server settings, depending on your server memory, harddisk, operating system or Java version.

This small HowTo addresses the administrator of your system and shows how, and where, to change the settings if you need more performance for your dimensions and stores. Continue reading this article and boost your system.

Database options

There are a number of options you can use to change the database behavior and to optimize it for your needs and server environment:

• btreeMode: Disk access Mode (0 = standard, 1 = memory mapped, default = 0)
• btreeReadBuffer: Buffer size for read only databases (default = 512)
• btreeWriteBuffer: Buffer size for write databases (default = 1024)
• btreeNodeSize: Node size in bytes for btrees (default = 4096)
• btreeCompression: Zip compresson for btrees (0 = no compression, 1 = weakest, 10 = strongest, default = 0)

All options are integer values and have to be set or changed in the iolap.xml file in the configuration of your application server. E.g. in a Tomcat server, this options have to be created or changed in the file <tomcat home>/conf/Catalina/locahost/iolap.xml

If you open the iolap.xml file, the properties will probably not be there. This is, because the iolap.xml file only overrides the default setting of the application and doesn’t need to contain values which haven’t been customized for your installation. If you want to change the database-setting, copy the following code into the file:

<Environment name="btreeMode" type="java.lang.Integer" value="0"/>

<Environment name="btreeReadBuffer" type="java.lang.Integer" value="512"/>

<Environment name="btreeWriteBuffer" type="java.lang.Integer" value="1024"/>

<Environment name="btreeNodeSize" type="java.lang.Integer" value="4096"/>

<Environment name="btreeCompression" type="java.lang.Integer" value="0"/>

After you have added the entries to the configuration file, you can edit them by changing the „value“ attribute. Usually, the application automatically restarts with the new settings after you changed and saved the file. If the changes don’t have any effect you may need to restart the application server.

Some of the properties (btreeNodeSize and btreeCompression) force the system to drop the old database files and to rebuild them.

Node-Size

The databases load and store their data in packages called „nodes.“  The node size is the minimum amount of data being read or written on the harddisk of your system. You can increase the node size of your databases by changing the „btreeNodeSize“ property in the configuration. The faster your harddisk is, the larger the node size can be.

The default size is „4096“. You can try larger values like „8192“ or „16384“ and test the performance change of your system. You shouldn’t decrease the node size below 4096, otherwise the system may cause errors if some data stored in a database is larger than a single node.

Buffer Size

The most recent nodes of a database are cached in the memory of your system. The properties „btreeReadBuffer“ and „btreeWriteBuffer“ control the number of nodes cached in the memory for each dimension or store. The property „btreeWriteBuffer“ controls the buffer size for dimensions and stores while they are created or reloded. The „btreeReadBuffer“ the size of dimension and stores while their model isn’t synchronizing.

Increasing the buffer sizes results in less disk activity but increases the memory consumption of your system. The amount of memory used by a database is buffer size * node size. E.g. with the default settings (node size = 4096, read buffer size = 512), each database uses 512 x 4096 bytes = 2 megabytes of memory.

If you have lots of memory available, you can theoretically increase the buffer size to a really big number, e.g. to „16384“. This would load the whole database into the memory and dramatically decrease the disk access of your system (unless your model rebuilds).

BTree Mode

The BTree Mode controls the way databases access the harddisk of your system. You can change the mode by altering the „btreeMode“ property. There are two possible values:

• 0 for classic random file access (default)
• 1 for memory mapped file access.

While the „classic“ access reads and writes the data in an old fashioned manner, the memory mapped mode uses the possibility of the operating system to map the harddisk data directly in the memory of your instantOLAP server. The result is, depending on your machine, a 20-40% faster disk access.

However, this feature is disabled by default. This is because you need at least Java 1.4 to use it (which you definitley should have) and it consumes much more memory if you use a Java version less than 1.7. So if you want to use it, your server should either have enough memory or you should install Java 7.

BTree Compression

instantOLAP databases compress their data using different techniques and the result are pretty small data files. However, if your harddisk size is really limited, you can also enable zip compression for your databases.

Use the property „btreeCompression“ to enable compression and to control the compression strength. With the default value „0“, compression is disabled. With values from „1“ to „10“ you can control the strengh whereby „1“ stands for the weakest but fastest compression and „10“ for the strongest and slowest.

Database compression can be really time consuming and working with compressed databases is round about half as fast as with uncompressed. Therefore, you should only activate compression if you really need to use less harddisk space.