Below concepts involve Data Blocks in some way or the other,
so to understand how Data Blocks behave it’s good to understand these concepts:
Let's Begin!!
- Compression
- Clean and Dirty Blocks
- Fragmentation
- Restructure
Compression:
When Essbase stores blocks to disk, it can compress the data
blocks using one of the following compression methods, this is based on the
type of data that is being loaded into the Essbase database. Essbase provides
several options for data compression:
1. Bitmap compression: This is the default compression
method. Essbase stores only non-missing values and uses a bitmapping scheme. A
bitmap uses one bit for each cell in the data block, whether the cell value is
missing or non-missing. When a data block is not compressed, Essbase uses 8
bytes to store every non-missing cell.
2. Run-length encoding (RLE): Essbase compresses repetitive,
consecutive values --any value that repeats three or more times consecutively,
including zeroes and #MISSING values. Each data value that is repeated three or
more times uses 8 bytes plus a 16 byte repetition factor.
3. zlib compression: Essbase builds a data dictionary based
on the actual data being compressed. This method is used in packages like PNG,
Zip, and gzip. Generally, the more dense or heterogeneous the data is, the
better zlib will compress it in comparison to bitmap or RLE compression.
4. Index Value Pair compression: Essbase applies this
compression if the block density is less than 3%.Index Value Pair addresses
compression on databases with larger block sizes, where the blocks are highly sparse.
zlib does not use this.
5. No compression. Essbase does not compress data blocks
when they are written to disk.
*In most cases, bitmap compression conserves disk space more
efficiently.
RLE vs. Bitmap Compression, which is better..
RLE Compression is generally used over Bitmap when:
1. Average block density very low (< 3%).
2. Database has many consecutive repeating Values.
Disabling Compression..
You may want to disable data compression if blocks have very
high density (90% or greater) and have few consecutive, repeating data values.
Under these conditions, enabling compression consumes resources unnecessarily.
Don't use compression if disc space/memory is not an issue compared to your
application. It can become a drain on the processor.
Clean and Dirty Blocks:
Dirty Blocks: updated blocks
Clean Blocks: calculated & non updated blocks
The concept of clean and dirty blocks is basically created
to understand what happens in an intelligent calculation.
While writing Calculation scripts we often use a SET
command, SET UPDATECALC .When we use SET UPDATECALC OFF in the housekeeping
section of the calculation script we are trying to turn Intelligent Calculation
“off” because..
Using intelligent calculation, Essbase calculates only dirty
blocks, such as updated data blocks and their dependent parents. Therefore, the
calculation is very efficient.
All data blocks in the database are marked as either clean
or dirty. If a data block is clean, then Essbase knows that the block does not
need to be recalculated.
If we put this setting as off, Essbase will do a full
calculation which is more time taking and makes our scripts inefficient.
In addition, to ensure the accuracy of your calculation
results, u might also consider carefully the effect of the SET
CLEARUPDATESTATUS AFTER command on your calculation.
Read more: http://docs.oracle.com/cd/E12032_01/doc/epm.921/html_techref/calc/set_clearupdatestatus.htm
Fragmentation:
Fragmentation occurs when Essbase writes a data block to a
new location on disk and leaves unused space in the former location of the data
block. Block size increases because data from a data load or calculation is
appended to the blocks; the blocks must therefore be written to the end of a
data file.
Fragmentation is likely to occur in the following
situations:
·
Databases that are constantly updating with data.
·
Databases that frequently update and
recalculate dense members.
·
Databases that contain a significant number
of Dynamic Calc and Store members.
·
Databases that use an isolation level of
uncommitted access with commit block set to zero
·
Databases that execute calculations around the
clock
·
Poorly designed Data loads
I If you experience performance slowdowns, check to see if
there is too much fragmentation of the database; if there is, you can take
steps to reduce it.
Steps to reduce fragmentation:
·
Take Level zero Backup of the database
·
Clear all data
·
Do a dense restructure.
·
Reload the data and do a Calc All.
·
In addition to this if you are having troubles
related to fragmentation because of Data Load designs; optimize your data load
files.
Restructure:
Restructuring can also be defined as reconstruction. Any
kind of changes to an Essbase outline cause changes to either Essbase index
file or data files or both and require restructuring .
TYPES OF RESTRUCTURES
·
Implicit
·
Explicit
·
Dense
·
Sparse
·
Outline only
IMPLICIT RESTRUCTURE: Essbase initiates an implicit
restructure after an outline is changed, whether done with the outline editor,
through an automated build, or Planning database refresh. The type of restructure that is performed
depends on the type of changes made to the outline.
EXPLICIT RESTRUCTURE: Explicit restructures occur when a
user requests a restructure to occur.
This can be done in Essbase Administration Services or via MaxL and
forces a full restructure (DENSE).
DENSE RESTRUCTURE: If
a member of a dense dimension is moved, deleted, or added, Essbase restructures
the blocks in the data files and creates new data files. When Essbase
restructures the data blocks, it regenerates the index automatically so that
index entries point to the new data blocks. Empty blocks are not removed.
Essbase marks all restructured blocks as dirty, so after a dense restructure
you must recalculate the database. Dense restructuring, the most time-consuming
of the restructures, can take a long time to complete for large databases.
SPARSE RESTRUCTURE:
If a member of a sparse dimension is moved, deleted, or added, Essbase
restructures the index and creates new index files. Restructuring the index is
relatively fast; the time required depends on the index size. Sparse
restructures are typically fast, but depend on the size of the index
file(s). Sparse restructures are faster
than dense restructures.
OUTLINE ONLY: If a
change affects only the database outline, Essbase does not restructure the index
or data files. Member name changes, creation of aliases, and dynamic
calculation formula changes are examples of changes that affect only the
database outline. Outline restructures are very quick and typically take
seconds.
Changes in which no restructure is required to outline:
• Deleting
members of a sparse dimension, but don’t forget to calculate the member!
• Deleting
members of an attribute dimension.
• Deleting shared members from a
sparse or dense dimension, again calculation is must in this case!!
• Adding
members to an attribute dimension.
• Adding
shared members to a sparse or dense dimension.
• Moving a
member in an attribute dimension.
• Changing a
member formula, calculate if needed!
• Defining
a sparse dynamic calc member as dynamic calc and store member.
• Defining
a dense or sparse dynamic calc and store member as dynamic calc.
• Defining
a regular dense dimension member as dynamic calc and store.
• Defining a sparse dimension which
is dynamic calc and store member as regular member.
• Defining
a dense dimension which is dynamic calc and store member as regular member.
• Changing
the order of an attribute dimension.
• Creating,
deleting, clearing, renaming, or importing an alias table.
• Creating or
deleting a UDA.
Cheers!!
Anvi.S