Difference between revisions of "CompressMemoryUsedBy"

m (CompressMemoryUsedBy(A))
 
Line 5: Line 5:
  
 
== CompressMemoryUsedBy(A) ==
 
== CompressMemoryUsedBy(A) ==
 +
This function reduces the memory used by array «A» when it is sparse (contains a lot of Null or zero values) or has other kinds of repetition.  It doesn't change the logical contents of «A», so you'll see no difference in results, other than possibly a drop in memory usage.
  
In some cases, this function is able to reduce the amount of memory consumed by array «A»The logical contents of «A» is unchanged, so you won't see a difference, other than possibly a drop in memory usage.
+
Analytica often takes advantage of sparseness or repetition within an array to reduce memory usage automaticallyBut, sometimes an array is created in a way that it cannot detect the sparseness without a time-consuming computation. In such cases when you know or suspect that a large array has sparse or repeated values, you can use CompressMemoryUsedBy(A) to force the compression.
  
In Analytica's internal representation of arrays, it is able to represent certain types of sparseness.  There are basically two forms that may occur: Shared subarrays and constant subvectors.   
+
It detects and takes advantage of three kinds of sparsity or repetition: Shared subarrays, constant subvectors, and repeated text values.   
 
+
A ''shared subarray'' might occur from an expression such as:
+
  
 +
Here is n example of an expression with ''shared subarrays'', where <code>A</code> and code>B</code> may be large arrays:
 
:<code>Array(I, [A, A, A, B, B])</code>
 
:<code>Array(I, [A, A, A, B, B])</code>
  
Under certain conditions (specifically, when <code>I</code> comes before all indexes of <code>A</code> and <code>B</code> in Analytica's internal canonical index order), the first three slices along <code>I</code> are the same subarray, allowing Analytica to store it only once and not three timesLikewise, the last two slices along <code>I</code> share a subarray.
+
In such cases, it computes and store the values of <code>A</code> and code>B</code> just once each, rather than separately for each value of <code>I</code>.  This works only when the Index <code>I</code> comes before all indexes of <code>A</code> and <code>B</code> in the internal canonical index order.
  
 
If the same array were created through a different sequence of operations, you might end up with a dense array, with every cell explicitly stored.  Under normal operations Analytica does not explicitly detect subarrays that could be potentially shared, since this can be a very expensive operation.  [[CompressMemoryUsedBy]] detects these and converts the internal representation into one that shares these common subarrays to save memory.
 
If the same array were created through a different sequence of operations, you might end up with a dense array, with every cell explicitly stored.  Under normal operations Analytica does not explicitly detect subarrays that could be potentially shared, since this can be a very expensive operation.  [[CompressMemoryUsedBy]] detects these and converts the internal representation into one that shares these common subarrays to save memory.
  
The second type of sparseness, ''constant subvectors'', occurs when all slices along an index contain precisely the same value.  For example, in the array:
+
A ''constant subvectors'' occurs when all slices along an index contain precisely the same value.  For example, in the array:
  
 
:{|class="wikitable"
 
:{|class="wikitable"
Line 31: Line 31:
 
|}
 
|}
  
When the <code>I</code> index is before <code>J</code> in Analytica's internal canonical index order, then the last two slices along <code>I</code> are constant along <code>J</code>.  Analytica saves memory (and computation time when this is used in a downstream computation) by storing the value <code>8</code> only once, and storing the value <code>4</code> only once.  Hence in this example, only 6 cells of the original 12 consume memory like this:
+
When the <code>I</code> index is before <code>J</code> in canonical index order, the last two slices along <code>I</code> are constant along <code>J</code>.  Analytica saves memory (and computation time when this is used in a downstream computation) by storing the value <code>8</code> just once for the second row, and <code>4</code> just once for the third rowSo it only 6 cells of the original 12 consume memory:
  
 
:{|class="wikitable"
 
:{|class="wikitable"
Line 46: Line 46:
 
[[CompressMemoryUsedBy]] recognizes constant-vector sparseness and replaces the original by a sparse vector when this is detected.
 
[[CompressMemoryUsedBy]] recognizes constant-vector sparseness and replaces the original by a sparse vector when this is detected.
  
[[CompressMemoryUsedBy]] also detects identical text strings when they occur in the same innermost vector, and changes these to share the same text.  However, it does not detect identical text strings in arbitrary points in the array (this is for space-time efficiency).
+
[[CompressMemoryUsedBy]] also detects identical text strings when they occur in the same innermost vector, and changes these to share the same text.  It does not detect identical text strings in arbitrary points in the array (this is for space-time efficiency).
  
 
==History==
 
==History==

Latest revision as of 17:55, 27 June 2020


CompressMemoryUsedBy(A)

This function reduces the memory used by array «A» when it is sparse (contains a lot of Null or zero values) or has other kinds of repetition. It doesn't change the logical contents of «A», so you'll see no difference in results, other than possibly a drop in memory usage.

Analytica often takes advantage of sparseness or repetition within an array to reduce memory usage automatically. But, sometimes an array is created in a way that it cannot detect the sparseness without a time-consuming computation. In such cases when you know or suspect that a large array has sparse or repeated values, you can use CompressMemoryUsedBy(A) to force the compression.

It detects and takes advantage of three kinds of sparsity or repetition: Shared subarrays, constant subvectors, and repeated text values.

Here is n example of an expression with shared subarrays, where A and code>B</code> may be large arrays:

Array(I, [A, A, A, B, B])

In such cases, it computes and store the values of A and code>B</code> just once each, rather than separately for each value of I. This works only when the Index I comes before all indexes of A and B in the internal canonical index order.

If the same array were created through a different sequence of operations, you might end up with a dense array, with every cell explicitly stored. Under normal operations Analytica does not explicitly detect subarrays that could be potentially shared, since this can be a very expensive operation. CompressMemoryUsedBy detects these and converts the internal representation into one that shares these common subarrays to save memory.

A constant subvectors occurs when all slices along an index contain precisely the same value. For example, in the array:

J ▶
I ▼ 3 5 7 6
8 8 8 8
4 4 4 4

When the I index is before J in canonical index order, the last two slices along I are constant along J. Analytica saves memory (and computation time when this is used in a downstream computation) by storing the value 8 just once for the second row, and 4 just once for the third row. So it only 6 cells of the original 12 consume memory:

J ▶
I ▼ 3 5 7 6
8
4

CompressMemoryUsedBy recognizes constant-vector sparseness and replaces the original by a sparse vector when this is detected.

CompressMemoryUsedBy also detects identical text strings when they occur in the same innermost vector, and changes these to share the same text. It does not detect identical text strings in arbitrary points in the array (this is for space-time efficiency).

History

This function was introduced in Analytica 4.3.

See Also

Comments


You are not allowed to post comments.