Module carray¶
Implementation of resizeable arrays of different types in Cython.
All arrays provide for the following operations:
- access by indexing.
- access through get/set function.
- appending values at the end of the array.
- reserving space for future appends.
- access to internal data through a numpy array.
Each array also provides an interface to its data through a numpy array.
This is done through the get_npy_array
function. The returned numpy
array can be used just like any other numpy array but for the following
restrictions:
- the array may not be resized.
- references of this array should not be kept.
- slices of this array may not be made.
The numpy array may however be copied and used in any manner.
-
class
cyarray.carray.
BaseArray
¶ Bases:
object
Base class for managed C-arrays.
-
__reduce_cython__
¶
-
__setstate_cython__
¶
-
align_array
¶ Rearrange the array contents according to the new indices.
-
alloc
¶ ‘long’
Type: alloc
-
copy_subset
¶ Copy subset of values from source to self.
-
copy_values
¶ Copy values of indexed particles from self to dest.
-
extend
¶ Extend the array with data from in_array.
-
get_c_type
¶ Return the c data type of this array.
-
get_npy_array
¶ do not keep its reference.
Type: Returns a numpy array of the data
-
length
¶ ‘long’
Type: length
-
remove
¶ Remove the particles with indices in index_list.
-
reserve
¶ Resizes the internal data to required size.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes the array to the new size.
-
set_data
¶ Set data from the given numpy array.
If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Update the min and max values of the array.
-
-
class
cyarray.carray.
BaseArrayIter
(BaseArray arr)¶ Bases:
object
Iteration object to support iteration over BaseArray.
-
__reduce_cython__
¶
-
__setstate_cython__
¶
-
-
class
cyarray.carray.
DoubleArray
¶ Bases:
cyarray.carray.BaseArray
Represents an array of doubles
Mallocs a memory buffer of size (n*sizeof(double)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.
Parameters: n (long) – Length of the array. -
data
¶ Pointer to an integer array.
Type: pointer
-
length
¶ Size of the array itself.
Type: long
-
alloc
¶ Size of the data buffer allocated.
Type: long
Examples
>>> x = DoubleArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0
>>> x = DoubleArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0)
-
__reduce__
¶ Implemented to facilitate pickling.
-
__setstate__
¶ Load the carray from the dictionary d.
-
append
¶ Appends value to the end of the array.
-
copy_subset
¶ Copy a subset of values from src to self.
Parameters: - start_index (long) – the first index in self that corresponds to the 0th index in source
- end_index (long) – the first index in self from start_index that is not copied
- stride (int) – the stride along indices, basically copy over chunks of the given stride.
-
copy_values
¶ Copies values of indices in indices from self to dest.
No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.
-
extend
¶ Extend the array with data from in_array.
Parameters: in_array (ndarray) – a numpy array with data to be added to the current array. Notes
- accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
-
get
¶ Gets value stored at position idx.
-
get_c_type
¶ Return the c data type for this array as a string.
-
index
¶ Returns the index at which value is in self, else -1.
-
maximum
¶ ‘double’
Type: maximum
-
minimum
¶ ‘double’
Type: minimum
-
remove
¶ Remove the particles with indices in index_list.
Parameters: Notes
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed.
-
reserve
¶ Resizes the internal data to
size*sizeof(double)
bytes.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes internal data to
size*sizeof(double)
bytes and sets the length to the new size.
-
set
¶ Sets location idx to value.
-
set_view
¶ Create a view of a given a parent array from start to end.
Note that this excludes the end index.
Parameters: - parent (DoubleArray) – The parent array of which this is a view.
- start (long) – The starting index to start the view from.
- end (long) – The ending index to end the view at, excludes the end itself.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Updates the min and max values of the array.
-
-
class
cyarray.carray.
FloatArray
¶ Bases:
cyarray.carray.BaseArray
Represents an array of floats
Mallocs a memory buffer of size (n*sizeof(float)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.
Parameters: n (long) – Length of the array. -
data
¶ Pointer to an integer array.
Type: pointer
-
length
¶ Size of the array itself.
Type: long
-
alloc
¶ Size of the data buffer allocated.
Type: long
Examples
>>> x = FloatArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0
>>> x = FloatArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0)
-
__reduce__
¶ Implemented to facilitate pickling.
-
__setstate__
¶ Load the carray from the dictionary d.
-
append
¶ Appends value to the end of the array.
-
copy_subset
¶ Copy a subset of values from src to self.
Parameters: - start_index (long) – the first index in self that corresponds to the 0th index in source
- end_index (long) – the first index in self from start_index that is not copied
- stride (int) – the stride along indices, basically copy over chunks of the given stride.
-
copy_values
¶ Copies values of indices in indices from self to dest.
No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.
-
extend
¶ Extend the array with data from in_array.
Parameters: in_array (ndarray) – a numpy array with data to be added to the current array. Notes
- accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
-
get
¶ Gets value stored at position idx.
-
get_c_type
¶ Return the c data type for this array as a string.
-
index
¶ Returns the index at which value is in self, else -1.
-
maximum
¶ ‘float’
Type: maximum
-
minimum
¶ ‘float’
Type: minimum
-
remove
¶ Remove the particles with indices in index_list.
Parameters: Notes
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed.
-
reserve
¶ Resizes the internal data to
size*sizeof(float)
bytes.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes internal data to
size*sizeof(float)
bytes and sets the length to the new size.
-
set
¶ Sets location idx to value.
-
set_view
¶ Create a view of a given a parent array from start to end.
Note that this excludes the end index.
Parameters: - parent (FloatArray) – The parent array of which this is a view.
- start (long) – The starting index to start the view from.
- end (long) – The ending index to end the view at, excludes the end itself.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Updates the min and max values of the array.
-
-
class
cyarray.carray.
IntArray
¶ Bases:
cyarray.carray.BaseArray
Represents an array of ints
Mallocs a memory buffer of size (n*sizeof(int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.
Parameters: n (long) – Length of the array. -
data
¶ Pointer to an integer array.
Type: pointer
-
length
¶ Size of the array itself.
Type: long
-
alloc
¶ Size of the data buffer allocated.
Type: long
Examples
>>> x = IntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0
>>> x = IntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0)
-
__reduce__
¶ Implemented to facilitate pickling.
-
__setstate__
¶ Load the carray from the dictionary d.
-
append
¶ Appends value to the end of the array.
-
copy_subset
¶ Copy a subset of values from src to self.
Parameters: - start_index (long) – the first index in self that corresponds to the 0th index in source
- end_index (long) – the first index in self from start_index that is not copied
- stride (int) – the stride along indices, basically copy over chunks of the given stride.
-
copy_values
¶ Copies values of indices in indices from self to dest.
No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.
-
extend
¶ Extend the array with data from in_array.
Parameters: in_array (ndarray) – a numpy array with data to be added to the current array. Notes
- accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
-
get
¶ Gets value stored at position idx.
-
get_c_type
¶ Return the c data type for this array as a string.
-
index
¶ Returns the index at which value is in self, else -1.
-
maximum
¶ ‘int’
Type: maximum
-
minimum
¶ ‘int’
Type: minimum
-
remove
¶ Remove the particles with indices in index_list.
Parameters: Notes
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed.
-
reserve
¶ Resizes the internal data to
size*sizeof(int)
bytes.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes internal data to
size*sizeof(int)
bytes and sets the length to the new size.
-
set
¶ Sets location idx to value.
-
set_view
¶ Create a view of a given a parent array from start to end.
Note that this excludes the end index.
Parameters: - parent (IntArray) – The parent array of which this is a view.
- start (long) – The starting index to start the view from.
- end (long) – The ending index to end the view at, excludes the end itself.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Updates the min and max values of the array.
-
-
class
cyarray.carray.
LongArray
¶ Bases:
cyarray.carray.BaseArray
Represents an array of longs
Mallocs a memory buffer of size (n*sizeof(long)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.
Parameters: n (long) – Length of the array. -
data
¶ Pointer to an integer array.
Type: pointer
-
length
¶ Size of the array itself.
Type: long
-
alloc
¶ Size of the data buffer allocated.
Type: long
Examples
>>> x = LongArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0
>>> x = LongArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0)
-
__reduce__
¶ Implemented to facilitate pickling.
-
__setstate__
¶ Load the carray from the dictionary d.
-
append
¶ Appends value to the end of the array.
-
copy_subset
¶ Copy a subset of values from src to self.
Parameters: - start_index (long) – the first index in self that corresponds to the 0th index in source
- end_index (long) – the first index in self from start_index that is not copied
- stride (int) – the stride along indices, basically copy over chunks of the given stride.
-
copy_values
¶ Copies values of indices in indices from self to dest.
No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.
-
extend
¶ Extend the array with data from in_array.
Parameters: in_array (ndarray) – a numpy array with data to be added to the current array. Notes
- accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
-
get
¶ Gets value stored at position idx.
-
get_c_type
¶ Return the c data type for this array as a string.
-
index
¶ Returns the index at which value is in self, else -1.
-
maximum
¶ ‘long’
Type: maximum
-
minimum
¶ ‘long’
Type: minimum
-
remove
¶ Remove the particles with indices in index_list.
Parameters: Notes
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed.
-
reserve
¶ Resizes the internal data to
size*sizeof(long)
bytes.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes internal data to
size*sizeof(long)
bytes and sets the length to the new size.
-
set
¶ Sets location idx to value.
-
set_view
¶ Create a view of a given a parent array from start to end.
Note that this excludes the end index.
Parameters: - parent (LongArray) – The parent array of which this is a view.
- start (long) – The starting index to start the view from.
- end (long) – The ending index to end the view at, excludes the end itself.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Updates the min and max values of the array.
-
-
class
cyarray.carray.
UIntArray
¶ Bases:
cyarray.carray.BaseArray
Represents an array of unsigned ints
Mallocs a memory buffer of size (n*sizeof(unsigned int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.
Parameters: n (long) – Length of the array. -
data
¶ Pointer to an integer array.
Type: pointer
-
length
¶ Size of the array itself.
Type: long
-
alloc
¶ Size of the data buffer allocated.
Type: long
Examples
>>> x = UIntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0
>>> x = UIntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0)
-
__reduce__
¶ Implemented to facilitate pickling.
-
__setstate__
¶ Load the carray from the dictionary d.
-
append
¶ Appends value to the end of the array.
-
copy_subset
¶ Copy a subset of values from src to self.
Parameters: - start_index (long) – the first index in self that corresponds to the 0th index in source
- end_index (long) – the first index in self from start_index that is not copied
- stride (int) – the stride along indices, basically copy over chunks of the given stride.
-
copy_values
¶ Copies values of indices in indices from self to dest.
No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.
-
extend
¶ Extend the array with data from in_array.
Parameters: in_array (ndarray) – a numpy array with data to be added to the current array. Notes
- accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
-
get
¶ Gets value stored at position idx.
-
get_c_type
¶ Return the c data type for this array as a string.
-
index
¶ Returns the index at which value is in self, else -1.
-
maximum
¶ ‘unsigned int’
Type: maximum
-
minimum
¶ ‘unsigned int’
Type: minimum
-
remove
¶ Remove the particles with indices in index_list.
Parameters: Notes
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed.
-
reserve
¶ Resizes the internal data to
size*sizeof(unsigned int)
bytes.
-
reset
¶ Reset the length of the array to 0.
-
resize
¶ Resizes internal data to
size*sizeof(unsigned int)
bytes and sets the length to the new size.
-
set
¶ Sets location idx to value.
-
set_view
¶ Create a view of a given a parent array from start to end.
Note that this excludes the end index.
Parameters: - parent (UIntArray) – The parent array of which this is a view.
- start (long) – The starting index to start the view from.
- end (long) – The ending index to end the view at, excludes the end itself.
-
squeeze
¶ Release any unused memory.
-
update_min_max
¶ Updates the min and max values of the array.
-
-
cyarray.carray.
py_aligned
¶ Align n items each having size (in bytes) item_size to 64 bits and return the appropriate number of items that would be aligned to 64 bytes.