OmniSciDB  8a228a1076
Data_Namespace Namespace Reference

Namespaces

 detail
 

Classes

class  AbstractBuffer
 An AbstractBuffer is a unit of data management for a data manager. More...
 
class  DataMgr
 
struct  MemoryData
 
struct  MemoryInfo
 
class  ProcBuddyinfoParser
 Parse /proc/buddyinfo into a Fragmentation health score. More...
 
class  ProcMeminfoParser
 Parse /proc/meminfo into key/value pairs. More...
 
class  TypedThrustAllocator
 a Templated version of Data_Namespace::ThrustAllocator that can be used as a custom allocator in thrust device containers such as thrust::device_vector. Note that this class derives from thrust::mr::allocator in order to meet the requirements of an Allocator More...
 

Typedefs

using DataMgrAllocationPolicy = typename thrust::detail::allocator_aware_execution_policy< thrust::device_execution_policy >::execute_with_allocator_type< ThrustAllocator >::type
 This is a custom device execution policy to allow a custom allocator backed by a DataMgr instance for any temp allocations done in thrust calls. This execution policy is required when using thrust functions that do host <-> device transfers internally, such as thrust::equal_range(). Without the use of this custom policy, thrust::equal_range() will crash. For more clarity, if you were to use the thrust::device(allocator) as the execution policy for thrust::equal_range, where allocator is a ThrustAllocator instance, thrust will crash because that policy by default uses a cuda default stream wrapper class (execute_on_stream_base): More...
 
template<typename T >
using ThrustAllocatorDeviceVector = thrust::device_vector< T, TypedThrustAllocator< T > >
 

Enumerations

enum  MemoryLevel { DISK_LEVEL = 0, CPU_LEVEL = 1, GPU_LEVEL = 2 }
 

Functions

std::ostream & operator<< (std::ostream &os, const DataMgr::SystemMemoryUsage &mem_info)
 

Typedef Documentation

◆ DataMgrAllocationPolicy

using Data_Namespace::DataMgrAllocationPolicy = typedef typename thrust::detail::allocator_aware_execution_policy< thrust::device_execution_policy>::execute_with_allocator_type<ThrustAllocator>::type

This is a custom device execution policy to allow a custom allocator backed by a DataMgr instance for any temp allocations done in thrust calls. This execution policy is required when using thrust functions that do host <-> device transfers internally, such as thrust::equal_range(). Without the use of this custom policy, thrust::equal_range() will crash. For more clarity, if you were to use the thrust::device(allocator) as the execution policy for thrust::equal_range, where allocator is a ThrustAllocator instance, thrust will crash because that policy by default uses a cuda default stream wrapper class (execute_on_stream_base):

https://github.com/thrust/thrust/blob/cuda-10.2/thrust/system/cuda/detail/par.h#L74

The get_stream friend function in that wrapper class attempts to get a stream from a wrapper that is not defined, and therefore segfaults.

So this custom device policy is a way around that problem. It ensures that execute_on_stream_base is not used as a default stream policy and instead uses the default stream.

If streams are ever intended to be used in the future for any of this thrust work, this will need to be adjusted, and likely care required in getting thrust::equal_range to work.

Note the above crash scenario was noticed while using cuda 10.2 on 06/03/2020.

Definition at line 50 of file DataMgrAllocationPolicy.h.

◆ ThrustAllocatorDeviceVector

template<typename T >
using Data_Namespace::ThrustAllocatorDeviceVector = typedef thrust::device_vector<T, TypedThrustAllocator<T> >

Definition at line 164 of file TypedThrustAllocator.h.

Enumeration Type Documentation

◆ MemoryLevel

Enumerator
DISK_LEVEL 
CPU_LEVEL 
GPU_LEVEL 

Definition at line 21 of file MemoryLevel.h.

Function Documentation

◆ operator<<()

std::ostream & Data_Namespace::operator<< ( std::ostream &  os,
const DataMgr::SystemMemoryUsage mem_info 
)

Definition at line 555 of file DataMgr.cpp.

References Data_Namespace::DataMgr::SystemMemoryUsage::frag, Data_Namespace::DataMgr::SystemMemoryUsage::free, Data_Namespace::DataMgr::SystemMemoryUsage::regular, Data_Namespace::DataMgr::SystemMemoryUsage::resident, Data_Namespace::DataMgr::SystemMemoryUsage::shared, Data_Namespace::DataMgr::SystemMemoryUsage::total, and Data_Namespace::DataMgr::SystemMemoryUsage::vtotal.

555  {
556  os << "jsonlog ";
557  os << "{";
558  os << " \"name\": \"CPU Memory Info\",";
559  os << " \"TotalMB\": " << mem_info.total / (1024. * 1024.) << ",";
560  os << " \"FreeMB\": " << mem_info.free / (1024. * 1024.) << ",";
561  os << " \"ProcessMB\": " << mem_info.resident / (1024. * 1024.) << ",";
562  os << " \"VirtualMB\": " << mem_info.vtotal / (1024. * 1024.) << ",";
563  os << " \"ProcessPlusSwapMB\": " << mem_info.regular / (1024. * 1024.) << ",";
564  os << " \"ProcessSharedMB\": " << mem_info.shared / (1024. * 1024.) << ",";
565  os << " \"FragmentationPercent\": " << mem_info.frag;
566  os << " }";
567  return os;
568 }