OmniSciDB  72c90bc290
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Data_Namespace Namespace Reference

Namespaces

 anonymous_namespace{DataMgr.cpp}
 
 detail
 

Classes

class  AbstractBuffer
 An AbstractBuffer is a unit of data management for a data manager. More...
 
class  TypedThrustAllocator
 a Templated version of Data_Namespace::ThrustAllocator that can be used as a custom allocator in thrust device containers such as thrust::device_vector. Note that this class derives from thrust::mr::allocator in order to meet the requirements of an Allocator More...
 
struct  MemoryData
 
struct  MemoryInfo
 
class  ProcMeminfoParser
 Parse /proc/meminfo into key/value pairs. More...
 
class  ProcBuddyinfoParser
 Parse /proc/buddyinfo into a Fragmentation health score. More...
 
class  DataMgr
 

Typedefs

using DataMgrAllocationPolicy = typename thrust::detail::allocator_aware_execution_policy< thrust::device_execution_policy >::execute_with_allocator_type< ThrustAllocator >::type
 This is a custom device execution policy to allow a custom allocator backed by a DataMgr instance for any temp allocations done in thrust calls. This execution policy is required when using thrust functions that do host <-> device transfers internally, such as thrust::equal_range(). Without the use of this custom policy, thrust::equal_range() will crash. For more clarity, if you were to use the thrust::device(allocator) as the execution policy for thrust::equal_range, where allocator is a ThrustAllocator instance, thrust will crash because that policy by default uses a cuda default stream wrapper class (execute_on_stream_base): More...
 
template<typename T >
using ThrustAllocatorDeviceVector = thrust::device_vector< T, TypedThrustAllocator< T >>
 
using CpuTierSizeVector = std::vector< size_t >
 

Enumerations

enum  CpuTier { DRAM = 0, PMEM = 1 }
 
enum  MemoryLevel { DISK_LEVEL = 0, CPU_LEVEL = 1, GPU_LEVEL = 2 }
 

Functions

std::ostream & operator<< (std::ostream &os, const DataMgr::SystemMemoryUsage &mem_info)
 

Variables

constexpr size_t numCpuTiers = 2
 

Typedef Documentation

using Data_Namespace::CpuTierSizeVector = typedef std::vector<size_t>

Definition at line 28 of file TieredCpuBufferMgr.h.

using Data_Namespace::DataMgrAllocationPolicy = typedef typename thrust::detail::allocator_aware_execution_policy< thrust::device_execution_policy>::execute_with_allocator_type<ThrustAllocator>::type

This is a custom device execution policy to allow a custom allocator backed by a DataMgr instance for any temp allocations done in thrust calls. This execution policy is required when using thrust functions that do host <-> device transfers internally, such as thrust::equal_range(). Without the use of this custom policy, thrust::equal_range() will crash. For more clarity, if you were to use the thrust::device(allocator) as the execution policy for thrust::equal_range, where allocator is a ThrustAllocator instance, thrust will crash because that policy by default uses a cuda default stream wrapper class (execute_on_stream_base):

https://github.com/thrust/thrust/blob/cuda-10.2/thrust/system/cuda/detail/par.h#L74

The get_stream friend function in that wrapper class attempts to get a stream from a wrapper that is not defined, and therefore segfaults.

So this custom device policy is a way around that problem. It ensures that execute_on_stream_base is not used as a default stream policy and instead uses the default stream.

If streams are ever intended to be used in the future for any of this thrust work, this will need to be adjusted, and likely care required in getting thrust::equal_range to work.

Note the above crash scenario was noticed while using cuda 10.2 on 06/03/2020.

Definition at line 51 of file DataMgrAllocationPolicy.h.

template<typename T >
using Data_Namespace::ThrustAllocatorDeviceVector = typedef thrust::device_vector<T, TypedThrustAllocator<T>>

Definition at line 164 of file TypedThrustAllocator.h.

Enumeration Type Documentation

Enumerator
DRAM 
PMEM 

Definition at line 27 of file TieredCpuBufferMgr.h.

Enumerator
DISK_LEVEL 
CPU_LEVEL 
GPU_LEVEL 

Definition at line 21 of file MemoryLevel.h.

Function Documentation

std::ostream & Data_Namespace::operator<< ( std::ostream &  os,
const DataMgr::SystemMemoryUsage mem_info 
)

Definition at line 662 of file DataMgr.cpp.

References Data_Namespace::DataMgr::SystemMemoryUsage::frag, Data_Namespace::DataMgr::SystemMemoryUsage::free, Data_Namespace::DataMgr::SystemMemoryUsage::regular, Data_Namespace::DataMgr::SystemMemoryUsage::resident, Data_Namespace::DataMgr::SystemMemoryUsage::shared, Data_Namespace::DataMgr::SystemMemoryUsage::total, and Data_Namespace::DataMgr::SystemMemoryUsage::vtotal.

662  {
663  os << "jsonlog ";
664  os << "{";
665  os << " \"name\": \"CPU Memory Info\",";
666  os << " \"TotalMB\": " << mem_info.total / (1024. * 1024.) << ",";
667  os << " \"FreeMB\": " << mem_info.free / (1024. * 1024.) << ",";
668  os << " \"ProcessMB\": " << mem_info.resident / (1024. * 1024.) << ",";
669  os << " \"VirtualMB\": " << mem_info.vtotal / (1024. * 1024.) << ",";
670  os << " \"ProcessPlusSwapMB\": " << mem_info.regular / (1024. * 1024.) << ",";
671  os << " \"ProcessSharedMB\": " << mem_info.shared / (1024. * 1024.) << ",";
672  os << " \"FragmentationPercent\": " << mem_info.frag;
673  os << " }";
674  return os;
675 }

Variable Documentation

constexpr size_t Data_Namespace::numCpuTiers = 2