OmniSciDB  340b00dbf6
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Data_Namespace::AbstractBuffer Class Referenceabstract

An AbstractBuffer is a unit of data management for a data manager. More...

#include <AbstractBuffer.h>

+ Inheritance diagram for Data_Namespace::AbstractBuffer:
+ Collaboration diagram for Data_Namespace::AbstractBuffer:

Public Member Functions

 AbstractBuffer (const int device_id)
 
 AbstractBuffer (const int device_id, const SQLTypeInfo sql_type)
 
virtual ~AbstractBuffer ()
 
virtual void read (int8_t *const dst, const size_t num_bytes, const size_t offset=0, const MemoryLevel dst_buffer_type=CPU_LEVEL, const int dst_device_id=-1)=0
 
virtual void write (int8_t *src, const size_t num_bytes, const size_t offset=0, const MemoryLevel src_buffer_type=CPU_LEVEL, const int src_device_id=-1)=0
 
virtual void reserve (size_t num_bytes)=0
 
virtual void append (int8_t *src, const size_t num_bytes, const MemoryLevel src_buffer_type=CPU_LEVEL, const int device_id=-1)=0
 
virtual int8_t * getMemoryPtr ()=0
 
virtual size_t pageCount () const =0
 
virtual size_t pageSize () const =0
 
virtual size_t reservedSize () const =0
 
virtual MemoryLevel getType () const =0
 
virtual int pin ()
 
virtual int unPin ()
 
virtual int getPinCount ()
 
size_t size () const
 
int getDeviceId () const
 
bool isDirty () const
 
bool isAppended () const
 
bool isUpdated () const
 
bool hasEncoder () const
 
SQLTypeInfo getSqlType () const
 
void setSqlType (const SQLTypeInfo &sql_type)
 
EncodergetEncoder () const
 
void setDirty ()
 
void setUpdated ()
 
void setAppended ()
 
void setSize (const size_t size)
 
void clearDirtyBits ()
 
void initEncoder (const SQLTypeInfo &tmp_sql_type)
 
void syncEncoder (const AbstractBuffer *src_buffer)
 
void copyTo (AbstractBuffer *destination_buffer, const size_t num_bytes=0)
 
void resetToEmpty ()
 

Protected Attributes

std::unique_ptr< Encoderencoder_
 
SQLTypeInfo sql_type_
 
size_t size_
 
int device_id_
 

Private Attributes

bool is_dirty_
 
bool is_appended_
 
bool is_updated_
 

Detailed Description

An AbstractBuffer is a unit of data management for a data manager.

Definition at line 47 of file AbstractBuffer.h.

Constructor & Destructor Documentation

Data_Namespace::AbstractBuffer::AbstractBuffer ( const int  device_id)
inline

Definition at line 49 of file AbstractBuffer.h.

50  : encoder_(nullptr)
51  , size_(0)
52  , device_id_(device_id)
53  , is_dirty_(false)
54  , is_appended_(false)
55  , is_updated_(false) {}
std::unique_ptr< Encoder > encoder_
Data_Namespace::AbstractBuffer::AbstractBuffer ( const int  device_id,
const SQLTypeInfo  sql_type 
)
inline

Definition at line 57 of file AbstractBuffer.h.

References initEncoder().

58  : size_(0)
59  , device_id_(device_id)
60  , is_dirty_(false)
61  , is_appended_(false)
62  , is_updated_(false) {
63  initEncoder(sql_type);
64  }
void initEncoder(const SQLTypeInfo &tmp_sql_type)

+ Here is the call graph for this function:

virtual Data_Namespace::AbstractBuffer::~AbstractBuffer ( )
inlinevirtual

Definition at line 65 of file AbstractBuffer.h.

65 {}

Member Function Documentation

void Data_Namespace::AbstractBuffer::clearDirtyBits ( )
inline

Definition at line 120 of file AbstractBuffer.h.

References is_appended_, is_dirty_, and is_updated_.

Referenced by File_Namespace::FileMgr::putBuffer(), and Buffer_Namespace::BufferMgr::putBuffer().

120  {
121  is_appended_ = false;
122  is_updated_ = false;
123  is_dirty_ = false;
124  }

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::copyTo ( AbstractBuffer destination_buffer,
const size_t  num_bytes = 0 
)

Definition at line 44 of file AbstractBuffer.cpp.

References getDeviceId(), getMemoryPtr(), getType(), isUpdated(), read(), reserve(), setSize(), size(), and syncEncoder().

Referenced by foreign_storage::CachingForeignStorageMgr::fetchBuffer(), PersistentStorageMgr::fetchBuffer(), File_Namespace::FileMgr::fetchBuffer(), and Buffer_Namespace::BufferMgr::fetchBuffer().

44  {
45  size_t chunk_size = (num_bytes == 0) ? size() : num_bytes;
46  destination_buffer->reserve(chunk_size);
47  if (isUpdated()) {
48  read(destination_buffer->getMemoryPtr(),
49  chunk_size,
50  0,
51  destination_buffer->getType(),
52  destination_buffer->getDeviceId());
53  } else {
54  read(destination_buffer->getMemoryPtr() + destination_buffer->size(),
55  chunk_size - destination_buffer->size(),
56  destination_buffer->size(),
57  destination_buffer->getType(),
58  destination_buffer->getDeviceId());
59  }
60  destination_buffer->setSize(chunk_size);
61  destination_buffer->syncEncoder(this);
62 }
virtual void read(int8_t *const dst, const size_t num_bytes, const size_t offset=0, const MemoryLevel dst_buffer_type=CPU_LEVEL, const int dst_device_id=-1)=0

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

int Data_Namespace::AbstractBuffer::getDeviceId ( ) const
inline

Definition at line 95 of file AbstractBuffer.h.

References device_id_.

Referenced by Data_Namespace::DataMgr::copy(), copyTo(), Data_Namespace::DataMgr::free(), File_Namespace::FileMgr::putBuffer(), and Buffer_Namespace::BufferMgr::putBuffer().

+ Here is the caller graph for this function:

Encoder* Data_Namespace::AbstractBuffer::getEncoder ( ) const
inline

Definition at line 102 of file AbstractBuffer.h.

References CHECK, encoder_, and hasEncoder().

Referenced by Chunk_NS::Chunk::appendData(), Chunk_NS::Chunk::getChunkBuffer(), Chunk_NS::Chunk::getNumElemsForBytesInsertData(), foreign_storage::TypedParquetInPlaceEncoder< int64_t, int32_t >::getUpdatedStats(), Chunk_NS::Chunk::initEncoder(), foreign_storage::anonymous_namespace{ParquetDataWrapper.cpp}::reduce_metadata(), and foreign_storage::anonymous_namespace{ForeignStorageCache.cpp}::set_metadata_for_buffer().

102  {
103  CHECK(hasEncoder());
104  return encoder_.get();
105  }
std::unique_ptr< Encoder > encoder_
#define CHECK(condition)
Definition: Logger.h:197

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

virtual int Data_Namespace::AbstractBuffer::getPinCount ( )
inlinevirtual

Reimplemented in Buffer_Namespace::Buffer.

Definition at line 91 of file AbstractBuffer.h.

Referenced by ThrustAllocator::allocate(), and ThrustAllocator::allocateScopedBuffer().

91 { return 0; }

+ Here is the caller graph for this function:

SQLTypeInfo Data_Namespace::AbstractBuffer::getSqlType ( ) const
inline

Definition at line 100 of file AbstractBuffer.h.

References sql_type_.

Referenced by Encoder::getMetadata(), FixedLengthArrayNoneEncoder::is_null(), FixedLengthArrayNoneEncoder::update_elem_stats(), and ArrayNoneEncoder::update_elem_stats().

100 { return sql_type_; }

+ Here is the caller graph for this function:

virtual MemoryLevel Data_Namespace::AbstractBuffer::getType ( ) const
pure virtual
bool Data_Namespace::AbstractBuffer::hasEncoder ( ) const
inline

Definition at line 99 of file AbstractBuffer.h.

References encoder_.

Referenced by getEncoder(), syncEncoder(), and File_Namespace::FileBuffer::writeMetadata().

99 { return (encoder_ != nullptr); }
std::unique_ptr< Encoder > encoder_

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::initEncoder ( const SQLTypeInfo tmp_sql_type)

Definition at line 26 of file AbstractBuffer.cpp.

References Encoder::Create(), encoder_, logger::FATAL, SQLTypeInfo::get_type_name(), LOG_IF, and sql_type_.

Referenced by AbstractBuffer(), foreign_storage::ParquetEncoder::createMetadata(), foreign_storage::TypedParquetInPlaceEncoder< int64_t, int32_t >::getUpdatedStats(), Chunk_NS::Chunk::initEncoder(), File_Namespace::FileBuffer::readMetadata(), foreign_storage::anonymous_namespace{ParquetDataWrapper.cpp}::reduce_metadata(), foreign_storage::anonymous_namespace{ForeignStorageCache.cpp}::set_metadata_for_buffer(), and syncEncoder().

26  {
27  sql_type_ = tmp_sql_type;
28  encoder_.reset(Encoder::Create(this, sql_type_));
29  LOG_IF(FATAL, encoder_ == nullptr)
30  << "Failed to create encoder for SQL Type " << sql_type_.get_type_name();
31 }
static Encoder * Create(Data_Namespace::AbstractBuffer *buffer, const SQLTypeInfo sqlType)
Definition: Encoder.cpp:26
#define LOG_IF(severity, condition)
Definition: Logger.h:287
std::string get_type_name() const
Definition: sqltypes.h:433
std::unique_ptr< Encoder > encoder_

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

bool Data_Namespace::AbstractBuffer::isAppended ( ) const
inline

Definition at line 97 of file AbstractBuffer.h.

References is_appended_.

Referenced by File_Namespace::FileMgr::putBuffer(), and Buffer_Namespace::BufferMgr::putBuffer().

+ Here is the caller graph for this function:

bool Data_Namespace::AbstractBuffer::isDirty ( ) const
inline
bool Data_Namespace::AbstractBuffer::isUpdated ( ) const
inline

Definition at line 98 of file AbstractBuffer.h.

References is_updated_.

Referenced by copyTo(), File_Namespace::FileMgr::putBuffer(), and Buffer_Namespace::BufferMgr::putBuffer().

+ Here is the caller graph for this function:

virtual size_t Data_Namespace::AbstractBuffer::pageCount ( ) const
pure virtual
virtual size_t Data_Namespace::AbstractBuffer::pageSize ( ) const
pure virtual
virtual int Data_Namespace::AbstractBuffer::pin ( )
inlinevirtual

Reimplemented in Buffer_Namespace::Buffer.

Definition at line 89 of file AbstractBuffer.h.

Referenced by Buffer_Namespace::BufferMgr::fetchBuffer().

89 { return 0; }

+ Here is the caller graph for this function:

virtual void Data_Namespace::AbstractBuffer::read ( int8_t *const  dst,
const size_t  num_bytes,
const size_t  offset = 0,
const MemoryLevel  dst_buffer_type = CPU_LEVEL,
const int  dst_device_id = -1 
)
pure virtual

Implemented in File_Namespace::FileBuffer, Buffer_Namespace::Buffer, ForeignStorageBuffer, and foreign_storage::ForeignStorageBuffer.

Referenced by StringNoneEncoder::appendData(), ArrayNoneEncoder::appendData(), and copyTo().

+ Here is the caller graph for this function:

virtual void Data_Namespace::AbstractBuffer::reserve ( size_t  num_bytes)
pure virtual
virtual size_t Data_Namespace::AbstractBuffer::reservedSize ( ) const
pure virtual
void Data_Namespace::AbstractBuffer::resetToEmpty ( )

Definition at line 64 of file AbstractBuffer.cpp.

References encoder_, and size_.

64  {
65  encoder_ = nullptr;
66  size_ = 0;
67 }
std::unique_ptr< Encoder > encoder_
void Data_Namespace::AbstractBuffer::setAppended ( )
inline

Definition at line 114 of file AbstractBuffer.h.

References is_appended_, and is_dirty_.

Referenced by ForeignStorageBuffer::append(), Buffer_Namespace::Buffer::append(), File_Namespace::FileBuffer::append(), Buffer_Namespace::Buffer::write(), and File_Namespace::FileBuffer::write().

114  {
115  is_appended_ = true;
116  is_dirty_ = true;
117  }

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::setDirty ( )
inline

Definition at line 107 of file AbstractBuffer.h.

References is_dirty_.

Referenced by StringNoneEncoder::appendData(), FixedLengthArrayNoneEncoder::appendData(), ArrayNoneEncoder::appendData(), File_Namespace::FileMgr::init(), Buffer_Namespace::Buffer::write(), and File_Namespace::FileBuffer::write().

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::setSize ( const size_t  size)
inline

Definition at line 119 of file AbstractBuffer.h.

References size(), and size_.

Referenced by copyTo(), ForeignStorageBufferMgr::fetchBuffer(), File_Namespace::FileMgr::init(), File_Namespace::FileMgr::putBuffer(), and foreign_storage::anonymous_namespace{ForeignStorageCache.cpp}::set_metadata_for_buffer().

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::setSqlType ( const SQLTypeInfo sql_type)
inline

Definition at line 101 of file AbstractBuffer.h.

References sql_type_.

101 { sql_type_ = sql_type; }
void Data_Namespace::AbstractBuffer::setUpdated ( )
inline

Definition at line 109 of file AbstractBuffer.h.

References is_dirty_, and is_updated_.

Referenced by foreign_storage::ForeignStorageCache::cacheChunk(), foreign_storage::ForeignStorageCache::cacheMetadataVec(), foreign_storage::anonymous_namespace{ForeignStorageCache.cpp}::set_metadata_for_buffer(), Buffer_Namespace::Buffer::write(), and File_Namespace::FileBuffer::write().

109  {
110  is_updated_ = true;
111  is_dirty_ = true;
112  }

+ Here is the caller graph for this function:

void Data_Namespace::AbstractBuffer::syncEncoder ( const AbstractBuffer src_buffer)

Definition at line 33 of file AbstractBuffer.cpp.

References encoder_, hasEncoder(), initEncoder(), and sql_type_.

Referenced by copyTo(), ForeignStorageBufferMgr::fetchBuffer(), File_Namespace::FileMgr::init(), File_Namespace::FileMgr::putBuffer(), and Buffer_Namespace::BufferMgr::putBuffer().

33  {
34  if (src_buffer->hasEncoder()) {
35  if (!hasEncoder()) {
36  initEncoder(src_buffer->sql_type_);
37  }
38  encoder_->copyMetadata(src_buffer->encoder_.get());
39  } else {
40  encoder_ = nullptr;
41  }
42 }
void initEncoder(const SQLTypeInfo &tmp_sql_type)
std::unique_ptr< Encoder > encoder_

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

virtual int Data_Namespace::AbstractBuffer::unPin ( )
inlinevirtual

Reimplemented in Buffer_Namespace::Buffer.

Definition at line 90 of file AbstractBuffer.h.

Referenced by Buffer_Namespace::BufferMgr::fetchBuffer(), and Chunk_NS::Chunk::unpinBuffer().

90 { return 0; }

+ Here is the caller graph for this function:

virtual void Data_Namespace::AbstractBuffer::write ( int8_t *  src,
const size_t  num_bytes,
const size_t  offset = 0,
const MemoryLevel  src_buffer_type = CPU_LEVEL,
const int  src_device_id = -1 
)
pure virtual

Implemented in File_Namespace::FileBuffer, Buffer_Namespace::Buffer, ForeignStorageBuffer, and foreign_storage::ForeignStorageBuffer.

Referenced by DateDaysEncoder< T, V >::appendData(), FixedLengthEncoder< T, V >::appendData(), NoneEncoder< T >::appendData(), Data_Namespace::DataMgr::copy(), and Buffer_Namespace::BufferMgr::putBuffer().

+ Here is the caller graph for this function:

Member Data Documentation

int Data_Namespace::AbstractBuffer::device_id_
protected
std::unique_ptr<Encoder> Data_Namespace::AbstractBuffer::encoder_
protected
bool Data_Namespace::AbstractBuffer::is_appended_
private

Definition at line 139 of file AbstractBuffer.h.

Referenced by clearDirtyBits(), isAppended(), and setAppended().

bool Data_Namespace::AbstractBuffer::is_dirty_
private

Definition at line 138 of file AbstractBuffer.h.

Referenced by clearDirtyBits(), isDirty(), setAppended(), setDirty(), and setUpdated().

bool Data_Namespace::AbstractBuffer::is_updated_
private

Definition at line 140 of file AbstractBuffer.h.

Referenced by clearDirtyBits(), isUpdated(), and setUpdated().


The documentation for this class was generated from the following files: