OmniSciDB  94e8789169
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
File_Namespace::FileBuffer Class Reference

Represents/provides access to contiguous data stored in the file system. More...

#include <FileBuffer.h>

+ Inheritance diagram for File_Namespace::FileBuffer:
+ Collaboration diagram for File_Namespace::FileBuffer:

Public Member Functions

 FileBuffer (FileMgr *fm, const size_t pageSize, const ChunkKey &chunkKey, const size_t initialSize=0)
 Constructs a FileBuffer object. More...
 
 FileBuffer (FileMgr *fm, const size_t pageSize, const ChunkKey &chunkKey, const SQLTypeInfo sqlType, const size_t initialSize=0)
 
 FileBuffer (FileMgr *fm, const ChunkKey &chunkKey, const std::vector< HeaderInfo >::const_iterator &headerStartIt, const std::vector< HeaderInfo >::const_iterator &headerEndIt)
 
 ~FileBuffer () override
 Destructor. More...
 
Page addNewMultiPage (const int32_t epoch)
 
void reserve (const size_t numBytes) override
 
void freeMetadataPages ()
 
size_t freeChunkPages ()
 
void freePages ()
 
void freePagesBeforeEpoch (const int32_t targetEpoch)
 
void read (int8_t *const dst, const size_t numBytes=0, const size_t offset=0, const MemoryLevel dstMemoryLevel=CPU_LEVEL, const int32_t deviceId=-1) override
 
void write (int8_t *src, const size_t numBytes, const size_t offset=0, const MemoryLevel srcMemoryLevel=CPU_LEVEL, const int32_t deviceId=-1) override
 Writes the contents of source (src) into new versions of the affected logical pages. More...
 
void append (int8_t *src, const size_t numBytes, const MemoryLevel srcMemoryLevel=CPU_LEVEL, const int32_t deviceId=-1) override
 
void copyPage (Page &srcPage, Page &destPage, const size_t numBytes, const size_t offset=0)
 
Data_Namespace::MemoryLevel getType () const override
 
int8_t * getMemoryPtr () override
 Not implemented for FileMgr – throws a runtime_error. More...
 
size_t pageCount () const override
 Returns the number of pages in the FileBuffer. More...
 
size_t pageSize () const override
 Returns the size in bytes of each page in the FileBuffer. More...
 
virtual size_t pageDataSize () const
 Returns the size in bytes of the data portion of each page in the FileBuffer. More...
 
virtual size_t reservedHeaderSize () const
 
virtual std::vector< MultiPagegetMultiPage () const
 Returns vector of MultiPages in the FileBuffer. More...
 
size_t reservedSize () const override
 Returns the total number of bytes allocated for the FileBuffer. More...
 
size_t numMetadataPages () const
 Returns the total number of used bytes in the FileBuffer. More...
 
- Public Member Functions inherited from Data_Namespace::AbstractBuffer
 AbstractBuffer (const int device_id)
 
 AbstractBuffer (const int device_id, const SQLTypeInfo sql_type)
 
virtual ~AbstractBuffer ()
 
virtual void read (int8_t *const dst, const size_t num_bytes, const size_t offset=0, const MemoryLevel dst_buffer_type=CPU_LEVEL, const int dst_device_id=-1)=0
 
virtual void write (int8_t *src, const size_t num_bytes, const size_t offset=0, const MemoryLevel src_buffer_type=CPU_LEVEL, const int src_device_id=-1)=0
 
virtual void append (int8_t *src, const size_t num_bytes, const MemoryLevel src_buffer_type=CPU_LEVEL, const int device_id=-1)=0
 
virtual void setMemoryPtr (int8_t *new_ptr)
 
virtual int pin ()
 
virtual int unPin ()
 
virtual int getPinCount ()
 
size_t size () const
 
int getDeviceId () const
 
bool isDirty () const
 
bool isAppended () const
 
bool isUpdated () const
 
bool hasEncoder () const
 
SQLTypeInfo getSqlType () const
 
void setSqlType (const SQLTypeInfo &sql_type)
 
EncodergetEncoder () const
 
void setDirty ()
 
void setUpdated ()
 
void setAppended ()
 
void setSize (const size_t size)
 
void clearDirtyBits ()
 
void initEncoder (const SQLTypeInfo &tmp_sql_type)
 
void syncEncoder (const AbstractBuffer *src_buffer)
 
void copyTo (AbstractBuffer *destination_buffer, const size_t num_bytes=0)
 
void resetToEmpty ()
 

Private Member Functions

void writeHeader (Page &page, const int32_t pageId, const int32_t epoch, const bool writeMetadata=false)
 Write header writes header at top of page in format. More...
 
void writeMetadata (const int32_t epoch)
 
void readMetadata (const Page &page)
 
void calcHeaderBuffer ()
 
void freePage (const Page &page, const bool isRolloff)
 
void freePagesBeforeEpochForMultiPage (MultiPage &multiPage, const int32_t targetEpoch, const int32_t currentEpoch)
 

Private Attributes

FileMgrfm_
 
MultiPage metadataPages_
 
std::vector< MultiPagemultiPages_
 
size_t pageSize_
 
size_t pageDataSize_
 
size_t reservedHeaderSize_
 
ChunkKey chunkKey_
 

Static Private Attributes

static size_t headerBufferOffset_ = 32
 

Friends

class FileMgr
 

Additional Inherited Members

- Protected Attributes inherited from Data_Namespace::AbstractBuffer
std::unique_ptr< Encoderencoder_
 
SQLTypeInfo sql_type_
 
size_t size_
 
int device_id_
 

Detailed Description

Represents/provides access to contiguous data stored in the file system.

The FileBuffer consists of logical pages, which can map to any identically-sized page in any file of the underlying file system. A page's metadata (file and page number) are stored in MultiPage objects, and each MultiPage includes page metadata for multiple versions of the same page.

Note that a "Chunk" is brought into a FileBuffer by the FileMgr.

Note(s): Forbid Copying Idiom 4.1

Definition at line 56 of file FileBuffer.h.

Constructor & Destructor Documentation

File_Namespace::FileBuffer::FileBuffer ( FileMgr fm,
const size_t  pageSize,
const ChunkKey chunkKey,
const size_t  initialSize = 0 
)

Constructs a FileBuffer object.

Definition at line 39 of file FileBuffer.cpp.

References calcHeaderBuffer(), CHECK, fm_, pageDataSize_, pageSize_, and reservedHeaderSize_.

43  : AbstractBuffer(fm->getDeviceId())
44  , fm_(fm)
47  , chunkKey_(chunkKey) {
48  // Create a new FileBuffer
49  CHECK(fm_);
52  //@todo reintroduce initialSize - need to develop easy way of
53  // differentiating these pre-allocated pages from "written-to" pages
54  /*
55  if (initalSize > 0) {
56  // should expand to initialSize bytes
57  size_t initialNumPages = (initalSize + pageSize_ -1) / pageSize_;
58  int32_t epoch = fm_->epoch();
59  for (size_t pageNum = 0; pageNum < initialNumPages; ++pageNum) {
60  Page page = addNewMultiPage(epoch);
61  writeHeader(page,pageNum,epoch);
62  }
63  }
64  */
65 }
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
AbstractBuffer(const int device_id)
size_t pageSize() const override
Returns the size in bytes of each page in the FileBuffer.
Definition: FileBuffer.h:131
#define CHECK(condition)
Definition: Logger.h:197

+ Here is the call graph for this function:

File_Namespace::FileBuffer::FileBuffer ( FileMgr fm,
const size_t  pageSize,
const ChunkKey chunkKey,
const SQLTypeInfo  sqlType,
const size_t  initialSize = 0 
)

Definition at line 67 of file FileBuffer.cpp.

References calcHeaderBuffer(), CHECK, fm_, pageDataSize_, pageSize_, and reservedHeaderSize_.

72  : AbstractBuffer(fm->getDeviceId(), sqlType)
73  , fm_(fm)
76  , chunkKey_(chunkKey) {
77  CHECK(fm_);
80 }
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
AbstractBuffer(const int device_id)
size_t pageSize() const override
Returns the size in bytes of each page in the FileBuffer.
Definition: FileBuffer.h:131
#define CHECK(condition)
Definition: Logger.h:197

+ Here is the call graph for this function:

File_Namespace::FileBuffer::FileBuffer ( FileMgr fm,
const ChunkKey chunkKey,
const std::vector< HeaderInfo >::const_iterator &  headerStartIt,
const std::vector< HeaderInfo >::const_iterator &  headerEndIt 
)

Definition at line 82 of file FileBuffer.cpp.

References calcHeaderBuffer(), CHECK, File_Namespace::MultiPage::current(), logger::FATAL, File_Namespace::Page::fileId, fm_, LOG, metadataPages_, multiPages_, File_Namespace::EpochedPage::page, pageDataSize_, pageSize_, File_Namespace::MultiPage::push(), readMetadata(), reservedHeaderSize_, and show_chunk().

86  : AbstractBuffer(fm->getDeviceId())
87  , fm_(fm)
89  , pageSize_(0)
90  , chunkKey_(chunkKey) {
91  // We are being assigned an existing FileBuffer on disk
92 
93  CHECK(fm_);
95  int32_t lastPageId = -1;
96  // Page lastMetadataPage;
97  for (auto vecIt = headerStartIt; vecIt != headerEndIt; ++vecIt) {
98  int32_t curPageId = vecIt->pageId;
99 
100  // We only want to read last metadata page
101  if (curPageId == -1) { // stats page
102  metadataPages_.push(vecIt->page, vecIt->versionEpoch);
103  } else {
104  if (curPageId != lastPageId) {
105  // protect from bad data on disk, and give diagnostics
106  if (curPageId != lastPageId + 1) {
107  LOG(FATAL) << "Failure reading DB file " << show_chunk(chunkKey)
108  << " Current page " << curPageId << " last page " << lastPageId
109  << " epoch " << vecIt->versionEpoch;
110  }
111  if (lastPageId == -1) {
112  // If we are on first real page
113  CHECK(metadataPages_.current().page.fileId != -1); // was initialized
116  }
117  MultiPage multiPage(pageSize_);
118  multiPages_.push_back(multiPage);
119  lastPageId = curPageId;
120  }
121  multiPages_.back().push(vecIt->page, vecIt->versionEpoch);
122  }
123  if (curPageId == -1) { // meaning there was only a metadata page
126  }
127  }
128 }
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
#define LOG(tag)
Definition: Logger.h:188
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
std::string show_chunk(const ChunkKey &key)
Definition: types.h:73
AbstractBuffer(const int device_id)
int32_t fileId
Definition: Page.h:47
void push(const Page &page, const int epoch)
Pushes a new page with epoch value.
Definition: Page.h:95
#define CHECK(condition)
Definition: Logger.h:197
void readMetadata(const Page &page)
Definition: FileBuffer.cpp:415
EpochedPage current() const
Returns a reference to the most recent version of the page.
Definition: Page.h:87

+ Here is the call graph for this function:

File_Namespace::FileBuffer::~FileBuffer ( )
override

Destructor.

Definition at line 130 of file FileBuffer.cpp.

130  {
131  // need to free pages
132  // NOP
133 }

Member Function Documentation

Page File_Namespace::FileBuffer::addNewMultiPage ( const int32_t  epoch)

Definition at line 388 of file FileBuffer.cpp.

References fm_, multiPages_, pageSize_, File_Namespace::MultiPage::push(), and File_Namespace::FileMgr::requestFreePage().

Referenced by append(), reserve(), and write().

388  {
389  Page page = fm_->requestFreePage(pageSize_, false);
390  MultiPage multiPage(pageSize_);
391  multiPage.push(page, epoch);
392  multiPages_.emplace_back(multiPage);
393  return page;
394 }
Page requestFreePage(size_t pagesize, const bool isMetadata)
Definition: FileMgr.cpp:871
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::append ( int8_t *  src,
const size_t  numBytes,
const MemoryLevel  srcMemoryLevel = CPU_LEVEL,
const int32_t  deviceId = -1 
)
override

Definition at line 472 of file FileBuffer.cpp.

References addNewMultiPage(), CHECK, File_Namespace::FileMgr::epoch(), File_Namespace::Page::fileId, fm_, File_Namespace::FileMgr::getFileInfoForFileId(), multiPages_, pageDataSize_, File_Namespace::Page::pageNum, pageSize_, reservedHeaderSize_, Data_Namespace::AbstractBuffer::setAppended(), Data_Namespace::AbstractBuffer::size_, File_Namespace::FileInfo::write(), and writeHeader().

Referenced by File_Namespace::FileMgr::putBuffer().

475  {
476  setAppended();
477 
478  size_t startPage = size_ / pageDataSize_;
479  size_t startPageOffset = size_ % pageDataSize_;
480  size_t numPagesToWrite =
481  (numBytes + startPageOffset + pageDataSize_ - 1) / pageDataSize_;
482  size_t bytesLeft = numBytes;
483  int8_t* curPtr = src; // a pointer to the current location in dst being written to
484  size_t initialNumPages = multiPages_.size();
485  size_ = size_ + numBytes;
486  int32_t epoch = fm_->epoch();
487  for (size_t pageNum = startPage; pageNum < startPage + numPagesToWrite; ++pageNum) {
488  Page page;
489  if (pageNum >= initialNumPages) {
490  page = addNewMultiPage(epoch);
491  writeHeader(page, pageNum, epoch);
492  } else {
493  // we already have a new page at current
494  // epoch for this page - just grab this page
495  page = multiPages_[pageNum].current().page;
496  }
497  CHECK(page.fileId >= 0); // make sure page was initialized
498  FileInfo* fileInfo = fm_->getFileInfoForFileId(page.fileId);
499  size_t bytesWritten;
500  if (pageNum == startPage) {
501  bytesWritten = fileInfo->write(
502  page.pageNum * pageSize_ + startPageOffset + reservedHeaderSize_,
503  min(pageDataSize_ - startPageOffset, bytesLeft),
504  curPtr);
505  } else {
506  bytesWritten = fileInfo->write(page.pageNum * pageSize_ + reservedHeaderSize_,
507  min(pageDataSize_, bytesLeft),
508  curPtr);
509  }
510  curPtr += bytesWritten;
511  bytesLeft -= bytesWritten;
512  }
513  CHECK(bytesLeft == 0);
514 }
Page addNewMultiPage(const int32_t epoch)
Definition: FileBuffer.cpp:388
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
size_t write(const size_t offset, const size_t size, int8_t *buf)
Definition: FileInfo.cpp:60
FileInfo * getFileInfoForFileId(const int32_t fileId)
Definition: FileMgr.h:187
void writeHeader(Page &page, const int32_t pageId, const int32_t epoch, const bool writeMetadata=false)
Write header writes header at top of page in format.
Definition: FileBuffer.cpp:396
int32_t epoch()
Returns current value of epoch - should be one greater than recorded at last checkpoint.
Definition: FileMgr.h:245
#define CHECK(condition)
Definition: Logger.h:197

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::calcHeaderBuffer ( )
private

Definition at line 146 of file FileBuffer.cpp.

References chunkKey_, headerBufferOffset_, and reservedHeaderSize_.

Referenced by FileBuffer().

146  {
147  // 3 * sizeof(int32_t) is for headerSize, for pageId and versionEpoch
148  // sizeof(size_t) is for chunkSize
149  reservedHeaderSize_ = (chunkKey_.size() + 3) * sizeof(int32_t);
150  size_t headerMod = reservedHeaderSize_ % headerBufferOffset_;
151  if (headerMod > 0) {
153  }
154 }
static size_t headerBufferOffset_
Definition: FileBuffer.h:174

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::copyPage ( Page srcPage,
Page destPage,
const size_t  numBytes,
const size_t  offset = 0 
)

Definition at line 368 of file FileBuffer.cpp.

References CHECK, checked_malloc(), File_Namespace::Page::fileId, fm_, File_Namespace::FileMgr::getFileInfoForFileId(), pageDataSize_, File_Namespace::Page::pageNum, pageSize_, File_Namespace::FileInfo::read(), reservedHeaderSize_, and File_Namespace::FileInfo::write().

Referenced by write().

371  {
372  // FILE *srcFile = fm_->files_[srcPage.fileId]->f;
373  // FILE *destFile = fm_->files_[destPage.fileId]->f;
374  CHECK(offset + numBytes < pageDataSize_);
375  FileInfo* srcFileInfo = fm_->getFileInfoForFileId(srcPage.fileId);
376  FileInfo* destFileInfo = fm_->getFileInfoForFileId(destPage.fileId);
377 
378  int8_t* buffer = reinterpret_cast<int8_t*>(checked_malloc(numBytes));
379  size_t bytesRead = srcFileInfo->read(
380  srcPage.pageNum * pageSize_ + offset + reservedHeaderSize_, numBytes, buffer);
381  CHECK(bytesRead == numBytes);
382  size_t bytesWritten = destFileInfo->write(
383  destPage.pageNum * pageSize_ + offset + reservedHeaderSize_, numBytes, buffer);
384  CHECK(bytesWritten == numBytes);
385  free(buffer);
386 }
void * checked_malloc(const size_t size)
Definition: checked_alloc.h:44
FileInfo * getFileInfoForFileId(const int32_t fileId)
Definition: FileMgr.h:187
size_t read(const size_t offset, const size_t size, int8_t *buf)
Definition: FileInfo.cpp:66
#define CHECK(condition)
Definition: Logger.h:197

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

size_t File_Namespace::FileBuffer::freeChunkPages ( )

Definition at line 172 of file FileBuffer.cpp.

References freePage(), and multiPages_.

Referenced by foreign_storage::ForeignStorageCache::eraseChunk(), foreign_storage::ForeignStorageCache::evictChunkByIterator(), and freePages().

172  {
173  size_t num_pages_freed = multiPages_.size();
174  for (auto multiPageIt = multiPages_.begin(); multiPageIt != multiPages_.end();
175  ++multiPageIt) {
176  for (auto pageIt = multiPageIt->pageVersions.begin();
177  pageIt != multiPageIt->pageVersions.end();
178  ++pageIt) {
179  freePage(pageIt->page, false /* isRolloff */);
180  }
181  }
182  multiPages_.clear();
183  return num_pages_freed;
184 }
void freePage(const Page &page, const bool isRolloff)
Definition: FileBuffer.cpp:156
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::freeMetadataPages ( )

Definition at line 161 of file FileBuffer.cpp.

References freePage(), metadataPages_, File_Namespace::MultiPage::pageVersions, and File_Namespace::MultiPage::pop().

Referenced by freePages().

161  {
162  for (auto metaPageIt = metadataPages_.pageVersions.begin();
163  metaPageIt != metadataPages_.pageVersions.end();
164  ++metaPageIt) {
165  freePage(metaPageIt->page, false /* isRolloff */);
166  }
167  while (metadataPages_.pageVersions.size() > 0) {
169  }
170 }
void freePage(const Page &page, const bool isRolloff)
Definition: FileBuffer.cpp:156
void pop()
Purges the oldest Page.
Definition: Page.h:100
std::deque< EpochedPage > pageVersions
Definition: Page.h:74

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::freePage ( const Page page,
const bool  isRolloff 
)
private

Definition at line 156 of file FileBuffer.cpp.

References File_Namespace::Page::fileId, fm_, File_Namespace::FileInfo::freePage(), File_Namespace::FileMgr::getFileInfoForFileId(), and File_Namespace::Page::pageNum.

Referenced by freeChunkPages(), freeMetadataPages(), and freePagesBeforeEpochForMultiPage().

156  {
157  FileInfo* fileInfo = fm_->getFileInfoForFileId(page.fileId);
158  fileInfo->freePage(page.pageNum, isRolloff);
159 }
void freePage(int32_t pageId, const bool isRolloff)
Definition: FileInfo.cpp:199
FileInfo * getFileInfoForFileId(const int32_t fileId)
Definition: FileMgr.h:187

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::freePages ( )

Definition at line 186 of file FileBuffer.cpp.

References freeChunkPages(), and freeMetadataPages().

186  {
188  freeChunkPages();
189 }

+ Here is the call graph for this function:

void File_Namespace::FileBuffer::freePagesBeforeEpoch ( const int32_t  targetEpoch)

Definition at line 201 of file FileBuffer.cpp.

References CHECK_LE, File_Namespace::FileMgr::epoch(), fm_, freePagesBeforeEpochForMultiPage(), metadataPages_, and multiPages_.

201  {
202  // This method is only safe to be called within a checkpoint, after the sync and epoch
203  // increment where a failure at any point32_t in the process would lead to a safe
204  // rollback
205  const int32_t currentEpoch = fm_->epoch();
206  CHECK_LE(targetEpoch, currentEpoch);
207  freePagesBeforeEpochForMultiPage(metadataPages_, targetEpoch, currentEpoch);
208  for (auto& multiPage : multiPages_) {
209  freePagesBeforeEpochForMultiPage(multiPage, targetEpoch, currentEpoch);
210  }
211 }
void freePagesBeforeEpochForMultiPage(MultiPage &multiPage, const int32_t targetEpoch, const int32_t currentEpoch)
Definition: FileBuffer.cpp:191
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
#define CHECK_LE(x, y)
Definition: Logger.h:208
int32_t epoch()
Returns current value of epoch - should be one greater than recorded at last checkpoint.
Definition: FileMgr.h:245

+ Here is the call graph for this function:

void File_Namespace::FileBuffer::freePagesBeforeEpochForMultiPage ( MultiPage multiPage,
const int32_t  targetEpoch,
const int32_t  currentEpoch 
)
private

Definition at line 191 of file FileBuffer.cpp.

References freePage(), and File_Namespace::MultiPage::freePagesBeforeEpoch().

Referenced by freePagesBeforeEpoch().

193  {
194  std::vector<EpochedPage> epochedPagesToFree =
195  multiPage.freePagesBeforeEpoch(targetEpoch, currentEpoch);
196  for (const auto& epochedPageToFree : epochedPagesToFree) {
197  freePage(epochedPageToFree.page, true /* isRolloff */);
198  }
199 }
void freePage(const Page &page, const bool isRolloff)
Definition: FileBuffer.cpp:156

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

int8_t* File_Namespace::FileBuffer::getMemoryPtr ( )
inlineoverridevirtual

Not implemented for FileMgr – throws a runtime_error.

Implements Data_Namespace::AbstractBuffer.

Definition at line 122 of file FileBuffer.h.

References logger::FATAL, and LOG.

122  {
123  LOG(FATAL) << "Operation not supported.";
124  return nullptr; // satisfy return-type warning
125  }
#define LOG(tag)
Definition: Logger.h:188
virtual std::vector<MultiPage> File_Namespace::FileBuffer::getMultiPage ( ) const
inlinevirtual

Returns vector of MultiPages in the FileBuffer.

Definition at line 141 of file FileBuffer.h.

Referenced by File_Namespace::FileMgr::init(), and read().

141 { return multiPages_; }
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176

+ Here is the caller graph for this function:

Data_Namespace::MemoryLevel File_Namespace::FileBuffer::getType ( ) const
inlineoverridevirtual

Implements Data_Namespace::AbstractBuffer.

Definition at line 119 of file FileBuffer.h.

References Data_Namespace::DISK_LEVEL.

size_t File_Namespace::FileBuffer::numMetadataPages ( ) const
inline

Returns the total number of used bytes in the FileBuffer.

Definition at line 149 of file FileBuffer.h.

149 { return metadataPages_.pageVersions.size(); };
std::deque< EpochedPage > pageVersions
Definition: Page.h:74
size_t File_Namespace::FileBuffer::pageCount ( ) const
inlineoverridevirtual

Returns the number of pages in the FileBuffer.

Implements Data_Namespace::AbstractBuffer.

Definition at line 128 of file FileBuffer.h.

128 { return multiPages_.size(); }
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
virtual size_t File_Namespace::FileBuffer::pageDataSize ( ) const
inlinevirtual

Returns the size in bytes of the data portion of each page in the FileBuffer.

Definition at line 134 of file FileBuffer.h.

Referenced by File_Namespace::FileMgr::init(), and File_Namespace::readForThread().

134 { return pageDataSize_; }

+ Here is the caller graph for this function:

size_t File_Namespace::FileBuffer::pageSize ( ) const
inlineoverridevirtual

Returns the size in bytes of each page in the FileBuffer.

Implements Data_Namespace::AbstractBuffer.

Definition at line 131 of file FileBuffer.h.

Referenced by File_Namespace::FileMgr::init(), File_Namespace::readForThread(), and writeHeader().

131 { return pageSize_; }

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::read ( int8_t *const  dst,
const size_t  numBytes = 0,
const size_t  offset = 0,
const MemoryLevel  dstMemoryLevel = CPU_LEVEL,
const int32_t  deviceId = -1 
)
override

Definition at line 265 of file FileBuffer.cpp.

References CHECK, Data_Namespace::CPU_LEVEL, logger::FATAL, fm_, getMultiPage(), File_Namespace::FileMgr::getNumReaderThreads(), generate_TableFunctionsFactory_init::i, LOG, File_Namespace::readThreadDS::multiPages, multiPages_, pageDataSize_, File_Namespace::readForThread(), File_Namespace::readThreadDS::t_bytesLeft, File_Namespace::readThreadDS::t_curPtr, File_Namespace::readThreadDS::t_endPage, File_Namespace::readThreadDS::t_fm, File_Namespace::readThreadDS::t_isFirstPage, File_Namespace::readThreadDS::t_startPage, and File_Namespace::readThreadDS::t_startPageOffset.

269  {
270  if (dstBufferType != CPU_LEVEL) {
271  LOG(FATAL) << "Unsupported Buffer type";
272  }
273 
274  // variable declarations
275  size_t startPage = offset / pageDataSize_;
276  size_t startPageOffset = offset % pageDataSize_;
277  size_t numPagesToRead =
278  (numBytes + startPageOffset + pageDataSize_ - 1) / pageDataSize_;
279  /*
280  if (startPage + numPagesToRead > multiPages_.size()) {
281  cout << "Start page: " << startPage << endl;
282  cout << "Num pages to read: " << numPagesToRead << endl;
283  cout << "Num multipages: " << multiPages_.size() << endl;
284  cout << "Offset: " << offset << endl;
285  cout << "Num bytes: " << numBytes << endl;
286  }
287  */
288 
289  CHECK(startPage + numPagesToRead <= multiPages_.size());
290 
291  size_t numPagesPerThread = 0;
292  size_t numBytesCurrent = numBytes; // total number of bytes still to be read
293  size_t bytesRead = 0; // total number of bytes already being read
294  size_t bytesLeftForThread = 0; // number of bytes to be read in the thread
295  size_t numExtraPages = 0; // extra pages to be assigned one per thread as needed
296  size_t numThreads = fm_->getNumReaderThreads();
297  std::vector<readThreadDS>
298  threadDSArr; // array of threadDS, needed to avoid racing conditions
299 
300  if (numPagesToRead > numThreads) {
301  numPagesPerThread = numPagesToRead / numThreads;
302  numExtraPages = numPagesToRead - (numThreads * numPagesPerThread);
303  } else {
304  numThreads = numPagesToRead;
305  numPagesPerThread = 1;
306  }
307 
308  /* set threadDS for the first thread */
309  readThreadDS threadDS;
310  threadDS.t_fm = fm_;
311  threadDS.t_startPage = offset / pageDataSize_;
312  if (numExtraPages > 0) {
313  threadDS.t_endPage = threadDS.t_startPage + numPagesPerThread + 1;
314  numExtraPages--;
315  } else {
316  threadDS.t_endPage = threadDS.t_startPage + numPagesPerThread;
317  }
318  threadDS.t_curPtr = dst;
319  threadDS.t_startPageOffset = offset % pageDataSize_;
320  threadDS.t_isFirstPage = true;
321 
322  bytesLeftForThread = min(((threadDS.t_endPage - threadDS.t_startPage) * pageDataSize_ -
323  threadDS.t_startPageOffset),
324  numBytesCurrent);
325  threadDS.t_bytesLeft = bytesLeftForThread;
326  threadDS.multiPages = getMultiPage();
327 
328  if (numThreads == 1) {
329  bytesRead += readForThread(this, threadDS);
330  } else {
331  std::vector<std::future<size_t>> threads;
332 
333  for (size_t i = 0; i < numThreads; i++) {
334  threadDSArr.push_back(threadDS);
335  threads.push_back(
336  std::async(std::launch::async, readForThread, this, threadDSArr[i]));
337 
338  // calculate elements of threadDS
339  threadDS.t_fm = fm_;
340  threadDS.t_isFirstPage = false;
341  threadDS.t_curPtr += bytesLeftForThread;
342  threadDS.t_startPage +=
343  threadDS.t_endPage -
344  threadDS.t_startPage; // based on # of pages read on previous iteration
345  if (numExtraPages > 0) {
346  threadDS.t_endPage = threadDS.t_startPage + numPagesPerThread + 1;
347  numExtraPages--;
348  } else {
349  threadDS.t_endPage = threadDS.t_startPage + numPagesPerThread;
350  }
351  numBytesCurrent -= bytesLeftForThread;
352  bytesLeftForThread = min(
353  ((threadDS.t_endPage - threadDS.t_startPage) * pageDataSize_), numBytesCurrent);
354  threadDS.t_bytesLeft = bytesLeftForThread;
355  threadDS.multiPages = getMultiPage();
356  }
357 
358  for (auto& p : threads) {
359  p.wait();
360  }
361  for (auto& p : threads) {
362  bytesRead += p.get();
363  }
364  }
365  CHECK(bytesRead == numBytes);
366 }
virtual std::vector< MultiPage > getMultiPage() const
Returns vector of MultiPages in the FileBuffer.
Definition: FileBuffer.h:141
static size_t readForThread(FileBuffer *fileBuffer, const readThreadDS threadDS)
Definition: FileBuffer.cpp:224
#define LOG(tag)
Definition: Logger.h:188
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
#define CHECK(condition)
Definition: Logger.h:197
size_t getNumReaderThreads()
Returns number of threads defined by parameter num-reader-threads which should be used during initial...
Definition: FileMgr.h:278

+ Here is the call graph for this function:

void File_Namespace::FileBuffer::readMetadata ( const Page page)
private

Definition at line 415 of file FileBuffer.cpp.

References CHECK, Data_Namespace::AbstractBuffer::encoder_, generate_TableFunctionsFactory_init::f, File_Namespace::Page::fileId, fm_, File_Namespace::FileMgr::getFileForFileId(), Data_Namespace::AbstractBuffer::initEncoder(), METADATA_PAGE_SIZE, METADATA_VERSION, NUM_METADATA, File_Namespace::Page::pageNum, pageSize_, reservedHeaderSize_, SQLTypeInfo::set_comp_param(), SQLTypeInfo::set_compression(), SQLTypeInfo::set_dimension(), SQLTypeInfo::set_notnull(), SQLTypeInfo::set_scale(), SQLTypeInfo::set_size(), SQLTypeInfo::set_subtype(), SQLTypeInfo::set_type(), Data_Namespace::AbstractBuffer::size_, Data_Namespace::AbstractBuffer::sql_type_, and setup::version.

Referenced by FileBuffer().

415  {
416  FILE* f = fm_->getFileForFileId(page.fileId);
417  fseek(f, page.pageNum * METADATA_PAGE_SIZE + reservedHeaderSize_, SEEK_SET);
418  fread((int8_t*)&pageSize_, sizeof(size_t), 1, f);
419  fread((int8_t*)&size_, sizeof(size_t), 1, f);
420  vector<int32_t> typeData(
421  NUM_METADATA); // assumes we will encode hasEncoder, bufferType,
422  // encodingType, encodingBits all as int
423  fread((int8_t*)&(typeData[0]), sizeof(int32_t), typeData.size(), f);
424  int32_t version = typeData[0];
425  CHECK(version == METADATA_VERSION); // add backward compatibility code here
426  bool has_encoder = static_cast<bool>(typeData[1]);
427  if (has_encoder) {
428  sql_type_.set_type(static_cast<SQLTypes>(typeData[2]));
429  sql_type_.set_subtype(static_cast<SQLTypes>(typeData[3]));
430  sql_type_.set_dimension(typeData[4]);
431  sql_type_.set_scale(typeData[5]);
432  sql_type_.set_notnull(static_cast<bool>(typeData[6]));
433  sql_type_.set_compression(static_cast<EncodingType>(typeData[7]));
434  sql_type_.set_comp_param(typeData[8]);
435  sql_type_.set_size(typeData[9]);
437  encoder_->readMetadata(f);
438  }
439 }
void set_compression(EncodingType c)
Definition: sqltypes.h:411
void set_size(int s)
Definition: sqltypes.h:409
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
HOST DEVICE void set_subtype(SQLTypes st)
Definition: sqltypes.h:402
void initEncoder(const SQLTypeInfo &tmp_sql_type)
void set_scale(int s)
Definition: sqltypes.h:406
FILE * getFileForFileId(const int32_t fileId)
Returns FILE pointer associated with requested fileId.
Definition: FileMgr.cpp:988
void set_comp_param(int p)
Definition: sqltypes.h:412
version
Definition: setup.py:65
#define NUM_METADATA
Definition: FileBuffer.h:35
void set_dimension(int d)
Definition: sqltypes.h:403
std::unique_ptr< Encoder > encoder_
void set_notnull(bool n)
Definition: sqltypes.h:408
#define CHECK(condition)
Definition: Logger.h:197
#define METADATA_VERSION
Definition: FileBuffer.h:36
HOST DEVICE void set_type(SQLTypes t)
Definition: sqltypes.h:401

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::reserve ( const size_t  numBytes)
overridevirtual

Implements Data_Namespace::AbstractBuffer.

Definition at line 135 of file FileBuffer.cpp.

References addNewMultiPage(), File_Namespace::FileMgr::epoch(), fm_, multiPages_, pageSize_, and writeHeader().

135  {
136  size_t numPagesRequested = (numBytes + pageSize_ - 1) / pageSize_;
137  size_t numCurrentPages = multiPages_.size();
138  int32_t epoch = fm_->epoch();
139 
140  for (size_t pageNum = numCurrentPages; pageNum < numPagesRequested; ++pageNum) {
141  Page page = addNewMultiPage(epoch);
142  writeHeader(page, pageNum, epoch);
143  }
144 }
Page addNewMultiPage(const int32_t epoch)
Definition: FileBuffer.cpp:388
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
void writeHeader(Page &page, const int32_t pageId, const int32_t epoch, const bool writeMetadata=false)
Write header writes header at top of page in format.
Definition: FileBuffer.cpp:396
int32_t epoch()
Returns current value of epoch - should be one greater than recorded at last checkpoint.
Definition: FileMgr.h:245

+ Here is the call graph for this function:

virtual size_t File_Namespace::FileBuffer::reservedHeaderSize ( ) const
inlinevirtual

Returns the size in bytes of the reserved header portion of each page in the FileBuffer.

Definition at line 138 of file FileBuffer.h.

Referenced by File_Namespace::FileMgr::init(), and File_Namespace::readForThread().

138 { return reservedHeaderSize_; }

+ Here is the caller graph for this function:

size_t File_Namespace::FileBuffer::reservedSize ( ) const
inlineoverridevirtual

Returns the total number of bytes allocated for the FileBuffer.

Implements Data_Namespace::AbstractBuffer.

Definition at line 144 of file FileBuffer.h.

144 { return multiPages_.size() * pageSize_; }
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
void File_Namespace::FileBuffer::write ( int8_t *  src,
const size_t  numBytes,
const size_t  offset = 0,
const MemoryLevel  srcMemoryLevel = CPU_LEVEL,
const int32_t  deviceId = -1 
)
override

Writes the contents of source (src) into new versions of the affected logical pages.

This method will write the contents of source (src) into new version of the affected logical pages. New pages are only appended if the value of epoch (in FileMgr)

Definition at line 516 of file FileBuffer.cpp.

References addNewMultiPage(), CHECK, copyPage(), Data_Namespace::CPU_LEVEL, File_Namespace::FileMgr::epoch(), logger::FATAL, File_Namespace::Page::fileId, fm_, File_Namespace::FileMgr::getFileInfoForFileId(), LOG, multiPages_, pageDataSize_, File_Namespace::Page::pageNum, pageSize_, File_Namespace::FileMgr::requestFreePage(), reservedHeaderSize_, Data_Namespace::AbstractBuffer::setAppended(), Data_Namespace::AbstractBuffer::setDirty(), Data_Namespace::AbstractBuffer::setUpdated(), Data_Namespace::AbstractBuffer::size_, File_Namespace::FileInfo::write(), and writeHeader().

Referenced by File_Namespace::FileMgr::putBuffer().

520  {
521  if (srcBufferType != CPU_LEVEL) {
522  LOG(FATAL) << "Unsupported Buffer type";
523  }
524 
525  bool tempIsAppended = false;
526  setDirty();
527  if (offset < size_) {
528  setUpdated();
529  }
530  if (offset + numBytes > size_) {
531  tempIsAppended = true; // because is_appended_ could have already been true - to
532  // avoid rewriting header
533  setAppended();
534  size_ = offset + numBytes;
535  }
536 
537  size_t startPage = offset / pageDataSize_;
538  size_t startPageOffset = offset % pageDataSize_;
539  size_t numPagesToWrite =
540  (numBytes + startPageOffset + pageDataSize_ - 1) / pageDataSize_;
541  size_t bytesLeft = numBytes;
542  int8_t* curPtr = src; // a pointer to the current location in dst being written to
543  size_t initialNumPages = multiPages_.size();
544  int32_t epoch = fm_->epoch();
545 
546  if (startPage >
547  initialNumPages) { // means there is a gap we need to allocate pages for
548  for (size_t pageNum = initialNumPages; pageNum < startPage; ++pageNum) {
549  Page page = addNewMultiPage(epoch);
550  writeHeader(page, pageNum, epoch);
551  }
552  }
553  for (size_t pageNum = startPage; pageNum < startPage + numPagesToWrite; ++pageNum) {
554  Page page;
555  if (pageNum >= initialNumPages) {
556  page = addNewMultiPage(epoch);
557  writeHeader(page, pageNum, epoch);
558  } else if (multiPages_[pageNum].current().epoch <
559  epoch) { // need to create new page b/c this current one lags epoch and we
560  // can't overwrite it also need to copy if we are on first or
561  // last page
562  Page lastPage = multiPages_[pageNum].current().page;
563  page = fm_->requestFreePage(pageSize_, false);
564  multiPages_[pageNum].push(page, epoch);
565  if (pageNum == startPage && startPageOffset > 0) {
566  // copyPage takes care of header offset so don't worry
567  // about it
568  copyPage(lastPage, page, startPageOffset, 0);
569  }
570  if (pageNum == startPage + numPagesToWrite &&
571  bytesLeft > 0) { // bytesLeft should always > 0
572  copyPage(lastPage,
573  page,
574  pageDataSize_ - bytesLeft,
575  bytesLeft); // these would be empty if we're appending but we won't
576  // worry about it right now
577  }
578  writeHeader(page, pageNum, epoch);
579  } else {
580  // we already have a new page at current
581  // epoch for this page - just grab this page
582  page = multiPages_[pageNum].current().page;
583  }
584  CHECK(page.fileId >= 0); // make sure page was initialized
585  FileInfo* fileInfo = fm_->getFileInfoForFileId(page.fileId);
586  size_t bytesWritten;
587  if (pageNum == startPage) {
588  bytesWritten = fileInfo->write(
589  page.pageNum * pageSize_ + startPageOffset + reservedHeaderSize_,
590  min(pageDataSize_ - startPageOffset, bytesLeft),
591  curPtr);
592  } else {
593  bytesWritten = fileInfo->write(page.pageNum * pageSize_ + reservedHeaderSize_,
594  min(pageDataSize_, bytesLeft),
595  curPtr);
596  }
597  curPtr += bytesWritten;
598  bytesLeft -= bytesWritten;
599  if (tempIsAppended && pageNum == startPage + numPagesToWrite - 1) { // if last page
600  //@todo below can lead to undefined - we're overwriting num
601  // bytes valid at checkpoint
602  writeHeader(page, 0, multiPages_[0].current().epoch, true);
603  }
604  }
605  CHECK(bytesLeft == 0);
606 }
Page requestFreePage(size_t pagesize, const bool isMetadata)
Definition: FileMgr.cpp:871
#define LOG(tag)
Definition: Logger.h:188
Page addNewMultiPage(const int32_t epoch)
Definition: FileBuffer.cpp:388
std::vector< MultiPage > multiPages_
Definition: FileBuffer.h:176
size_t write(const size_t offset, const size_t size, int8_t *buf)
Definition: FileInfo.cpp:60
FileInfo * getFileInfoForFileId(const int32_t fileId)
Definition: FileMgr.h:187
void writeHeader(Page &page, const int32_t pageId, const int32_t epoch, const bool writeMetadata=false)
Write header writes header at top of page in format.
Definition: FileBuffer.cpp:396
int32_t epoch()
Returns current value of epoch - should be one greater than recorded at last checkpoint.
Definition: FileMgr.h:245
#define CHECK(condition)
Definition: Logger.h:197
void copyPage(Page &srcPage, Page &destPage, const size_t numBytes, const size_t offset=0)
Definition: FileBuffer.cpp:368

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::writeHeader ( Page page,
const int32_t  pageId,
const int32_t  epoch,
const bool  writeMetadata = false 
)
private

Write header writes header at top of page in format.

Definition at line 396 of file FileBuffer.cpp.

References chunkKey_, gpu_enabled::copy(), File_Namespace::Page::fileId, fm_, File_Namespace::FileMgr::getFileInfoForFileId(), METADATA_PAGE_SIZE, File_Namespace::Page::pageNum, pageSize(), pageSize_, and File_Namespace::FileInfo::write().

Referenced by append(), File_Namespace::FileMgr::init(), reserve(), write(), and writeMetadata().

399  {
400  int32_t intHeaderSize = chunkKey_.size() + 3; // does not include chunkSize
401  vector<int32_t> header(intHeaderSize);
402  // in addition to chunkkey we need size of header, pageId, version
403  header[0] =
404  (intHeaderSize - 1) * sizeof(int32_t); // don't need to include size of headerSize
405  // value - sizeof(size_t) is for chunkSize
406  std::copy(chunkKey_.begin(), chunkKey_.end(), header.begin() + 1);
407  header[intHeaderSize - 2] = pageId;
408  header[intHeaderSize - 1] = epoch;
409  FileInfo* fileInfo = fm_->getFileInfoForFileId(page.fileId);
411  fileInfo->write(
412  page.pageNum * pageSize, (intHeaderSize) * sizeof(int32_t), (int8_t*)&header[0]);
413 }
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
void writeMetadata(const int32_t epoch)
Definition: FileBuffer.cpp:441
DEVICE auto copy(ARGS &&...args)
Definition: gpu_enabled.h:51
size_t pageSize() const override
Returns the size in bytes of each page in the FileBuffer.
Definition: FileBuffer.h:131
FileInfo * getFileInfoForFileId(const int32_t fileId)
Definition: FileMgr.h:187

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::FileBuffer::writeMetadata ( const int32_t  epoch)
private

Definition at line 441 of file FileBuffer.cpp.

References Data_Namespace::AbstractBuffer::encoder_, generate_TableFunctionsFactory_init::f, File_Namespace::Page::fileId, fm_, SQLTypeInfo::get_comp_param(), SQLTypeInfo::get_compression(), SQLTypeInfo::get_dimension(), SQLTypeInfo::get_notnull(), SQLTypeInfo::get_scale(), SQLTypeInfo::get_size(), SQLTypeInfo::get_subtype(), SQLTypeInfo::get_type(), File_Namespace::FileMgr::getFileForFileId(), Data_Namespace::AbstractBuffer::hasEncoder(), METADATA_PAGE_SIZE, METADATA_VERSION, metadataPages_, NUM_METADATA, File_Namespace::Page::pageNum, pageSize_, File_Namespace::MultiPage::push(), File_Namespace::FileMgr::requestFreePage(), reservedHeaderSize_, Data_Namespace::AbstractBuffer::size_, Data_Namespace::AbstractBuffer::sql_type_, and writeHeader().

441  {
442  // Right now stats page is size_ (in bytes), bufferType, encodingType,
443  // encodingDataType, numElements
444  Page page = fm_->requestFreePage(METADATA_PAGE_SIZE, true);
445  writeHeader(page, -1, epoch, true);
446  FILE* f = fm_->getFileForFileId(page.fileId);
447  fseek(f, page.pageNum * METADATA_PAGE_SIZE + reservedHeaderSize_, SEEK_SET);
448  fwrite((int8_t*)&pageSize_, sizeof(size_t), 1, f);
449  fwrite((int8_t*)&size_, sizeof(size_t), 1, f);
450  vector<int32_t> typeData(
451  NUM_METADATA); // assumes we will encode hasEncoder, bufferType,
452  // encodingType, encodingBits all as int32_t
453  typeData[0] = METADATA_VERSION;
454  typeData[1] = static_cast<int32_t>(hasEncoder());
455  if (hasEncoder()) {
456  typeData[2] = static_cast<int32_t>(sql_type_.get_type());
457  typeData[3] = static_cast<int32_t>(sql_type_.get_subtype());
458  typeData[4] = sql_type_.get_dimension();
459  typeData[5] = sql_type_.get_scale();
460  typeData[6] = static_cast<int32_t>(sql_type_.get_notnull());
461  typeData[7] = static_cast<int32_t>(sql_type_.get_compression());
462  typeData[8] = sql_type_.get_comp_param();
463  typeData[9] = sql_type_.get_size();
464  }
465  fwrite((int8_t*)&(typeData[0]), sizeof(int32_t), typeData.size(), f);
466  if (hasEncoder()) { // redundant
467  encoder_->writeMetadata(f);
468  }
469  metadataPages_.push(page, epoch);
470 }
HOST DEVICE SQLTypes get_subtype() const
Definition: sqltypes.h:312
#define METADATA_PAGE_SIZE
Definition: FileBuffer.h:37
HOST DEVICE int get_size() const
Definition: sqltypes.h:321
Page requestFreePage(size_t pagesize, const bool isMetadata)
Definition: FileMgr.cpp:871
HOST DEVICE int get_scale() const
Definition: sqltypes.h:316
HOST DEVICE SQLTypes get_type() const
Definition: sqltypes.h:311
FILE * getFileForFileId(const int32_t fileId)
Returns FILE pointer associated with requested fileId.
Definition: FileMgr.cpp:988
void writeHeader(Page &page, const int32_t pageId, const int32_t epoch, const bool writeMetadata=false)
Write header writes header at top of page in format.
Definition: FileBuffer.cpp:396
HOST DEVICE EncodingType get_compression() const
Definition: sqltypes.h:319
#define NUM_METADATA
Definition: FileBuffer.h:35
void push(const Page &page, const int epoch)
Pushes a new page with epoch value.
Definition: Page.h:95
HOST DEVICE int get_dimension() const
Definition: sqltypes.h:313
std::unique_ptr< Encoder > encoder_
HOST DEVICE int get_comp_param() const
Definition: sqltypes.h:320
HOST DEVICE bool get_notnull() const
Definition: sqltypes.h:318
#define METADATA_VERSION
Definition: FileBuffer.h:36

+ Here is the call graph for this function:

Friends And Related Function Documentation

friend class FileMgr
friend

Definition at line 57 of file FileBuffer.h.

Member Data Documentation

ChunkKey File_Namespace::FileBuffer::chunkKey_
private

Definition at line 180 of file FileBuffer.h.

Referenced by calcHeaderBuffer(), and writeHeader().

FileMgr* File_Namespace::FileBuffer::fm_
private
size_t File_Namespace::FileBuffer::headerBufferOffset_ = 32
staticprivate

Definition at line 174 of file FileBuffer.h.

Referenced by calcHeaderBuffer().

MultiPage File_Namespace::FileBuffer::metadataPages_
private

Definition at line 175 of file FileBuffer.h.

Referenced by FileBuffer(), freeMetadataPages(), freePagesBeforeEpoch(), and writeMetadata().

std::vector<MultiPage> File_Namespace::FileBuffer::multiPages_
private
size_t File_Namespace::FileBuffer::pageDataSize_
private

Definition at line 178 of file FileBuffer.h.

Referenced by append(), copyPage(), FileBuffer(), read(), and write().

size_t File_Namespace::FileBuffer::pageSize_
private
size_t File_Namespace::FileBuffer::reservedHeaderSize_
private

The documentation for this class was generated from the following files: