File_Namespace::FileInfo Struct Reference

#include <FileInfo.h>

Collaboration diagram for File_Namespace::FileInfo:

Public Member Functions
	FileInfo (FileMgr *fileMgr, const int fileId, FILE *f, const size_t pageSize, const size_t numPages, const bool init=false)
	Constructor. More...
	~FileInfo ()
	Destructor. More...
void	initNewFile ()
	Adds all pages to freePages and zeroes first four bytes of header. More...
void	freePageDeferred (int pageId)
void	freePage (int pageId)
int	getFreePage ()
size_t	write (const size_t offset, const size_t size, int8_t *buf)
size_t	read (const size_t offset, const size_t size, int8_t *buf)
void	openExistingFile (std::vector< HeaderInfo > &headerVec, const int fileMgrEpoch)
void	print (bool pagesummary)
	Prints a summary of the file to stdout. More...
size_t	size ()
	Returns the number of bytes used by the file. More...
int	syncToDisk ()
size_t	available ()
	Returns the number of free bytes available. More...
size_t	numFreePages ()
	Returns the number of free pages available. More...
size_t	used ()
	Returns the amount of used bytes; size() - available() More...

Public Attributes
FileMgr *	fileMgr
int	fileId
FILE *	f
	unique file identifier (i.e., used for a file name) More...
size_t	pageSize
	file stream object for the represented file More...
size_t	numPages
	the fixed size of each page in the file More...
std::set< size_t >	freePages
	the number of pages in the file More...
std::mutex	freePagesMutex_
	set of page numbers of free pages More...
std::mutex	readWriteMutex_

Detailed Description

Definition at line 51 of file FileInfo.h.

Constructor & Destructor Documentation

File_Namespace::FileInfo::FileInfo	(	FileMgr *	fileMgr,
		const int	fileId,
		FILE *	f,
		const size_t	pageSize,
		const size_t	numPages,
		const bool	init = false
	)

Constructor.

Definition at line 28 of file FileInfo.cpp.

References initNewFile().

    : fileMgr(fileMgr), fileId(fileId), f(f), pageSize(pageSize), numPages(numPages) {
  if (init) {
    initNewFile();
  }
}

Here is the call graph for this function:

File_Namespace::FileInfo::~FileInfo

(

)

Destructor.

Definition at line 40 of file FileInfo.cpp.

References File_Namespace::close(), and f.

                    {
  // close file, if applicable
  if (f) {
    close(f);
  }
}

Here is the call graph for this function:

Member Function Documentation

size_t File_Namespace::FileInfo::available ( )

inline

Returns the number of free bytes available.

Definition at line 104 of file FileInfo.h.

References freePages, and pageSize.

Referenced by print(), and used().

{ return freePages.size() * pageSize; }

Here is the caller graph for this function:

void File_Namespace::FileInfo::freePage

(

int

pageId

)

Definition at line 202 of file FileInfo.cpp.

References CHECK, DELETE_CONTINGENT, File_Namespace::FileMgr::epoch(), f, fileMgr, File_Namespace::FileMgr::free_page(), freePages, freePagesMutex_, pageSize, and File_Namespace::write().

Referenced by File_Namespace::FileBuffer::freeChunkPages(), and File_Namespace::FileBuffer::freeMetadataPages().

                                  {
#define RESILIENT_PAGE_HEADER
#ifdef RESILIENT_PAGE_HEADER
  int epoch_freed_page[2] = {DELETE_CONTINGENT, fileMgr->epoch()};
  File_Namespace::write(f,
                        pageId * pageSize + sizeof(int),
                        sizeof(epoch_freed_page),
                        (int8_t*)epoch_freed_page);
  fileMgr->free_page(std::make_pair(this, pageId));
#else
  int zeroVal = 0;
  int8_t* zeroAddr = reinterpret_cast<int8_t*>(&zeroVal);
  File_Namespace::write(f, pageId * pageSize, sizeof(int), zeroAddr);
  std::lock_guard<std::mutex> lock(freePagesMutex_);
  freePages.insert(pageId);
#endif  // RESILIENT_PAGE_HEADER

#ifdef ENABLE_CRASH_CORRUPTION_TEST
  signal(SIGUSR2, sighandler);
  if (goto_crash)
    CHECK(pageId % 8 != 4);
#endif
}

Here is the call graph for this function:

Here is the caller graph for this function:

void File_Namespace::FileInfo::freePageDeferred

(

int

pageId

)

Definition at line 187 of file FileInfo.cpp.

References freePages, and freePagesMutex_.

                                          {
  std::lock_guard<std::mutex> lock(freePagesMutex_);
  freePages.insert(pageId);
}

int File_Namespace::FileInfo::getFreePage

(

)

Definition at line 226 of file FileInfo.cpp.

References freePages, and freePagesMutex_.

Referenced by File_Namespace::FileMgr::requestFreePage(), and File_Namespace::FileMgr::requestFreePages().

                          {
  // returns -1 if there is no free page
  std::lock_guard<std::mutex> lock(freePagesMutex_);
  if (freePages.size() == 0) {
    return -1;
  }
  auto pageIt = freePages.begin();
  int pageNum = *pageIt;
  freePages.erase(pageIt);
  return pageNum;
}

Here is the caller graph for this function:

void File_Namespace::FileInfo::initNewFile

(

)

Adds all pages to freePages and zeroes first four bytes of header.

Definition at line 47 of file FileInfo.cpp.

References f, freePages, numPages, pageSize, and File_Namespace::write().

Referenced by FileInfo().

                           {
  // initialize pages and free page list
  // Also zeroes out first four bytes of every header

  int headerSize = 0;
  int8_t* headerSizePtr = (int8_t*)(&headerSize);
  for (size_t pageId = 0; pageId < numPages; ++pageId) {
    File_Namespace::write(f, pageId * pageSize, sizeof(int), headerSizePtr);
    freePages.insert(pageId);
  }
}

Here is the call graph for this function:

Here is the caller graph for this function:

size_t File_Namespace::FileInfo::numFreePages ( )

inline

Returns the number of free pages available.

Definition at line 107 of file FileInfo.h.

References freePages, and freePagesMutex_.

                               {
    std::lock_guard<std::mutex> lock(freePagesMutex_);
    return freePages.size();
  }

void File_Namespace::FileInfo::openExistingFile	(	std::vector< HeaderInfo > &	headerVec,
		const int	fileMgrEpoch
	)

Definition at line 69 of file FileInfo.cpp.

References CHECK_EQ, CHECK_GE, DELETE_CONTINGENT, File_Namespace::FileMgr::epoch(), f, fileId, fileMgr, freePages, File_Namespace::FileMgr::get_fileMgrKey(), LOG, numPages, pageSize, show_chunk(), VLOG, logger::WARNING, and File_Namespace::write().

Referenced by File_Namespace::FileMgr::openExistingFile().

                                                        {
  // HeaderInfo is defined in Page.h
  ChunkKey oldChunkKey(4);
  int oldPageId = -99;
  int oldVersionEpoch = -99;
  int skipped = 0;
  for (size_t pageNum = 0; pageNum < numPages; ++pageNum) {
    int headerSize;

    constexpr size_t MAX_INTS_TO_READ{10};  // currently use 1+6 ints
    int ints[MAX_INTS_TO_READ];
    CHECK_EQ(fseek(f, pageNum * pageSize, SEEK_SET), 0);
    CHECK_EQ(fread(ints, sizeof(int), MAX_INTS_TO_READ, f), MAX_INTS_TO_READ);

    headerSize = ints[0];
    if (0 != headerSize) {
      if (DELETE_CONTINGENT == ints[1]) {
        if (fileMgr->epoch() > ints[2]) {
          int zero{0};
          File_Namespace::write(f, pageNum * pageSize, sizeof(int), (int8_t*)&zero);
          headerSize = 0;
        }
      }
    }

    if (headerSize != 0) {
      // headerSize doesn't include headerSize itself
      // We're tying ourself to headers of ints here
      size_t numHeaderElems = headerSize / sizeof(int);
      CHECK_GE(numHeaderElems, size_t(2));
      // size_t chunkSize;
      // We don't want to read headerSize in our header - so start
      // reading 4 bytes past it

      // always derive dbid/tbid from FileMgr
      ChunkKey chunkKey(&ints[1], &ints[1 + numHeaderElems - 2]);
      chunkKey[0] = fileMgr->get_fileMgrKey().first;
      chunkKey[1] = fileMgr->get_fileMgrKey().second;
      // recover page in case a crash failed deletion of this page
      if (DELETE_CONTINGENT == ints[1]) {
        File_Namespace::write(
            f, pageNum * pageSize + sizeof(int), 2 * sizeof(int), (int8_t*)&chunkKey[0]);
      }

      // cout << "Chunk key: " << show_chunk(chunkKey) << endl;
      // Last two elements of header are always PageId and Version
      // epoch - these are not in the chunk key so seperate them
      int pageId = ints[1 + numHeaderElems - 2];
      // cout << "Page id: " << pageId << endl;
      int versionEpoch = ints[1 + numHeaderElems - 1];
      if (chunkKey != oldChunkKey || oldPageId != pageId - (1 + skipped)) {
        if (skipped > 0) {
          VLOG(4) << "FId.PSz: " << fileId << "." << pageSize
                  << " Chunk key: " << show_chunk(oldChunkKey)
                  << " Page id from : " << oldPageId << " to : " << oldPageId + skipped
                  << " Epoch: " << oldVersionEpoch;
        } else if (oldPageId != -99) {
          VLOG(4) << "FId.PSz: " << fileId << "." << pageSize
                  << " Chunk key: " << show_chunk(oldChunkKey)
                  << " Page id: " << oldPageId << " Epoch: " << oldVersionEpoch;
        }
        oldPageId = pageId;
        oldVersionEpoch = versionEpoch;
        oldChunkKey = chunkKey;
        skipped = 0;
      } else {
        skipped++;
      }
      // read(f,pageNum*pageSize+sizeof(int),headerSize-2*sizeof(int),(int8_t
      // *)(&chunkKey[0])); read(f,pageNum*pageSize+sizeof(int) + headerSize -
      // 2*sizeof(int),sizeof(int),(int8_t *)(&pageId));
      // read(f,pageNum*pageSize+sizeof(int) + headerSize -
      // sizeof(int),sizeof(int),(int8_t *)(&versionEpoch));
      // read(f,pageNum*pageSize+sizeof(int) + headerSize -
      // sizeof(size_t),sizeof(size_t),(int8_t *)(&chunkSize));

      /* Check if version epoch is equal to
       * or greater (note: should never be greater)
       * than FileMgr epoch_ - this means that this
       * page wasn't checkpointed and thus we should
       * not use it
       */
      if (versionEpoch >= fileMgrEpoch) {
        // First write 0 to first four bytes of
        // header to mark as free
        headerSize = 0;
        File_Namespace::write(f, pageNum * pageSize, sizeof(int), (int8_t*)&headerSize);
        // Now add page to free list
        freePages.insert(pageNum);
        LOG(WARNING) << "Was not checkpointed: Chunk key: " << show_chunk(chunkKey)
                     << " Page id: " << pageId << " Epoch: " << versionEpoch
                     << " FileMgrEpoch " << fileMgrEpoch << endl;

      } else {  // page was checkpointed properly
        Page page(fileId, pageNum);
        headerVec.emplace_back(chunkKey, pageId, versionEpoch, page);
        // std::cout << "Inserted into headerVec" << std::endl;
      }
    } else {  // no header for this page - insert into free list
      freePages.insert(pageNum);
    }
  }
  // printlast
  if (oldPageId != -99) {
    if (skipped > 0) {
      VLOG(4) << "FId.PSz: " << fileId << "." << pageSize
              << " Chunk key: " << show_chunk(oldChunkKey)
              << " Page id from : " << oldPageId << " to : " << oldPageId + skipped
              << " Epoch: " << oldVersionEpoch;
    } else {
      VLOG(4) << "FId.PSz: " << fileId << "." << pageSize
              << " Chunk key: " << show_chunk(oldChunkKey) << " Page id: " << oldPageId
              << " Epoch: " << oldVersionEpoch;
    }
  }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void File_Namespace::FileInfo::print

(

bool

pagesummary

)

Prints a summary of the file to stdout.

Definition at line 238 of file FileInfo.cpp.

References available(), fileId, size(), and used().

                                     {
  std::cout << "File: " << fileId << std::endl;
  std::cout << "Size: " << size() << std::endl;
  std::cout << "Used: " << used() << std::endl;
  std::cout << "Free: " << available() << std::endl;
  if (!pagesummary) {
    return;
  }

  // for (size_t i = 0; i < pages.size(); ++i) {
  //    // @todo page summary
  //}
}

Here is the call graph for this function:

size_t File_Namespace::FileInfo::read	(	const size_t	offset,
		const size_t	size,
		int8_t *	buf
	)

Definition at line 64 of file FileInfo.cpp.

References f, File_Namespace::read(), and readWriteMutex_.

Referenced by File_Namespace::FileBuffer::copyPage(), File_Namespace::FileMgr::copyPage(), and File_Namespace::readForThread().

                                                                         {
  std::lock_guard<std::mutex> lock(readWriteMutex_);
  return File_Namespace::read(f, offset, size, buf);
}

Here is the call graph for this function:

Here is the caller graph for this function:

size_t File_Namespace::FileInfo::size ( )

inline

Returns the number of bytes used by the file.

Definition at line 89 of file FileInfo.h.

References numPages, and pageSize.

Referenced by print(), and used().

{ return pageSize * numPages; }

Here is the caller graph for this function:

int File_Namespace::FileInfo::syncToDisk ( )

inline

Definition at line 91 of file FileInfo.h.

References f, logger::FATAL, omnisci::fsync(), and LOG.

                          {
    if (fflush(f) != 0) {
      LOG(FATAL) << "Error trying to flush changes to disk, the error was: "
                 << std::strerror(errno);
    }
#ifdef __APPLE__
    return fcntl(fileno(f), 51);
#else
    return fsync(fileno(f));
#endif
  }

Here is the call graph for this function:

size_t File_Namespace::FileInfo::used ( )

inline

Returns the amount of used bytes; size() - available()

Definition at line 113 of file FileInfo.h.

References available(), and size().

Referenced by print().

{ return size() - available(); }

Here is the call graph for this function:

Here is the caller graph for this function:

size_t File_Namespace::FileInfo::write	(	const size_t	offset,
		const size_t	size,
		int8_t *	buf
	)

Definition at line 59 of file FileInfo.cpp.

References f, readWriteMutex_, and File_Namespace::write().

Referenced by File_Namespace::FileBuffer::append(), File_Namespace::FileBuffer::copyPage(), File_Namespace::FileMgr::copyPage(), File_Namespace::FileBuffer::write(), and File_Namespace::FileBuffer::writeHeader().

                                                                          {
  std::lock_guard<std::mutex> lock(readWriteMutex_);
  return File_Namespace::write(f, offset, size, buf);
}

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

FILE* File_Namespace::FileInfo::f

unique file identifier (i.e., used for a file name)

Definition at line 54 of file FileInfo.h.

Referenced by freePage(), initNewFile(), openExistingFile(), read(), syncToDisk(), write(), and ~FileInfo().

int File_Namespace::FileInfo::fileId

Definition at line 53 of file FileInfo.h.

Referenced by openExistingFile(), print(), File_Namespace::FileMgr::requestFreePage(), and File_Namespace::FileMgr::requestFreePages().

FileMgr* File_Namespace::FileInfo::fileMgr

Definition at line 52 of file FileInfo.h.

Referenced by freePage(), and openExistingFile().

std::set<size_t> File_Namespace::FileInfo::freePages

the number of pages in the file

Definition at line 59 of file FileInfo.h.

Referenced by available(), freePage(), freePageDeferred(), getFreePage(), initNewFile(), numFreePages(), and openExistingFile().

std::mutex File_Namespace::FileInfo::freePagesMutex_

set of page numbers of free pages

Definition at line 60 of file FileInfo.h.

Referenced by freePage(), freePageDeferred(), getFreePage(), and numFreePages().

size_t File_Namespace::FileInfo::numPages

the fixed size of each page in the file

Definition at line 56 of file FileInfo.h.

Referenced by initNewFile(), openExistingFile(), and size().

size_t File_Namespace::FileInfo::pageSize

file stream object for the represented file

Definition at line 55 of file FileInfo.h.

Referenced by available(), freePage(), initNewFile(), openExistingFile(), and size().

std::mutex File_Namespace::FileInfo::readWriteMutex_

Definition at line 61 of file FileInfo.h.

Referenced by read(), and write().

---

The documentation for this struct was generated from the following files:

• /home/jenkins-slave/workspace/core-os-doxygen/DataMgr/FileMgr/FileInfo.h
• /home/jenkins-slave/workspace/core-os-doxygen/DataMgr/FileMgr/FileInfo.cpp