OmniSciDB  a987f07e93
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
File_Namespace::TableFileMgr Class Reference

#include <CachingFileMgr.h>

+ Collaboration diagram for File_Namespace::TableFileMgr:

Public Member Functions

 TableFileMgr (const std::string &table_path)
 
 ~TableFileMgr ()
 
void incrementEpoch ()
 increment the epoch for this subdir (not synced to disk). More...
 
void writeAndSyncEpochToDisk ()
 Write and flush the epoch to the epoch file on disk. More...
 
int32_t getEpoch () const
 Returns the current epoch (locked) More...
 
void removeDiskContent () const
 Removes all disk data for the subdir. More...
 
size_t getReservedSpace () const
 Returns the disk space used (in bytes) for the subdir. More...
 
void deleteWrapperFile () const
 Deletes only the wrapper file on disk. More...
 
void writeWrapperFile (const std::string &doc) const
 Writes wrapper file to disk. More...
 
bool hasWrapperFile () const
 

Private Attributes

std::string table_path_
 
std::string epoch_file_path_
 
std::string wrapper_file_path_
 
Epoch epoch_
 
bool is_checkpointed_ = true
 
FILE * epoch_file_ = nullptr
 
heavyai::shared_mutex table_mutex_
 

Detailed Description

Definition at line 92 of file CachingFileMgr.h.

Constructor & Destructor Documentation

File_Namespace::TableFileMgr::TableFileMgr ( const std::string &  table_path)

Definition at line 755 of file CachingFileMgr.cpp.

References Epoch::byte_size(), CHECK, File_Namespace::create(), epoch_, epoch_file_, epoch_file_path_, heavyai::file_size(), incrementEpoch(), File_Namespace::open(), File_Namespace::read(), Epoch::storage_ptr(), table_path_, and writeAndSyncEpochToDisk().

756  : table_path_(table_path)
759  , epoch_(Epoch())
760  , is_checkpointed_(true) {
761  if (!bf::exists(table_path_)) {
762  bf::create_directory(table_path_);
763  } else {
764  CHECK(bf::is_directory(table_path_)) << "Specified path '" << table_path_
765  << "' for cache table data is not a directory.";
766  }
767  if (bf::exists(epoch_file_path_)) {
768  CHECK(bf::is_regular_file(epoch_file_path_))
769  << "Found epoch file '" << epoch_file_path_ << "' which is not a regular file";
771  << "Found epoch file '" << epoch_file_path_ << "' which is not of expected size";
774  } else {
777  incrementEpoch();
778  }
779 }
int8_t * storage_ptr()
Definition: Epoch.h:61
void writeAndSyncEpochToDisk()
Write and flush the epoch to the epoch file on disk.
FILE * create(const std::string &basePath, const int fileId, const size_t pageSize, const size_t numPages)
Definition: File.cpp:57
size_t read(FILE *f, const size_t offset, const size_t size, int8_t *buf)
Reads the specified number of bytes from the offset position in file f into buf.
Definition: File.cpp:142
void incrementEpoch()
increment the epoch for this subdir (not synced to disk).
static constexpr char EPOCH_FILENAME[]
Definition: FileMgr.h:387
Definition: Epoch.h:30
static size_t byte_size()
Definition: Epoch.h:63
static constexpr char WRAPPER_FILE_NAME[]
FILE * open(int fileId)
Opens/creates the file with the given id; returns NULL on error.
Definition: File.cpp:107
#define CHECK(condition)
Definition: Logger.h:289
size_t file_size(const int fd)
Definition: heavyai_fs.cpp:33

+ Here is the call graph for this function:

File_Namespace::TableFileMgr::~TableFileMgr ( )
inline

Definition at line 95 of file CachingFileMgr.h.

References File_Namespace::close(), and epoch_file_.

95 { close(epoch_file_); }
void close(FILE *f)
Closes the file pointed to by the FILE pointer.
Definition: File.cpp:128

+ Here is the call graph for this function:

Member Function Documentation

void File_Namespace::TableFileMgr::deleteWrapperFile ( ) const

Deletes only the wrapper file on disk.

Definition at line 825 of file CachingFileMgr.cpp.

References table_mutex_, and wrapper_file_path_.

825  {
827  bf::remove_all(wrapper_file_path_);
828 }
heavyai::shared_mutex table_mutex_
std::unique_lock< T > unique_lock
int32_t File_Namespace::TableFileMgr::getEpoch ( ) const

Returns the current epoch (locked)

Definition at line 790 of file CachingFileMgr.cpp.

References Epoch::ceiling(), epoch_, and table_mutex_.

790  {
792  return static_cast<int32_t>(epoch_.ceiling());
793 }
int32_t ceiling() const
Definition: Epoch.h:44
std::shared_lock< T > shared_lock
heavyai::shared_mutex table_mutex_

+ Here is the call graph for this function:

size_t File_Namespace::TableFileMgr::getReservedSpace ( ) const

Returns the disk space used (in bytes) for the subdir.

Definition at line 814 of file CachingFileMgr.cpp.

References heavyai::file_size(), table_mutex_, and table_path_.

814  {
816  size_t space = 0;
817  for (const auto& file : bf::recursive_directory_iterator(table_path_)) {
818  if (bf::is_regular_file(file.path())) {
819  space += bf::file_size(file.path());
820  }
821  }
822  return space;
823 }
std::shared_lock< T > shared_lock
heavyai::shared_mutex table_mutex_
size_t file_size(const int fd)
Definition: heavyai_fs.cpp:33

+ Here is the call graph for this function:

bool File_Namespace::TableFileMgr::hasWrapperFile ( ) const

Checks if data wrapper file has been written to disk/cached.

Definition at line 840 of file CachingFileMgr.cpp.

References table_mutex_, and wrapper_file_path_.

840  {
842  return bf::exists(wrapper_file_path_);
843 }
std::shared_lock< T > shared_lock
heavyai::shared_mutex table_mutex_
void File_Namespace::TableFileMgr::incrementEpoch ( )

increment the epoch for this subdir (not synced to disk).

Definition at line 781 of file CachingFileMgr.cpp.

References Epoch::ceiling(), CHECK, epoch_, Epoch::increment(), is_checkpointed_, Epoch::max_allowable_epoch(), and table_mutex_.

Referenced by TableFileMgr().

781  {
783  epoch_.increment();
784  is_checkpointed_ = false;
786  << "Epoch greater than maximum allowed value (" << epoch_.ceiling() << " > "
787  << Epoch::max_allowable_epoch() << ").";
788 }
int32_t ceiling() const
Definition: Epoch.h:44
static int64_t max_allowable_epoch()
Definition: Epoch.h:69
heavyai::shared_mutex table_mutex_
int32_t increment()
Definition: Epoch.h:54
std::unique_lock< T > unique_lock
#define CHECK(condition)
Definition: Logger.h:289

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::TableFileMgr::removeDiskContent ( ) const

Removes all disk data for the subdir.

Definition at line 809 of file CachingFileMgr.cpp.

References table_mutex_, and table_path_.

809  {
811  bf::remove_all(table_path_);
812 }
heavyai::shared_mutex table_mutex_
std::unique_lock< T > unique_lock
void File_Namespace::TableFileMgr::writeAndSyncEpochToDisk ( )

Write and flush the epoch to the epoch file on disk.

Definition at line 795 of file CachingFileMgr.cpp.

References Epoch::byte_size(), CHECK, epoch_, epoch_file_, heavyai::fsync(), is_checkpointed_, Epoch::storage_ptr(), table_mutex_, and File_Namespace::write().

Referenced by TableFileMgr().

795  {
798  int32_t status = fflush(epoch_file_);
799  CHECK(status == 0) << "Could not flush epoch file to disk";
800 #ifdef __APPLE__
801  status = fcntl(fileno(epoch_file_), 51);
802 #else
803  status = heavyai::fsync(fileno(epoch_file_));
804 #endif
805  CHECK(status == 0) << "Could not sync epoch file to disk";
806  is_checkpointed_ = true;
807 }
int8_t * storage_ptr()
Definition: Epoch.h:61
size_t write(FILE *f, const size_t offset, const size_t size, const int8_t *buf)
Writes the specified number of bytes to the offset position in file f from buf.
Definition: File.cpp:150
heavyai::shared_mutex table_mutex_
std::unique_lock< T > unique_lock
static size_t byte_size()
Definition: Epoch.h:63
int fsync(int fd)
Definition: heavyai_fs.cpp:62
#define CHECK(condition)
Definition: Logger.h:289

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

void File_Namespace::TableFileMgr::writeWrapperFile ( const std::string &  doc) const

Writes wrapper file to disk.

Definition at line 830 of file CachingFileMgr.cpp.

References table_mutex_, and wrapper_file_path_.

830  {
832  std::ofstream ofs(wrapper_file_path_);
833  if (!ofs) {
834  throw std::runtime_error{"Error trying to create file \"" + wrapper_file_path_ +
835  "\". The error was: " + std::strerror(errno)};
836  }
837  ofs << doc;
838 }
heavyai::shared_mutex table_mutex_
std::unique_lock< T > unique_lock

Member Data Documentation

Epoch File_Namespace::TableFileMgr::epoch_
private

Definition at line 141 of file CachingFileMgr.h.

Referenced by getEpoch(), incrementEpoch(), TableFileMgr(), and writeAndSyncEpochToDisk().

FILE* File_Namespace::TableFileMgr::epoch_file_ = nullptr
private

Definition at line 143 of file CachingFileMgr.h.

Referenced by TableFileMgr(), writeAndSyncEpochToDisk(), and ~TableFileMgr().

std::string File_Namespace::TableFileMgr::epoch_file_path_
private

Definition at line 139 of file CachingFileMgr.h.

Referenced by TableFileMgr().

bool File_Namespace::TableFileMgr::is_checkpointed_ = true
private

Definition at line 142 of file CachingFileMgr.h.

Referenced by incrementEpoch(), and writeAndSyncEpochToDisk().

heavyai::shared_mutex File_Namespace::TableFileMgr::table_mutex_
mutableprivate
std::string File_Namespace::TableFileMgr::table_path_
private

Definition at line 138 of file CachingFileMgr.h.

Referenced by getReservedSpace(), removeDiskContent(), and TableFileMgr().

std::string File_Namespace::TableFileMgr::wrapper_file_path_
private

Definition at line 140 of file CachingFileMgr.h.

Referenced by deleteWrapperFile(), hasWrapperFile(), and writeWrapperFile().


The documentation for this class was generated from the following files: