OmniSciDB  8fa3bf436f
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
foreign_storage::ForeignDataWrapper Class Referenceabstract

#include <ForeignDataWrapper.h>

+ Inheritance diagram for foreign_storage::ForeignDataWrapper:

Public Types

enum  ParallelismLevel { NONE, INTRA_FRAGMENT, INTER_FRAGMENT }
 

Public Member Functions

 ForeignDataWrapper ()=default
 
virtual ~ForeignDataWrapper ()=default
 
virtual void populateChunkMetadata (ChunkMetadataVector &chunk_metadata_vector)=0
 
virtual void populateChunkBuffers (const ChunkToBufferMap &required_buffers, const ChunkToBufferMap &optional_buffers)=0
 
virtual void serializeDataWrapperInternals (const std::string &file_path) const =0
 
virtual void restoreDataWrapperInternals (const std::string &file_path, const ChunkMetadataVector &chunk_metadata)=0
 
virtual bool isRestored () const =0
 
virtual void validateServerOptions (const ForeignServer *foreign_server) const =0
 
virtual void validateTableOptions (const ForeignTable *foreign_table) const =0
 
virtual const std::set
< std::string_view > & 
getSupportedTableOptions () const =0
 
virtual void validateUserMappingOptions (const UserMapping *user_mapping, const ForeignServer *foreign_server) const =0
 
virtual const std::set
< std::string_view > & 
getSupportedUserMappingOptions () const =0
 
virtual void validateSchema (const std::list< ColumnDescriptor > &columns) const
 
virtual ParallelismLevel getCachedParallelismLevel () const
 
virtual ParallelismLevel getNonCachedParallelismLevel () const
 

Detailed Description

Definition at line 30 of file ForeignDataWrapper.h.

Member Enumeration Documentation

ParallelismLevel describes the desired level of parallelism of the data wrapper. This level controls which optional_buffers are passed to populateChunkBuffers with the following behaviour:

NONE - no additional optional buffers are passed in

INTRA_FRAGMENT - additional optional buffers which are in the same fragment as the required buffers

INTER_FRAGMENT - additional optional buffers which may be in different fragments than those of the required buffers

Note, the optional buffers are passed in with the intention of allowing the data wrapper to employ parallelism in retrieving them. Each subsequent level allows for a greater degree of parallelism but does not have to be supported.

Enumerator
NONE 
INTRA_FRAGMENT 
INTER_FRAGMENT 

Definition at line 125 of file ForeignDataWrapper.h.

Constructor & Destructor Documentation

foreign_storage::ForeignDataWrapper::ForeignDataWrapper ( )
default
virtual foreign_storage::ForeignDataWrapper::~ForeignDataWrapper ( )
virtualdefault

Member Function Documentation

virtual ParallelismLevel foreign_storage::ForeignDataWrapper::getCachedParallelismLevel ( ) const
inlinevirtual

Gets the desired level of parallelism for the data wrapper when a cache is in use. This affects the optional buffers that the data wrapper is made aware of during data requests.

Reimplemented in foreign_storage::CsvDataWrapper, and foreign_storage::ParquetDataWrapper.

Definition at line 132 of file ForeignDataWrapper.h.

References NONE.

virtual ParallelismLevel foreign_storage::ForeignDataWrapper::getNonCachedParallelismLevel ( ) const
inlinevirtual

Gets the desired level of parallelism for the data wrapper when no cache is in use. This affects the optional buffers that the data wrapper is made aware of during data requests.

Reimplemented in foreign_storage::CsvDataWrapper, and foreign_storage::ParquetDataWrapper.

Definition at line 139 of file ForeignDataWrapper.h.

References NONE.

virtual const std::set<std::string_view>& foreign_storage::ForeignDataWrapper::getSupportedTableOptions ( ) const
pure virtual

Gets the set of supported table options for the data wrapper.

Implemented in foreign_storage::CsvDataWrapper, and foreign_storage::AbstractFileStorageDataWrapper.

virtual const std::set<std::string_view>& foreign_storage::ForeignDataWrapper::getSupportedUserMappingOptions ( ) const
pure virtual

Gets the set of supported user mapping options for the data wrapper.

Implemented in foreign_storage::AbstractFileStorageDataWrapper.

virtual bool foreign_storage::ForeignDataWrapper::isRestored ( ) const
pure virtual
virtual void foreign_storage::ForeignDataWrapper::populateChunkBuffers ( const ChunkToBufferMap required_buffers,
const ChunkToBufferMap optional_buffers 
)
pure virtual

Populates given chunk buffers identified by chunk keys. All provided chunk buffers are expected to be for the same fragment.

Parameters
required_buffers- chunk buffers that must always be populated
optional_buffers- chunk buffers that can be optionally populated, if the data wrapper has to scan through chunk data anyways (typically for row wise data formats)

Implemented in foreign_storage::ParquetDataWrapper, and foreign_storage::CsvDataWrapper.

Referenced by foreign_storage::CachingForeignStorageMgr::populateChunkBuffersSafely().

+ Here is the caller graph for this function:

virtual void foreign_storage::ForeignDataWrapper::populateChunkMetadata ( ChunkMetadataVector chunk_metadata_vector)
pure virtual

Populates given chunk metadata vector with metadata for all chunks in related foreign table.

Parameters
chunk_metadata_vector- vector that will be populated with chunk metadata

Implemented in foreign_storage::ParquetDataWrapper, and foreign_storage::CsvDataWrapper.

virtual void foreign_storage::ForeignDataWrapper::restoreDataWrapperInternals ( const std::string &  file_path,
const ChunkMetadataVector chunk_metadata 
)
pure virtual

Restore internal state of datawrapper

Parameters
file_path- location of file created by serializeMetadata
chunk_metadata_vector- vector of chunk metadata recovered from disk

Implemented in foreign_storage::CsvDataWrapper, and foreign_storage::ParquetDataWrapper.

virtual void foreign_storage::ForeignDataWrapper::serializeDataWrapperInternals ( const std::string &  file_path) const
pure virtual

Serialize internal state of wrapper into file at given path if implemented

Parameters
file_path- location to save file to

Implemented in foreign_storage::CsvDataWrapper, and foreign_storage::ParquetDataWrapper.

virtual void foreign_storage::ForeignDataWrapper::validateSchema ( const std::list< ColumnDescriptor > &  columns) const
inlinevirtual

Verifies the schema is supported by this foreign table

Parameters
columns- column descriptors for this table

Definition at line 106 of file ForeignDataWrapper.h.

106 {};
virtual void foreign_storage::ForeignDataWrapper::validateServerOptions ( const ForeignServer foreign_server) const
pure virtual

Checks that the options for the given foreign server object are valid.

Parameters
foreign_server- foreign server object containing options to be validated

Implemented in foreign_storage::AbstractFileStorageDataWrapper.

Referenced by foreign_storage::ForeignServer::validateStorageParameters().

+ Here is the caller graph for this function:

virtual void foreign_storage::ForeignDataWrapper::validateTableOptions ( const ForeignTable foreign_table) const
pure virtual

Checks that the options for the given foreign table object are valid.

Parameters
foreign_table- foreign table object containing options to be validated

Implemented in foreign_storage::CsvDataWrapper, and foreign_storage::AbstractFileStorageDataWrapper.

virtual void foreign_storage::ForeignDataWrapper::validateUserMappingOptions ( const UserMapping *  user_mapping,
const ForeignServer foreign_server 
) const
pure virtual

Checks that the options for the given user mapping object are valid.

Parameters
user_mapping- user mapping object containing options to be validated

Implemented in foreign_storage::AbstractFileStorageDataWrapper.


The documentation for this class was generated from the following file: