OmniSciDB  085a039ca4
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
foreign_storage::CsvDataWrapper Class Reference

#include <CsvDataWrapper.h>

+ Inheritance diagram for foreign_storage::CsvDataWrapper:
+ Collaboration diagram for foreign_storage::CsvDataWrapper:

Public Member Functions

 CsvDataWrapper ()
 CsvDataWrapper (const int db_id, const ForeignTable *foreign_table)
 CsvDataWrapper (const int db_id, const ForeignTable *foreign_table, const UserMapping *user_mapping, const bool disable_cache=false)
void validateTableOptions (const ForeignTable *foreign_table) const override
const std::set
< std::string_view > & 
getSupportedTableOptions () const override
- Public Member Functions inherited from foreign_storage::AbstractTextFileDataWrapper
 AbstractTextFileDataWrapper ()
 AbstractTextFileDataWrapper (const int db_id, const ForeignTable *foreign_table)
 AbstractTextFileDataWrapper (const int db_id, const ForeignTable *foreign_table, const UserMapping *user_mapping, const bool disable_cache)
void populateChunkMetadata (ChunkMetadataVector &chunk_metadata_vector) override
void populateChunkBuffers (const ChunkToBufferMap &required_buffers, const ChunkToBufferMap &optional_buffers, AbstractBuffer *delete_buffer) override
std::string getSerializedDataWrapper () const override
void restoreDataWrapperInternals (const std::string &file_path, const ChunkMetadataVector &chunk_metadata) override
bool isRestored () const override
ParallelismLevel getCachedParallelismLevel () const override
ParallelismLevel getNonCachedParallelismLevel () const override
void createRenderGroupAnalyzers () override
 Create RenderGroupAnalyzers for poly columns. More...
- Public Member Functions inherited from foreign_storage::AbstractFileStorageDataWrapper
 AbstractFileStorageDataWrapper ()
void validateServerOptions (const ForeignServer *foreign_server) const override
void validateUserMappingOptions (const UserMapping *user_mapping, const ForeignServer *foreign_server) const override
const std::set
< std::string_view > & 
getSupportedUserMappingOptions () const override
- Public Member Functions inherited from foreign_storage::ForeignDataWrapper
 ForeignDataWrapper ()=default
virtual ~ForeignDataWrapper ()=default
virtual const std::set
< std::string > 
getAlterableTableOptions () const
virtual void validateSchema (const std::list< ColumnDescriptor > &columns) const

Static Public Member Functions

static bool validateAndGetIsS3Select (const ForeignTable *foreign_table)

Protected Member Functions

const TextFileBufferParsergetFileBufferParser () const override

Private Member Functions

std::set< std::string_view > getAllCsvTableOptions () const

Static Private Attributes

static const std::set
< std::string_view > 
static const CsvFileBufferParser csv_file_buffer_parser_ {}

Additional Inherited Members

- Public Types inherited from foreign_storage::ForeignDataWrapper
- Static Public Attributes inherited from foreign_storage::AbstractFileStorageDataWrapper
static const std::string STORAGE_TYPE_KEY = "STORAGE_TYPE"
static const std::string BASE_PATH_KEY = "BASE_PATH"
static const std::string FILE_PATH_KEY = "FILE_PATH"
static const std::string REGEX_PATH_FILTER_KEY = "REGEX_PATH_FILTER"
static const std::string LOCAL_FILE_STORAGE_TYPE = "LOCAL_FILE"
static const std::string S3_STORAGE_TYPE = "AWS_S3"
static const std::string FILE_SORT_ORDER_BY_KEY = shared::FILE_SORT_ORDER_BY_KEY
static const std::string FILE_SORT_REGEX_KEY = shared::FILE_SORT_REGEX_KEY
static const std::array
< std::string, 1 > 
- Static Protected Member Functions inherited from foreign_storage::AbstractFileStorageDataWrapper
static std::string getFullFilePath (const ForeignTable *foreign_table)
 Returns the path to the source file/dir of the table. Depending on options this may result from a concatenation of server and table path options. More...

Detailed Description

Definition at line 23 of file CsvDataWrapper.h.

Constructor & Destructor Documentation

foreign_storage::CsvDataWrapper::CsvDataWrapper ( )
foreign_storage::CsvDataWrapper::CsvDataWrapper ( const int  db_id,
const ForeignTable foreign_table 

Definition at line 25 of file CsvDataWrapper.cpp.

foreign_storage::CsvDataWrapper::CsvDataWrapper ( const int  db_id,
const ForeignTable foreign_table,
const UserMapping user_mapping,
const bool  disable_cache = false 

Definition at line 28 of file CsvDataWrapper.cpp.

32  : AbstractTextFileDataWrapper(db_id, foreign_table, user_mapping, disable_cache) {}

Member Function Documentation

std::set< std::string_view > foreign_storage::CsvDataWrapper::getAllCsvTableOptions ( ) const

Definition at line 45 of file CsvDataWrapper.cpp.

References csv_table_options_, and foreign_storage::AbstractFileStorageDataWrapper::getSupportedTableOptions().

Referenced by getSupportedTableOptions().

45  {
46  std::set<std::string_view> supported_table_options(
49  supported_table_options.insert(csv_table_options_.begin(), csv_table_options_.end());
50  return supported_table_options;
51 }
static const std::set< std::string_view > csv_table_options_
const std::set< std::string_view > & getSupportedTableOptions() const override

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

const TextFileBufferParser & foreign_storage::CsvDataWrapper::getFileBufferParser ( ) const

Implements foreign_storage::AbstractTextFileDataWrapper.

Definition at line 53 of file CsvDataWrapper.cpp.

References csv_file_buffer_parser_.

53  {
55 }
static const CsvFileBufferParser csv_file_buffer_parser_
const std::set< std::string_view > & foreign_storage::CsvDataWrapper::getSupportedTableOptions ( ) const

Gets the set of supported table options for the data wrapper.

Reimplemented from foreign_storage::AbstractFileStorageDataWrapper.

Definition at line 40 of file CsvDataWrapper.cpp.

References getAllCsvTableOptions().

40  {
41  static const auto supported_table_options = getAllCsvTableOptions();
42  return supported_table_options;
43 }
std::set< std::string_view > getAllCsvTableOptions() const

+ Here is the call graph for this function:

bool foreign_storage::CsvDataWrapper::validateAndGetIsS3Select ( const ForeignTable foreign_table)

Definition at line 57 of file CsvDataWrapper.cpp.

References foreign_storage::ForeignTable::foreign_server, foreign_storage::OptionsContainer::options, foreign_storage::AbstractFileStorageDataWrapper::S3_STORAGE_TYPE, and foreign_storage::AbstractFileStorageDataWrapper::STORAGE_TYPE_KEY.

Referenced by foreign_storage::ForeignDataWrapperFactory::create(), foreign_storage::ForeignDataWrapperFactory::createForValidation(), and validateTableOptions().

57  {
58  static constexpr const char* S3_DIRECT = "S3_DIRECT";
59  static constexpr const char* S3_SELECT = "S3_SELECT";
60  static constexpr const char* S3_ACCESS_TYPE = "S3_ACCESS_TYPE";
61  auto access_type = foreign_table->options.find(S3_ACCESS_TYPE);
63  if (access_type != foreign_table->options.end()) {
64  auto& server_options = foreign_table->foreign_server->options;
65  if (server_options.find(AbstractFileStorageDataWrapper::STORAGE_TYPE_KEY)->second !=
67  throw std::runtime_error{
68  "The \"" + std::string{S3_ACCESS_TYPE} +
69  "\" option is only valid for foreign tables using servers with \"" +
70  AbstractFileStorageDataWrapper::STORAGE_TYPE_KEY + "\" option value of \"" +
72  }
73  if (access_type->second != S3_DIRECT && access_type->second != S3_SELECT) {
74  throw std::runtime_error{
75  "Invalid value provided for the \"" + std::string{S3_ACCESS_TYPE} +
76  "\" option. Value must be one of the following: " + S3_DIRECT + ", " +
77  S3_SELECT + "."};
78  }
79  return (access_type->second == S3_SELECT);
80  } else {
81  return false;
82  }
83 }

+ Here is the caller graph for this function:

void foreign_storage::CsvDataWrapper::validateTableOptions ( const ForeignTable foreign_table) const

Checks that the options for the given foreign table object are valid.

foreign_table- foreign table object containing options to be validated

Reimplemented from foreign_storage::AbstractFileStorageDataWrapper.

Definition at line 34 of file CsvDataWrapper.cpp.

References csv_file_buffer_parser_, foreign_storage::CsvFileBufferParser::validateAndGetCopyParams(), validateAndGetIsS3Select(), and foreign_storage::AbstractFileStorageDataWrapper::validateTableOptions().

34  {
37  validateAndGetIsS3Select(foreign_table);
38 }
static bool validateAndGetIsS3Select(const ForeignTable *foreign_table)
void validateTableOptions(const ForeignTable *foreign_table) const override
import_export::CopyParams validateAndGetCopyParams(const ForeignTable *foreign_table) const override
static const CsvFileBufferParser csv_file_buffer_parser_

+ Here is the call graph for this function:

Member Data Documentation

const CsvFileBufferParser foreign_storage::CsvDataWrapper::csv_file_buffer_parser_ {}

Definition at line 46 of file CsvDataWrapper.h.

Referenced by getFileBufferParser(), and validateTableOptions().

const std::set< std::string_view > foreign_storage::CsvDataWrapper::csv_table_options_
Initial value:

Definition at line 45 of file CsvDataWrapper.h.

Referenced by getAllCsvTableOptions().

The documentation for this class was generated from the following files: