OmniSciDB  95562058bd
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
foreign_storage::CsvReader Class Referenceabstract

#include <CsvReader.h>

+ Inheritance diagram for foreign_storage::CsvReader:
+ Collaboration diagram for foreign_storage::CsvReader:

Public Member Functions

 CsvReader (const std::string &file_path, const import_export::CopyParams &copy_params)
 
virtual ~CsvReader ()=default
 
virtual size_t read (void *buffer, size_t max_size)=0
 
virtual bool isScanFinished ()=0
 
virtual size_t readRegion (void *buffer, size_t offset, size_t size)=0
 
virtual size_t getRemainingSize ()=0
 
virtual bool isRemainingSizeKnown ()=0
 
virtual void checkForMoreRows (size_t file_offset, const ForeignServer *server_options=nullptr, const UserMapping *user_mapping=nullptr)
 
virtual void serialize (rapidjson::Value &value, rapidjson::Document::AllocatorType &allocator) const =0
 

Protected Attributes

import_export::CopyParams copy_params_
 
std::string file_path_
 

Detailed Description

Definition at line 34 of file CsvReader.h.

Constructor & Destructor Documentation

foreign_storage::CsvReader::CsvReader ( const std::string &  file_path,
const import_export::CopyParams copy_params 
)
inline

Definition at line 36 of file CsvReader.h.

37  : copy_params_(copy_params), file_path_(file_path){};
import_export::CopyParams copy_params_
Definition: CsvReader.h:102
virtual foreign_storage::CsvReader::~CsvReader ( )
virtualdefault

Member Function Documentation

virtual void foreign_storage::CsvReader::checkForMoreRows ( size_t  file_offset,
const ForeignServer server_options = nullptr,
const UserMapping *  user_mapping = nullptr 
)
inlinevirtual

Rescan the target files Throws an exception if the rescan fails (ie files are not in a valid appended state or not supported)

Parameters
file_offset- where to resume the scan from (end of the last row) as not all of the bytes may have been consumed by the upstream compoennet
server_options- only needed for S3 backed CSV
user_mapping- only needed for S3 backed CSV

Reimplemented in foreign_storage::LocalMultiFileReader, foreign_storage::CompressedFileReader, foreign_storage::SingleFileReader, and foreign_storage::MultiS3Reader.

Definition at line 84 of file CsvReader.h.

86  {
87  throw std::runtime_error{"APPEND mode not yet supported for this table."};
88  }
virtual size_t foreign_storage::CsvReader::getRemainingSize ( )
pure virtual
virtual bool foreign_storage::CsvReader::isRemainingSizeKnown ( )
pure virtual
virtual bool foreign_storage::CsvReader::isScanFinished ( )
pure virtual
Returns
true if the entire CSV has been read

Implemented in foreign_storage::MultiFileReader, foreign_storage::CompressedFileReader, foreign_storage::SingleFileReader, and foreign_storage::CsvReaderS3.

Referenced by foreign_storage::dispatch_metadata_scan_requests(), and import_export::delimited_parser::find_row_end_pos().

+ Here is the caller graph for this function:

virtual size_t foreign_storage::CsvReader::read ( void *  buffer,
size_t  max_size 
)
pure virtual

Read up to max_size bytes from archive into buffer starting starting from the end of the last read

Parameters
buffer- buffer to load into
max_size- maximum number of bytes to read into the buffer
Returns
number of bytes actually read

Implemented in foreign_storage::MultiFileReader, foreign_storage::CompressedFileReader, foreign_storage::SingleFileReader, and foreign_storage::CsvReaderS3.

Referenced by foreign_storage::dispatch_metadata_scan_requests(), and import_export::delimited_parser::find_row_end_pos().

+ Here is the caller graph for this function:

virtual size_t foreign_storage::CsvReader::readRegion ( void *  buffer,
size_t  offset,
size_t  size 
)
pure virtual

Read up to max_size bytes from archive, starting at given offset isScanFinished() must return true to use readRegion

Parameters
buffer- buffer to load into
offset- starting point into the archive to read
size- maximum number of bytes to read into the buffer
Returns
number of bytes actually read

Implemented in foreign_storage::MultiFileReader, foreign_storage::CompressedFileReader, foreign_storage::SingleFileReader, and foreign_storage::CsvReaderS3.

Referenced by foreign_storage::parse_file_regions().

+ Here is the caller graph for this function:

virtual void foreign_storage::CsvReader::serialize ( rapidjson::Value &  value,
rapidjson::Document::AllocatorType &  allocator 
) const
pure virtual

Serialize internal state to given json object This Json will later be used to restore the reader state through a constructor must be called when isScanFinished() is true

Parameters
value- json object to store needed state to this function can store any needed data or none
allocator- allocator to use for json contruction

Implemented in foreign_storage::MultiFileReader, foreign_storage::CompressedFileReader, foreign_storage::SingleFileReader, foreign_storage::CsvReaderS3, and foreign_storage::MultiS3Reader.

Member Data Documentation


The documentation for this class was generated from the following file: