FixedLengthArrayNoneEncoder Class Reference

#include <FixedLengthArrayNoneEncoder.h>

Inheritance diagram for FixedLengthArrayNoneEncoder:

Collaboration diagram for FixedLengthArrayNoneEncoder:

Public Member Functions
	FixedLengthArrayNoneEncoder (AbstractBuffer *buffer, size_t as)
size_t	getNumElemsForBytesInsertData (const std::vector< ArrayDatum > *srcData, const int start_idx, const size_t numAppendElems, const size_t byteLimit, const bool replicating=false)
ChunkMetadata	appendData (int8_t *&srcData, const size_t numAppendElems, const SQLTypeInfo &, const bool replicating=false) override
ChunkMetadata	appendData (const std::vector< ArrayDatum > *srcData, const int start_idx, const size_t numAppendElems, const bool replicating=false)
void	getMetadata (ChunkMetadata &chunkMetadata) override
ChunkMetadata	getMetadata (const SQLTypeInfo &ti) override
void	updateStats (const int64_t, const bool) override
void	updateStats (const double, const bool) override
void	reduceStats (const Encoder &) override
void	writeMetadata (FILE *f) override
void	readMetadata (FILE *f) override
void	copyMetadata (const Encoder *copyFromEncoder) override
void	updateMetadata (int8_t *array)
Public Member Functions inherited from Encoder
	Encoder (Data_Namespace::AbstractBuffer *buffer)
virtual	~Encoder ()
virtual bool	resetChunkStats (const ChunkStats &)
	: Reset chunk level stats (min, max, nulls) using new values from the argument. More...
size_t	getNumElems () const
void	setNumElems (const size_t num_elems)

Static Public Member Functions
static bool	is_null (const SQLTypeInfo &type, int8_t *array)
Static Public Member Functions inherited from Encoder
static Encoder *	Create (Data_Namespace::AbstractBuffer *buffer, const SQLTypeInfo sqlType)

Public Attributes
Datum	elem_min
Datum	elem_max
bool	has_nulls
bool	initialized

Private Member Functions
bool	is_null (int8_t *array)
void	update_elem_stats (const ArrayDatum &array)

Private Attributes
std::mutex	EncoderMutex_
size_t	array_size

Additional Inherited Members
Protected Attributes inherited from Encoder
size_t	num_elems_
Data_Namespace::AbstractBuffer *	buffer_
DecimalOverflowValidator	decimal_overflow_validator_
DateDaysOverflowValidator	date_days_overflow_validator_

Detailed Description

Definition at line 41 of file FixedLengthArrayNoneEncoder.h.

Constructor & Destructor Documentation

FixedLengthArrayNoneEncoder::FixedLengthArrayNoneEncoder ( AbstractBuffer *  buffer, size_t  as  )

inline

Definition at line 43 of file FixedLengthArrayNoneEncoder.h.

      : Encoder(buffer), has_nulls(false), initialized(false), array_size(as) {}

Member Function Documentation

ChunkMetadata FixedLengthArrayNoneEncoder::appendData ( int8_t *&  srcData, const size_t  numAppendElems, const SQLTypeInfo &  , const bool  replicating = false  )

inlineoverridevirtual

Implements Encoder.

Definition at line 58 of file FixedLengthArrayNoneEncoder.h.

References CHECK().

Referenced by Chunk_NS::Chunk::appendData().

                                                                    {
    CHECK(false);  // should never be called for arrays
    return ChunkMetadata{};
  }

Here is the call graph for this function:

Here is the caller graph for this function:

ChunkMetadata FixedLengthArrayNoneEncoder::appendData ( const std::vector< ArrayDatum > *  srcData, const int  start_idx, const size_t  numAppendElems, const bool  replicating = false  )

inline

Definition at line 66 of file FixedLengthArrayNoneEncoder.h.

References Data_Namespace::AbstractBuffer::append(), array_size, Encoder::buffer_, CHECK_EQ, getMetadata(), Data_Namespace::AbstractBuffer::isDirty(), Encoder::num_elems_, Data_Namespace::AbstractBuffer::reserve(), Data_Namespace::AbstractBuffer::setDirty(), and update_elem_stats().

                                                           {
    size_t data_size = array_size * numAppendElems;
    buffer_->reserve(data_size);

    for (size_t i = start_idx; i < start_idx + numAppendElems; i++) {
      size_t len = (*srcData)[replicating ? 0 : i].length;
      // Length of the appended array should be equal to the fixed length,
      // all others should have been discarded, assert if something slips through
      CHECK_EQ(len, array_size);
      // NULL arrays have been filled with subtype's NULL sentinels,
      // should be appended as regular data, same size
      buffer_->append((*srcData)[replicating ? 0 : i].pointer, len);

      // keep Chunk statistics with array elements
      update_elem_stats((*srcData)[replicating ? 0 : i]);
    }
    // make sure buffer_ is flushed even if no new data is appended to it
    // (e.g. empty strings) because the metadata needs to be flushed.
    if (!buffer_->isDirty()) {
      buffer_->setDirty();
    }

    num_elems_ += numAppendElems;
    ChunkMetadata chunkMetadata;
    getMetadata(chunkMetadata);
    return chunkMetadata;
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::copyMetadata ( const Encoder *  copyFromEncoder )

inlineoverridevirtual

Implements Encoder.

Definition at line 132 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, Encoder::getNumElems(), has_nulls, initialized, and Encoder::num_elems_.

                                                             {
    num_elems_ = copyFromEncoder->getNumElems();
    auto array_encoder =
        dynamic_cast<const FixedLengthArrayNoneEncoder*>(copyFromEncoder);
    elem_min = array_encoder->elem_min;
    elem_max = array_encoder->elem_max;
    has_nulls = array_encoder->has_nulls;
    initialized = array_encoder->initialized;
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::getMetadata ( ChunkMetadata &  chunkMetadata )

inlineoverridevirtual

Reimplemented from Encoder.

Definition at line 97 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, ChunkMetadata::fillChunkStats(), Encoder::getMetadata(), and has_nulls.

Referenced by appendData().

                                                          {
    Encoder::getMetadata(chunkMetadata);  // call on parent class
    chunkMetadata.fillChunkStats(elem_min, elem_max, has_nulls);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

ChunkMetadata FixedLengthArrayNoneEncoder::getMetadata ( const SQLTypeInfo &  ti )

inlineoverridevirtual

Implements Encoder.

Definition at line 103 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, and has_nulls.

                                                            {
    ChunkMetadata chunk_metadata{ti, 0, 0, ChunkStats{elem_min, elem_max, has_nulls}};
    return chunk_metadata;
  }

size_t FixedLengthArrayNoneEncoder::getNumElemsForBytesInsertData ( const std::vector< ArrayDatum > *  srcData, const int  start_idx, const size_t  numAppendElems, const size_t  byteLimit, const bool  replicating = false  )

inline

Definition at line 46 of file FixedLengthArrayNoneEncoder.h.

References array_size.

Referenced by Chunk_NS::Chunk::getNumElemsForBytesInsertData().

                                                                       {
    size_t dataSize = numAppendElems * array_size;
    if (dataSize > byteLimit) {
      dataSize = byteLimit;
    }
    return dataSize / array_size;
  }

Here is the caller graph for this function:

static bool FixedLengthArrayNoneEncoder::is_null ( const SQLTypeInfo &  type, int8_t *  array  )

inlinestatic

Definition at line 146 of file FixedLengthArrayNoneEncoder.h.

References CHECK_EQ, SQLTypeInfoCore< TYPE_FACET_PACK >::get_compression(), SQLTypeInfoCore< TYPE_FACET_PACK >::get_notnull(), SQLTypeInfoCore< TYPE_FACET_PACK >::get_subtype(), kBIGINT, kBOOLEAN, kCHAR, kDATE, kDECIMAL, kDOUBLE, kENCODING_DICT, kFLOAT, kINT, kNUMERIC, kSMALLINT, kTEXT, kTIME, kTIMESTAMP, kTINYINT, kVARCHAR, NULL_ARRAY_BIGINT, NULL_ARRAY_BOOLEAN, NULL_ARRAY_DOUBLE, NULL_ARRAY_FLOAT, NULL_ARRAY_INT, NULL_ARRAY_SMALLINT, NULL_ARRAY_TINYINT, and UNREACHABLE.

Referenced by Fragmenter_Namespace::FixedLenArrayChunkConverter::convertToColumnarFormat(), and updateMetadata().

                                                              {
    if (type.get_notnull()) {
      return false;
    }
    switch (type.get_subtype()) {
      case kBOOLEAN: {
        const bool* bool_array = (bool*)array;
        return ((int8_t)bool_array[0] == NULL_ARRAY_BOOLEAN);
      }
      case kINT: {
        const int32_t* int_array = (int32_t*)array;
        return (int_array[0] == NULL_ARRAY_INT);
      }
      case kSMALLINT: {
        const int16_t* smallint_array = (int16_t*)array;
        return (smallint_array[0] == NULL_ARRAY_SMALLINT);
      }
      case kTINYINT: {
        const int8_t* tinyint_array = (int8_t*)array;
        return (tinyint_array[0] == NULL_ARRAY_TINYINT);
      }
      case kBIGINT:
      case kNUMERIC:
      case kDECIMAL: {
        const int64_t* bigint_array = (int64_t*)array;
        return (bigint_array[0] == NULL_ARRAY_BIGINT);
      }
      case kFLOAT: {
        const float* flt_array = (float*)array;
        return (flt_array[0] == NULL_ARRAY_FLOAT);
      }
      case kDOUBLE: {
        const double* dbl_array = (double*)array;
        return (dbl_array[0] == NULL_ARRAY_DOUBLE);
      }
      case kTIME:
      case kTIMESTAMP:
      case kDATE: {
        const int64_t* tm_array = reinterpret_cast<int64_t*>(array);
        return (tm_array[0] == NULL_ARRAY_BIGINT);
      }
      case kCHAR:
      case kVARCHAR:
      case kTEXT: {
        CHECK_EQ(type.get_compression(), kENCODING_DICT);
        const int32_t* int_array = (int32_t*)array;
        return (int_array[0] == NULL_ARRAY_INT);
      }
      default:
        UNREACHABLE();
    }
    return false;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool FixedLengthArrayNoneEncoder::is_null ( int8_t *  array )

inlineprivate

Definition at line 209 of file FixedLengthArrayNoneEncoder.h.

References Encoder::buffer_, is_null(), and Data_Namespace::AbstractBuffer::sql_type.

Referenced by is_null().

{ return is_null(buffer_->sql_type, array); }

Here is the call graph for this function:

Here is the caller graph for this function:

void FixedLengthArrayNoneEncoder::readMetadata ( FILE *  f )

inlineoverridevirtual

Implements Encoder.

Definition at line 123 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, has_nulls, initialized, and Encoder::num_elems_.

                                      {
    // assumes pointer is already in right place
    fread((int8_t*)&num_elems_, sizeof(size_t), 1, f);
    fread((int8_t*)&elem_min, sizeof(Datum), 1, f);
    fread((int8_t*)&elem_max, sizeof(Datum), 1, f);
    fread((int8_t*)&has_nulls, sizeof(bool), 1, f);
    fread((int8_t*)&initialized, sizeof(bool), 1, f);
  }

void FixedLengthArrayNoneEncoder::reduceStats ( const Encoder &  )

inlineoverridevirtual

Implements Encoder.

Definition at line 112 of file FixedLengthArrayNoneEncoder.h.

References CHECK().

{ CHECK(false); }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::update_elem_stats ( const ArrayDatum &  array )

inlineprivate

Definition at line 211 of file FixedLengthArrayNoneEncoder.h.

References Datum::bigintval, Datum::boolval, Encoder::buffer_, CHECK_EQ, Encoder::decimal_overflow_validator_, Datum::doubleval, elem_max, elem_min, Datum::floatval, SQLTypeInfoCore< TYPE_FACET_PACK >::get_compression(), SQLTypeInfoCore< TYPE_FACET_PACK >::get_subtype(), has_nulls, initialized, Datum::intval, kBIGINT, kBOOLEAN, kCHAR, kDATE, kDECIMAL, kDOUBLE, kENCODING_DICT, kFLOAT, kINT, kNUMERIC, kSMALLINT, kTEXT, kTIME, kTIMESTAMP, kTINYINT, kVARCHAR, NULL_BIGINT, NULL_BOOLEAN, NULL_DOUBLE, NULL_FLOAT, NULL_INT, NULL_SMALLINT, NULL_TINYINT, Datum::smallintval, Data_Namespace::AbstractBuffer::sql_type, Datum::tinyintval, UNREACHABLE, and DecimalOverflowValidator::validate().

Referenced by appendData(), and updateMetadata().

                                                  {
    if (array.is_null) {
      has_nulls = true;
    }
    switch (buffer_->sql_type.get_subtype()) {
      case kBOOLEAN: {
        if (!initialized) {
          elem_min.boolval = true;
          elem_max.boolval = false;
        }
        if (array.is_null) {
          break;
        }
        const bool* bool_array = (bool*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(bool); i++) {
          if ((int8_t)bool_array[i] == NULL_BOOLEAN) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.boolval = std::min(elem_min.boolval, bool_array[i]);
            elem_max.boolval = std::max(elem_max.boolval, bool_array[i]);
          } else {
            elem_min.boolval = bool_array[i];
            elem_max.boolval = bool_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kINT: {
        if (!initialized) {
          elem_min.intval = 1;
          elem_max.intval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int32_t* int_array = (int32_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int32_t); i++) {
          if (int_array[i] == NULL_INT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.intval = std::min(elem_min.intval, int_array[i]);
            elem_max.intval = std::max(elem_max.intval, int_array[i]);
          } else {
            elem_min.intval = int_array[i];
            elem_max.intval = int_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kSMALLINT: {
        if (!initialized) {
          elem_min.smallintval = 1;
          elem_max.smallintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int16_t* smallint_array = (int16_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int16_t); i++) {
          if (smallint_array[i] == NULL_SMALLINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.smallintval = std::min(elem_min.smallintval, smallint_array[i]);
            elem_max.smallintval = std::max(elem_max.smallintval, smallint_array[i]);
          } else {
            elem_min.smallintval = smallint_array[i];
            elem_max.smallintval = smallint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kTINYINT: {
        if (!initialized) {
          elem_min.tinyintval = 1;
          elem_max.tinyintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int8_t* tinyint_array = (int8_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int8_t); i++) {
          if (tinyint_array[i] == NULL_TINYINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.tinyintval = std::min(elem_min.tinyintval, tinyint_array[i]);
            elem_max.tinyintval = std::max(elem_max.tinyintval, tinyint_array[i]);
          } else {
            elem_min.tinyintval = tinyint_array[i];
            elem_max.tinyintval = tinyint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kBIGINT:
      case kNUMERIC:
      case kDECIMAL: {
        if (!initialized) {
          elem_min.bigintval = 1;
          elem_max.bigintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int64_t* bigint_array = (int64_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int64_t); i++) {
          if (bigint_array[i] == NULL_BIGINT) {
            has_nulls = true;
          } else if (initialized) {
            decimal_overflow_validator_.validate(bigint_array[i]);
            elem_min.bigintval = std::min(elem_min.bigintval, bigint_array[i]);
            elem_max.bigintval = std::max(elem_max.bigintval, bigint_array[i]);
          } else {
            decimal_overflow_validator_.validate(bigint_array[i]);
            elem_min.bigintval = bigint_array[i];
            elem_max.bigintval = bigint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kFLOAT: {
        if (!initialized) {
          elem_min.floatval = 1.0;
          elem_max.floatval = 0.0;
        }
        if (array.is_null) {
          break;
        }
        const float* flt_array = (float*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(float); i++) {
          if (flt_array[i] == NULL_FLOAT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.floatval = std::min(elem_min.floatval, flt_array[i]);
            elem_max.floatval = std::max(elem_max.floatval, flt_array[i]);
          } else {
            elem_min.floatval = flt_array[i];
            elem_max.floatval = flt_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kDOUBLE: {
        if (!initialized) {
          elem_min.doubleval = 1.0;
          elem_max.doubleval = 0.0;
        }
        if (array.is_null) {
          break;
        }
        const double* dbl_array = (double*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(double); i++) {
          if (dbl_array[i] == NULL_DOUBLE) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.doubleval = std::min(elem_min.doubleval, dbl_array[i]);
            elem_max.doubleval = std::max(elem_max.doubleval, dbl_array[i]);
          } else {
            elem_min.doubleval = dbl_array[i];
            elem_max.doubleval = dbl_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kTIME:
      case kTIMESTAMP:
      case kDATE: {
        if (!initialized) {
          elem_min.bigintval = 1;
          elem_max.bigintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int64_t* tm_array = reinterpret_cast<int64_t*>(array.pointer);
        for (size_t i = 0; i < array.length / sizeof(int64_t); i++) {
          if (tm_array[i] == NULL_BIGINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.bigintval = std::min(elem_min.bigintval, tm_array[i]);
            elem_max.bigintval = std::max(elem_max.bigintval, tm_array[i]);
          } else {
            elem_min.bigintval = tm_array[i];
            elem_max.bigintval = tm_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kCHAR:
      case kVARCHAR:
      case kTEXT: {
        CHECK_EQ(buffer_->sql_type.get_compression(), kENCODING_DICT);
        if (!initialized) {
          elem_min.intval = 1;
          elem_max.intval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int32_t* int_array = (int32_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int32_t); i++) {
          if (int_array[i] == NULL_INT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.intval = std::min(elem_min.intval, int_array[i]);
            elem_max.intval = std::max(elem_max.intval, int_array[i]);
          } else {
            elem_min.intval = int_array[i];
            elem_max.intval = int_array[i];
            initialized = true;
          }
        }
        break;
      }
      default:
        UNREACHABLE();
    }
  };

Here is the call graph for this function:

Here is the caller graph for this function:

void FixedLengthArrayNoneEncoder::updateMetadata ( int8_t *  array )

inline

Definition at line 142 of file FixedLengthArrayNoneEncoder.h.

References array_size, is_null(), and update_elem_stats().

                                     {
    update_elem_stats(ArrayDatum(array_size, array, is_null(array), DoNothingDeleter()));
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::updateStats ( const int64_t  , const bool    )

inlineoverridevirtual

Implements Encoder.

Definition at line 108 of file FixedLengthArrayNoneEncoder.h.

References CHECK().

{ CHECK(false); }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::updateStats ( const double  , const bool    )

inlineoverridevirtual

Implements Encoder.

Definition at line 110 of file FixedLengthArrayNoneEncoder.h.

References CHECK().

{ CHECK(false); }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::writeMetadata ( FILE *  f )

inlineoverridevirtual

Implements Encoder.

Definition at line 114 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, has_nulls, initialized, and Encoder::num_elems_.

                                       {
    // assumes pointer is already in right place
    fwrite((int8_t*)&num_elems_, sizeof(size_t), 1, f);
    fwrite((int8_t*)&elem_min, sizeof(Datum), 1, f);
    fwrite((int8_t*)&elem_max, sizeof(Datum), 1, f);
    fwrite((int8_t*)&has_nulls, sizeof(bool), 1, f);
    fwrite((int8_t*)&initialized, sizeof(bool), 1, f);
  }

Member Data Documentation

size_t FixedLengthArrayNoneEncoder::array_size

private

Definition at line 207 of file FixedLengthArrayNoneEncoder.h.

Referenced by appendData(), getNumElemsForBytesInsertData(), and updateMetadata().

Datum FixedLengthArrayNoneEncoder::elem_max

Definition at line 201 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), readMetadata(), update_elem_stats(), and writeMetadata().

Datum FixedLengthArrayNoneEncoder::elem_min

Definition at line 200 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), readMetadata(), update_elem_stats(), and writeMetadata().

std::mutex FixedLengthArrayNoneEncoder::EncoderMutex_

private

Definition at line 206 of file FixedLengthArrayNoneEncoder.h.

bool FixedLengthArrayNoneEncoder::has_nulls

Definition at line 202 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), readMetadata(), update_elem_stats(), and writeMetadata().

bool FixedLengthArrayNoneEncoder::initialized

Definition at line 203 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), readMetadata(), update_elem_stats(), and writeMetadata().

---

The documentation for this class was generated from the following file:

• /home/jenkins-slave/workspace/core-os-doxygen/DataMgr/FixedLengthArrayNoneEncoder.h