Fragmenter_Namespace::InsertDataLoader Struct Reference

#include <InsertDataLoader.h>

Collaboration diagram for Fragmenter_Namespace::InsertDataLoader:

Classes
struct	DistributedConnector

Public Member Functions
	InsertDataLoader (DistributedConnector &connector)
void	insertData (const Catalog_Namespace::SessionInfo &session_info, InsertData &insert_data)

Private Member Functions
void	moveToNextLeaf ()

Private Attributes
size_t	leaf_count_
size_t	current_leaf_index_
DistributedConnector &	connector_

Detailed Description

Definition at line 25 of file InsertDataLoader.h.

Constructor & Destructor Documentation

Fragmenter_Namespace::InsertDataLoader::InsertDataLoader ( DistributedConnector &  connector )

inline

Definition at line 35 of file InsertDataLoader.h.

      : leaf_count_(connector.leafCount())
      , current_leaf_index_(0)
      , connector_(connector){};

Member Function Documentation

void Fragmenter_Namespace::InsertDataLoader::insertData	(	const Catalog_Namespace::SessionInfo &	session_info,
		InsertData &	insert_data
	)

Definition at line 284 of file InsertDataLoader.cpp.

References cat(), CHECK, Fragmenter_Namespace::computeRowIndicesOfShards(), connector_, Fragmenter_Namespace::copyDataOfShard(), current_leaf_index_, Catalog_Namespace::SessionInfo::getCatalog(), Fragmenter_Namespace::InsertDataLoader::DistributedConnector::insertDataToLeaf(), Fragmenter_Namespace::InsertDataLoader::DistributedConnector::leafCount(), moveToNextLeaf(), and Fragmenter_Namespace::InsertData::tableId.

Referenced by Parser::InsertIntoTableAsSelectStmt::populateData().

                                                           {
  const auto& cat = session_info.getCatalog();
  const auto* td = cat.getMetadataForTable(insert_data.tableId);

  CHECK(td);
  if (td->nShards == 0) {
    connector_.insertDataToLeaf(session_info, current_leaf_index_, insert_data);
  } else {
    // we have a sharded target table, start spreading to physical tables
    auto rowIndicesOfShards =
        computeRowIndicesOfShards(cat, connector_.leafCount(), insert_data);

    auto insertShardData =
        [this, &session_info, &insert_data, &cat, &td, &rowIndicesOfShards](
            size_t shardId) {
          const auto shard_tables = cat.getPhysicalTablesDescriptors(td);
          auto stardTableIdx = shardId % td->nShards;
          auto shardLeafIdx = shardId / td->nShards;

          const auto& rowIndicesOfShard = rowIndicesOfShards[shardId];
          ShardDataOwner shardDataOwner;

          InsertData shardData = copyDataOfShard(
              cat, shardDataOwner, insert_data, stardTableIdx, rowIndicesOfShard);
          connector_.insertDataToLeaf(session_info, shardLeafIdx, shardData);
        };

    std::vector<std::future<void>> worker_threads;
    for (size_t shardId = 0; shardId < rowIndicesOfShards.size(); shardId++) {
      if (rowIndicesOfShards[shardId].size() > 0) {
        worker_threads.push_back(
            std::async(std::launch::async, insertShardData, shardId));
      }
    }
    for (auto& child : worker_threads) {
      child.wait();
    }
    for (auto& child : worker_threads) {
      child.get();
    }
  }

  moveToNextLeaf();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void Fragmenter_Namespace::InsertDataLoader::moveToNextLeaf ( )

inlineprivate

Definition at line 44 of file InsertDataLoader.h.

References current_leaf_index_, and leaf_count_.

Referenced by insertData().

                        {
    current_leaf_index_++;
    if (current_leaf_index_ >= leaf_count_) {
      current_leaf_index_ = 0;
    }
  }

Here is the caller graph for this function:

Member Data Documentation

DistributedConnector& Fragmenter_Namespace::InsertDataLoader::connector_

private

Definition at line 53 of file InsertDataLoader.h.

Referenced by insertData().

size_t Fragmenter_Namespace::InsertDataLoader::current_leaf_index_

private

Definition at line 52 of file InsertDataLoader.h.

Referenced by insertData(), and moveToNextLeaf().

size_t Fragmenter_Namespace::InsertDataLoader::leaf_count_

private

Definition at line 51 of file InsertDataLoader.h.

Referenced by moveToNextLeaf().

---

The documentation for this struct was generated from the following files:

• /home/jenkins-slave/workspace/core-os-doxygen/Fragmenter/InsertDataLoader.h
• /home/jenkins-slave/workspace/core-os-doxygen/Fragmenter/InsertDataLoader.cpp