OmniSciDB  c0231cc57d
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
RelAlgDag Class Reference

#include <RelAlgDag.h>

+ Inheritance diagram for RelAlgDag:
+ Collaboration diagram for RelAlgDag:

Public Types

enum  BuildState { BuildState::kNotBuilt, BuildState::kBuiltNotOptimized, BuildState::kBuiltOptimized }
 

Public Member Functions

 RelAlgDag ()
 
BuildState getBuildState () const
 
void eachNode (std::function< void(RelAlgNode const *)> const &) const
 
const RelAlgNodegetRootNode () const
 
std::shared_ptr< const RelAlgNodegetRootNodeShPtr () const
 
void registerSubquery (std::shared_ptr< RexSubQuery > subquery)
 
const std::vector
< std::shared_ptr< RexSubQuery > > & 
getSubqueries () const
 
void registerQueryHints (std::shared_ptr< RelAlgNode > node, Hints *hints_delivered, RegisteredQueryHint &global_query_hint)
 
std::optional
< RegisteredQueryHint
getQueryHint (const RelAlgNode *node) const
 
std::unordered_map< size_t,
std::unordered_map< unsigned,
RegisteredQueryHint > > & 
getQueryHints ()
 
const RegisteredQueryHintgetGlobalHints () const
 
void setGlobalQueryHints (const RegisteredQueryHint &global_hints)
 
void resetQueryExecutionState ()
 

Private Attributes

BuildState build_state_
 
std::vector< std::shared_ptr
< RelAlgNode > > 
nodes_
 
std::vector< std::shared_ptr
< RexSubQuery > > 
subqueries_
 
std::unordered_map< size_t,
std::unordered_map< unsigned,
RegisteredQueryHint > > 
query_hint_
 
RegisteredQueryHint global_hints_
 

Friends

struct RelAlgDagModifier
 

Detailed Description

Class defining an in-memory, easy-to-navigate internal representation of a relational algebra DAG interpreted from a JSON provided by Calcite. Must be built through the RelAlgDagBuilder interface.

Definition at line 2323 of file RelAlgDag.h.

Member Enumeration Documentation

enum RelAlgDag::BuildState
strong
Enumerator
kNotBuilt 
kBuiltNotOptimized 
kBuiltOptimized 

Definition at line 2325 of file RelAlgDag.h.

2325 { kNotBuilt, kBuiltNotOptimized, kBuiltOptimized };

Constructor & Destructor Documentation

RelAlgDag::RelAlgDag ( )
inline

Definition at line 2327 of file RelAlgDag.h.

References kNotBuilt.

Member Function Documentation

void RelAlgDag::eachNode ( std::function< void(RelAlgNode const *)> const &  callback) const

Definition at line 3209 of file RelAlgDag.cpp.

References nodes_.

3209  {
3210  for (auto const& node : nodes_) {
3211  if (node) {
3212  callback(node.get());
3213  }
3214  }
3215 }
std::vector< std::shared_ptr< RelAlgNode > > nodes_
Definition: RelAlgDag.h:2607
BuildState RelAlgDag::getBuildState ( ) const
inline

Definition at line 2329 of file RelAlgDag.h.

References build_state_.

Referenced by RelAlgDagBuilder::optimizeDag().

2329 { return build_state_; }
BuildState build_state_
Definition: RelAlgDag.h:2605

+ Here is the caller graph for this function:

const RegisteredQueryHint& RelAlgDag::getGlobalHints ( ) const
inline

Definition at line 2593 of file RelAlgDag.h.

References global_hints_.

2593 { return global_hints_; }
RegisteredQueryHint global_hints_
Definition: RelAlgDag.h:2614
std::optional<RegisteredQueryHint> RelAlgDag::getQueryHint ( const RelAlgNode node) const
inline

Definition at line 2565 of file RelAlgDag.h.

References RelAlgNode::getId(), global_hints_, RegisteredQueryHint::isAnyQueryHintDelivered(), query_hint_, and RelAlgNode::toHash().

2565  {
2566  auto node_it = query_hint_.find(node->toHash());
2567  if (node_it != query_hint_.end()) {
2568  auto const& registered_query_hint_map = node_it->second;
2569  auto hint_it = registered_query_hint_map.find(node->getId());
2570  if (hint_it != registered_query_hint_map.end()) {
2571  auto const& registered_query_hint = hint_it->second;
2573  // apply global hint to the registered query hint for this query block
2574  return std::make_optional(registered_query_hint || global_hints_);
2575  } else {
2576  return std::make_optional(registered_query_hint);
2577  }
2578  }
2579  }
2581  // if no hint is registered from this query block
2582  // we return global hint instead
2583  return std::make_optional(global_hints_);
2584  }
2585  return std::nullopt;
2586  }
std::unordered_map< size_t, std::unordered_map< unsigned, RegisteredQueryHint > > query_hint_
Definition: RelAlgDag.h:2613
unsigned getId() const
Definition: RelAlgDag.h:814
RegisteredQueryHint global_hints_
Definition: RelAlgDag.h:2614
virtual size_t toHash() const =0
bool isAnyQueryHintDelivered() const
Definition: QueryHint.h:256

+ Here is the call graph for this function:

std::unordered_map<size_t, std::unordered_map<unsigned, RegisteredQueryHint> >& RelAlgDag::getQueryHints ( )
inline

Definition at line 2589 of file RelAlgDag.h.

References query_hint_.

2589  {
2590  return query_hint_;
2591  }
std::unordered_map< size_t, std::unordered_map< unsigned, RegisteredQueryHint > > query_hint_
Definition: RelAlgDag.h:2613
const RelAlgNode& RelAlgDag::getRootNode ( ) const
inline

Returns the root node of the DAG.

Definition at line 2336 of file RelAlgDag.h.

References CHECK, and nodes_.

2336  {
2337  CHECK(nodes_.size());
2338  const auto& last_ptr = nodes_.back();
2339  CHECK(last_ptr);
2340  return *last_ptr;
2341  }
std::vector< std::shared_ptr< RelAlgNode > > nodes_
Definition: RelAlgDag.h:2607
#define CHECK(condition)
Definition: Logger.h:222
std::shared_ptr<const RelAlgNode> RelAlgDag::getRootNodeShPtr ( ) const
inline

Definition at line 2343 of file RelAlgDag.h.

References CHECK, and nodes_.

2343  {
2344  CHECK(nodes_.size());
2345  return nodes_.back();
2346  }
std::vector< std::shared_ptr< RelAlgNode > > nodes_
Definition: RelAlgDag.h:2607
#define CHECK(condition)
Definition: Logger.h:222
const std::vector<std::shared_ptr<RexSubQuery> >& RelAlgDag::getSubqueries ( ) const
inline

Gets all registered subqueries. Only the root DAG can contain subqueries.

Definition at line 2359 of file RelAlgDag.h.

References subqueries_.

2359  {
2360  return subqueries_;
2361  }
std::vector< std::shared_ptr< RexSubQuery > > subqueries_
Definition: RelAlgDag.h:2608
void RelAlgDag::registerQueryHints ( std::shared_ptr< RelAlgNode node,
Hints hints_delivered,
RegisteredQueryHint global_query_hint 
)
inline

Definition at line 2364 of file RelAlgDag.h.

References RegisteredQueryHint::aggregate_tree_fanout, CHECK, CHECK_EQ, RegisteredQueryHint::columnar_output, RegisteredQueryHint::cpu_mode, g_enable_columnar_output, g_enable_data_recycler, g_use_query_resultset_cache, kAggregateTreeFanout, kColumnarOutput, kCpuMode, RegisteredQueryHint::keep_result, RegisteredQueryHint::keep_table_function_result, kKeepResult, kKeepTableFuncResult, kOverlapsAllowGpuBuild, kOverlapsBucketThreshold, kOverlapsKeysPerBin, kOverlapsMaxSize, kOverlapsNoCache, kRowwiseOutput, RegisteredQueryHint::overlaps_allow_gpu_build, RegisteredQueryHint::overlaps_bucket_threshold, RegisteredQueryHint::overlaps_keys_per_bin, RegisteredQueryHint::overlaps_max_size, RegisteredQueryHint::overlaps_no_cache, query_hint_, RegisteredQueryHint::registerHint(), RegisteredQueryHint::rowwise_output, and VLOG.

2366  {
2367  std::optional<bool> has_global_columnar_output_hint = std::nullopt;
2368  std::optional<bool> has_global_rowwise_output_hint = std::nullopt;
2369  RegisteredQueryHint query_hint;
2370  for (auto it = hints_delivered->begin(); it != hints_delivered->end(); it++) {
2371  auto target = it->second;
2372  auto hint_type = it->first;
2373  switch (hint_type) {
2374  case QueryHint::kCpuMode: {
2375  query_hint.registerHint(QueryHint::kCpuMode);
2376  query_hint.cpu_mode = true;
2377  if (target.isGlobalHint()) {
2378  global_query_hint.registerHint(QueryHint::kCpuMode);
2379  global_query_hint.cpu_mode = true;
2380  }
2381  break;
2382  }
2384  has_global_columnar_output_hint = target.isGlobalHint();
2385  break;
2386  }
2388  has_global_rowwise_output_hint = target.isGlobalHint();
2389  break;
2390  }
2392  CHECK(target.getListOptions().size() == 1);
2393  double overlaps_bucket_threshold = std::stod(target.getListOptions()[0]);
2394  if (overlaps_bucket_threshold >= 0.0 && overlaps_bucket_threshold <= 90.0) {
2396  query_hint.overlaps_bucket_threshold = overlaps_bucket_threshold;
2397  if (target.isGlobalHint()) {
2399  global_query_hint.overlaps_bucket_threshold = overlaps_bucket_threshold;
2400  }
2401  } else {
2402  VLOG(1) << "Skip the given query hint \"overlaps_bucket_threshold\" ("
2403  << overlaps_bucket_threshold
2404  << ") : the hint value should be within 0.0 ~ 90.0";
2405  }
2406  break;
2407  }
2409  CHECK(target.getListOptions().size() == 1);
2410  std::stringstream ss(target.getListOptions()[0]);
2411  int overlaps_max_size;
2412  ss >> overlaps_max_size;
2413  if (overlaps_max_size >= 0) {
2415  query_hint.overlaps_max_size = (size_t)overlaps_max_size;
2416  if (target.isGlobalHint()) {
2417  global_query_hint.registerHint(QueryHint::kOverlapsMaxSize);
2418  global_query_hint.overlaps_max_size = (size_t)overlaps_max_size;
2419  }
2420  } else {
2421  VLOG(1) << "Skip the query hint \"overlaps_max_size\" (" << overlaps_max_size
2422  << ") : the hint value should be larger than or equal to zero";
2423  }
2424  break;
2425  }
2428  query_hint.overlaps_allow_gpu_build = true;
2429  if (target.isGlobalHint()) {
2431  global_query_hint.overlaps_allow_gpu_build = true;
2432  }
2433  break;
2434  }
2437  query_hint.overlaps_no_cache = true;
2438  if (target.isGlobalHint()) {
2439  global_query_hint.registerHint(QueryHint::kOverlapsNoCache);
2440  global_query_hint.overlaps_no_cache = true;
2441  }
2442  VLOG(1) << "Skip auto tuner and hashtable caching for overlaps join.";
2443  break;
2444  }
2446  CHECK(target.getListOptions().size() == 1);
2447  double overlaps_keys_per_bin = std::stod(target.getListOptions()[0]);
2448  if (overlaps_keys_per_bin > 0.0 &&
2449  overlaps_keys_per_bin < std::numeric_limits<double>::max()) {
2451  query_hint.overlaps_keys_per_bin = overlaps_keys_per_bin;
2452  if (target.isGlobalHint()) {
2453  global_query_hint.registerHint(QueryHint::kOverlapsKeysPerBin);
2454  global_query_hint.overlaps_keys_per_bin = overlaps_keys_per_bin;
2455  }
2456  } else {
2457  VLOG(1) << "Skip the given query hint \"overlaps_keys_per_bin\" ("
2458  << overlaps_keys_per_bin
2459  << ") : the hint value should be larger than zero";
2460  }
2461  break;
2462  }
2463  case QueryHint::kKeepResult: {
2465  VLOG(1) << "Skip query hint \'keep_result\' because neither data recycler "
2466  "nor query resultset recycler is enabled";
2467  } else {
2469  query_hint.keep_result = true;
2470  if (target.isGlobalHint()) {
2471  global_query_hint.registerHint(QueryHint::kKeepResult);
2472  global_query_hint.keep_result = true;
2473  }
2474  }
2475  break;
2476  }
2479  VLOG(1) << "Skip query hint \'keep_table_function_result\' because neither "
2480  "data recycler "
2481  "nor query resultset recycler is enabled";
2482  } else {
2483  // we assume table function's hint is handled as global hint by default
2484  global_query_hint.registerHint(QueryHint::kKeepTableFuncResult);
2485  global_query_hint.keep_table_function_result = true;
2486  }
2487  break;
2488  }
2490  CHECK_EQ(1u, target.getListOptions().size());
2491  int aggregate_tree_fanout = std::stoi(target.getListOptions()[0]);
2492  if (aggregate_tree_fanout < 0) {
2493  VLOG(1) << "A fan-out of an aggregate tree should be larger than zero";
2494  } else if (aggregate_tree_fanout > 1024) {
2495  VLOG(1) << "Too large fanout is provided (i.e., fanout < 1024)";
2496  } else {
2498  query_hint.aggregate_tree_fanout = aggregate_tree_fanout;
2499  if (target.isGlobalHint()) {
2500  global_query_hint.registerHint(QueryHint::kAggregateTreeFanout);
2501  global_query_hint.aggregate_tree_fanout = aggregate_tree_fanout;
2502  }
2503  }
2504  }
2505  default:
2506  break;
2507  }
2508  }
2509  // we have four cases depending on 1) g_enable_columnar_output flag
2510  // and 2) query hint status: columnar_output and rowwise_output
2511  // case 1. g_enable_columnar_output = true
2512  // case 1.a) columnar_output = true (so rowwise_output = false);
2513  // case 1.b) rowwise_output = true (so columnar_output = false);
2514  // case 2. g_enable_columnar_output = false
2515  // case 2.a) columnar_output = true (so rowwise_output = false);
2516  // case 2.b) rowwise_output = true (so columnar_output = false);
2517  // case 1.a --> use columnar output
2518  // case 1.b --> use rowwise output
2519  // case 2.a --> use columnar output
2520  // case 2.b --> use rowwise output
2521  if (has_global_columnar_output_hint.has_value() &&
2522  has_global_rowwise_output_hint.has_value()) {
2523  VLOG(1)
2524  << "Two hints 1) columnar output and 2) rowwise output are enabled together, "
2525  << "so skip them and use the runtime configuration "
2526  "\"g_enable_columnar_output\"";
2527  } else if (has_global_columnar_output_hint.has_value() &&
2528  !has_global_rowwise_output_hint.has_value()) {
2530  VLOG(1) << "We already enable columnar output by default "
2531  "(g_enable_columnar_output = true), so skip this columnar output hint";
2532  } else {
2534  query_hint.columnar_output = true;
2535  if (*has_global_columnar_output_hint) {
2536  global_query_hint.registerHint(QueryHint::kColumnarOutput);
2537  global_query_hint.columnar_output = true;
2538  }
2539  }
2540  } else if (!has_global_columnar_output_hint.has_value() &&
2541  has_global_rowwise_output_hint.has_value()) {
2542  if (!g_enable_columnar_output) {
2543  VLOG(1) << "We already use the default rowwise output (g_enable_columnar_output "
2544  "= false), so skip this rowwise output hint";
2545  } else {
2547  query_hint.rowwise_output = true;
2548  if (*has_global_rowwise_output_hint) {
2549  global_query_hint.registerHint(QueryHint::kRowwiseOutput);
2550  global_query_hint.rowwise_output = true;
2551  }
2552  }
2553  }
2554  auto node_key = node->toHash();
2555  auto it = query_hint_.find(node_key);
2556  if (it == query_hint_.end()) {
2557  std::unordered_map<unsigned, RegisteredQueryHint> hint_map;
2558  hint_map.emplace(node->getId(), query_hint);
2559  query_hint_.emplace(node_key, hint_map);
2560  } else {
2561  it->second.emplace(node->getId(), query_hint);
2562  }
2563  }
#define CHECK_EQ(x, y)
Definition: Logger.h:230
std::unordered_map< size_t, std::unordered_map< unsigned, RegisteredQueryHint > > query_hint_
Definition: RelAlgDag.h:2613
bool g_use_query_resultset_cache
Definition: Execute.cpp:148
bool overlaps_allow_gpu_build
Definition: QueryHint.h:241
double overlaps_keys_per_bin
Definition: QueryHint.h:243
bool g_enable_data_recycler
Definition: Execute.cpp:146
bool g_enable_columnar_output
Definition: Execute.cpp:99
bool keep_table_function_result
Definition: QueryHint.h:233
void registerHint(const QueryHint hint)
Definition: QueryHint.h:261
size_t overlaps_max_size
Definition: QueryHint.h:240
#define CHECK(condition)
Definition: Logger.h:222
double overlaps_bucket_threshold
Definition: QueryHint.h:239
size_t aggregate_tree_fanout
Definition: QueryHint.h:236
#define VLOG(n)
Definition: Logger.h:316

+ Here is the call graph for this function:

void RelAlgDag::registerSubquery ( std::shared_ptr< RexSubQuery subquery)
inline

Registers a subquery with a root DAG builder. Should only be called during DAG building and registration should only occur on the root.

Definition at line 2352 of file RelAlgDag.h.

References subqueries_.

Referenced by anonymous_namespace{RelAlgDag.cpp}::parse_subquery().

2352  {
2353  subqueries_.push_back(subquery);
2354  }
std::vector< std::shared_ptr< RexSubQuery > > subqueries_
Definition: RelAlgDag.h:2608

+ Here is the caller graph for this function:

void RelAlgDag::resetQueryExecutionState ( )

Gets all registered subqueries. Only the root DAG can contain subqueries.

Definition at line 3217 of file RelAlgDag.cpp.

References nodes_.

3217  {
3218  for (auto& node : nodes_) {
3219  if (node) {
3220  node->resetQueryExecutionState();
3221  }
3222  }
3223 }
std::vector< std::shared_ptr< RelAlgNode > > nodes_
Definition: RelAlgDag.h:2607
void RelAlgDag::setGlobalQueryHints ( const RegisteredQueryHint global_hints)
inline

Definition at line 2595 of file RelAlgDag.h.

References global_hints_.

2595  {
2596  global_hints_ = global_hints;
2597  }
RegisteredQueryHint global_hints_
Definition: RelAlgDag.h:2614

Friends And Related Function Documentation

friend struct RelAlgDagModifier
friend

Definition at line 2616 of file RelAlgDag.h.

Member Data Documentation

BuildState RelAlgDag::build_state_
private

Definition at line 2605 of file RelAlgDag.h.

Referenced by getBuildState(), and RelAlgDagModifier::setBuildState().

RegisteredQueryHint RelAlgDag::global_hints_
private

Definition at line 2614 of file RelAlgDag.h.

Referenced by getGlobalHints(), getQueryHint(), and setGlobalQueryHints().

std::vector<std::shared_ptr<RelAlgNode> > RelAlgDag::nodes_
private
std::unordered_map<size_t, std::unordered_map<unsigned, RegisteredQueryHint> > RelAlgDag::query_hint_
private
std::vector<std::shared_ptr<RexSubQuery> > RelAlgDag::subqueries_
private

Definition at line 2608 of file RelAlgDag.h.

Referenced by getSubqueries(), RelAlgDagModifier::getSubqueries(), and registerSubquery().


The documentation for this class was generated from the following files: