com.mapd.calcite.parser.MapDParser Class Reference

Collaboration diagram for com.mapd.calcite.parser.MapDParser:

Public Member Functions
	MapDParser (String dataDir, final Supplier< MapDSqlOperatorTable > mapDSqlOperatorTable, int mapdPort, SockTransportProperties skT)
void	clearMemo ()
void	setUser (MapDUser mapdUser)
Pair< String, SqlIdentifierCapturer >	process (String sql, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
String	optimizeRAQuery (String query, final MapDParserOptions parserOptions) throws IOException
String	processSql (String sql, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
String	processSql (final SqlNode sqlNode, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
MapDPlanner.CompletionResult	getCompletionHints (String sql, int cursor, List< String > visible_tables)
Set< String >	resolveSelectIdentifiers (SqlIdentifierCapturer capturer)
SqlIdentifierCapturer	captureIdentifiers (String sql, boolean legacy_syntax) throws SqlParseException
SqlIdentifierCapturer	captureIdentifiers (SqlNode node) throws SqlParseException
int	getCallCount ()
void	updateMetaData (String schema, String table)

Static Public Attributes
static final ThreadLocal < MapDParser >	CURRENT_PARSER = new ThreadLocal<>()

Protected Member Functions
RelDataTypeSystem	createTypeSystem ()

Package Functions
RelRoot	queryToRelNode (final String sql, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
RelRoot	convertSqlToRelNode (final SqlNode sqlNode, final MapDPlanner mapDPlanner, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException

Package Attributes
SqlNode	sqlNode_

Static Package Attributes
static final Logger	MAPDLOGGER = LoggerFactory.getLogger(MapDParser.class)

Private Member Functions
MapDPlanner	getPlanner ()
boolean	isCorrelated (SqlNode expression)
MapDPlanner	getPlanner (final boolean allowSubQueryExpansion, final boolean allowPushdownJoinCondition)
String	getTableName (SqlNode node)
SqlSelect	rewriteSimpleUpdateAsSelect (final SqlUpdate update)
LogicalTableModify	getDummyUpdate (SqlUpdate update) throws SqlParseException, ValidationException, RelConversionException
RelRoot	rewriteUpdateAsSelect (SqlUpdate update, MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
RelRoot	replaceIsTrue (final RelDataTypeFactory typeFactory, RelRoot root)
SqlNode	parseSql (String sql, final boolean legacy_syntax, Planner planner) throws SqlParseException
void	desugar (SqlSelect select_node, RelDataTypeFactory typeFactory)
SqlNode	expandCase (SqlCase old_case_node, RelDataTypeFactory typeFactory)
SqlOrderBy	desugar (SqlSelect select_node, SqlOrderBy order_by_node, RelDataTypeFactory typeFactory)
void	desugarExpression (SqlNode node, RelDataTypeFactory typeFactory)
SqlNode	expand (final SqlNode node, final java.util.Map< String, SqlNode > id_to_expr, RelDataTypeFactory typeFactory)
SqlNodeList	expand (final SqlNodeList group_by_list, final java.util.Map< String, SqlNode > id_to_expr, RelDataTypeFactory typeFactory)
SqlNode	expandVariance (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)
SqlNode	expandVariance (final SqlParserPos pos, final SqlNode operand, boolean biased, boolean sqrt, boolean flt, RelDataTypeFactory typeFactory)
SqlNode	expandCovariance (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)
SqlNode	expandCovariance (SqlParserPos pos, final SqlNode operand0, final SqlNode operand1, boolean pop, boolean flt, RelDataTypeFactory typeFactory)
SqlNode	expandCorrelation (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)

Private Attributes
final Supplier < MapDSqlOperatorTable >	mapDSqlOperatorTable
final String	dataDir
int	callCount = 0
final int	mapdPort
MapDUser	mapdUser
SockTransportProperties	sock_transport_properties = null

Static Private Attributes
static final EnumSet< SqlKind >	SCALAR
static final EnumSet< SqlKind >	EXISTS = EnumSet.of(SqlKind.EXISTS)
static final EnumSet< SqlKind >	DELETE = EnumSet.of(SqlKind.DELETE)
static final EnumSet< SqlKind >	UPDATE = EnumSet.of(SqlKind.UPDATE)
static final EnumSet< SqlKind >	IN = EnumSet.of(SqlKind.IN)
static final EnumSet< SqlKind >	ARRAY_VALUE
static Map< String, Boolean >	SubqueryCorrMemo = new ConcurrentHashMap<>()
static final Context	MAPD_CONNECTION_CONTEXT

Detailed Description

Author

michael

Definition at line 142 of file MapDParser.java.

Constructor & Destructor Documentation

com.mapd.calcite.parser.MapDParser.MapDParser ( String  dataDir, final Supplier< MapDSqlOperatorTable >  mapDSqlOperatorTable, int  mapdPort, SockTransportProperties  skT  )

inline

Definition at line 166 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDParser.mapdPort, and com.mapd.calcite.parser.MapDParser.mapDSqlOperatorTable.

Referenced by com.mapd.calcite.parser.MapDParser.isCorrelated().

                                       {
    this.dataDir = dataDir;
    this.mapDSqlOperatorTable = mapDSqlOperatorTable;
    this.mapdPort = mapdPort;
    this.sock_transport_properties = skT;
  }

Here is the caller graph for this function:

Member Function Documentation

SqlIdentifierCapturer com.mapd.calcite.parser.MapDParser.captureIdentifiers ( String  sql, boolean  legacy_syntax  ) throws SqlParseException

inline

Definition at line 1413 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.process().

                                   {
    try {
      Planner planner = getPlanner();
      SqlNode node = parseSql(sql, legacy_syntax, planner);
      return captureIdentifiers(node);
    } catch (Exception | Error e) {
      MAPDLOGGER.error("Error parsing sql: " + sql, e);
      return new SqlIdentifierCapturer();
    }
  }

Here is the caller graph for this function:

SqlIdentifierCapturer com.mapd.calcite.parser.MapDParser.captureIdentifiers ( SqlNode  node ) throws SqlParseException

inline

Definition at line 1425 of file MapDParser.java.

                                                                                         {
    try {
      SqlIdentifierCapturer capturer = new SqlIdentifierCapturer();
      capturer.scan(node);
      return capturer;
    } catch (Exception | Error e) {
      MAPDLOGGER.error("Error parsing sql: " + node, e);
      return new SqlIdentifierCapturer();
    }
  }

void com.mapd.calcite.parser.MapDParser.clearMemo ( )

inline

Definition at line 176 of file MapDParser.java.

                          {
    SubqueryCorrMemo.clear();
  }

RelRoot com.mapd.calcite.parser.MapDParser.convertSqlToRelNode ( final SqlNode  sqlNode, final MapDPlanner  mapDPlanner, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inlinepackage

Definition at line 772 of file MapDParser.java.

References DELETE, f, setup.root, and UPDATE.

Referenced by com.mapd.calcite.parser.MapDParser.processSql().

                                                                                {
    SqlNode node = sqlNode;
    MapDPlanner planner = mapDPlanner;
    boolean allowCorrelatedSubQueryExpansion = true;
    boolean allowPushdownJoinCondition = true;
    boolean patchUpdateToDelete = false;

    if (node.isA(DELETE)) {
      SqlDelete sqlDelete = (SqlDelete) node;
      node = new SqlUpdate(node.getParserPosition(),
              sqlDelete.getTargetTable(),
              SqlNodeList.EMPTY,
              SqlNodeList.EMPTY,
              sqlDelete.getCondition(),
              sqlDelete.getSourceSelect(),
              sqlDelete.getAlias());

      patchUpdateToDelete = true;
    }

    if (node.isA(UPDATE)) {
      SqlUpdate update = (SqlUpdate) node;
      update = (SqlUpdate) planner.validate(update);
      RelRoot root = rewriteUpdateAsSelect(update, parserOptions);

      if (patchUpdateToDelete) {
        LogicalTableModify modify = (LogicalTableModify) root.rel;

        try {
          Field f = TableModify.class.getDeclaredField("operation");
          f.setAccessible(true);
          f.set(modify, Operation.DELETE);
        } catch (Throwable e) {
          throw new RuntimeException(e);
        }

        root = RelRoot.of(modify, SqlKind.DELETE);
      }

      return root;
    }

    if (parserOptions.isLegacySyntax()) {
      // close original planner
      planner.close();
      // create a new one
      planner = getPlanner(allowCorrelatedSubQueryExpansion, allowPushdownJoinCondition);
      node = parseSql(
              node.toSqlString(CalciteSqlDialect.DEFAULT).toString(), false, planner);
    }

    SqlNode validateR = planner.validate(node);
    planner.setFilterPushDownInfo(parserOptions.getFilterPushDownInfo());
    RelRoot relR = planner.rel(validateR);
    relR = replaceIsTrue(planner.getTypeFactory(), relR);
    planner.close();

    if (!parserOptions.isViewOptimizeEnabled()) {
      return relR;
    } else {
      // check to see if a view is involved in the query
      boolean foundView = false;
      MapDSchema schema = new MapDSchema(
              dataDir, this, mapdPort, mapdUser, sock_transport_properties);
      SqlIdentifierCapturer capturer = captureIdentifiers(sqlNode);
      for (String name : capturer.selects) {
        MapDTable table = (MapDTable) schema.getTable(name);
        if (null == table) {
          throw new RuntimeException("table/view not found: " + name);
        }
        if (table instanceof MapDView) {
          foundView = true;
        }
      }

      if (!foundView) {
        return relR;
      }

      HepProgramBuilder builder = new HepProgramBuilder();
      builder.addRuleInstance(CoreRules.JOIN_PROJECT_BOTH_TRANSPOSE_INCLUDE_OUTER);
      builder.addRuleInstance(CoreRules.FILTER_MERGE);
      builder.addRuleInstance(CoreRules.FILTER_PROJECT_TRANSPOSE);
      builder.addRuleInstance(CoreRules.PROJECT_MERGE);
      builder.addRuleInstance(ProjectProjectRemoveRule.INSTANCE);

      HepPlanner hepPlanner = new HepPlanner(builder.build());
      final RelNode root = relR.project();
      hepPlanner.setRoot(root);
      final RelNode newRel = hepPlanner.findBestExp();

      return RelRoot.of(newRel, relR.kind);
    }
  }

Here is the caller graph for this function:

RelDataTypeSystem com.mapd.calcite.parser.MapDParser.createTypeSystem ( )

inlineprotected

Definition at line 1447 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

                                                 {
    final MapDTypeSystem typeSystem = new MapDTypeSystem();
    return typeSystem;
  }

Here is the caller graph for this function:

void com.mapd.calcite.parser.MapDParser.desugar ( SqlSelect  select_node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 963 of file MapDParser.java.

                                                                              {
    desugar(select_node, null, typeFactory);
  }

SqlOrderBy com.mapd.calcite.parser.MapDParser.desugar ( SqlSelect  select_node, SqlOrderBy  order_by_node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1013 of file MapDParser.java.

References i.

                                          {
    MAPDLOGGER.debug("desugar: before: " + select_node.toString());
    desugarExpression(select_node.getFrom(), typeFactory);
    desugarExpression(select_node.getWhere(), typeFactory);
    SqlNodeList select_list = select_node.getSelectList();
    SqlNodeList new_select_list = new SqlNodeList(select_list.getParserPosition());
    java.util.Map<String, SqlNode> id_to_expr = new java.util.HashMap<String, SqlNode>();
    for (SqlNode proj : select_list) {
      if (!(proj instanceof SqlBasicCall)) {
        if (proj instanceof SqlCase) {
          new_select_list.add(expandCase((SqlCase) proj, typeFactory));
        } else {
          new_select_list.add(proj);
        }
      } else {
        assert proj instanceof SqlBasicCall;
        SqlBasicCall proj_call = (SqlBasicCall) proj;
        if (proj_call.operands.length > 0) {
          for (int i = 0; i < proj_call.operands.length; i++) {
            if (proj_call.operand(i) instanceof SqlCase) {
              SqlNode new_op = expandCase(proj_call.operand(i), typeFactory);
              proj_call.setOperand(i, new_op);
            }
          }
        }
        new_select_list.add(expand(proj_call, id_to_expr, typeFactory));
      }
    }
    select_node.setSelectList(new_select_list);
    SqlNodeList group_by_list = select_node.getGroup();
    if (group_by_list != null) {
      select_node.setGroupBy(expand(group_by_list, id_to_expr, typeFactory));
    }
    SqlNode having = select_node.getHaving();
    if (having != null) {
      expand(having, id_to_expr, typeFactory);
    }
    SqlOrderBy new_order_by_node = null;
    if (order_by_node != null && order_by_node.orderList != null
            && order_by_node.orderList.size() > 0) {
      SqlNodeList new_order_by_list =
              expand(order_by_node.orderList, id_to_expr, typeFactory);
      new_order_by_node = new SqlOrderBy(order_by_node.getParserPosition(),
              select_node,
              new_order_by_list,
              order_by_node.offset,
              order_by_node.fetch);
    }

    MAPDLOGGER.debug("desugar:  after: " + select_node.toString());
    return new_order_by_node;
  }

void com.mapd.calcite.parser.MapDParser.desugarExpression ( SqlNode  node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1068 of file MapDParser.java.

                                                                               {
    if (node instanceof SqlSelect) {
      desugar((SqlSelect) node, typeFactory);
      return;
    }
    if (!(node instanceof SqlBasicCall)) {
      return;
    }
    SqlBasicCall basic_call = (SqlBasicCall) node;
    for (SqlNode operator : basic_call.getOperands()) {
      if (operator instanceof SqlOrderBy) {
        desugarExpression(((SqlOrderBy) operator).query, typeFactory);
      } else {
        desugarExpression(operator, typeFactory);
      }
    }
  }

SqlNode com.mapd.calcite.parser.MapDParser.expand ( final SqlNode  node, final java.util.Map< String, SqlNode >  id_to_expr, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1086 of file MapDParser.java.

References i.

                                          {
    MAPDLOGGER.debug("expand: " + node.toString());
    if (node instanceof SqlBasicCall) {
      SqlBasicCall node_call = (SqlBasicCall) node;
      SqlNode[] operands = node_call.getOperands();
      for (int i = 0; i < operands.length; ++i) {
        node_call.setOperand(i, expand(operands[i], id_to_expr, typeFactory));
      }
      SqlNode expanded_variance = expandVariance(node_call, typeFactory);
      if (expanded_variance != null) {
        return expanded_variance;
      }
      SqlNode expanded_covariance = expandCovariance(node_call, typeFactory);
      if (expanded_covariance != null) {
        return expanded_covariance;
      }
      SqlNode expanded_correlation = expandCorrelation(node_call, typeFactory);
      if (expanded_correlation != null) {
        return expanded_correlation;
      }
    }
    if (node instanceof SqlSelect) {
      SqlSelect select_node = (SqlSelect) node;
      desugar(select_node, typeFactory);
    }
    return node;
  }

SqlNodeList com.mapd.calcite.parser.MapDParser.expand ( final SqlNodeList  group_by_list, final java.util.Map< String, SqlNode >  id_to_expr, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1116 of file MapDParser.java.

                                          {
    SqlNodeList new_group_by_list = new SqlNodeList(new SqlParserPos(-1, -1));
    for (SqlNode group_by : group_by_list) {
      if (!(group_by instanceof SqlIdentifier)) {
        new_group_by_list.add(expand(group_by, id_to_expr, typeFactory));
        continue;
      }
      SqlIdentifier group_by_id = ((SqlIdentifier) group_by);
      if (id_to_expr.containsKey(group_by_id.toString())) {
        new_group_by_list.add(id_to_expr.get(group_by_id.toString()));
      } else {
        new_group_by_list.add(group_by);
      }
    }
    return new_group_by_list;
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandCase ( SqlCase  old_case_node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 967 of file MapDParser.java.

                                                                                    {
    SqlNodeList newWhenList =
            new SqlNodeList(old_case_node.getWhenOperands().getParserPosition());
    SqlNodeList newThenList =
            new SqlNodeList(old_case_node.getThenOperands().getParserPosition());
    java.util.Map<String, SqlNode> id_to_expr = new java.util.HashMap<String, SqlNode>();
    for (SqlNode node : old_case_node.getWhenOperands()) {
      SqlNode newCall = expand(node, id_to_expr, typeFactory);
      if (null != newCall) {
        newWhenList.add(newCall);
      } else {
        newWhenList.add(node);
      }
    }
    for (SqlNode node : old_case_node.getThenOperands()) {
      SqlNode newCall = expand(node, id_to_expr, typeFactory);
      if (null != newCall) {
        newThenList.add(newCall);
      } else {
        newThenList.add(node);
      }
    }
    SqlNode new_else_operand = old_case_node.getElseOperand();
    if (null != new_else_operand) {
      SqlNode candidate_else_operand =
              expand(old_case_node.getElseOperand(), id_to_expr, typeFactory);
      if (null != candidate_else_operand) {
        new_else_operand = candidate_else_operand;
      }
    }
    SqlNode new_value_operand = old_case_node.getValueOperand();
    if (null != new_value_operand) {
      SqlNode candidate_value_operand =
              expand(old_case_node.getValueOperand(), id_to_expr, typeFactory);
      if (null != candidate_value_operand) {
        new_value_operand = candidate_value_operand;
      }
    }
    SqlNode newCaseNode = SqlCase.createSwitched(old_case_node.getParserPosition(),
            new_value_operand,
            newWhenList,
            newThenList,
            new_else_operand);
    return newCaseNode;
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandCorrelation ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1366 of file MapDParser.java.

                                                                        {
    // Expand correlation coefficient
    if (proj_call.operandCount() != 2) {
      return null;
    }
    boolean flt;
    if (proj_call.getOperator().isName("CORR", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("CORRELATION")) {
      // expand correlation coefficient
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("CORR_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("CORRELATION_FLOAT")) {
      // expand correlation coefficient
      flt = true;
    } else {
      return null;
    }
    // corr(x, y) ==> (avg(x * y) - avg(x) * avg(y)) / (stddev_pop(x) *
    // stddev_pop(y))
    // ==> covar_pop(x, y) / (stddev_pop(x) * stddev_pop(y))
    final SqlNode operand0 = proj_call.operand(0);
    final SqlNode operand1 = proj_call.operand(1);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode covariance =
            expandCovariance(pos, operand0, operand1, true, flt, typeFactory);
    SqlNode stddev0 = expandVariance(pos, operand0, true, true, flt, typeFactory);
    SqlNode stddev1 = expandVariance(pos, operand1, true, true, flt, typeFactory);
    final SqlNode mulStddev =
            SqlStdOperatorTable.MULTIPLY.createCall(pos, stddev0, stddev1);
    final SqlNumericLiteral zero = SqlLiteral.createExactNumeric("0.0", pos);
    final SqlNode mulStddevEqZero =
            SqlStdOperatorTable.EQUALS.createCall(pos, mulStddev, zero);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    SqlNodeList whenList1 = new SqlNodeList(pos);
    SqlNodeList thenList1 = new SqlNodeList(pos);
    whenList1.add(mulStddevEqZero);
    thenList1.add(nul);
    final SqlNode denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList1, thenList1, mulStddev);
    final SqlNode expanded_proj_call =
            SqlStdOperatorTable.DIVIDE.createCall(pos, covariance, denominator);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandCovariance ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1274 of file MapDParser.java.

                                                                        {
    // Expand covariance aggregates
    if (proj_call.operandCount() != 2) {
      return null;
    }
    boolean pop;
    boolean flt;
    if (proj_call.getOperator().isName("COVAR_POP", false)) {
      pop = true;
      flt = false;
    } else if (proj_call.getOperator().isName("COVAR_SAMP", false)) {
      pop = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("COVAR_POP_FLOAT")) {
      pop = true;
      flt = true;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("COVAR_SAMP_FLOAT")) {
      pop = false;
      flt = true;
    } else {
      return null;
    }
    final SqlNode operand0 = proj_call.operand(0);
    final SqlNode operand1 = proj_call.operand(1);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode expanded_proj_call =
            expandCovariance(pos, operand0, operand1, pop, flt, typeFactory);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandCovariance ( SqlParserPos  pos, final SqlNode  operand0, final SqlNode  operand1, boolean  pop, boolean  flt, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1307 of file MapDParser.java.

References count, and DOUBLE.

                                          {
    // covar_pop(x, y) ==> avg(x * y) - avg(x) * avg(y)
    // covar_samp(x, y) ==> (sum(x * y) - sum(x) * avg(y))
    // ((case count(x) when 1 then NULL else count(x) - 1 end))
    final SqlNode arg0 = SqlStdOperatorTable.CAST.createCall(operand0.getParserPosition(),
            operand0,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode arg1 = SqlStdOperatorTable.CAST.createCall(operand1.getParserPosition(),
            operand1,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode mulArg = SqlStdOperatorTable.MULTIPLY.createCall(pos, arg0, arg1);
    final SqlNode avgArg1 = SqlStdOperatorTable.AVG.createCall(pos, arg1);
    if (pop) {
      final SqlNode avgMulArg = SqlStdOperatorTable.AVG.createCall(pos, mulArg);
      final SqlNode avgArg0 = SqlStdOperatorTable.AVG.createCall(pos, arg0);
      final SqlNode mulAvgAvg =
              SqlStdOperatorTable.MULTIPLY.createCall(pos, avgArg0, avgArg1);
      final SqlNode covarPop =
              SqlStdOperatorTable.MINUS.createCall(pos, avgMulArg, mulAvgAvg);
      return SqlStdOperatorTable.CAST.createCall(pos,
              covarPop,
              SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                      flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    }
    final SqlNode sumMulArg = SqlStdOperatorTable.SUM.createCall(pos, mulArg);
    final SqlNode sumArg0 = SqlStdOperatorTable.SUM.createCall(pos, arg0);
    final SqlNode mulSumAvg =
            SqlStdOperatorTable.MULTIPLY.createCall(pos, sumArg0, avgArg1);
    final SqlNode sub = SqlStdOperatorTable.MINUS.createCall(pos, sumMulArg, mulSumAvg);
    final SqlNode count = SqlStdOperatorTable.COUNT.createCall(pos, operand0);
    final SqlNumericLiteral one = SqlLiteral.createExactNumeric("1", pos);
    final SqlNode countEqOne = SqlStdOperatorTable.EQUALS.createCall(pos, count, one);
    final SqlNode countMinusOne = SqlStdOperatorTable.MINUS.createCall(pos, count, one);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    SqlNodeList whenList1 = new SqlNodeList(pos);
    SqlNodeList thenList1 = new SqlNodeList(pos);
    whenList1.add(countEqOne);
    thenList1.add(nul);
    final SqlNode int_denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList1, thenList1, countMinusOne);
    final SqlNode denominator = SqlStdOperatorTable.CAST.createCall(pos,
            int_denominator,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode covarSamp =
            SqlStdOperatorTable.DIVIDE.createCall(pos, sub, denominator);
    return SqlStdOperatorTable.CAST.createCall(pos,
            covarSamp,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandVariance ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1135 of file MapDParser.java.

                                                                        {
    // Expand variance aggregates that are not supported natively
    if (proj_call.operandCount() != 1) {
      return null;
    }
    boolean biased;
    boolean sqrt;
    boolean flt;
    if (proj_call.getOperator().isName("STDDEV_POP", false)) {
      biased = true;
      sqrt = true;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_POP_FLOAT")) {
      biased = true;
      sqrt = true;
      flt = true;
    } else if (proj_call.getOperator().isName("STDDEV_SAMP", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("STDDEV")) {
      biased = false;
      sqrt = true;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_SAMP_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_FLOAT")) {
      biased = false;
      sqrt = true;
      flt = true;
    } else if (proj_call.getOperator().isName("VAR_POP", false)) {
      biased = true;
      sqrt = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("VAR_POP_FLOAT")) {
      biased = true;
      sqrt = false;
      flt = true;
    } else if (proj_call.getOperator().isName("VAR_SAMP", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("VARIANCE")) {
      biased = false;
      sqrt = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("VAR_SAMP_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("VARIANCE_FLOAT")) {
      biased = false;
      sqrt = false;
      flt = true;
    } else {
      return null;
    }
    final SqlNode operand = proj_call.operand(0);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode expanded_proj_call =
            expandVariance(pos, operand, biased, sqrt, flt, typeFactory);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

SqlNode com.mapd.calcite.parser.MapDParser.expandVariance ( final SqlParserPos  pos, final SqlNode  operand, boolean  biased, boolean  sqrt, boolean  flt, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1192 of file MapDParser.java.

References count, DOUBLE, and run_benchmark_import.result.

                                          {
    // stddev_pop(x) ==>
    // power(
    // (sum(x * x) - sum(x) * sum(x) / (case count(x) when 0 then NULL else count(x)
    // end)) / (case count(x) when 0 then NULL else count(x) end), .5)
    //
    // stddev_samp(x) ==>
    // power(
    // (sum(x * x) - sum(x) * sum(x) / (case count(x) when 0 then NULL else count(x)
    // )) / ((case count(x) when 1 then NULL else count(x) - 1 end)), .5)
    //
    // var_pop(x) ==>
    // (sum(x * x) - sum(x) * sum(x) / ((case count(x) when 0 then NULL else
    // count(x)
    // end))) / ((case count(x) when 0 then NULL else count(x) end))
    //
    // var_samp(x) ==>
    // (sum(x * x) - sum(x) * sum(x) / ((case count(x) when 0 then NULL else
    // count(x)
    // end))) / ((case count(x) when 1 then NULL else count(x) - 1 end))
    //
    final SqlNode arg = SqlStdOperatorTable.CAST.createCall(pos,
            operand,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode argSquared = SqlStdOperatorTable.MULTIPLY.createCall(pos, arg, arg);
    final SqlNode sumArgSquared = SqlStdOperatorTable.SUM.createCall(pos, argSquared);
    final SqlNode sum = SqlStdOperatorTable.SUM.createCall(pos, arg);
    final SqlNode sumSquared = SqlStdOperatorTable.MULTIPLY.createCall(pos, sum, sum);
    final SqlNode count = SqlStdOperatorTable.COUNT.createCall(pos, arg);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    final SqlNumericLiteral zero = SqlLiteral.createExactNumeric("0", pos);
    final SqlNode countEqZero = SqlStdOperatorTable.EQUALS.createCall(pos, count, zero);
    SqlNodeList whenList = new SqlNodeList(pos);
    SqlNodeList thenList = new SqlNodeList(pos);
    whenList.add(countEqZero);
    thenList.add(nul);
    final SqlNode int_denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList, thenList, count);
    final SqlNode denominator = SqlStdOperatorTable.CAST.createCall(pos,
            int_denominator,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode avgSumSquared =
            SqlStdOperatorTable.DIVIDE.createCall(pos, sumSquared, denominator);
    final SqlNode diff =
            SqlStdOperatorTable.MINUS.createCall(pos, sumArgSquared, avgSumSquared);
    final SqlNode denominator1;
    if (biased) {
      denominator1 = denominator;
    } else {
      final SqlNumericLiteral one = SqlLiteral.createExactNumeric("1", pos);
      final SqlNode countEqOne = SqlStdOperatorTable.EQUALS.createCall(pos, count, one);
      final SqlNode countMinusOne = SqlStdOperatorTable.MINUS.createCall(pos, count, one);
      SqlNodeList whenList1 = new SqlNodeList(pos);
      SqlNodeList thenList1 = new SqlNodeList(pos);
      whenList1.add(countEqOne);
      thenList1.add(nul);
      final SqlNode int_denominator1 = SqlStdOperatorTable.CASE.createCall(
              null, pos, null, whenList1, thenList1, countMinusOne);
      denominator1 = SqlStdOperatorTable.CAST.createCall(pos,
              int_denominator1,
              SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                      flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    }
    final SqlNode div = SqlStdOperatorTable.DIVIDE.createCall(pos, diff, denominator1);
    SqlNode result = div;
    if (sqrt) {
      final SqlNumericLiteral half = SqlLiteral.createExactNumeric("0.5", pos);
      result = SqlStdOperatorTable.POWER.createCall(pos, div, half);
    }
    return SqlStdOperatorTable.CAST.createCall(pos,
            result,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
  }

int com.mapd.calcite.parser.MapDParser.getCallCount ( )

inline

Definition at line 1436 of file MapDParser.java.

Referenced by com.mapd.parser.server.CalciteParserFactory.validateObject().

                            {
    return callCount;
  }

Here is the caller graph for this function:

MapDPlanner.CompletionResult com.mapd.calcite.parser.MapDParser.getCompletionHints ( String  sql, int  cursor, List< String >  visible_tables  )

inline

Definition at line 438 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getPlanner().

                                                               {
    return getPlanner().getCompletionHints(sql, cursor, visible_tables);
  }

Here is the call graph for this function:

LogicalTableModify com.mapd.calcite.parser.MapDParser.getDummyUpdate ( SqlUpdate  update ) throws SqlParseException, ValidationException, RelConversionException

inlineprivate

Definition at line 576 of file MapDParser.java.

                                                                                {
    SqlIdentifier targetTable = (SqlIdentifier) update.getTargetTable();
    String targetTableName = targetTable.names.get(targetTable.names.size() - 1);
    MapDPlanner planner = getPlanner();
    String dummySql = "DELETE FROM " + targetTableName;
    SqlNode dummyNode = planner.parse(dummySql);
    dummyNode = planner.validate(dummyNode);
    RelRoot dummyRoot = planner.rel(dummyNode);
    LogicalTableModify dummyModify = (LogicalTableModify) dummyRoot.rel;
    return dummyModify;
  }

MapDPlanner com.mapd.calcite.parser.MapDParser.getPlanner ( )

inlineprivate

Definition at line 214 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getCompletionHints(), com.mapd.calcite.parser.MapDParser.optimizeRAQuery(), com.mapd.calcite.parser.MapDParser.process(), and com.mapd.calcite.parser.MapDParser.processSql().

                                   {
    return getPlanner(true, true);
  }

Here is the caller graph for this function:

MapDPlanner com.mapd.calcite.parser.MapDParser.getPlanner ( final boolean  allowSubQueryExpansion, final boolean  allowPushdownJoinCondition  )

inlineprivate

Definition at line 240 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.createTypeSystem(), com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDParser.EXISTS, f, com.mapd.calcite.parser.MapDUser.getDB(), com.mapd.parser.hint.OmniSciHintStrategyTable.HINT_STRATEGY_TABLE, com.mapd.calcite.parser.MapDParser.IN, com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.MAPD_CONNECTION_CONTEXT, com.mapd.calcite.parser.MapDParser.mapdPort, com.mapd.calcite.parser.MapDParser.mapdUser, setup.root, com.mapd.calcite.parser.MapDParser.SCALAR, com.mapd.calcite.parser.MapDParser.sock_transport_properties, t, toString(), and run_benchmark_import.type.

                                                    {
    final MapDSchema mapd =
            new MapDSchema(dataDir, this, mapdPort, mapdUser, sock_transport_properties);
    final SchemaPlus rootSchema = Frameworks.createRootSchema(true);

    BiPredicate<SqlNode, SqlNode> expandPredicate = new BiPredicate<SqlNode, SqlNode>() {
      @Override
      public boolean test(SqlNode root, SqlNode expression) {
        if (!allowSubQueryExpansion) {
          return false;
        }

        // special handling of sub-queries
        if (expression.isA(SCALAR) || expression.isA(EXISTS) || expression.isA(IN)) {
          // only expand if it is correlated.

          if (expression.isA(EXISTS)) {
            // always expand subquery by EXISTS clause
            return true;
          }

          if (expression.isA(IN)) {
            // expand subquery by IN clause
            // but correlated subquery by NOT_IN clause is not available
            // currently due to a lack of supporting in Calcite
            boolean found_expression = false;
            if (expression instanceof SqlCall) {
              SqlCall call = (SqlCall) expression;
              if (call.getOperandList().size() == 2) {
                // if IN clause is correlated, its second operand of corresponding
                // expression is SELECT clause which indicates a correlated subquery.
                // Here, an expression "f.val IN (SELECT ...)" has two operands.
                // Since we have interest in its subquery, so try to check whether
                // the second operand, i.e., call.getOperandList().get(1)
                // is a type of SqlSelect and also is correlated.
                // Note that the second operand of non-correlated IN clause
                // does not have SqlSelect as its second operand
                if (call.getOperandList().get(1) instanceof SqlSelect) {
                  expression = call.getOperandList().get(1);
                  SqlSelect select_call = (SqlSelect) expression;
                  if (select_call.hasWhere()) {
                    found_expression = true;
                  }
                }
              }
            }
            if (!found_expression) {
              return false;
            }
          }

          if (isCorrelated(expression)) {
            SqlSelect select = null;
            if (expression instanceof SqlCall) {
              SqlCall call = (SqlCall) expression;
              if (call.getOperator().equals(SqlStdOperatorTable.SCALAR_QUERY)) {
                expression = call.getOperandList().get(0);
              }
            }

            if (expression instanceof SqlSelect) {
              select = (SqlSelect) expression;
            }

            if (null != select) {
              if (null != select.getFetch() || null != select.getOffset()
                      || (null != select.getOrderList()
                              && select.getOrderList().size() != 0)) {
                throw new CalciteException(
                        "Correlated sub-queries with ordering not supported.", null);
              }
            }
            return true;
          }
        }

        // per default we do not want to expand
        return false;
      }
    };

    BiPredicate<SqlNode, Join> pushdownJoinPredicate = new BiPredicate<SqlNode, Join>() {
      @Override
      public boolean test(SqlNode t, Join u) {
        if (!allowPushdownJoinCondition) {
          return false;
        }

        return !hasGeoColumns(u.getRowType());
      }

      private boolean hasGeoColumns(RelDataType type) {
        for (RelDataTypeField f : type.getFieldList()) {
          if ("any".equalsIgnoreCase(f.getType().getFamily().toString())) {
            // any indicates geo types at the moment
            return true;
          }
        }

        return false;
      }
    };

    final FrameworkConfig config =
            Frameworks.newConfigBuilder()
                    .defaultSchema(rootSchema.add(mapdUser.getDB(), mapd))
                    .operatorTable(mapDSqlOperatorTable.get())
                    .parserConfig(SqlParser.configBuilder()
                                          .setConformance(SqlConformanceEnum.LENIENT)
                                          .setUnquotedCasing(Casing.UNCHANGED)
                                          .setCaseSensitive(false)
                                          // allow identifiers of up to 512 chars
                                          .setIdentifierMaxLength(512)
                                          .setParserFactory(ExtendedSqlParser.FACTORY)
                                          .build())
                    .sqlToRelConverterConfig(
                            SqlToRelConverter
                                    .configBuilder()
                                    // enable sub-query expansion (de-correlation)
                                    .withExpandPredicate(expandPredicate)
                                    // allow as many as possible IN operator values
                                    .withInSubQueryThreshold(Integer.MAX_VALUE)
                                    .withPushdownJoinCondition(pushdownJoinPredicate)
                                    .withHintStrategyTable(
                                            OmniSciHintStrategyTable.HINT_STRATEGY_TABLE)
                                    .build())
                    .typeSystem(createTypeSystem())
                    .context(MAPD_CONNECTION_CONTEXT)
                    .build();
    MapDPlanner planner = new MapDPlanner(config);
    planner.setRestriction(mapdUser.getRestriction());
    return planner;
  }

Here is the call graph for this function:

String com.mapd.calcite.parser.MapDParser.getTableName ( SqlNode  node )

inlineprivate

Definition at line 465 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect().

                                            {
    if (node.isA(EnumSet.of(SqlKind.AS))) {
      node = ((SqlCall) node).getOperandList().get(1);
    }
    if (node instanceof SqlIdentifier) {
      SqlIdentifier id = (SqlIdentifier) node;
      return id.names.get(id.names.size() - 1);
    }
    return null;
  }

Here is the caller graph for this function:

boolean com.mapd.calcite.parser.MapDParser.isCorrelated ( SqlNode  expression )

inlineprivate

Definition at line 218 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, DEFAULT, com.mapd.calcite.parser.MapDParser.MapDParser(), com.mapd.calcite.parser.MapDParser.mapdPort, com.mapd.calcite.parser.MapDParser.mapDSqlOperatorTable, com.mapd.calcite.parser.MapDParser.mapdUser, run_benchmark_import.parser, and com.mapd.calcite.parser.MapDParser.sock_transport_properties.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), and com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect().

                                                   {
    String queryString = expression.toSqlString(CalciteSqlDialect.DEFAULT).getSql();
    Boolean isCorrelatedSubquery = SubqueryCorrMemo.get(queryString);
    if (null != isCorrelatedSubquery) {
      return isCorrelatedSubquery;
    }

    try {
      MapDParser parser = new MapDParser(
              dataDir, mapDSqlOperatorTable, mapdPort, sock_transport_properties);
      MapDParserOptions options = new MapDParserOptions();
      parser.setUser(mapdUser);
      parser.processSql(expression, options);
    } catch (Exception e) {
      // if we are not able to parse, then assume correlated
      SubqueryCorrMemo.put(queryString, true);
      return true;
    }
    SubqueryCorrMemo.put(queryString, false);
    return false;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

String com.mapd.calcite.parser.MapDParser.optimizeRAQuery ( String  query, final MapDParserOptions  parserOptions  ) throws IOException

inline

Definition at line 389 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.mapdPort, com.mapd.calcite.parser.MapDParser.mapdUser, com.mapd.calcite.parser.MapDParser.replaceIsTrue(), and com.mapd.calcite.parser.MapDParser.sock_transport_properties.

                             {
    MapDSchema schema =
            new MapDSchema(dataDir, this, mapdPort, mapdUser, sock_transport_properties);
    MapDPlanner planner = getPlanner(true, true);

    planner.setFilterPushDownInfo(parserOptions.getFilterPushDownInfo());
    RelRoot optRel = planner.optimizeRaQuery(query, schema);
    optRel = replaceIsTrue(planner.getTypeFactory(), optRel);
    return MapDSerializer.toString(optRel.project());
  }

Here is the call graph for this function:

SqlNode com.mapd.calcite.parser.MapDParser.parseSql ( String  sql, final boolean  legacy_syntax, Planner  planner  ) throws SqlParseException

inlineprivate

Definition at line 918 of file MapDParser.java.

References if().

Referenced by com.mapd.calcite.parser.MapDParser.process(), and com.mapd.calcite.parser.MapDParser.processSql().

                                   {
    SqlNode parseR = null;
    try {
      parseR = planner.parse(sql);
      MAPDLOGGER.debug(" node is \n" + parseR.toString());
    } catch (SqlParseException ex) {
      MAPDLOGGER.error("failed to parse SQL '" + sql + "' \n" + ex.toString());
      throw ex;
    }

    if (!legacy_syntax) {
      return parseR;
    }

    RelDataTypeFactory typeFactory = planner.getTypeFactory();
    SqlSelect select_node = null;
    if (parseR instanceof SqlSelect) {
      select_node = (SqlSelect) parseR;
      desugar(select_node, typeFactory);
    } else if (parseR instanceof SqlOrderBy) {
      SqlOrderBy order_by_node = (SqlOrderBy) parseR;
      if (order_by_node.query instanceof SqlSelect) {
        select_node = (SqlSelect) order_by_node.query;
        SqlOrderBy new_order_by_node = desugar(select_node, order_by_node, typeFactory);
        if (new_order_by_node != null) {
          return new_order_by_node;
        }
      } else if (order_by_node.query instanceof SqlWith) {
        SqlWith old_with_node = (SqlWith) order_by_node.query;
        if (old_with_node.body instanceof SqlSelect) {
          select_node = (SqlSelect) old_with_node.body;
          desugar(select_node, typeFactory);
        }
      }
    } else if (parseR instanceof SqlWith) {
      SqlWith old_with_node = (SqlWith) parseR;
      if (old_with_node.body instanceof SqlSelect) {
        select_node = (SqlSelect) old_with_node.body;
        desugar(select_node, typeFactory);
      }
    }
    return parseR;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

Pair<String, SqlIdentifierCapturer> com.mapd.calcite.parser.MapDParser.process ( String  sql, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inline

Definition at line 379 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.captureIdentifiers(), com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.parseSql(), com.mapd.calcite.parser.MapDParser.processSql(), and run_benchmark_import.res.

                                                                                {
    final MapDPlanner planner = getPlanner(true, true);
    final SqlNode sqlNode = parseSql(sql, parserOptions.isLegacySyntax(), planner);
    String res = processSql(sqlNode, parserOptions);
    SqlIdentifierCapturer capture = captureIdentifiers(sqlNode);
    return new Pair<String, SqlIdentifierCapturer>(res, capture);
  }

Here is the call graph for this function:

String com.mapd.calcite.parser.MapDParser.processSql ( String  sql, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inline

Definition at line 401 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.callCount, com.mapd.calcite.parser.MapDParser.getPlanner(), and com.mapd.calcite.parser.MapDParser.parseSql().

Referenced by com.mapd.calcite.parser.MapDParser.process().

                                                                                {
    callCount++;

    final MapDPlanner planner = getPlanner(true, true);
    final SqlNode sqlNode = parseSql(sql, parserOptions.isLegacySyntax(), planner);

    return processSql(sqlNode, parserOptions);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

String com.mapd.calcite.parser.MapDParser.processSql ( final SqlNode  sqlNode, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inline

Definition at line 411 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.callCount, com.mapd.calcite.parser.MapDParser.convertSqlToRelNode(), com.mapd.calcite.parser.MapDParser.getPlanner(), and run_benchmark_import.res.

                                                                                {
    callCount++;

    if (sqlNode instanceof JsonSerializableDdl) {
      return ((JsonSerializableDdl) sqlNode).toJsonString();
    }

    if (sqlNode instanceof SqlDdl) {
      return sqlNode.toString();
    }

    final MapDPlanner planner = getPlanner(true, true);
    planner.advanceToValidate();

    final RelRoot sqlRel = convertSqlToRelNode(sqlNode, planner, parserOptions);
    RelNode project = sqlRel.project();

    if (parserOptions.isExplain()) {
      return RelOptUtil.toString(sqlRel.project());
    }

    String res = MapDSerializer.toString(project);

    return res;
  }

Here is the call graph for this function:

RelRoot com.mapd.calcite.parser.MapDParser.queryToRelNode ( final String  sql, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inlinepackage

Definition at line 765 of file MapDParser.java.

                                                                                {
    final MapDPlanner planner = getPlanner(true, true);
    final SqlNode sqlNode = parseSql(sql, parserOptions.isLegacySyntax(), planner);
    return convertSqlToRelNode(sqlNode, planner, parserOptions);
  }

RelRoot com.mapd.calcite.parser.MapDParser.replaceIsTrue ( final RelDataTypeFactory  typeFactory, RelRoot  root  )

inlineprivate

Definition at line 870 of file MapDParser.java.

References AND, i, and OR.

Referenced by com.mapd.calcite.parser.MapDParser.optimizeRAQuery().

                                                                                    {
    final RexShuttle callShuttle = new RexShuttle() {
      RexBuilder builder = new RexBuilder(typeFactory);

      public RexNode visitCall(RexCall call) {
        call = (RexCall) super.visitCall(call);
        if (call.getKind() == SqlKind.IS_TRUE) {
          return builder.makeCall(SqlStdOperatorTable.AND,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NOT_NULL, call.getOperands().get(0)),
                  call.getOperands().get(0));
        } else if (call.getKind() == SqlKind.IS_NOT_TRUE) {
          return builder.makeCall(SqlStdOperatorTable.OR,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NULL, call.getOperands().get(0)),
                  builder.makeCall(SqlStdOperatorTable.NOT, call.getOperands().get(0)));
        } else if (call.getKind() == SqlKind.IS_FALSE) {
          return builder.makeCall(SqlStdOperatorTable.AND,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NOT_NULL, call.getOperands().get(0)),
                  builder.makeCall(SqlStdOperatorTable.NOT, call.getOperands().get(0)));
        } else if (call.getKind() == SqlKind.IS_NOT_FALSE) {
          return builder.makeCall(SqlStdOperatorTable.OR,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NULL, call.getOperands().get(0)),
                  call.getOperands().get(0));
        }

        return call;
      }
    };

    RelNode node = root.rel.accept(new RelShuttleImpl() {
      @Override
      protected RelNode visitChild(RelNode parent, int i, RelNode child) {
        RelNode node = super.visitChild(parent, i, child);
        return node.accept(callShuttle);
      }
    });

    return new RelRoot(node,
            root.validatedRowType,
            root.kind,
            root.fields,
            root.collation,
            Collections.emptyList());
  }

Here is the caller graph for this function:

Set<String> com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers ( SqlIdentifierCapturer  capturer )

inline

Definition at line 443 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDView.getAccessedObjects(), com.mapd.calcite.parser.MapDSchema.getTable(), com.mapd.calcite.parser.MapDParser.mapdPort, com.mapd.calcite.parser.MapDParser.mapdUser, setup.name, org.apache.calcite.prepare.SqlIdentifierCapturer.selects, com.mapd.calcite.parser.MapDParser.sock_transport_properties, and test_readcsv.table.

                                                                              {
    MapDSchema schema =
            new MapDSchema(dataDir, this, mapdPort, mapdUser, sock_transport_properties);
    HashSet<String> resolved = new HashSet<>();

    for (String name : capturer.selects) {
      MapDTable table = (MapDTable) schema.getTable(name);
      if (null == table) {
        throw new RuntimeException("table/view not found: " + name);
      }

      if (table instanceof MapDView) {
        MapDView view = (MapDView) table;
        resolved.addAll(resolveSelectIdentifiers(view.getAccessedObjects()));
      } else {
        resolved.add(name);
      }
    }

    return resolved;
  }

Here is the call graph for this function:

SqlSelect com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect ( final SqlUpdate  update )

inlineprivate

Definition at line 476 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getTableName(), com.mapd.calcite.parser.MapDParser.isCorrelated(), and join().

                                                                        {
    SqlNode where = update.getCondition();

    if (update.getSourceExpressionList().size() != 1) {
      return null;
    }

    if (!(update.getSourceExpressionList().get(0) instanceof SqlSelect)) {
      return null;
    }

    final SqlSelect inner = (SqlSelect) update.getSourceExpressionList().get(0);

    if (null != inner.getGroup() || null != inner.getFetch() || null != inner.getOffset()
            || (null != inner.getOrderList() && inner.getOrderList().size() != 0)
            || (null != inner.getGroup() && inner.getGroup().size() != 0)
            || null == getTableName(inner.getFrom())) {
      return null;
    }

    if (!isCorrelated(inner)) {
      return null;
    }

    final String updateTableName = getTableName(update.getTargetTable());

    if (null != where) {
      where = where.accept(new SqlShuttle() {
        @Override
        public SqlNode visit(SqlIdentifier id) {
          if (id.isSimple()) {
            id = new SqlIdentifier(Arrays.asList(updateTableName, id.getSimple()),
                    id.getParserPosition());
          }

          return id;
        }
      });
    }

    SqlJoin join = new SqlJoin(ZERO,
            update.getTargetTable(),
            SqlLiteral.createBoolean(false, ZERO),
            SqlLiteral.createSymbol(JoinType.LEFT, ZERO),
            inner.getFrom(),
            SqlLiteral.createSymbol(JoinConditionType.ON, ZERO),
            inner.getWhere());

    SqlNode select0 = inner.getSelectList().get(0);

    boolean wrapInSingleValue = true;
    if (select0 instanceof SqlCall) {
      SqlCall selectExprCall = (SqlCall) select0;
      if (Util.isSingleValue(selectExprCall)) {
        wrapInSingleValue = false;
      }
    }

    if (wrapInSingleValue) {
      if (select0.isA(EnumSet.of(SqlKind.AS))) {
        select0 = ((SqlCall) select0).getOperandList().get(0);
      }
      select0 = new SqlBasicCall(
              SqlStdOperatorTable.SINGLE_VALUE, new SqlNode[] {select0}, ZERO);
    }

    SqlNodeList selectList = new SqlNodeList(ZERO);
    selectList.add(select0);
    selectList.add(new SqlBasicCall(SqlStdOperatorTable.AS,
            new SqlNode[] {new SqlBasicCall(
                                   new SqlUnresolvedFunction(
                                           new SqlIdentifier("OFFSET_IN_FRAGMENT", ZERO),
                                           null,
                                           null,
                                           null,
                                           null,
                                           SqlFunctionCategory.USER_DEFINED_FUNCTION),
                                   new SqlNode[0],
                                   SqlParserPos.ZERO),
                    new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO)},
            ZERO));

    SqlNodeList groupBy = new SqlNodeList(ZERO);
    groupBy.add(new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO));

    SqlSelect select = new SqlSelect(ZERO,
            null,
            selectList,
            join,
            where,
            groupBy,
            null,
            null,
            null,
            null,
            null,
            null);
    return select;
  }

Here is the call graph for this function:

RelRoot com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect ( SqlUpdate  update, MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inlineprivate

Definition at line 589 of file MapDParser.java.

References com.mapd.calcite.parser.MapDSqlOperatorTable.ARRAY_VALUE_CONSTRUCTOR, i, setup.name, and Void.

                                                                                {
    int correlatedQueriesCount[] = new int[1];
    SqlBasicVisitor<Void> correlatedQueriesCounter = new SqlBasicVisitor<Void>() {
      @Override
      public Void visit(SqlCall call) {
        if (call.isA(SCALAR)
                && ((call instanceof SqlBasicCall && call.operandCount() == 1
                            && !call.operand(0).isA(SCALAR))
                        || !(call instanceof SqlBasicCall))) {
          if (isCorrelated(call)) {
            correlatedQueriesCount[0]++;
          }
        }
        return super.visit(call);
      }
    };

    update.accept(correlatedQueriesCounter);
    if (correlatedQueriesCount[0] > 1) {
      throw new CalciteException(
              "table modifications with multiple correlated sub-queries not supported.",
              null);
    }

    boolean allowPushdownJoinCondition = false;
    SqlNodeList sourceExpression = new SqlNodeList(SqlParserPos.ZERO);
    LogicalTableModify dummyModify = getDummyUpdate(update);
    RelOptTable targetTable = dummyModify.getTable();
    RelDataType targetTableType = targetTable.getRowType();

    SqlSelect select = rewriteSimpleUpdateAsSelect(update);
    boolean applyRexCast = null == select;

    if (null == select) {
      for (int i = 0; i < update.getSourceExpressionList().size(); i++) {
        SqlNode targetColumn = update.getTargetColumnList().get(i);
        SqlNode expression = update.getSourceExpressionList().get(i);

        if (!(targetColumn instanceof SqlIdentifier)) {
          throw new RuntimeException("Unknown identifier type!");
        }
        SqlIdentifier id = (SqlIdentifier) targetColumn;
        RelDataType fieldType =
                targetTableType.getField(id.names.get(id.names.size() - 1), false, false)
                        .getType();

        if (expression.isA(ARRAY_VALUE) && null != fieldType.getComponentType()) {
          // apply a cast to all array value elements

          SqlDataTypeSpec elementType = new SqlDataTypeSpec(
                  new SqlBasicTypeNameSpec(fieldType.getComponentType().getSqlTypeName(),
                          fieldType.getPrecision(),
                          fieldType.getScale(),
                          null == fieldType.getCharset() ? null
                                                         : fieldType.getCharset().name(),
                          SqlParserPos.ZERO),
                  SqlParserPos.ZERO);
          SqlCall array_expression = (SqlCall) expression;
          ArrayList<SqlNode> values = new ArrayList<>();

          for (SqlNode value : array_expression.getOperandList()) {
            if (value.isA(EnumSet.of(SqlKind.LITERAL))) {
              SqlNode casted_value = new SqlBasicCall(SqlStdOperatorTable.CAST,
                      new SqlNode[] {value, elementType},
                      value.getParserPosition());
              values.add(casted_value);
            } else {
              values.add(value);
            }
          }

          expression = new SqlBasicCall(MapDSqlOperatorTable.ARRAY_VALUE_CONSTRUCTOR,
                  values.toArray(new SqlNode[0]),
                  expression.getParserPosition());
        }
        sourceExpression.add(expression);
      }

      sourceExpression.add(new SqlBasicCall(SqlStdOperatorTable.AS,
              new SqlNode[] {
                      new SqlBasicCall(new SqlUnresolvedFunction(
                                               new SqlIdentifier("OFFSET_IN_FRAGMENT",
                                                       SqlParserPos.ZERO),
                                               null,
                                               null,
                                               null,
                                               null,
                                               SqlFunctionCategory.USER_DEFINED_FUNCTION),
                              new SqlNode[0],
                              SqlParserPos.ZERO),
                      new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO)},
              ZERO));

      select = new SqlSelect(SqlParserPos.ZERO,
              null,
              sourceExpression,
              update.getTargetTable(),
              update.getCondition(),
              null,
              null,
              null,
              null,
              null,
              null,
              null);
    }

    MapDPlanner planner = getPlanner(true, allowPushdownJoinCondition);
    SqlNode node = null;
    try {
      node = planner.parse(select.toSqlString(CalciteSqlDialect.DEFAULT).getSql());
      node = planner.validate(node);
    } catch (Exception e) {
      MAPDLOGGER.error("Error processing UPDATE rewrite, rewritten stmt was: "
              + select.toSqlString(CalciteSqlDialect.DEFAULT).getSql());
      throw e;
    }

    RelRoot root = planner.rel(node);
    LogicalProject project = (LogicalProject) root.project();

    ArrayList<String> fields = new ArrayList<String>();
    ArrayList<RexNode> nodes = new ArrayList<RexNode>();
    final RexBuilder builder = new RexBuilder(planner.getTypeFactory());

    for (SqlNode n : update.getTargetColumnList()) {
      if (n instanceof SqlIdentifier) {
        SqlIdentifier id = (SqlIdentifier) n;
        fields.add(id.names.get(id.names.size() - 1));
      } else {
        throw new RuntimeException("Unknown identifier type!");
      }
    }

    // The magical number here when processing the projection
    // is skipping the OFFSET_IN_FRAGMENT() expression used by
    // update and delete
    int idx = 0;
    for (RexNode exp : project.getProjects()) {
      if (applyRexCast && idx + 1 < project.getProjects().size()) {
        RelDataType expectedFieldType =
                targetTableType.getField(fields.get(idx), false, false).getType();
        if (!exp.getType().equals(expectedFieldType) && !exp.isA(ARRAY_VALUE)) {
          exp = builder.makeCast(expectedFieldType, exp);
        }
      }

      nodes.add(exp);
      idx++;
    }

    ArrayList<RexNode> inputs = new ArrayList<RexNode>();
    int n = 0;
    for (int i = 0; i < fields.size(); i++) {
      inputs.add(
              new RexInputRef(n, project.getRowType().getFieldList().get(n).getType()));
      n++;
    }

    fields.add("EXPR$DELETE_OFFSET_IN_FRAGMENT");
    inputs.add(new RexInputRef(n, project.getRowType().getFieldList().get(n).getType()));

    project = project.copy(
            project.getTraitSet(), project.getInput(), nodes, project.getRowType());

    LogicalTableModify modify = LogicalTableModify.create(targetTable,
            dummyModify.getCatalogReader(),
            project,
            Operation.UPDATE,
            fields,
            inputs,
            true);
    return RelRoot.of(modify, SqlKind.UPDATE);
  }

void com.mapd.calcite.parser.MapDParser.setUser ( MapDUser  mapdUser )

inline

Definition at line 375 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.mapdUser.

                                         {
    this.mapdUser = mapdUser;
  }

void com.mapd.calcite.parser.MapDParser.updateMetaData ( String  schema, String  table  )

inline

Definition at line 1440 of file MapDParser.java.

References test_readcsv.table.

                                                          {
    MAPDLOGGER.debug("schema :" + schema + " table :" + table);
    MapDSchema mapd =
            new MapDSchema(dataDir, this, mapdPort, null, sock_transport_properties);
    mapd.updateMetaData(schema, table);
  }

Member Data Documentation

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.ARRAY_VALUE

staticprivate

Initial value:

=
          EnumSet.of(SqlKind.ARRAY_VALUE_CONSTRUCTOR)

Definition at line 150 of file MapDParser.java.

int com.mapd.calcite.parser.MapDParser.callCount = 0

private

Definition at line 158 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.processSql().

final ThreadLocal<MapDParser> com.mapd.calcite.parser.MapDParser.CURRENT_PARSER = new ThreadLocal<>()

static

Definition at line 143 of file MapDParser.java.

final String com.mapd.calcite.parser.MapDParser.dataDir

private

Definition at line 156 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.MapDParser(), com.mapd.calcite.parser.MapDParser.optimizeRAQuery(), and com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers().

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.DELETE = EnumSet.of(SqlKind.DELETE)

staticprivate

Definition at line 147 of file MapDParser.java.

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.EXISTS = EnumSet.of(SqlKind.EXISTS)

staticprivate

Definition at line 146 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.IN = EnumSet.of(SqlKind.IN)

staticprivate

Definition at line 149 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

final Context com.mapd.calcite.parser.MapDParser.MAPD_CONNECTION_CONTEXT

staticprivate

Definition at line 180 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

final Logger com.mapd.calcite.parser.MapDParser.MAPDLOGGER = LoggerFactory.getLogger(MapDParser.class)

staticpackage

Definition at line 153 of file MapDParser.java.

final int com.mapd.calcite.parser.MapDParser.mapdPort

private

Definition at line 159 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.MapDParser(), com.mapd.calcite.parser.MapDParser.optimizeRAQuery(), and com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers().

final Supplier<MapDSqlOperatorTable> com.mapd.calcite.parser.MapDParser.mapDSqlOperatorTable

private

Definition at line 155 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.isCorrelated(), and com.mapd.calcite.parser.MapDParser.MapDParser().

MapDUser com.mapd.calcite.parser.MapDParser.mapdUser

private

Definition at line 160 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.optimizeRAQuery(), com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers(), and com.mapd.calcite.parser.MapDParser.setUser().

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.SCALAR

staticprivate

Initial value:

=
          EnumSet.of(SqlKind.SCALAR_QUERY, SqlKind.SELECT)

Definition at line 144 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

SockTransportProperties com.mapd.calcite.parser.MapDParser.sock_transport_properties = null

private

Definition at line 162 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.optimizeRAQuery(), and com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers().

SqlNode com.mapd.calcite.parser.MapDParser.sqlNode_

package

Definition at line 161 of file MapDParser.java.

Map<String, Boolean> com.mapd.calcite.parser.MapDParser.SubqueryCorrMemo = new ConcurrentHashMap<>()

staticprivate

Definition at line 164 of file MapDParser.java.

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.UPDATE = EnumSet.of(SqlKind.UPDATE)

staticprivate

Definition at line 148 of file MapDParser.java.

---

The documentation for this class was generated from the following file:

• /home/jenkins-slave/workspace/core-os-doxygen/java/calcite/src/main/java/com/mapd/calcite/parser/MapDParser.java