com.mapd.calcite.parser.MapDParser Class Reference

Collaboration diagram for com.mapd.calcite.parser.MapDParser:

Public Member Functions
	MapDParser (String dataDir, final Map< String, ExtensionFunction > extSigs, int mapdPort, SockTransportProperties skT)
void	clearMemo ()
void	setUser (MapDUser mapdUser)
String	getRelAlgebra (String sql, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
MapDPlanner.CompletionResult	getCompletionHints (String sql, int cursor, List< String > visible_tables)
Set< String >	resolveSelectIdentifiers (SqlIdentifierCapturer capturer)
SqlIdentifierCapturer	captureIdentifiers (String sql, boolean legacy_syntax) throws SqlParseException
int	getCallCount ()
void	updateMetaData (String schema, String table)

Static Public Attributes
static final ThreadLocal< MapDParser >	CURRENT_PARSER = new ThreadLocal<>()

Protected Member Functions
SqlOperatorTable	createOperatorTable (final Map< String, ExtensionFunction > extSigs)
RelDataTypeSystem	createTypeSystem ()

Package Functions
RelRoot	queryToSqlNode (final String sql, final MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException

Package Attributes
SqlNode	sqlNode_

Static Package Attributes
static final Logger	MAPDLOGGER = LoggerFactory.getLogger(MapDParser.class)

Private Member Functions
MapDPlanner	getPlanner ()
boolean	isCorrelated (SqlNode expression)
MapDPlanner	getPlanner (final boolean allowSubQueryExpansion, final boolean allowPushdownJoinCondition)
String	getTableName (SqlNode node)
SqlSelect	rewriteSimpleUpdateAsSelect (final SqlUpdate update)
LogicalTableModify	getDummyUpdate (SqlUpdate update) throws SqlParseException, ValidationException, RelConversionException
RelRoot	rewriteUpdateAsSelect (SqlUpdate update, MapDParserOptions parserOptions) throws SqlParseException, ValidationException, RelConversionException
RelRoot	replaceIsTrue (final RelDataTypeFactory typeFactory, RelRoot root)
SqlNode	processSQL (String sql, final boolean legacy_syntax, Planner planner) throws SqlParseException
void	desugar (SqlSelect select_node, RelDataTypeFactory typeFactory)
SqlOrderBy	desugar (SqlSelect select_node, SqlOrderBy order_by_node, RelDataTypeFactory typeFactory)
void	desugarExpression (SqlNode node, RelDataTypeFactory typeFactory)
SqlNode	expand (final SqlNode node, final java.util.Map< String, SqlNode > id_to_expr, RelDataTypeFactory typeFactory)
SqlNodeList	expand (final SqlNodeList group_by_list, final java.util.Map< String, SqlNode > id_to_expr, RelDataTypeFactory typeFactory)
SqlNode	expandVariance (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)
SqlNode	expandVariance (final SqlParserPos pos, final SqlNode operand, boolean biased, boolean sqrt, boolean flt, RelDataTypeFactory typeFactory)
SqlNode	expandCovariance (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)
SqlNode	expandCovariance (SqlParserPos pos, final SqlNode operand0, final SqlNode operand1, boolean pop, boolean flt, RelDataTypeFactory typeFactory)
SqlNode	expandCorrelation (final SqlBasicCall proj_call, RelDataTypeFactory typeFactory)

Static Private Member Functions
static SqlNode	getUnaliasedExpression (final SqlNode node)
static boolean	isSelectStar (SqlNode node)
static SqlSelect	getSelectChild (SqlNode node)

Private Attributes
final Map< String, ExtensionFunction >	extSigs
final String	dataDir
int	callCount = 0
final int	mapdPort
MapDUser	mapdUser
SockTransportProperties	sock_transport_properties = null

Static Private Attributes
static final EnumSet< SqlKind >	SCALAR
static final EnumSet< SqlKind >	EXISTS = EnumSet.of(SqlKind.EXISTS)
static final EnumSet< SqlKind >	DELETE = EnumSet.of(SqlKind.DELETE)
static final EnumSet< SqlKind >	UPDATE = EnumSet.of(SqlKind.UPDATE)
static final EnumSet< SqlKind >	IN = EnumSet.of(SqlKind.IN)
static final EnumSet< SqlKind >	ARRAY_VALUE
static Map< String, Boolean >	SubqueryCorrMemo = new ConcurrentHashMap<>()
static final Context	MAPD_CONNECTION_CONTEXT

Detailed Description

Author

michael

Definition at line 114 of file MapDParser.java.

Constructor & Destructor Documentation

MapDParser()

com.mapd.calcite.parser.MapDParser.MapDParser ( String  dataDir, final Map< String, ExtensionFunction >  extSigs, int  mapdPort, SockTransportProperties  skT  )

inline

Definition at line 142 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDParser.extSigs, and com.mapd.calcite.parser.MapDParser.mapdPort.

Referenced by com.mapd.calcite.parser.MapDParser.isCorrelated().

                                       {
    System.setProperty(
            "saffron.default.charset", ConversionUtil.NATIVE_UTF16_CHARSET_NAME);
    System.setProperty(
            "saffron.default.nationalcharset", ConversionUtil.NATIVE_UTF16_CHARSET_NAME);
    System.setProperty("saffron.default.collation.name",
            ConversionUtil.NATIVE_UTF16_CHARSET_NAME + "$en_US");
    this.dataDir = dataDir;
    this.extSigs = extSigs;
    this.mapdPort = mapdPort;
    this.sock_transport_properties = skT;
  }

Here is the caller graph for this function:

Member Function Documentation

captureIdentifiers()

SqlIdentifierCapturer com.mapd.calcite.parser.MapDParser.captureIdentifiers ( String  sql, boolean  legacy_syntax  ) throws SqlParseException

inline

Definition at line 1334 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.processSQL(), and org.apache.calcite.prepare.SqlIdentifierCapturer.scan().

Referenced by com.mapd.calcite.parser.MapDView.MapDView(), com.mapd.parser.server.CalciteServerHandler.process(), and com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                   {
    try {
      Planner planner = getPlanner();
      SqlNode node = processSQL(sql, legacy_syntax, planner);
      SqlIdentifierCapturer capturer = new SqlIdentifierCapturer();
      capturer.scan(node);
      return capturer;
    } catch (Exception | Error e) {
      MAPDLOGGER.error("Error parsing sql: " + sql, e);
      return new SqlIdentifierCapturer();
    }
  }

Here is the call graph for this function:

Here is the caller graph for this function:

clearMemo()

void com.mapd.calcite.parser.MapDParser.clearMemo ( )

inline

Definition at line 158 of file MapDParser.java.

Referenced by com.mapd.parser.server.CalciteServerHandler.process().

                          {
    SubqueryCorrMemo.clear();
  }

Here is the caller graph for this function:

createOperatorTable()

SqlOperatorTable com.mapd.calcite.parser.MapDParser.createOperatorTable ( final Map< String, ExtensionFunction >  extSigs )

inlineprotected

Creates an operator table.

Parameters

extSigs

Returns

New operator table

Definition at line 1324 of file MapDParser.java.

References com.mapd.calcite.parser.MapDSqlOperatorTable.addUDF().

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

                                                        {
    final MapDSqlOperatorTable tempOpTab =
            new MapDSqlOperatorTable(SqlStdOperatorTable.instance());
    // MAT 11 Nov 2015
    // Example of how to add custom function
    MapDSqlOperatorTable.addUDF(tempOpTab, extSigs);
    return tempOpTab;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

createTypeSystem()

RelDataTypeSystem com.mapd.calcite.parser.MapDParser.createTypeSystem ( )

inlineprotected

Definition at line 1359 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner().

                                                 {
    final MapDTypeSystem typeSystem = new MapDTypeSystem();
    return typeSystem;
  }

Here is the caller graph for this function:

desugar() [1/2]

void com.mapd.calcite.parser.MapDParser.desugar ( SqlSelect  select_node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 927 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.desugarExpression(), com.mapd.calcite.parser.MapDParser.expand(), and com.mapd.calcite.parser.MapDParser.processSQL().

                                                                              {
    desugar(select_node, null, typeFactory);
  }

Here is the caller graph for this function:

desugar() [2/2]

SqlOrderBy com.mapd.calcite.parser.MapDParser.desugar ( SqlSelect  select_node, SqlOrderBy  order_by_node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 931 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.desugarExpression(), com.mapd.calcite.parser.MapDParser.expand(), and Experimental.String.

                                          {
    MAPDLOGGER.debug("desugar: before: " + select_node.toString());
    desugarExpression(select_node.getFrom(), typeFactory);
    desugarExpression(select_node.getWhere(), typeFactory);
    SqlNodeList select_list = select_node.getSelectList();
    SqlNodeList new_select_list = new SqlNodeList(select_list.getParserPosition());
    java.util.Map<String, SqlNode> id_to_expr = new java.util.HashMap<String, SqlNode>();
    for (SqlNode proj : select_list) {
      if (!(proj instanceof SqlBasicCall)) {
        new_select_list.add(proj);
        continue;
      }
      SqlBasicCall proj_call = (SqlBasicCall) proj;
      new_select_list.add(expand(proj_call, id_to_expr, typeFactory));
    }
    select_node.setSelectList(new_select_list);
    SqlNodeList group_by_list = select_node.getGroup();
    if (group_by_list != null) {
      select_node.setGroupBy(expand(group_by_list, id_to_expr, typeFactory));
    }
    SqlNode having = select_node.getHaving();
    if (having != null) {
      expand(having, id_to_expr, typeFactory);
    }
    SqlOrderBy new_order_by_node = null;
    if (order_by_node != null && order_by_node.orderList != null
            && order_by_node.orderList.size() > 0) {
      SqlNodeList new_order_by_list =
              expand(order_by_node.orderList, id_to_expr, typeFactory);
      new_order_by_node = new SqlOrderBy(order_by_node.getParserPosition(),
              select_node,
              new_order_by_list,
              order_by_node.offset,
              order_by_node.fetch);
    }

    MAPDLOGGER.debug("desugar:  after: " + select_node.toString());
    return new_order_by_node;
  }

Here is the call graph for this function:

desugarExpression()

void com.mapd.calcite.parser.MapDParser.desugarExpression ( SqlNode  node, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 973 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.desugar().

Referenced by com.mapd.calcite.parser.MapDParser.desugar().

                                                                               {
    if (node instanceof SqlSelect) {
      desugar((SqlSelect) node, typeFactory);
      return;
    }
    if (!(node instanceof SqlBasicCall)) {
      return;
    }
    SqlBasicCall basic_call = (SqlBasicCall) node;
    for (SqlNode operator : basic_call.getOperands()) {
      if (operator instanceof SqlOrderBy) {
        desugarExpression(((SqlOrderBy) operator).query, typeFactory);
      } else {
        desugarExpression(operator, typeFactory);
      }
    }
  }

Here is the call graph for this function:

Here is the caller graph for this function:

expand() [1/2]

SqlNode com.mapd.calcite.parser.MapDParser.expand ( final SqlNode  node, final java.util.Map< String, SqlNode >  id_to_expr, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 991 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.desugar(), com.mapd.calcite.parser.MapDParser.expandCorrelation(), com.mapd.calcite.parser.MapDParser.expandCovariance(), and com.mapd.calcite.parser.MapDParser.expandVariance().

Referenced by com.mapd.calcite.parser.MapDParser.desugar(), and com.mapd.calcite.parser.MapDParser.expand().

                                          {
    MAPDLOGGER.debug("expand: " + node.toString());
    if (node instanceof SqlBasicCall) {
      SqlBasicCall node_call = (SqlBasicCall) node;
      SqlNode[] operands = node_call.getOperands();
      for (int i = 0; i < operands.length; ++i) {
        node_call.setOperand(i, expand(operands[i], id_to_expr, typeFactory));
      }
      SqlNode expanded_variance = expandVariance(node_call, typeFactory);
      if (expanded_variance != null) {
        return expanded_variance;
      }
      SqlNode expanded_covariance = expandCovariance(node_call, typeFactory);
      if (expanded_covariance != null) {
        return expanded_covariance;
      }
      SqlNode expanded_correlation = expandCorrelation(node_call, typeFactory);
      if (expanded_correlation != null) {
        return expanded_correlation;
      }
    }
    if (node instanceof SqlSelect) {
      SqlSelect select_node = (SqlSelect) node;
      desugar(select_node, typeFactory);
    }
    return node;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

expand() [2/2]

SqlNodeList com.mapd.calcite.parser.MapDParser.expand ( final SqlNodeList  group_by_list, final java.util.Map< String, SqlNode >  id_to_expr, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1021 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.expand().

                                          {
    SqlNodeList new_group_by_list = new SqlNodeList(new SqlParserPos(-1, -1));
    for (SqlNode group_by : group_by_list) {
      if (!(group_by instanceof SqlIdentifier)) {
        new_group_by_list.add(expand(group_by, id_to_expr, typeFactory));
        continue;
      }
      SqlIdentifier group_by_id = ((SqlIdentifier) group_by);
      if (id_to_expr.containsKey(group_by_id.toString())) {
        new_group_by_list.add(id_to_expr.get(group_by_id.toString()));
      } else {
        new_group_by_list.add(group_by);
      }
    }
    return new_group_by_list;
  }

Here is the call graph for this function:

expandCorrelation()

SqlNode com.mapd.calcite.parser.MapDParser.expandCorrelation ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1271 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.expandCovariance(), and com.mapd.calcite.parser.MapDParser.expandVariance().

Referenced by com.mapd.calcite.parser.MapDParser.expand().

                                                                        {
    // Expand correlation coefficient
    if (proj_call.operandCount() != 2) {
      return null;
    }
    boolean flt;
    if (proj_call.getOperator().isName("CORR", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("CORRELATION")) {
      // expand correlation coefficient
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("CORR_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("CORRELATION_FLOAT")) {
      // expand correlation coefficient
      flt = true;
    } else {
      return null;
    }
    // corr(x, y) ==> (avg(x * y) - avg(x) * avg(y)) / (stddev_pop(x) *
    // stddev_pop(y))
    // ==> covar_pop(x, y) / (stddev_pop(x) * stddev_pop(y))
    final SqlNode operand0 = proj_call.operand(0);
    final SqlNode operand1 = proj_call.operand(1);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode covariance =
            expandCovariance(pos, operand0, operand1, true, flt, typeFactory);
    SqlNode stddev0 = expandVariance(pos, operand0, true, true, flt, typeFactory);
    SqlNode stddev1 = expandVariance(pos, operand1, true, true, flt, typeFactory);
    final SqlNode mulStddev =
            SqlStdOperatorTable.MULTIPLY.createCall(pos, stddev0, stddev1);
    final SqlNumericLiteral zero = SqlLiteral.createExactNumeric("0.0", pos);
    final SqlNode mulStddevEqZero =
            SqlStdOperatorTable.EQUALS.createCall(pos, mulStddev, zero);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    SqlNodeList whenList1 = new SqlNodeList(pos);
    SqlNodeList thenList1 = new SqlNodeList(pos);
    whenList1.add(mulStddevEqZero);
    thenList1.add(nul);
    final SqlNode denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList1, thenList1, mulStddev);
    final SqlNode expanded_proj_call =
            SqlStdOperatorTable.DIVIDE.createCall(pos, covariance, denominator);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

expandCovariance() [1/2]

SqlNode com.mapd.calcite.parser.MapDParser.expandCovariance ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1179 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.expand(), and com.mapd.calcite.parser.MapDParser.expandCorrelation().

                                                                        {
    // Expand covariance aggregates
    if (proj_call.operandCount() != 2) {
      return null;
    }
    boolean pop;
    boolean flt;
    if (proj_call.getOperator().isName("COVAR_POP", false)) {
      pop = true;
      flt = false;
    } else if (proj_call.getOperator().isName("COVAR_SAMP", false)) {
      pop = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("COVAR_POP_FLOAT")) {
      pop = true;
      flt = true;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("COVAR_SAMP_FLOAT")) {
      pop = false;
      flt = true;
    } else {
      return null;
    }
    final SqlNode operand0 = proj_call.operand(0);
    final SqlNode operand1 = proj_call.operand(1);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode expanded_proj_call =
            expandCovariance(pos, operand0, operand1, pop, flt, typeFactory);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

Here is the caller graph for this function:

expandCovariance() [2/2]

SqlNode com.mapd.calcite.parser.MapDParser.expandCovariance ( SqlParserPos  pos, final SqlNode  operand0, final SqlNode  operand1, boolean  pop, boolean  flt, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1212 of file MapDParser.java.

                                          {
    // covar_pop(x, y) ==> avg(x * y) - avg(x) * avg(y)
    // covar_samp(x, y) ==> (sum(x * y) - sum(x) * avg(y))
    // ((case count(x) when 1 then NULL else count(x) - 1 end))
    final SqlNode arg0 = SqlStdOperatorTable.CAST.createCall(operand0.getParserPosition(),
            operand0,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode arg1 = SqlStdOperatorTable.CAST.createCall(operand1.getParserPosition(),
            operand1,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode mulArg = SqlStdOperatorTable.MULTIPLY.createCall(pos, arg0, arg1);
    final SqlNode avgArg1 = SqlStdOperatorTable.AVG.createCall(pos, arg1);
    if (pop) {
      final SqlNode avgMulArg = SqlStdOperatorTable.AVG.createCall(pos, mulArg);
      final SqlNode avgArg0 = SqlStdOperatorTable.AVG.createCall(pos, arg0);
      final SqlNode mulAvgAvg =
              SqlStdOperatorTable.MULTIPLY.createCall(pos, avgArg0, avgArg1);
      final SqlNode covarPop =
              SqlStdOperatorTable.MINUS.createCall(pos, avgMulArg, mulAvgAvg);
      return SqlStdOperatorTable.CAST.createCall(pos,
              covarPop,
              SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                      flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    }
    final SqlNode sumMulArg = SqlStdOperatorTable.SUM.createCall(pos, mulArg);
    final SqlNode sumArg0 = SqlStdOperatorTable.SUM.createCall(pos, arg0);
    final SqlNode mulSumAvg =
            SqlStdOperatorTable.MULTIPLY.createCall(pos, sumArg0, avgArg1);
    final SqlNode sub = SqlStdOperatorTable.MINUS.createCall(pos, sumMulArg, mulSumAvg);
    final SqlNode count = SqlStdOperatorTable.COUNT.createCall(pos, operand0);
    final SqlNumericLiteral one = SqlLiteral.createExactNumeric("1", pos);
    final SqlNode countEqOne = SqlStdOperatorTable.EQUALS.createCall(pos, count, one);
    final SqlNode countMinusOne = SqlStdOperatorTable.MINUS.createCall(pos, count, one);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    SqlNodeList whenList1 = new SqlNodeList(pos);
    SqlNodeList thenList1 = new SqlNodeList(pos);
    whenList1.add(countEqOne);
    thenList1.add(nul);
    final SqlNode int_denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList1, thenList1, countMinusOne);
    final SqlNode denominator = SqlStdOperatorTable.CAST.createCall(pos,
            int_denominator,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode covarSamp =
            SqlStdOperatorTable.DIVIDE.createCall(pos, sub, denominator);
    return SqlStdOperatorTable.CAST.createCall(pos,
            covarSamp,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
  }

expandVariance() [1/2]

SqlNode com.mapd.calcite.parser.MapDParser.expandVariance ( final SqlBasicCall  proj_call, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1040 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.expand(), and com.mapd.calcite.parser.MapDParser.expandCorrelation().

                                                                        {
    // Expand variance aggregates that are not supported natively
    if (proj_call.operandCount() != 1) {
      return null;
    }
    boolean biased;
    boolean sqrt;
    boolean flt;
    if (proj_call.getOperator().isName("STDDEV_POP", false)) {
      biased = true;
      sqrt = true;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_POP_FLOAT")) {
      biased = true;
      sqrt = true;
      flt = true;
    } else if (proj_call.getOperator().isName("STDDEV_SAMP", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("STDDEV")) {
      biased = false;
      sqrt = true;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_SAMP_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("STDDEV_FLOAT")) {
      biased = false;
      sqrt = true;
      flt = true;
    } else if (proj_call.getOperator().isName("VAR_POP", false)) {
      biased = true;
      sqrt = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("VAR_POP_FLOAT")) {
      biased = true;
      sqrt = false;
      flt = true;
    } else if (proj_call.getOperator().isName("VAR_SAMP", false)
            || proj_call.getOperator().getName().equalsIgnoreCase("VARIANCE")) {
      biased = false;
      sqrt = false;
      flt = false;
    } else if (proj_call.getOperator().getName().equalsIgnoreCase("VAR_SAMP_FLOAT")
            || proj_call.getOperator().getName().equalsIgnoreCase("VARIANCE_FLOAT")) {
      biased = false;
      sqrt = false;
      flt = true;
    } else {
      return null;
    }
    final SqlNode operand = proj_call.operand(0);
    final SqlParserPos pos = proj_call.getParserPosition();
    SqlNode expanded_proj_call =
            expandVariance(pos, operand, biased, sqrt, flt, typeFactory);
    MAPDLOGGER.debug("Expanded select_list SqlCall: " + proj_call.toString());
    MAPDLOGGER.debug("to : " + expanded_proj_call.toString());
    return expanded_proj_call;
  }

Here is the caller graph for this function:

expandVariance() [2/2]

SqlNode com.mapd.calcite.parser.MapDParser.expandVariance ( final SqlParserPos  pos, final SqlNode  operand, boolean  biased, boolean  sqrt, boolean  flt, RelDataTypeFactory  typeFactory  )

inlineprivate

Definition at line 1097 of file MapDParser.java.

References run_benchmark_import.result.

                                          {
    // stddev_pop(x) ==>
    // power(
    // (sum(x * x) - sum(x) * sum(x) / (case count(x) when 0 then NULL else count(x)
    // end)) / (case count(x) when 0 then NULL else count(x) end), .5)
    //
    // stddev_samp(x) ==>
    // power(
    // (sum(x * x) - sum(x) * sum(x) / (case count(x) when 0 then NULL else count(x)
    // )) / ((case count(x) when 1 then NULL else count(x) - 1 end)), .5)
    //
    // var_pop(x) ==>
    // (sum(x * x) - sum(x) * sum(x) / ((case count(x) when 0 then NULL else
    // count(x)
    // end))) / ((case count(x) when 0 then NULL else count(x) end))
    //
    // var_samp(x) ==>
    // (sum(x * x) - sum(x) * sum(x) / ((case count(x) when 0 then NULL else
    // count(x)
    // end))) / ((case count(x) when 1 then NULL else count(x) - 1 end))
    //
    final SqlNode arg = SqlStdOperatorTable.CAST.createCall(pos,
            operand,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode argSquared = SqlStdOperatorTable.MULTIPLY.createCall(pos, arg, arg);
    final SqlNode sumArgSquared = SqlStdOperatorTable.SUM.createCall(pos, argSquared);
    final SqlNode sum = SqlStdOperatorTable.SUM.createCall(pos, arg);
    final SqlNode sumSquared = SqlStdOperatorTable.MULTIPLY.createCall(pos, sum, sum);
    final SqlNode count = SqlStdOperatorTable.COUNT.createCall(pos, arg);
    final SqlLiteral nul = SqlLiteral.createNull(pos);
    final SqlNumericLiteral zero = SqlLiteral.createExactNumeric("0", pos);
    final SqlNode countEqZero = SqlStdOperatorTable.EQUALS.createCall(pos, count, zero);
    SqlNodeList whenList = new SqlNodeList(pos);
    SqlNodeList thenList = new SqlNodeList(pos);
    whenList.add(countEqZero);
    thenList.add(nul);
    final SqlNode int_denominator = SqlStdOperatorTable.CASE.createCall(
            null, pos, null, whenList, thenList, count);
    final SqlNode denominator = SqlStdOperatorTable.CAST.createCall(pos,
            int_denominator,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    final SqlNode avgSumSquared =
            SqlStdOperatorTable.DIVIDE.createCall(pos, sumSquared, denominator);
    final SqlNode diff =
            SqlStdOperatorTable.MINUS.createCall(pos, sumArgSquared, avgSumSquared);
    final SqlNode denominator1;
    if (biased) {
      denominator1 = denominator;
    } else {
      final SqlNumericLiteral one = SqlLiteral.createExactNumeric("1", pos);
      final SqlNode countEqOne = SqlStdOperatorTable.EQUALS.createCall(pos, count, one);
      final SqlNode countMinusOne = SqlStdOperatorTable.MINUS.createCall(pos, count, one);
      SqlNodeList whenList1 = new SqlNodeList(pos);
      SqlNodeList thenList1 = new SqlNodeList(pos);
      whenList1.add(countEqOne);
      thenList1.add(nul);
      final SqlNode int_denominator1 = SqlStdOperatorTable.CASE.createCall(
              null, pos, null, whenList1, thenList1, countMinusOne);
      denominator1 = SqlStdOperatorTable.CAST.createCall(pos,
              int_denominator1,
              SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                      flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
    }
    final SqlNode div = SqlStdOperatorTable.DIVIDE.createCall(pos, diff, denominator1);
    SqlNode result = div;
    if (sqrt) {
      final SqlNumericLiteral half = SqlLiteral.createExactNumeric("0.5", pos);
      result = SqlStdOperatorTable.POWER.createCall(pos, div, half);
    }
    return SqlStdOperatorTable.CAST.createCall(pos,
            result,
            SqlTypeUtil.convertTypeToSpec(typeFactory.createSqlType(
                    flt ? SqlTypeName.FLOAT : SqlTypeName.DOUBLE)));
  }

getCallCount()

int com.mapd.calcite.parser.MapDParser.getCallCount ( )

inline

Definition at line 1348 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.callCount.

Referenced by com.mapd.parser.server.CalciteParserFactory.validateObject().

                            {
    return callCount;
  }

Here is the caller graph for this function:

getCompletionHints()

MapDPlanner.CompletionResult com.mapd.calcite.parser.MapDParser.getCompletionHints ( String  sql, int  cursor, List< String >  visible_tables  )

inline

Definition at line 363 of file MapDParser.java.

References org.apache.calcite.prepare.MapDPlanner.getCompletionHints(), and com.mapd.calcite.parser.MapDParser.getPlanner().

Referenced by com.mapd.parser.server.CalciteServerHandler.getCompletionHints().

                                                               {
    return getPlanner().getCompletionHints(sql, cursor, visible_tables);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

getDummyUpdate()

LogicalTableModify com.mapd.calcite.parser.MapDParser.getDummyUpdate ( SqlUpdate  update ) throws SqlParseException, ValidationException, RelConversionException

inlineprivate

Definition at line 488 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getPlanner(), org.apache.calcite.prepare.MapDPlanner.rel(), and Experimental.String.

Referenced by com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect().

                                                                                {
    SqlIdentifier targetTable = (SqlIdentifier) update.getTargetTable();
    String targetTableName = targetTable.names.get(targetTable.names.size() - 1);
    MapDPlanner planner = getPlanner();
    String dummySql = "DELETE FROM " + targetTableName;
    SqlNode dummyNode = planner.parse(dummySql);
    dummyNode = planner.validate(dummyNode);
    RelRoot dummyRoot = planner.rel(dummyNode);
    LogicalTableModify dummyModify = (LogicalTableModify) dummyRoot.rel;
    return dummyModify;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

getPlanner() [1/2]

MapDPlanner com.mapd.calcite.parser.MapDParser.getPlanner ( )

inlineprivate

Definition at line 189 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.captureIdentifiers(), com.mapd.calcite.parser.MapDParser.getCompletionHints(), com.mapd.calcite.parser.MapDParser.getDummyUpdate(), com.mapd.calcite.parser.MapDParser.queryToSqlNode(), and com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect().

                                   {
    return getPlanner(true, true);
  }

Here is the caller graph for this function:

getPlanner() [2/2]

MapDPlanner com.mapd.calcite.parser.MapDParser.getPlanner ( final boolean  allowSubQueryExpansion, final boolean  allowPushdownJoinCondition  )

inlineprivate

Definition at line 215 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.createOperatorTable(), com.mapd.calcite.parser.MapDParser.createTypeSystem(), com.mapd.calcite.parser.MapDUser.getDB(), Integer, com.mapd.calcite.parser.MapDParser.isCorrelated(), and run_benchmark_import.type.

                                                    {
    final MapDSchema mapd =
            new MapDSchema(dataDir, this, mapdPort, mapdUser, sock_transport_properties);
    final SchemaPlus rootSchema = Frameworks.createRootSchema(true);

    BiPredicate<SqlNode, SqlNode> expandPredicate = new BiPredicate<SqlNode, SqlNode>() {
      @Override
      public boolean test(SqlNode root, SqlNode expression) {
        if (!allowSubQueryExpansion) {
          return false;
        }

        // special handling of sub-queries
        if (expression.isA(SCALAR) || expression.isA(EXISTS) || expression.isA(IN)) {
          // only expand if it is correlated.

          if (expression.isA(EXISTS)) {
            // always expand subquery by EXISTS clause
            return true;
          }

          if (expression.isA(IN)) {
            // expand subquery by IN clause
            // but correlated subquery by NOT_IN clause is not available
            // currently due to a lack of supporting in Calcite
            boolean found_expression = false;
            if (expression instanceof SqlCall) {
              SqlCall call = (SqlCall) expression;
              if (call.getOperandList().size() == 2) {
                // if IN clause is correlated, its second operand of corresponding
                // expression is SELECT clause which indicates a correlated subquery.
                // Here, an expression "f.val IN (SELECT ...)" has two operands.
                // Since we have interest in its subquery, so try to check whether
                // the second operand, i.e., call.getOperandList().get(1)
                // is a type of SqlSelect and also is correlated.
                // Note that the second operand of non-correlated IN clause
                // does not have SqlSelect as its second operand
                if (call.getOperandList().get(1) instanceof SqlSelect) {
                  expression = call.getOperandList().get(1);
                  SqlSelect select_call = (SqlSelect) expression;
                  if (select_call.hasWhere()) {
                    found_expression = true;
                  }
                }
              }
            }
            if (!found_expression) {
              return false;
            }
          }

          if (isCorrelated(expression)) {
            SqlSelect select = null;
            if (expression instanceof SqlCall) {
              SqlCall call = (SqlCall) expression;
              if (call.getOperator().equals(SqlStdOperatorTable.SCALAR_QUERY)) {
                expression = call.getOperandList().get(0);
              }
            }

            if (expression instanceof SqlSelect) {
              select = (SqlSelect) expression;
            }

            if (null != select) {
              if (null != select.getFetch() || null != select.getOffset()
                      || (null != select.getOrderList()
                              && select.getOrderList().size() != 0)) {
                throw new CalciteException(
                        "Correlated sub-queries with ordering not supported.", null);
              }
            }
            return true;
          }
        }

        // per default we do not want to expand
        return false;
      }
    };

    BiPredicate<SqlNode, Join> pushdownJoinPredicate = new BiPredicate<SqlNode, Join>() {
      @Override
      public boolean test(SqlNode t, Join u) {
        if (!allowPushdownJoinCondition) {
          return false;
        }

        return !hasGeoColumns(u.getRowType());
      }

      private boolean hasGeoColumns(RelDataType type) {
        for (RelDataTypeField f : type.getFieldList()) {
          if ("any".equalsIgnoreCase(f.getType().getFamily().toString())) {
            // any indicates geo types at the moment
            return true;
          }
        }

        return false;
      }
    };

    final FrameworkConfig config =
            Frameworks.newConfigBuilder()
                    .defaultSchema(rootSchema.add(mapdUser.getDB(), mapd))
                    .operatorTable(createOperatorTable(extSigs))
                    .parserConfig(SqlParser.configBuilder()
                                          .setConformance(SqlConformanceEnum.LENIENT)
                                          .setUnquotedCasing(Casing.UNCHANGED)
                                          .setCaseSensitive(false)
                                          .build())
                    .sqlToRelConverterConfig(
                            SqlToRelConverter
                                    .configBuilder()
                                    // enable sub-query expansion (de-correlation)
                                    .withExpandPredicate(expandPredicate)
                                    // allow as many as possible IN operator values
                                    .withInSubQueryThreshold(Integer.MAX_VALUE)
                                    .withPushdownJoinCondition(pushdownJoinPredicate)
                                    .build())
                    .typeSystem(createTypeSystem())
                    .context(MAPD_CONNECTION_CONTEXT)
                    .build();
    return new MapDPlanner(config);
  }

Here is the call graph for this function:

getRelAlgebra()

String com.mapd.calcite.parser.MapDParser.getRelAlgebra ( String  sql, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inline

Definition at line 347 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.queryToSqlNode(), run_benchmark_import.res, Experimental.String, and com.mapd.calcite.parser.MapDSerializer.toString().

Referenced by com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.planner.tester.main(), and com.mapd.parser.server.CalciteServerHandler.process().

                                                                                {
    callCount++;
    final RelRoot sqlRel = queryToSqlNode(sql, parserOptions);

    RelNode project = sqlRel.project();

    if (parserOptions.isExplain()) {
      return RelOptUtil.toString(sqlRel.project());
    }

    String res = MapDSerializer.toString(project);

    return res;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

getSelectChild()

static SqlSelect com.mapd.calcite.parser.MapDParser.getSelectChild ( SqlNode  node )

inlinestaticprivate

Definition at line 882 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.isSelectStar(), and com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                                        {
    if (node instanceof SqlSelect) {
      return (SqlSelect) node;
    }
    if (node instanceof SqlOrderBy) {
      SqlOrderBy order_by_node = (SqlOrderBy) node;
      if (order_by_node.query instanceof SqlSelect) {
        return (SqlSelect) order_by_node.query;
      }
    }
    return null;
  }

Here is the caller graph for this function:

getTableName()

String com.mapd.calcite.parser.MapDParser.getTableName ( SqlNode  node )

inlineprivate

Definition at line 390 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect().

                                            {
    if (node.isA(EnumSet.of(SqlKind.AS))) {
      node = ((SqlCall) node).getOperandList().get(1);
    }
    if (node instanceof SqlIdentifier) {
      SqlIdentifier id = (SqlIdentifier) node;
      return id.names.get(id.names.size() - 1);
    }
    return null;
  }

Here is the caller graph for this function:

getUnaliasedExpression()

static SqlNode com.mapd.calcite.parser.MapDParser.getUnaliasedExpression ( final SqlNode  node )

inlinestaticprivate

Definition at line 853 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.isSelectStar(), and com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                                                    {
    if (node instanceof SqlBasicCall
            && ((SqlBasicCall) node).getOperator() instanceof SqlAsOperator) {
      SqlNode[] operands = ((SqlBasicCall) node).getOperands();
      return operands[0];
    }
    return node;
  }

Here is the caller graph for this function:

isCorrelated()

boolean com.mapd.calcite.parser.MapDParser.isCorrelated ( SqlNode  expression )

inlineprivate

Definition at line 193 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getRelAlgebra(), com.mapd.calcite.parser.MapDParser.MapDParser(), run_benchmark_import.parser, com.mapd.calcite.parser.MapDParser.setUser(), and Experimental.String.

Referenced by com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect(), and com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect().

                                                   {
    String queryString = expression.toSqlString(SqlDialect.CALCITE).getSql();
    Boolean isCorrelatedSubquery = SubqueryCorrMemo.get(queryString);
    if (null != isCorrelatedSubquery) {
      return isCorrelatedSubquery;
    }

    try {
      MapDParser parser =
              new MapDParser(dataDir, extSigs, mapdPort, sock_transport_properties);
      MapDParserOptions options = new MapDParserOptions();
      parser.setUser(mapdUser);
      parser.getRelAlgebra(expression.toSqlString(SqlDialect.CALCITE).getSql(), options);
    } catch (Exception e) {
      // if we are not able to parse, then assume correlated
      SubqueryCorrMemo.put(queryString, true);
      return true;
    }
    SubqueryCorrMemo.put(queryString, false);
    return false;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

isSelectStar()

static boolean com.mapd.calcite.parser.MapDParser.isSelectStar ( SqlNode  node )

inlinestaticprivate

Definition at line 862 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getSelectChild(), and com.mapd.calcite.parser.MapDParser.getUnaliasedExpression().

Referenced by com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                                    {
    SqlSelect select_node = getSelectChild(node);
    if (select_node == null) {
      return false;
    }
    SqlNode from = getUnaliasedExpression(select_node.getFrom());
    if (from instanceof SqlCall) {
      return false;
    }
    SqlNodeList proj_exprs = select_node.getSelectList();
    if (proj_exprs.size() != 1) {
      return false;
    }
    SqlNode proj_expr = proj_exprs.get(0);
    if (!(proj_expr instanceof SqlIdentifier)) {
      return false;
    }
    return ((SqlIdentifier) proj_expr).isStar();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

processSQL()

SqlNode com.mapd.calcite.parser.MapDParser.processSQL ( String  sql, final boolean  legacy_syntax, Planner  planner  ) throws SqlParseException

inlineprivate

Definition at line 895 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.desugar().

Referenced by com.mapd.calcite.parser.MapDParser.captureIdentifiers(), and com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                   {
    SqlNode parseR = null;
    try {
      parseR = planner.parse(sql);
      MAPDLOGGER.debug(" node is \n" + parseR.toString());
    } catch (SqlParseException ex) {
      MAPDLOGGER.error("failed to process SQL '" + sql + "' \n" + ex.toString());
      throw ex;
    }

    if (!legacy_syntax) {
      return parseR;
    }
    RelDataTypeFactory typeFactory = planner.getTypeFactory();
    SqlSelect select_node = null;
    if (parseR instanceof SqlSelect) {
      select_node = (SqlSelect) parseR;
      desugar(select_node, typeFactory);
    } else if (parseR instanceof SqlOrderBy) {
      SqlOrderBy order_by_node = (SqlOrderBy) parseR;
      if (order_by_node.query instanceof SqlSelect) {
        select_node = (SqlSelect) order_by_node.query;
        SqlOrderBy new_order_by_node = desugar(select_node, order_by_node, typeFactory);
        if (new_order_by_node != null) {
          return new_order_by_node;
        }
      }
    }
    return parseR;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

queryToSqlNode()

RelRoot com.mapd.calcite.parser.MapDParser.queryToSqlNode ( final String  sql, final MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inlinepackage

Definition at line 665 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.captureIdentifiers(), com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.getSelectChild(), com.mapd.calcite.parser.MapDSchema.getTable(), com.mapd.calcite.parser.MapDParser.getUnaliasedExpression(), com.mapd.calcite.parser.ProjectProjectRemoveRule.INSTANCE, com.mapd.calcite.parser.MapDParser.isSelectStar(), com.mapd.calcite.parser.MapDParser.processSQL(), com.mapd.calcite.parser.MapDParser.replaceIsTrue(), com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect(), org.apache.calcite.prepare.SqlIdentifierCapturer.selects, and Experimental.String.

Referenced by com.mapd.calcite.parser.MapDParser.getRelAlgebra(), and com.mapd.calcite.parser.MapDView.MapDView().

                                                                                {
    boolean allowCorrelatedSubQueryExpansion = true;
    boolean allowPushdownJoinCondition = true;
    boolean patchUpdateToDelete = false;
    MapDPlanner planner =
            getPlanner(allowCorrelatedSubQueryExpansion, allowPushdownJoinCondition);

    SqlNode node = processSQL(sql, parserOptions.isLegacySyntax(), planner);

    if (node.isA(DELETE)) {
      SqlDelete sqlDelete = (SqlDelete) node;
      node = new SqlUpdate(node.getParserPosition(),
              sqlDelete.getTargetTable(),
              SqlNodeList.EMPTY,
              SqlNodeList.EMPTY,
              sqlDelete.getCondition(),
              sqlDelete.getSourceSelect(),
              sqlDelete.getAlias());

      patchUpdateToDelete = true;
    }

    if (node.isA(UPDATE)) {
      SqlUpdate update = (SqlUpdate) node;
      update = (SqlUpdate) planner.validate(update);
      RelRoot root = rewriteUpdateAsSelect(update, parserOptions);

      if (patchUpdateToDelete) {
        LogicalTableModify modify = (LogicalTableModify) root.rel;

        try {
          Field f = TableModify.class.getDeclaredField("operation");
          f.setAccessible(true);
          f.set(modify, Operation.DELETE);
        } catch (Throwable e) {
          throw new RuntimeException(e);
        }

        root = RelRoot.of(modify, SqlKind.DELETE);
      }

      return root;
    }

    if (parserOptions.isLegacySyntax()) {
      // close original planner
      planner.close();
      // create a new one
      planner = getPlanner(allowCorrelatedSubQueryExpansion, allowPushdownJoinCondition);
      node = processSQL(node.toSqlString(SqlDialect.CALCITE).toString(), false, planner);
    }

    boolean is_select_star = isSelectStar(node);

    SqlNode validateR = planner.validate(node);
    SqlSelect validate_select = getSelectChild(validateR);

    // Hide rowid from select * queries
    if (parserOptions.isLegacySyntax() && is_select_star && validate_select != null) {
      SqlNodeList proj_exprs = ((SqlSelect) validateR).getSelectList();
      SqlNodeList new_proj_exprs = new SqlNodeList(proj_exprs.getParserPosition());
      for (SqlNode proj_expr : proj_exprs) {
        final SqlNode unaliased_proj_expr = getUnaliasedExpression(proj_expr);

        if (unaliased_proj_expr instanceof SqlIdentifier) {
          if ((((SqlIdentifier) unaliased_proj_expr).toString().toLowerCase())
                          .endsWith(".rowid")) {
            continue;
          }
        }
        new_proj_exprs.add(proj_expr);
      }
      validate_select.setSelectList(new_proj_exprs);

      // trick planner back into correct state for validate
      planner.close();
      // create a new one
      planner = getPlanner(allowCorrelatedSubQueryExpansion, allowPushdownJoinCondition);
      processSQL(validateR.toSqlString(SqlDialect.CALCITE).toString(), false, planner);
      // now validate the new modified SqlNode;
      validateR = planner.validate(validateR);
    }

    planner.setFilterPushDownInfo(parserOptions.getFilterPushDownInfo());
    RelRoot relR = planner.rel(validateR);
    relR = replaceIsTrue(planner.getTypeFactory(), relR);
    planner.close();

    if (!parserOptions.isViewOptimizeEnabled()) {
      return relR;
    } else {
      // check to see if a view is involved in the query
      boolean foundView = false;
      MapDSchema schema = new MapDSchema(
              dataDir, this, mapdPort, mapdUser, sock_transport_properties);
      SqlIdentifierCapturer capturer =
              captureIdentifiers(sql, parserOptions.isLegacySyntax());
      for (String name : capturer.selects) {
        MapDTable table = (MapDTable) schema.getTable(name);
        if (null == table) {
          throw new RuntimeException("table/view not found: " + name);
        }
        if (table instanceof MapDView) {
          foundView = true;
        }
      }

      if (!foundView) {
        return relR;
      }

      // do some calcite based optimization
      // will allow duplicate projects to merge
      ProjectMergeRule projectMergeRule =
              new ProjectMergeRule(true, RelFactories.LOGICAL_BUILDER);
      final Program program =
              Programs.hep(ImmutableList.of(FilterProjectTransposeRule.INSTANCE,
                                   projectMergeRule,
                                   ProjectProjectRemoveRule.INSTANCE,
                                   FilterMergeRule.INSTANCE,
                                   JoinProjectTransposeRule.LEFT_PROJECT_INCLUDE_OUTER,
                                   JoinProjectTransposeRule.RIGHT_PROJECT_INCLUDE_OUTER,
                                   JoinProjectTransposeRule.BOTH_PROJECT_INCLUDE_OUTER),
                      true,
                      DefaultRelMetadataProvider.INSTANCE);

      RelNode oldRel;
      RelNode newRel = relR.project();

      do {
        oldRel = newRel;
        newRel = program.run(null,
                oldRel,
                null,
                ImmutableList.<RelOptMaterialization>of(),
                ImmutableList.<RelOptLattice>of());
        // there must be a better way to compare these
      } while (!RelOptUtil.toString(oldRel).equals(RelOptUtil.toString(newRel)));
      RelRoot optRel = RelRoot.of(newRel, relR.kind);
      return optRel;
    }
  }

Here is the call graph for this function:

Here is the caller graph for this function:

replaceIsTrue()

RelRoot com.mapd.calcite.parser.MapDParser.replaceIsTrue ( final RelDataTypeFactory  typeFactory, RelRoot  root  )

inlineprivate

Definition at line 809 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                                                                    {
    final RexShuttle callShuttle = new RexShuttle() {
      RexBuilder builder = new RexBuilder(typeFactory);

      public RexNode visitCall(RexCall call) {
        call = (RexCall) super.visitCall(call);
        if (call.getKind() == SqlKind.IS_TRUE) {
          return builder.makeCall(SqlStdOperatorTable.AND,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NOT_NULL, call.getOperands().get(0)),
                  call.getOperands().get(0));
        } else if (call.getKind() == SqlKind.IS_NOT_TRUE) {
          return builder.makeCall(SqlStdOperatorTable.OR,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NULL, call.getOperands().get(0)),
                  builder.makeCall(SqlStdOperatorTable.NOT, call.getOperands().get(0)));
        } else if (call.getKind() == SqlKind.IS_FALSE) {
          return builder.makeCall(SqlStdOperatorTable.AND,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NOT_NULL, call.getOperands().get(0)),
                  builder.makeCall(SqlStdOperatorTable.NOT, call.getOperands().get(0)));
        } else if (call.getKind() == SqlKind.IS_NOT_FALSE) {
          return builder.makeCall(SqlStdOperatorTable.OR,
                  builder.makeCall(
                          SqlStdOperatorTable.IS_NULL, call.getOperands().get(0)),
                  call.getOperands().get(0));
        }

        return call;
      }
    };

    RelNode node = root.rel.accept(new RelShuttleImpl() {
      @Override
      protected RelNode visitChild(RelNode parent, int i, RelNode child) {
        RelNode node = super.visitChild(parent, i, child);
        return node.accept(callShuttle);
      }
    });

    return new RelRoot(
            node, root.validatedRowType, root.kind, root.fields, root.collation);
  }

Here is the caller graph for this function:

resolveSelectIdentifiers()

Set<String> com.mapd.calcite.parser.MapDParser.resolveSelectIdentifiers ( SqlIdentifierCapturer  capturer )

inline

Definition at line 368 of file MapDParser.java.

References com.mapd.calcite.parser.MapDView.getAccessedObjects(), com.mapd.calcite.parser.MapDSchema.getTable(), org.apache.calcite.prepare.SqlIdentifierCapturer.selects, and Experimental.String.

Referenced by com.mapd.parser.server.CalciteServerHandler.process().

                                                                              {
    MapDSchema schema =
            new MapDSchema(dataDir, this, mapdPort, mapdUser, sock_transport_properties);
    HashSet<String> resolved = new HashSet<>();

    for (String name : capturer.selects) {
      MapDTable table = (MapDTable) schema.getTable(name);
      if (null == table) {
        throw new RuntimeException("table/view not found: " + name);
      }

      if (table instanceof MapDView) {
        MapDView view = (MapDView) table;
        resolved.addAll(resolveSelectIdentifiers(view.getAccessedObjects()));
      } else {
        resolved.add(name);
      }
    }

    return resolved;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

rewriteSimpleUpdateAsSelect()

SqlSelect com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect ( final SqlUpdate  update )

inlineprivate

Definition at line 401 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.getTableName(), com.mapd.calcite.parser.MapDParser.isCorrelated(), join(), and Experimental.String.

Referenced by com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect().

                                                                        {
    SqlNode where = update.getCondition();

    if (update.getSourceExpressionList().size() != 1) {
      return null;
    }

    if (!(update.getSourceExpressionList().get(0) instanceof SqlSelect)) {
      return null;
    }

    final SqlSelect inner = (SqlSelect) update.getSourceExpressionList().get(0);

    if (null != inner.getGroup() || null != inner.getFetch() || null != inner.getOffset()
            || (null != inner.getOrderList() && inner.getOrderList().size() != 0)
            || (null != inner.getGroup() && inner.getGroup().size() != 0)
            || null == getTableName(inner.getFrom())) {
      return null;
    }

    if (!isCorrelated(inner)) {
      return null;
    }

    final String updateTableName = getTableName(update.getTargetTable());

    if (null != where) {
      where = where.accept(new SqlShuttle() {
        @Override
        public SqlNode visit(SqlIdentifier id) {
          if (id.isSimple()) {
            id = new SqlIdentifier(Arrays.asList(updateTableName, id.getSimple()),
                    id.getParserPosition());
          }

          return id;
        }
      });
    }

    SqlJoin join = new SqlJoin(ZERO,
            update.getTargetTable(),
            SqlLiteral.createBoolean(false, ZERO),
            SqlLiteral.createSymbol(JoinType.LEFT, ZERO),
            inner.getFrom(),
            SqlLiteral.createSymbol(JoinConditionType.ON, ZERO),
            inner.getWhere());

    SqlNode select0 = inner.getSelectList().get(0);

    boolean wrapInSingleValue = true;
    if (select0 instanceof SqlCall) {
      SqlCall selectExprCall = (SqlCall) select0;
      if (Util.isSingleValue(selectExprCall)) {
        wrapInSingleValue = false;
      }
    }

    if (wrapInSingleValue) {
      select0 = new SqlBasicCall(
              SqlStdOperatorTable.SINGLE_VALUE, new SqlNode[] {select0}, ZERO);
    }

    SqlNodeList selectList = new SqlNodeList(ZERO);
    selectList.add(select0);
    selectList.add(new SqlBasicCall(SqlStdOperatorTable.AS,
            new SqlNode[] {new SqlBasicCall(
                                   new SqlUnresolvedFunction(
                                           new SqlIdentifier("OFFSET_IN_FRAGMENT", ZERO),
                                           null,
                                           null,
                                           null,
                                           null,
                                           SqlFunctionCategory.USER_DEFINED_FUNCTION),
                                   new SqlNode[0],
                                   SqlParserPos.ZERO),
                    new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO)},
            ZERO));

    SqlNodeList groupBy = new SqlNodeList(ZERO);
    groupBy.add(new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO));

    SqlSelect select = new SqlSelect(
            ZERO, null, selectList, join, where, groupBy, null, null, null, null, null);
    return select;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

rewriteUpdateAsSelect()

RelRoot com.mapd.calcite.parser.MapDParser.rewriteUpdateAsSelect ( SqlUpdate  update, MapDParserOptions  parserOptions  ) throws SqlParseException, ValidationException, RelConversionException

inlineprivate

Definition at line 501 of file MapDParser.java.

References com.mapd.calcite.parser.MapDSqlOperatorTable.ARRAY_VALUE_CONSTRUCTOR, com.mapd.calcite.parser.MapDParser.getDummyUpdate(), com.mapd.calcite.parser.MapDParser.getPlanner(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.parser.MapDParser.rewriteSimpleUpdateAsSelect(), and Void.

Referenced by com.mapd.calcite.parser.MapDParser.queryToSqlNode().

                                                                                {
    int correlatedQueriesCount[] = new int[1];
    SqlBasicVisitor<Void> correlatedQueriesCounter = new SqlBasicVisitor<Void>() {
      @Override
      public Void visit(SqlCall call) {
        if (call.isA(SCALAR)
                && ((call instanceof SqlBasicCall && call.operandCount() == 1
                            && !call.operand(0).isA(SCALAR))
                        || !(call instanceof SqlBasicCall))) {
          if (isCorrelated(call)) {
            correlatedQueriesCount[0]++;
          }
        }
        return super.visit(call);
      }
    };

    update.accept(correlatedQueriesCounter);
    if (correlatedQueriesCount[0] > 1) {
      throw new CalciteException(
              "table modifications with multiple correlated sub-queries not supported.",
              null);
    }

    boolean allowPushdownJoinCondition = false;
    SqlNodeList sourceExpression = new SqlNodeList(SqlParserPos.ZERO);
    LogicalTableModify dummyModify = getDummyUpdate(update);
    RelOptTable targetTable = dummyModify.getTable();
    RelDataType targetTableType = targetTable.getRowType();

    SqlSelect select = rewriteSimpleUpdateAsSelect(update);
    boolean applyRexCast = null == select;

    if (null == select) {
      for (int i = 0; i < update.getSourceExpressionList().size(); i++) {
        SqlNode targetColumn = update.getTargetColumnList().get(i);
        SqlNode expression = update.getSourceExpressionList().get(i);

        if (!(targetColumn instanceof SqlIdentifier)) {
          throw new RuntimeException("Unknown identifier type!");
        }
        SqlIdentifier id = (SqlIdentifier) targetColumn;
        RelDataType fieldType =
                targetTableType.getField(id.names.get(id.names.size() - 1), false, false)
                        .getType();

        if (expression.isA(ARRAY_VALUE) && null != fieldType.getComponentType()) {
          // apply a cast to all array value elements

          SqlDataTypeSpec elementType = new SqlDataTypeSpec(
                  new SqlBasicTypeNameSpec(fieldType.getComponentType().getSqlTypeName(),
                          fieldType.getPrecision(),
                          fieldType.getScale(),
                          null == fieldType.getCharset() ? null
                                                         : fieldType.getCharset().name(),
                          SqlParserPos.ZERO),
                  SqlParserPos.ZERO);
          SqlCall array_expression = (SqlCall) expression;
          ArrayList<SqlNode> values = new ArrayList<>();

          for (SqlNode value : array_expression.getOperandList()) {
            if (value.isA(EnumSet.of(SqlKind.LITERAL))) {
              SqlNode casted_value = new SqlBasicCall(SqlStdOperatorTable.CAST,
                      new SqlNode[] {value, elementType},
                      value.getParserPosition());
              values.add(casted_value);
            } else {
              values.add(value);
            }
          }

          expression = new SqlBasicCall(MapDSqlOperatorTable.ARRAY_VALUE_CONSTRUCTOR,
                  values.toArray(new SqlNode[0]),
                  expression.getParserPosition());
        }
        sourceExpression.add(expression);
      }

      sourceExpression.add(new SqlBasicCall(SqlStdOperatorTable.AS,
              new SqlNode[] {
                      new SqlBasicCall(new SqlUnresolvedFunction(
                                               new SqlIdentifier("OFFSET_IN_FRAGMENT",
                                                       SqlParserPos.ZERO),
                                               null,
                                               null,
                                               null,
                                               null,
                                               SqlFunctionCategory.USER_DEFINED_FUNCTION),
                              new SqlNode[0],
                              SqlParserPos.ZERO),
                      new SqlIdentifier("EXPR$DELETE_OFFSET_IN_FRAGMENT", ZERO)},
              ZERO));

      select = new SqlSelect(SqlParserPos.ZERO,
              null,
              sourceExpression,
              update.getTargetTable(),
              update.getCondition(),
              null,
              null,
              null,
              null,
              null,
              null);
    }

    MapDPlanner planner = getPlanner(true, allowPushdownJoinCondition);
    SqlNode node = planner.parse(select.toSqlString(SqlDialect.CALCITE).getSql());
    node = planner.validate(node);
    RelRoot root = planner.rel(node);
    LogicalProject project = (LogicalProject) root.project();

    ArrayList<String> fields = new ArrayList<String>();
    ArrayList<RexNode> nodes = new ArrayList<RexNode>();
    final RexBuilder builder = new RexBuilder(planner.getTypeFactory());

    for (SqlNode n : update.getTargetColumnList()) {
      if (n instanceof SqlIdentifier) {
        SqlIdentifier id = (SqlIdentifier) n;
        fields.add(id.names.get(id.names.size() - 1));
      } else {
        throw new RuntimeException("Unknown identifier type!");
      }
    }

    int idx = 0;
    for (RexNode exp : project.getChildExps()) {
      if (applyRexCast && idx + 1 < project.getChildExps().size()) {
        RelDataType expectedFieldType =
                targetTableType.getField(fields.get(idx), false, false).getType();
        if (!exp.getType().equals(expectedFieldType) && !exp.isA(ARRAY_VALUE)) {
          exp = builder.makeCast(expectedFieldType, exp);
        }
      }

      nodes.add(exp);
      idx++;
    }

    ArrayList<RexNode> inputs = new ArrayList<RexNode>();
    int n = 0;
    for (int i = 0; i < fields.size(); i++) {
      inputs.add(
              new RexInputRef(n, project.getRowType().getFieldList().get(n).getType()));
      n++;
    }

    fields.add("EXPR$DELETE_OFFSET_IN_FRAGMENT");
    inputs.add(new RexInputRef(n, project.getRowType().getFieldList().get(n).getType()));

    project = project.copy(
            project.getTraitSet(), project.getInput(), nodes, project.getRowType());

    LogicalTableModify modify = LogicalTableModify.create(targetTable,
            dummyModify.getCatalogReader(),
            project,
            Operation.UPDATE,
            fields,
            inputs,
            true);
    return RelRoot.of(modify, SqlKind.UPDATE);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

setUser()

void com.mapd.calcite.parser.MapDParser.setUser ( MapDUser  mapdUser )

inline

Definition at line 343 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.mapdUser.

Referenced by com.mapd.parser.server.CalciteServerHandler.getCompletionHints(), com.mapd.calcite.parser.MapDParser.isCorrelated(), com.mapd.calcite.planner.tester.main(), and com.mapd.parser.server.CalciteServerHandler.process().

                                         {
    this.mapdUser = mapdUser;
  }

Here is the caller graph for this function:

updateMetaData()

void com.mapd.calcite.parser.MapDParser.updateMetaData ( String  schema, String  table  )

inline

Definition at line 1352 of file MapDParser.java.

References com.mapd.calcite.parser.MapDParser.dataDir, com.mapd.calcite.parser.MapDParser.mapdPort, com.mapd.calcite.parser.MapDParser.sock_transport_properties, and com.mapd.calcite.parser.MapDSchema.updateMetaData().

Referenced by com.mapd.parser.server.CalciteServerHandler.updateMetadata().

                                                          {
    MAPDLOGGER.debug("schema :" + schema + " table :" + table);
    MapDSchema mapd =
            new MapDSchema(dataDir, this, mapdPort, null, sock_transport_properties);
    mapd.updateMetaData(schema, table);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

ARRAY_VALUE

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.ARRAY_VALUE

staticprivate

Initial value:

=
          EnumSet.of(SqlKind.ARRAY_VALUE_CONSTRUCTOR)

Definition at line 122 of file MapDParser.java.

callCount

int com.mapd.calcite.parser.MapDParser.callCount = 0

private

Definition at line 134 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.getCallCount().

CURRENT_PARSER

final ThreadLocal<MapDParser> com.mapd.calcite.parser.MapDParser.CURRENT_PARSER = new ThreadLocal<>()

static

Definition at line 115 of file MapDParser.java.

dataDir

final String com.mapd.calcite.parser.MapDParser.dataDir

private

Definition at line 132 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.MapDParser(), and com.mapd.calcite.parser.MapDParser.updateMetaData().

DELETE

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.DELETE = EnumSet.of(SqlKind.DELETE)

staticprivate

Definition at line 119 of file MapDParser.java.

EXISTS

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.EXISTS = EnumSet.of(SqlKind.EXISTS)

staticprivate

Definition at line 118 of file MapDParser.java.

extSigs

final Map<String, ExtensionFunction> com.mapd.calcite.parser.MapDParser.extSigs

private

Definition at line 131 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.MapDParser().

IN

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.IN = EnumSet.of(SqlKind.IN)

staticprivate

Definition at line 121 of file MapDParser.java.

MAPD_CONNECTION_CONTEXT

final Context com.mapd.calcite.parser.MapDParser.MAPD_CONNECTION_CONTEXT

staticprivate

Initial value:

= new Context() {
    MapDRelDataTypeSystemImpl myTypeSystem = new MapDRelDataTypeSystemImpl();
    CalciteConnectionConfig config = new CalciteConnectionConfigImpl(new Properties()) {
      @SuppressWarnings("unchecked")
      public <T extends Object> T typeSystem(
              java.lang.Class<T> typeSystemClass, T defaultTypeSystem) {
        return (T) myTypeSystem;
      };

      public boolean caseSensitive() {
        return false;
      };

      public org.apache.calcite.sql.validate.SqlConformance conformance() {
        return SqlConformanceEnum.LENIENT;
      };
    };

    @Override
    public <C> C unwrap(Class<C> aClass) {
      if (aClass.isInstance(config)) {
        return aClass.cast(config);
      }
      return null;
    }
  }

Definition at line 162 of file MapDParser.java.

MAPDLOGGER

final Logger com.mapd.calcite.parser.MapDParser.MAPDLOGGER = LoggerFactory.getLogger(MapDParser.class)

staticpackage

Definition at line 125 of file MapDParser.java.

mapdPort

final int com.mapd.calcite.parser.MapDParser.mapdPort

private

Definition at line 135 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.MapDParser(), and com.mapd.calcite.parser.MapDParser.updateMetaData().

mapdUser

MapDUser com.mapd.calcite.parser.MapDParser.mapdUser

private

Definition at line 136 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.setUser().

SCALAR

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.SCALAR

staticprivate

Initial value:

=
          EnumSet.of(SqlKind.SCALAR_QUERY, SqlKind.SELECT)

Definition at line 116 of file MapDParser.java.

sock_transport_properties

SockTransportProperties com.mapd.calcite.parser.MapDParser.sock_transport_properties = null

private

Definition at line 138 of file MapDParser.java.

Referenced by com.mapd.calcite.parser.MapDParser.updateMetaData().

sqlNode_

SqlNode com.mapd.calcite.parser.MapDParser.sqlNode_

package

Definition at line 137 of file MapDParser.java.

SubqueryCorrMemo

Map<String, Boolean> com.mapd.calcite.parser.MapDParser.SubqueryCorrMemo = new ConcurrentHashMap<>()

staticprivate

Definition at line 140 of file MapDParser.java.

UPDATE

final EnumSet<SqlKind> com.mapd.calcite.parser.MapDParser.UPDATE = EnumSet.of(SqlKind.UPDATE)

staticprivate

Definition at line 120 of file MapDParser.java.

---

The documentation for this class was generated from the following file:

• /home/jenkins-slave/workspace/core-os-doxygen/java/calcite/src/main/java/com/mapd/calcite/parser/MapDParser.java