OmniSciDB  c1a53651b2
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Public Member Functions | Public Attributes | Private Attributes | List of all members
generate_TableFunctionsFactory_init.Parser Class Reference

Public Member Functions

def __init__
 
def tokens
 
def is_at_end
 
def current_token
 
def advance
 
def expect
 
def consume
 
def current_pos
 
def raise_parser_error
 
def match
 
def lookahead
 
def parse_udtf
 
def parse_args
 
def parse_arg
 
def parse_type
 
def parse_composed
 
def parse_primitive
 
def parse_templates
 
def parse_template
 
def parse_annotation
 
def parse_identifier
 
def parse_string
 
def parse_number
 
def parse
 

Public Attributes

 line
 

Private Attributes

 _tokens
 
 _curr
 

Detailed Description

Definition at line 1285 of file generate_TableFunctionsFactory_init.py.

Constructor & Destructor Documentation

def generate_TableFunctionsFactory_init.Parser.__init__ (   self,
  line 
)

Definition at line 1286 of file generate_TableFunctionsFactory_init.py.

1286 
1287  def __init__(self, line):
1288  self._tokens = Tokenize(line).tokens
1289  self._curr = 0
1290  self.line = line
generate_TableFunctionsFactory_init.Tokenize
Definition: generate_TableFunctionsFactory_init.py:460
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser.line
line
Definition: generate_TableFunctionsFactory_init.py:1289
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288
generate_TableFunctionsFactory_init.Parser.__init__
def __init__
Definition: generate_TableFunctionsFactory_init.py:1286

Member Function Documentation

def generate_TableFunctionsFactory_init.Parser.advance (   self)

Definition at line 1301 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr.

Referenced by generate_TableFunctionsFactory_init.Parser.consume(), and generate_TableFunctionsFactory_init.Parser.expect().

1301 
1302  def advance(self):
1303  self._curr += 1
generate_TableFunctionsFactory_init.Parser.advance
def advance
Definition: generate_TableFunctionsFactory_init.py:1301
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.consume (   self,
  expected_type 
) 
consumes the current token iff its type matches the
expected_type. Otherwise, an error is raised

Definition at line 1315 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Tokenize._tokens, generate_TableFunctionsFactory_init.Parser._tokens, generate_TableFunctionsFactory_init.Tokenize.advance(), generate_TableFunctionsFactory_init.Parser.advance(), anonymous_namespace{RelAlgDag.cpp}::RANodeIterator.advance(), generate_TableFunctionsFactory_init.Tokenize.current_token(), generate_TableFunctionsFactory_init.Parser.current_token(), and generate_TableFunctionsFactory_init.Parser.raise_parser_error().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_args(), generate_TableFunctionsFactory_init.Parser.parse_composed(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), generate_TableFunctionsFactory_init.Parser.parse_number(), generate_TableFunctionsFactory_init.Parser.parse_string(), generate_TableFunctionsFactory_init.Parser.parse_template(), generate_TableFunctionsFactory_init.Parser.parse_templates(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1315 
1316  def consume(self, expected_type):
1317  """consumes the current token iff its type matches the
1318  expected_type. Otherwise, an error is raised
1319  """
1320  curr_token = self.current_token()
1321  if curr_token.type == expected_type:
1322  self.advance()
1323  return curr_token
1324  else:
1325  expected_token = Token.tok_name(expected_type)
1326  self.raise_parser_error(
1327  'Token mismatch at function consume. '
1328  'Expected type "%s" but got token "%s"\n\n'
1329  'Tokens: %s\n' % (expected_token, curr_token, self._tokens)
1330  )
generate_TableFunctionsFactory_init.Parser.advance
def advance
Definition: generate_TableFunctionsFactory_init.py:1301
generate_TableFunctionsFactory_init.Parser.current_token
def current_token
Definition: generate_TableFunctionsFactory_init.py:1298
generate_TableFunctionsFactory_init.Parser.raise_parser_error
def raise_parser_error
Definition: generate_TableFunctionsFactory_init.py:1334
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.current_pos (   self)

Definition at line 1331 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr.

Referenced by generate_TableFunctionsFactory_init.Parser.raise_parser_error().

1331 
1332  def current_pos(self):
1333  return self._curr
generate_TableFunctionsFactory_init.Parser.current_pos
def current_pos
Definition: generate_TableFunctionsFactory_init.py:1331
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.current_token (   self)

Definition at line 1298 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Tokenize._tokens, and generate_TableFunctionsFactory_init.Parser._tokens.

Referenced by generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Parser.expect(), generate_TableFunctionsFactory_init.Parser.match(), and generate_TableFunctionsFactory_init.Parser.raise_parser_error().

1298 
1299  def current_token(self):
1300  return self._tokens[self._curr]
generate_TableFunctionsFactory_init.Parser.current_token
def current_token
Definition: generate_TableFunctionsFactory_init.py:1298
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.expect (   self,
  expected_type 
)

Definition at line 1304 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Tokenize._tokens, generate_TableFunctionsFactory_init.Parser._tokens, generate_TableFunctionsFactory_init.Tokenize.advance(), generate_TableFunctionsFactory_init.Parser.advance(), anonymous_namespace{RelAlgDag.cpp}::RANodeIterator.advance(), generate_TableFunctionsFactory_init.Tokenize.current_token(), and generate_TableFunctionsFactory_init.Parser.current_token().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_udtf().

1304 
1305  def expect(self, expected_type):
1306  curr_token = self.current_token()
1307  msg = "Expected token %s but got %s at pos %d.\n Tokens: %s" % (
1308  curr_token,
1309  Token.tok_name(expected_type),
1310  self._curr,
1311  self._tokens,
1312  )
1313  assert curr_token.type == expected_type, msg
1314  self.advance()
generate_TableFunctionsFactory_init.Parser.advance
def advance
Definition: generate_TableFunctionsFactory_init.py:1301
generate_TableFunctionsFactory_init.Parser.current_token
def current_token
Definition: generate_TableFunctionsFactory_init.py:1298
generate_TableFunctionsFactory_init.Parser.expect
def expect
Definition: generate_TableFunctionsFactory_init.py:1304
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.is_at_end (   self)

Definition at line 1295 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Tokenize._tokens, and generate_TableFunctionsFactory_init.Parser._tokens.

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_args(), generate_TableFunctionsFactory_init.Parser.parse_templates(), generate_TableFunctionsFactory_init.Parser.parse_type(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1295 
1296  def is_at_end(self):
1297  return self._curr >= len(self._tokens)
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.lookahead (   self)

Definition at line 1350 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Tokenize._tokens, and generate_TableFunctionsFactory_init.Parser._tokens.

Referenced by generate_TableFunctionsFactory_init.Parser.parse_arg().

1350 
1351  def lookahead(self):
1352  return self._tokens[self._curr + 1]
generate_TableFunctionsFactory_init.Parser.lookahead
def lookahead
Definition: generate_TableFunctionsFactory_init.py:1350
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.match (   self,
  expected_type 
)

Definition at line 1346 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Tokenize.current_token(), and generate_TableFunctionsFactory_init.Parser.current_token().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_args(), generate_TableFunctionsFactory_init.Parser.parse_composed(), generate_TableFunctionsFactory_init.Parser.parse_primitive(), generate_TableFunctionsFactory_init.Parser.parse_template(), generate_TableFunctionsFactory_init.Parser.parse_templates(), generate_TableFunctionsFactory_init.Parser.parse_type(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1346 
1347  def match(self, expected_type):
1348  curr_token = self.current_token()
1349  return curr_token.type == expected_type
generate_TableFunctionsFactory_init.Parser.current_token
def current_token
Definition: generate_TableFunctionsFactory_init.py:1298
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse (   self) 
fmt: off

udtf: IDENTIFIER "(" (args)? ")" ("|" annotation)* "->" args ("," templates)? ("|" "output_row_size" "=" primitive)?

args: arg ("," arg)*

arg: type IDENTIFIER? ("|" annotation)*

type: composed
    | primitive

composed: "Cursor" "<" arg ("," arg)* ">"
| IDENTIFIER "<" type ("," type)* ">"

primitive: IDENTIFIER
 | NUMBER
 | STRING

annotation: IDENTIFIER "=" IDENTIFIER ("<" NUMBER ("," NUMBER) ">")?
  | IDENTIFIER "=" "[" PRIMITIVE? ("," PRIMITIVE)* "]"
  | "require" "=" STRING

templates: template ("," template)
template: IDENTIFIER "=" "[" IDENTIFIER ("," IDENTIFIER)* "]"

IDENTIFIER: [A-Za-z_][A-Za-z0-9_]*
NUMBER: [0-9]+
STRING: \".*?\"

fmt: on

Definition at line 1611 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1611 
1612  def parse(self):
1613  """fmt: off
1614 
1615  udtf: IDENTIFIER "(" (args)? ")" ("|" annotation)* "->" args ("," templates)? ("|" "output_row_size" "=" primitive)?
1616 
1617  args: arg ("," arg)*
1618 
1619  arg: type IDENTIFIER? ("|" annotation)*
1620 
1621  type: composed
1622  | primitive
1623 
1624  composed: "Cursor" "<" arg ("," arg)* ">"
1625  | IDENTIFIER "<" type ("," type)* ">"
1626 
1627  primitive: IDENTIFIER
1628  | NUMBER
1629  | STRING
1630 
1631  annotation: IDENTIFIER "=" IDENTIFIER ("<" NUMBER ("," NUMBER) ">")?
1632  | IDENTIFIER "=" "[" PRIMITIVE? ("," PRIMITIVE)* "]"
1633  | "require" "=" STRING
1634 
1635  templates: template ("," template)
1636  template: IDENTIFIER "=" "[" IDENTIFIER ("," IDENTIFIER)* "]"
1637 
1638  IDENTIFIER: [A-Za-z_][A-Za-z0-9_]*
1639  NUMBER: [0-9]+
1640  STRING: \".*?\"
1641 
1642  fmt: on
1643  """
1644  self._curr = 0
1645  udtf = self.parse_udtf()
1646 
1647  # set parent
1648  udtf.parent = None
1649  d = deque()
1650  d.append(udtf)
1651  while d:
1652  node = d.pop()
1653  if isinstance(node, Iterable):
1654  for child in node:
1655  child.parent = node
1656  d.append(child)
1657  return udtf
1658 
1659 
# fmt: off
generate_TableFunctionsFactory_init.Parser.parse_udtf
def parse_udtf
Definition: generate_TableFunctionsFactory_init.py:1353
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288
generate_TableFunctionsFactory_init.Parser.parse
def parse
Definition: generate_TableFunctionsFactory_init.py:1611

+ Here is the call graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_annotation (   self) 
fmt: off

annotation: IDENTIFIER "=" IDENTIFIER ("<" NUMBER ("," NUMBER) ">")?
  | IDENTIFIER "=" "[" PRIMITIVE? ("," PRIMITIVE)* "]"
  | "require" "=" STRING

fmt: on

Definition at line 1541 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), generate_TableFunctionsFactory_init.Parser.parse_number(), generate_TableFunctionsFactory_init.Parser.parse_primitive(), and generate_TableFunctionsFactory_init.Parser.parse_string().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_arg(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1541 
1542  def parse_annotation(self):
1543  """fmt: off
1544 
1545  annotation: IDENTIFIER "=" IDENTIFIER ("<" NUMBER ("," NUMBER) ">")?
1546  | IDENTIFIER "=" "[" PRIMITIVE? ("," PRIMITIVE)* "]"
1547  | "require" "=" STRING
1548 
1549  fmt: on
1550  """
1551  key = self.parse_identifier()
1552  self.consume(Token.EQUAL)
1553 
1554  if key == "require":
1555  value = self.parse_string()
1556  elif not self.is_at_end() and self.match(Token.LSQB):
1557  value = []
1558  self.consume(Token.LSQB)
1559  if not self.match(Token.RSQB):
1560  value.append(self.parse_primitive())
1561  while self.match(Token.COMMA):
1562  self.consume(Token.COMMA)
1563  value.append(self.parse_primitive())
1564  self.consume(Token.RSQB)
1565  else:
1566  value = self.parse_identifier()
1567  if not self.is_at_end() and self.match(Token.LESS):
1568  self.consume(Token.LESS)
1569  if self.match(Token.GREATER):
1570  value += "<%s>" % (-1) # Signifies no input
1571  else:
1572  num1 = self.parse_number()
1573  if self.match(Token.COMMA):
1574  self.consume(Token.COMMA)
1575  num2 = self.parse_number()
1576  value += "<%s,%s>" % (num1, num2)
1577  else:
1578  value += "<%s>" % (num1)
1579  self.consume(Token.GREATER)
1580  return AnnotationNode(key, value)
generate_TableFunctionsFactory_init.AnnotationNode
Definition: generate_TableFunctionsFactory_init.py:1225
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.parse_annotation
def parse_annotation
Definition: generate_TableFunctionsFactory_init.py:1541
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_string
def parse_string
Definition: generate_TableFunctionsFactory_init.py:1591
generate_TableFunctionsFactory_init.Parser.parse_number
def parse_number
Definition: generate_TableFunctionsFactory_init.py:1601
generate_TableFunctionsFactory_init.Parser.parse_primitive
def parse_primitive
Definition: generate_TableFunctionsFactory_init.py:1490
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_arg (   self) 
fmt: off

arg: type IDENTIFIER? ("|" annotation)*

fmt: on

Definition at line 1423 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Tokenize.lookahead(), generate_TableFunctionsFactory_init.Parser.lookahead(), generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), and generate_TableFunctionsFactory_init.Parser.parse_type().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_args(), and generate_TableFunctionsFactory_init.Parser.parse_composed().

1423 
1424  def parse_arg(self):
1425  """fmt: off
1426 
1427  arg: type IDENTIFIER? ("|" annotation)*
1428 
1429  fmt: on
1430  """
1431  typ = self.parse_type()
1432 
1433  annotations = []
1434 
1435  if not self.is_at_end() and self.match(Token.IDENTIFIER):
1436  name = self.parse_identifier()
1437  annotations.append(AnnotationNode('name', name))
1438 
1439  while not self.is_at_end() and self.match(Token.VBAR):
1440  ahead = self.lookahead()
1441  if ahead.type == Token.IDENTIFIER and ahead.lexeme == 'output_row_size':
1442  break
1443  self.consume(Token.VBAR)
1444  annotations.append(self.parse_annotation())
1445 
1446  return ArgNode(typ, annotations)
generate_TableFunctionsFactory_init.Parser.lookahead
def lookahead
Definition: generate_TableFunctionsFactory_init.py:1350
generate_TableFunctionsFactory_init.AnnotationNode
Definition: generate_TableFunctionsFactory_init.py:1225
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.parse_annotation
def parse_annotation
Definition: generate_TableFunctionsFactory_init.py:1541
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.parse_type
def parse_type
Definition: generate_TableFunctionsFactory_init.py:1447
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.ArgNode
Definition: generate_TableFunctionsFactory_init.py:1083
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315
generate_TableFunctionsFactory_init.Parser.parse_arg
def parse_arg
Definition: generate_TableFunctionsFactory_init.py:1423

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_args (   self) 
fmt: off

args: arg IDENTIFIER ("," arg)*

fmt: on

Definition at line 1401 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_arg(), and generate_TableFunctionsFactory_init.Parser.parse_type().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_udtf().

1401 
1402  def parse_args(self):
1403  """fmt: off
1404 
1405  args: arg IDENTIFIER ("," arg)*
1406 
1407  fmt: on
1408  """
1409  args = []
1410  args.append(self.parse_arg())
1411  while not self.is_at_end() and self.match(Token.COMMA):
1412  curr = self._curr
1413  self.consume(Token.COMMA)
1414  self.parse_type() # assuming that we are not ending with COMMA
1415  if not self.is_at_end() and self.match(Token.EQUAL):
1416  # arg type cannot be assigned, so this must be a template specification
1417  self._curr = curr # step back and let the code below parse the templates
1418  break
1419  else:
1420  self._curr = curr + 1 # step back from self.parse_type(), parse_arg will parse it again
1421  args.append(self.parse_arg())
1422  return args
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.parse_type
def parse_type
Definition: generate_TableFunctionsFactory_init.py:1447
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_args
def parse_args
Definition: generate_TableFunctionsFactory_init.py:1401
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315
generate_TableFunctionsFactory_init.Parser.parse_arg
def parse_arg
Definition: generate_TableFunctionsFactory_init.py:1423
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_composed (   self) 
fmt: off

composed: "Cursor" "<" arg ("," arg)* ">"
| IDENTIFIER "<" type ("," type)* ">"

fmt: on

Definition at line 1467 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.is_identifier_cursor(), generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), and generate_TableFunctionsFactory_init.Parser.parse_type().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_type().

1467 
1468  def parse_composed(self):
1469  """fmt: off
1470 
1471  composed: "Cursor" "<" arg ("," arg)* ">"
1472  | IDENTIFIER "<" type ("," type)* ">"
1473 
1474  fmt: on
1475  """
1476  idtn = self.parse_identifier()
1477  self.consume(Token.LESS)
1478  if is_identifier_cursor(idtn):
1479  inner = [self.parse_arg()]
1480  while self.match(Token.COMMA):
1481  self.consume(Token.COMMA)
1482  inner.append(self.parse_arg())
1483  else:
1484  inner = [self.parse_type()]
1485  while self.match(Token.COMMA):
1486  self.consume(Token.COMMA)
1487  inner.append(self.parse_type())
1488  self.consume(Token.GREATER)
1489  return ComposedNode(idtn, inner)
generate_TableFunctionsFactory_init.ComposedNode
Definition: generate_TableFunctionsFactory_init.py:1177
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.parse_composed
def parse_composed
Definition: generate_TableFunctionsFactory_init.py:1467
generate_TableFunctionsFactory_init.Parser.parse_type
def parse_type
Definition: generate_TableFunctionsFactory_init.py:1447
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.is_identifier_cursor
def is_identifier_cursor
Definition: generate_TableFunctionsFactory_init.py:385
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315
generate_TableFunctionsFactory_init.Parser.parse_arg
def parse_arg
Definition: generate_TableFunctionsFactory_init.py:1423

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_identifier (   self) 
fmt: off

IDENTIFIER: [A-Za-z_][A-Za-z0-9_]*

fmt: on

Definition at line 1581 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_composed(), generate_TableFunctionsFactory_init.Parser.parse_primitive(), generate_TableFunctionsFactory_init.Parser.parse_template(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1581 
1582  def parse_identifier(self):
1583  """ fmt: off
1584 
1585  IDENTIFIER: [A-Za-z_][A-Za-z0-9_]*
1586 
1587  fmt: on
1588  """
1589  token = self.consume(Token.IDENTIFIER)
1590  return token.lexeme
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_number (   self) 
fmt: off

NUMBER: [0-9]+

fmt: on

Definition at line 1601 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), and generate_TableFunctionsFactory_init.Parser.parse_primitive().

1601 
1602  def parse_number(self):
1603  """ fmt: off
1604 
1605  NUMBER: [0-9]+
1606 
1607  fmt: on
1608  """
1609  token = self.consume(Token.NUMBER)
1610  return token.lexeme
generate_TableFunctionsFactory_init.Parser.parse_number
def parse_number
Definition: generate_TableFunctionsFactory_init.py:1601
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_primitive (   self) 
fmt: off

primitive: IDENTIFIER
 | NUMBER
 | STRING

fmt: on

Definition at line 1490 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), generate_TableFunctionsFactory_init.Parser.parse_number(), generate_TableFunctionsFactory_init.Parser.parse_string(), and generate_TableFunctionsFactory_init.Parser.raise_parser_error().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_type(), and generate_TableFunctionsFactory_init.Parser.parse_udtf().

1490 
1491  def parse_primitive(self):
1492  """fmt: off
1493 
1494  primitive: IDENTIFIER
1495  | NUMBER
1496  | STRING
1497 
1498  fmt: on
1499  """
1500  if self.match(Token.IDENTIFIER):
1501  lexeme = self.parse_identifier()
1502  elif self.match(Token.NUMBER):
1503  lexeme = self.parse_number()
1504  elif self.match(Token.STRING):
1505  lexeme = self.parse_string()
1506  else:
1507  raise self.raise_parser_error()
1508  return PrimitiveNode(lexeme)
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.raise_parser_error
def raise_parser_error
Definition: generate_TableFunctionsFactory_init.py:1334
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_string
def parse_string
Definition: generate_TableFunctionsFactory_init.py:1591
generate_TableFunctionsFactory_init.Parser.parse_number
def parse_number
Definition: generate_TableFunctionsFactory_init.py:1601
generate_TableFunctionsFactory_init.PrimitiveNode
Definition: generate_TableFunctionsFactory_init.py:1152
generate_TableFunctionsFactory_init.Parser.parse_primitive
def parse_primitive
Definition: generate_TableFunctionsFactory_init.py:1490

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_string (   self) 
fmt: off

STRING: \".*?\"

fmt: on

Definition at line 1591 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_annotation(), and generate_TableFunctionsFactory_init.Parser.parse_primitive().

1591 
1592  def parse_string(self):
1593  """ fmt: off
1594 
1595  STRING: \".*?\"
1596 
1597  fmt: on
1598  """
1599  token = self.consume(Token.STRING)
1600  return token.lexeme
generate_TableFunctionsFactory_init.Parser.parse_string
def parse_string
Definition: generate_TableFunctionsFactory_init.py:1591
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_template (   self) 
fmt: off

template: IDENTIFIER "=" "[" IDENTIFIER ("," IDENTIFIER)* "]"

fmt: on

Definition at line 1523 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Parser.match(), and generate_TableFunctionsFactory_init.Parser.parse_identifier().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_templates().

1523 
1524  def parse_template(self):
1525  """fmt: off
1526 
1527  template: IDENTIFIER "=" "[" IDENTIFIER ("," IDENTIFIER)* "]"
1528 
1529  fmt: on
1530  """
1531  key = self.parse_identifier()
1532  types = []
1533  self.consume(Token.EQUAL)
1534  self.consume(Token.LSQB)
1535  types.append(self.parse_identifier())
1536  while self.match(Token.COMMA):
1537  self.consume(Token.COMMA)
1538  types.append(self.parse_identifier())
1539  self.consume(Token.RSQB)
1540  return TemplateNode(key, tuple(types))
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_template
def parse_template
Definition: generate_TableFunctionsFactory_init.py:1523
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315
generate_TableFunctionsFactory_init.TemplateNode
Definition: generate_TableFunctionsFactory_init.py:1247

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_templates (   self) 
fmt: off

templates: template ("," template)*

fmt: on

Definition at line 1509 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.match(), and generate_TableFunctionsFactory_init.Parser.parse_template().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_udtf().

1509 
1510  def parse_templates(self):
1511  """fmt: off
1512 
1513  templates: template ("," template)*
1514 
1515  fmt: on
1516  """
1517  T = []
1518  T.append(self.parse_template())
1519  while not self.is_at_end() and self.match(Token.COMMA):
1520  self.consume(Token.COMMA)
1521  T.append(self.parse_template())
1522  return T
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_template
def parse_template
Definition: generate_TableFunctionsFactory_init.py:1523
generate_TableFunctionsFactory_init.Parser.parse_templates
def parse_templates
Definition: generate_TableFunctionsFactory_init.py:1509
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_type (   self) 
fmt: off

type: composed
    | primitive

fmt: on

Definition at line 1447 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser._curr, generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_composed(), and generate_TableFunctionsFactory_init.Parser.parse_primitive().

Referenced by generate_TableFunctionsFactory_init.Parser.parse_arg(), generate_TableFunctionsFactory_init.Parser.parse_args(), and generate_TableFunctionsFactory_init.Parser.parse_composed().

1447 
1448  def parse_type(self):
1449  """fmt: off
1450 
1451  type: composed
1452  | primitive
1453 
1454  fmt: on
1455  """
1456  curr = self._curr # save state
1457  primitive = self.parse_primitive()
1458  if self.is_at_end():
1459  return primitive
1460 
1461  if not self.match(Token.LESS):
1462  return primitive
1463 
1464  self._curr = curr # return state
1465 
1466  return self.parse_composed()
generate_TableFunctionsFactory_init.Parser.parse_composed
def parse_composed
Definition: generate_TableFunctionsFactory_init.py:1467
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.parse_type
def parse_type
Definition: generate_TableFunctionsFactory_init.py:1447
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.parse_primitive
def parse_primitive
Definition: generate_TableFunctionsFactory_init.py:1490
generate_TableFunctionsFactory_init.Parser._curr
_curr
Definition: generate_TableFunctionsFactory_init.py:1288

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.parse_udtf (   self) 
fmt: off

udtf: IDENTIFIER "(" (args)? ")" ("|" annotation)* "->" args ("," templates)? ("|" "output_row_size" "=" primitive)?

fmt: on

Definition at line 1353 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Parser.expect(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.UdtfNode.line, generate_TableFunctionsFactory_init.Parser.line, generate_TableFunctionsFactory_init.Parser.match(), generate_TableFunctionsFactory_init.Parser.parse_annotation(), generate_TableFunctionsFactory_init.Parser.parse_args(), generate_TableFunctionsFactory_init.Parser.parse_identifier(), generate_TableFunctionsFactory_init.Parser.parse_primitive(), and generate_TableFunctionsFactory_init.Parser.parse_templates().

Referenced by generate_TableFunctionsFactory_init.Parser.parse().

1353 
1354  def parse_udtf(self):
1355  """fmt: off
1356 
1357  udtf: IDENTIFIER "(" (args)? ")" ("|" annotation)* "->" args ("," templates)? ("|" "output_row_size" "=" primitive)?
1358 
1359  fmt: on
1360  """
1361  name = self.parse_identifier()
1362  self.expect(Token.LPAR) # (
1363  input_args = []
1364  if not self.match(Token.RPAR):
1365  input_args = self.parse_args()
1366  self.expect(Token.RPAR) # )
1367  annotations = []
1368  while not self.is_at_end() and self.match(Token.VBAR): # |
1369  self.consume(Token.VBAR)
1370  annotations.append(self.parse_annotation())
1371  self.expect(Token.RARROW) # ->
1372  output_args = self.parse_args()
1373 
1374  templates = None
1375  if not self.is_at_end() and self.match(Token.COMMA):
1376  self.consume(Token.COMMA)
1377  templates = self.parse_templates()
1378 
1379  sizer = None
1380  if not self.is_at_end() and self.match(Token.VBAR):
1381  self.consume(Token.VBAR)
1382  idtn = self.parse_identifier()
1383  assert idtn == "output_row_size", idtn
1384  self.consume(Token.EQUAL)
1385  node = self.parse_primitive()
1386  key = "kPreFlightParameter"
1387  sizer = AnnotationNode(key, value=node.type)
1388 
1389  # set arg_pos
1390  i = 0
1391  for arg in input_args:
1392  arg.arg_pos = i
1393  arg.kind = "input"
1394  i += arg.type.cursor_length() if arg.type.is_cursor() else 1
1395 
1396  for i, arg in enumerate(output_args):
1397  arg.arg_pos = i
1398  arg.kind = "output"
1399 
1400  return UdtfNode(name, input_args, output_args, annotations, templates, sizer, self.line)
generate_TableFunctionsFactory_init.AnnotationNode
Definition: generate_TableFunctionsFactory_init.py:1225
generate_TableFunctionsFactory_init.Parser.parse_identifier
def parse_identifier
Definition: generate_TableFunctionsFactory_init.py:1581
generate_TableFunctionsFactory_init.Parser.parse_annotation
def parse_annotation
Definition: generate_TableFunctionsFactory_init.py:1541
generate_TableFunctionsFactory_init.Parser.is_at_end
def is_at_end
Definition: generate_TableFunctionsFactory_init.py:1295
generate_TableFunctionsFactory_init.Parser.match
def match
Definition: generate_TableFunctionsFactory_init.py:1346
generate_TableFunctionsFactory_init.Parser.expect
def expect
Definition: generate_TableFunctionsFactory_init.py:1304
generate_TableFunctionsFactory_init.Parser.parse_templates
def parse_templates
Definition: generate_TableFunctionsFactory_init.py:1509
generate_TableFunctionsFactory_init.Parser.parse_args
def parse_args
Definition: generate_TableFunctionsFactory_init.py:1401
generate_TableFunctionsFactory_init.Parser.line
line
Definition: generate_TableFunctionsFactory_init.py:1289
generate_TableFunctionsFactory_init.UdtfNode
Definition: generate_TableFunctionsFactory_init.py:1026
generate_TableFunctionsFactory_init.Parser.parse_primitive
def parse_primitive
Definition: generate_TableFunctionsFactory_init.py:1490
generate_TableFunctionsFactory_init.Parser.consume
def consume
Definition: generate_TableFunctionsFactory_init.py:1315
generate_TableFunctionsFactory_init.Parser.parse_udtf
def parse_udtf
Definition: generate_TableFunctionsFactory_init.py:1353

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.raise_parser_error (   self,
  msg = None 
)

Definition at line 1334 of file generate_TableFunctionsFactory_init.py.

References ChunkIter.current_pos, generate_TableFunctionsFactory_init.Parser.current_pos(), generate_TableFunctionsFactory_init.Tokenize.current_token(), generate_TableFunctionsFactory_init.Parser.current_token(), generate_TableFunctionsFactory_init.Tokenize.tokens(), and generate_TableFunctionsFactory_init.Parser.tokens().

Referenced by generate_TableFunctionsFactory_init.Parser.consume(), and generate_TableFunctionsFactory_init.Parser.parse_primitive().

1334 
1335  def raise_parser_error(self, msg=None):
1336  if not msg:
1337  token = self.current_token()
1338  pos = self.current_pos()
1339  tokens = self.tokens
1340  msg = "\n\nError while trying to parse token %s at pos %d.\n" "Tokens: %s" % (
1341  token,
1342  pos,
1343  tokens,
1344  )
1345  raise ParserException(msg)
generate_TableFunctionsFactory_init.Parser.current_token
def current_token
Definition: generate_TableFunctionsFactory_init.py:1298
generate_TableFunctionsFactory_init.Parser.current_pos
def current_pos
Definition: generate_TableFunctionsFactory_init.py:1331
generate_TableFunctionsFactory_init.Parser.raise_parser_error
def raise_parser_error
Definition: generate_TableFunctionsFactory_init.py:1334
generate_TableFunctionsFactory_init.Parser.tokens
def tokens
Definition: generate_TableFunctionsFactory_init.py:1292
generate_TableFunctionsFactory_init.ParserException
Definition: generate_TableFunctionsFactory_init.py:396

+ Here is the call graph for this function:

+ Here is the caller graph for this function:

def generate_TableFunctionsFactory_init.Parser.tokens (   self)

Definition at line 1292 of file generate_TableFunctionsFactory_init.py.

References generate_TableFunctionsFactory_init.Tokenize._tokens, and generate_TableFunctionsFactory_init.Parser._tokens.

Referenced by generate_TableFunctionsFactory_init.Parser.raise_parser_error().

1292 
1293  def tokens(self):
1294  return self._tokens
generate_TableFunctionsFactory_init.Parser._tokens
_tokens
Definition: generate_TableFunctionsFactory_init.py:1287
generate_TableFunctionsFactory_init.Parser.tokens
def tokens
Definition: generate_TableFunctionsFactory_init.py:1292

+ Here is the caller graph for this function:

Member Data Documentation

generate_TableFunctionsFactory_init.Parser._curr
private

Definition at line 1288 of file generate_TableFunctionsFactory_init.py.

Referenced by generate_TableFunctionsFactory_init.Parser.advance(), generate_TableFunctionsFactory_init.Parser.current_pos(), generate_TableFunctionsFactory_init.Parser.current_token(), generate_TableFunctionsFactory_init.Parser.expect(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.lookahead(), generate_TableFunctionsFactory_init.Parser.parse(), generate_TableFunctionsFactory_init.Parser.parse_args(), and generate_TableFunctionsFactory_init.Parser.parse_type().

generate_TableFunctionsFactory_init.Parser._tokens
private

Definition at line 1287 of file generate_TableFunctionsFactory_init.py.

Referenced by generate_TableFunctionsFactory_init.Parser.consume(), generate_TableFunctionsFactory_init.Parser.current_token(), generate_TableFunctionsFactory_init.Parser.expect(), generate_TableFunctionsFactory_init.Parser.is_at_end(), generate_TableFunctionsFactory_init.Parser.lookahead(), and generate_TableFunctionsFactory_init.Parser.tokens().

generate_TableFunctionsFactory_init.Parser.line

Definition at line 1289 of file generate_TableFunctionsFactory_init.py.

Referenced by generate_TableFunctionsFactory_init.Tokenize.add_token(), generate_TableFunctionsFactory_init.Tokenize.current_token(), generate_TableFunctionsFactory_init.Tokenize.is_at_end(), generate_TableFunctionsFactory_init.Tokenize.lookahead(), generate_TableFunctionsFactory_init.Parser.parse_udtf(), generate_TableFunctionsFactory_init.Tokenize.peek(), and generate_TableFunctionsFactory_init.Tokenize.raise_tokenize_error().

The documentation for this class was generated from the following file: