NAME

pirparser.c - parser for Parrot Intermediate Representation

THOUGHTS FOR LATER

• Add error correcting stuff, so parsing can continue after simple errors.
• Maybe use #define's to implement small functions. Effectively, these routines will be inlined in compilation_unit() or instruction(), but when using #define's it's still code that looks good.
• Skip first token of statements if it's sure that the token has already been checked by the caller. For instance, no need to match 'goto' again, if it was already checked in compilation_unit().
• Maybe use compilers/bcg for back-end, if that would fit well.
• Clean up grammar after discussion.

+ For instance, why have :postcomp and :immediate, having same meaning?, and why ".sub" and ".pcc_sub"?

+ why "object" as type? Why custom names as type, such as '.local Array x' -- this is not needed, and makes the code look more like a HLL. Just stick to 'pmc', which works fine.

+ why allow an optional comma between sub pragmas? param pragmas don't have that so remove this opt. comma as well for consistency.

+ unify '.sym' and '.local'. Just allow one, and think of something else for local labels in macros.

PARSER INTERNALS

The parser_state structure has the following fields:

 typedef struct parser_state {
    struct     lexer_state *lexer;     -- the lexer
    token      curtoken;               -- the current token as returned by the lexer
    char     **heredoc_ids;            -- array for holding heredoc arguments
    unsigned   heredoc_index;          -- index to keep track of heredoc ids in the array
    unsigned   parse_errors;           -- counter for parse_errors
    pirvtable *vtable;                 -- vtable holding pointers for output routines
 }

PARSER API

• void exit_parser()

Clean up and exit the program normally.

• get_parse_errors()

return the number of parse errors.

• new_parser()

constructor for a parser_state object. The specified filename is parsed. The semantic actions in vtable are called at certain points in the code. The vtable is constructed in the parser's client code.

• struct lexer_state const *get_lexer()

returns the specified parser's lexer

• token get_token()

returns the specified parser's current token

HELPER FUNCTIONS

• static void resize_heredoc_args()

Reallocate memory for the array holding heredoc arguments. If needed, the array is resized to twice its previous size. So, initially it's MAX_HEREDOC_ARGS, after the first resize(), it's 2 times MAX_HEREDOC_ARGS, after the second time it's 2 * 2 * MAX_HEREDOC_ARGS, etc.

• static void syntax_error()

Handle all syntax error through this function. numargs is the number of variable arguments. All arguments should be of type "char *" !!!

• static void match()

checks whether the current token is the same as the expected token. If so, all is ok, and the next token is fetched. If not, an appropiate syntax error is reported.

GRAMMAR

Conventions

The following conventions are used:

 [ foo ]       indicate an optional foo element
 { foo }       indicate zero or more foo elements
 ( foo | bar ) either foo or bar
 IDENTIFIER    match a token of type IDENTIFIER
 'string'      match the literal 'string'

Grammar rules

Expressions

  expression -> ( IDENTIFIER | INTC | NUMC | STRINGC | register )

  string_value -> SREG | PASM_SREG | STRINGC

  method -> IDENTIFIER | STRINGC

  target   -> register | IDENTIFIER

  type -> 'int' | 'num' | 'pmc' | 'string'

  key -> '-' expression | '..' expression | expression [ '..' [ expression ] ]

  keylist -> '[' key { (';'|',') key } ']'

  arg_flags -> { arg_flag }

  arg_flag -> ':flat' | ':named' [ '(' STRINGC ')' ]

  argument -> HEREDOCID | expression arg_flags | STRINGC ('=>' expression | arg_flags)

  argument_list -> argument { ',' argument }

  arguments -> '(' [argument_list] ')' heredoc_arguments

  heredoc_arugments -> { HEREDOC_STRING }

Statements

  methodcall -> INVOCANT_IDENT method arguments

  arith_expr -> [ binop expression ]

  binop      -> '+'  | '-' | '*'  | '/' | '//' | '%'  | '~~'  | '~'
              | '&&' | '&' | '||' | '|' | '<<' | '>>' | '>>>' | '.'

  parrot_instruction -> PARROT_OP [ expression {',' expression } ]

  assignment -> '=' ( unop expression
                    | expression arith_expr
                    | target ( keylist | [ '->' method ] arguments )
                    | STRINGC arguments
                    | 'global' STRINGC
                    | heredocstring
                    | methodcall
                    | 'null'
                    | parrot_instruction
                    )

  unop       -> '-' | '!' | '~'

  return_statement -> '.return' ( arguments
                                | target ['->' method] arguments
                                | methodcall
                                )
                                '\n'

  yield_statement -> '.yield' arguments '\n'

  close_ns -> '.endnamespace' IDENTIFIER '\n'

  open_ns -> '.namespace' IDENTIFIER '\n'

  local_id_list -> local_id { ',' local_id }

  local_id  -> IDENTIFIER [':unique_reg']

  declaration_list -> type local_id_list '\n'

  sym_declaration -> '.sym' declaration_list

  local_declaration -> '.local' declaration_list

  lex_declaration -> '.lex' STRINGC ',' target '\n'

  conditional_expression -> expression [cond_op expression]

  cond_op -> '>' | '>=' | '<' | '<=' | '==' | '!='

  jump_statement -> 'goto' IDENTIFIER '\n'

  goto_statement -> jump_statement

  unless_statement -> 'unless' (['null'] expression | conditional_expression) jump_statement

  if_statement -> 'if' (['null'] expression | conditional_expression) jump_statement

  const_definition -> 'int' IDENTIFIER '=' INTC
                    | 'num' IDENTIFIER '=' NUMC
                    | 'pmc' IDENTIFIER '=' STRINGC
                    | 'string' IDENTIFIER '=' STRINGC

  param_flags -> { param_flag }

  param_flag -> ':slurpy'
              | ':named'['(' STRINGC ')']
              | ':unique_reg'
              | ':optional'
              | ':opt_flag'

  invokable -> IDENTIFIER | PREG

  long-invocation -> '.pcc_begin' '\n'
                     { '.arg' expression arg_flags }
                     ( '.pcc_call'|'.nci_call') invokable '\n'
                     | '.invocant' invokable '\n'
                       '.meth_call' method '\n'
                     )
                     { (local_declaration | '.result' target param_flags '\n') }
                     '.pcc_end' '\n'

  long_return_statement -> '.pcc_begin_return' '\n'
                           { '.return' expression arg_flags '\n' }
                           '.pcc_end_return' '\n'

  long_yield_statement -> '.pcc_begin_yield' '\n'
                          { '.return' expression arg_flags '\n' }
                          '.pcc_end_yield' '\n'

  target_statement -> target ( '=' assignment
                             | augmented_op expression
                             | keylist '=' expression
                             | '->' method arguments
                             | arguments
                             )
                             '\n'

  augmented_op     -> '+=' | '-=' | '%=' | '/='  | '//=' | '*='  | '.='
                    | '~=' | '&=' | '|=' | '**=' | '<<=' | '>>=' | '>>>='

  target_list -> '(' target param_flags {',' target param_flags } ')'

  multi-result-invocation -> target_list '=' (invokable arguments | methodcall) '\n'

  invokable -> IDENTIFIER | PREG | STRINGC

  macro_expansion -> MACRO_IDENT [ '(' [ expression { ',' expression } ')' ] '\n'

  get_results_instr -> '.get_results' target_list '\n'

  global_assignment -> 'global' string_value '=' (IDENTIFIER|PREG) '\n'

  instructions -> {instruction}

  instruction  -> {LABEL ['\n']} instr

  instr -> if_statement
         | unless_statement
         | local_declaration
         | sym_declaration
         | lex_declaration
         | '.globalconst' const_definition
         | '.const' const_definition
         | open_ns
         | close_ns
         | return_statement
         | yield_statement
         | macro_expansion
         | target_statement
         | STRINGC arguments
         | methodcall
         | long_invocation
         | long_return_statement
         | long_yield_statement
         | 'null' var
         | get_results_instruction
         | global_assignment
         | '\n'

Compilation Units

  global_definition -> '.global' IDENTIFIER

  multi-type-list -> '(' [multi-type {',' multi-type } ] ')'

  multi-type -> IDENTIFIER | STRINGC | keylist | type

  sub_flags -> [sub_flag { [','] sub_flag } ]

  sub_flag  -> ':anon'
             | ':init'
             | ':load'
             | ':main'
             | ':method'
             | ':lex'
             | ':outer' '(' ( STRINGC | IDENTIFIER )  ')'
             | ':vtable' '(' STRINGC ')'
             | ':multi' multi-type-list
             | ':postcomp'
             | ':immediate'

  parameters -> { '.param' (register | type IDENTIFIER) [param_flag] '\n' }

  sub_definition -> '.sub' (IDENTIFIER | STRINGC) subflags '\n' parameters instructions '.end'

  emit_block -> '.emit' '\n' { parrot_instruction '\n' } '.eom'

  macro_parameters -> [ '(' [ id {',' id} ] ')' ]

 macro-definition -> '.macro' IDENTIFIER macro_parameters '\n' macro_body '.endm'

  include -> '.include' STRINGC

  pragma -> '.pragma' 'n_operators' INTC

  hll_specifier -> '.HLL' STRINGC ',' STRINGC

  hll_mapping -> '.HLL_map' INTC ',' INTC

  namespace_declaration -> '.namespace' [ '[' STRINGC { (','|';') STRINGC ']' ]

  loadlib -> '.loadlib' STRINGC

  compilation_unit -> global_definition
                    | sub_definition
                    | '.const' const_definition
                    | emit_block
                    | include
                    | macro_definition
                    | pragma
                    | loadlib
                    | namespace_declaration
                    | hll_specifier
                    | hll_mapping

  program -> {'\n'} compilation_unit { '\n' compilation_unit } EOF

  TOP -> program