Dhaka::Lexer

The lexer generator. To generate a lexer from a lexer specification MyLexerSpecification:

lexer = Dhaka::Lexer.new(MyLexerSpecification)

To compile this lexer as MyLexer to a string of Ruby source:

lexer.compile_to_ruby_source_as(:MyLexer)

Attributes

specification[R]

Public Class Methods

new(specification) click to toggle source

Creates a new lexer from a given specification.

# File lib/dhaka/lexer/lexer.rb, line 11
def initialize(specification)
  dfas           = {}
  @specification = specification
  specification.items.each do |pattern, item|
    dfas[pattern] = LexerSupport::DFA.new(pattern)
  end
  super(ItemSet.new(dfas.values.collect{|dfa| dfa.start_state}))
end

Public Instance Methods

action_for_pattern(pattern) click to toggle source

# File lib/dhaka/lexer/lexer.rb, line 38
def action_for_pattern pattern #:nodoc
  @specification.items[pattern].action
end

compile_to_ruby_source_as(lexer_class_name) click to toggle source

Compiles the lexer to Ruby code that when executed, reloads all the states and actions of the lexer into a class named lexer_class_name.

# File lib/dhaka/lexer/lexer.rb, line 22
def compile_to_ruby_source_as lexer_class_name
  result  =   "class #{lexer_class_name} < Dhaka::CompiledLexer\n\n"
  result <<   "  self.specification = #{specification.name}\n\n"
  result <<   "  start_with #{start_state.object_id}\n\n"
  @states.each do |key, state|
    result << "#{state.compile_to_ruby_source}\n\n"
  end
  result <<   "end"
  result
end

lex(input) click to toggle source

Returns a LexerRun that tokenizes input.

# File lib/dhaka/lexer/lexer.rb, line 34
def lex input
  LexerRun.new(self, input)
end

Home Classes Methods

In Files

Parent

Methods

Class/Module Index

Dhaka::Lexer

Attributes

Public Class Methods

Public Instance Methods