common treetop pattern for parsing lists of things

May 8th, 2008 •
tags: peg, ruby, treetop

rule expression_list
  expression tail:(SPACE expression)* {
    def eval(env = {})
      expressions.inject([]) do |exprs, expr|
        exprs << expr.eval(env)
      end
    end

    def expressions
      [ expression ] + tail.elements.map { |e| e.expression }
    end
  }
end

comment and keyword handling in treetop

May 8th, 2008 •
tags: peg, ruby, treetop

# whitespace
rule white
[ \r\t\n]+
end

# mandatory space
rule SPACE
(white / comment)+
end

# optional space
rule space
SPACE?
end

# anything but not white
rule non_white_char
!white .
end

# typical keyword rule
rule var_keyword
'var' !non_white_char
end

rule comment
comment_to_eol / multiline_comment
end

rule multiline_comment
'/*' (!'*/' . )* '*/'
end

rule comment_to_eol
# TODO find out why this doesn't work in specs
#'#' (!"\n" .)+ "\n"

'#' (!"\n" .)*
end

(ab)using ruby within treetop(?)

May 8th, 2008 •
tags: peg, ruby, treetop

rule case_expression

case_keyword SPACE case_exp:expression SPACE
when_expression_list SPACE
else_keyword SPACE else_exp:statement_list SPACE
end_keyword

{
def eval(env = {})
case_val = case_exp.eval(env)
else_val = else_exp.eval(env)
Kernel.eval <<-CASE_STMT
lambda do
case #{case_val.is_a?(String) ? "'#{case_val}'" : case_val}
#{ruby_when_expressions(env)}
else #{else_val.is_a?(String) ? "'#{else_val}'" : else_val}
end
end [] # call this lambda immediately
CASE_STMT
end

def ruby_when_expressions(env = {})
when_expression_list.eval(env).inject('') do |ruby, e|
# possible string values have been wrapped in '' already
ruby << "when #{e[:condition]} then #{e[:expression]} "
end
end
}

end

interactive treetop

March 27th, 2008 •
tags: peg, ruby, treetop

# All credits go to Clifford Heath
# --------------------------------

require 'rubygems'
require 'treetop'
require 'my_grammar'
require 'readline'

parser = MyGrammarParser.new
while line = Readline::readline("? ", [])
  begin
    p parser.parse(line)
    rescue => e
    puts e
  end
end
puts

treetop >= 1.2.3 works nice with rails

March 9th, 2008 •
tags: rails, treetop

treetop-1.2.3 and polyglot-0.2.1 solve the problems treetop previously had when used from a rails app. The dependencies on ruby facets were removed, and the load_error bug was fixed in polyglot

treetop-1.2.2 rails workaround

March 5th, 2008 •
tags: rails, treetop

This was originally posted to the treetop-development group on google

First of all thx a lot for this wonderful treetop! Now I can finally
start to write small parsers because it’s the ruby I love :)

Anyways, here are my comments on the issues encountered by Nick. (I
use treetop from within a rails app)
I admit I didn’t have to dig as deep as him (thx for all the digging
and the post in this group, it saved HOURS for me most probably!)

1) To get my app running in development mode, I had to comment out the
following in treetop-1.2.2/lib/treetop.rb

# require “facets/stylize”

The funny thing is, if I don’t do that, it doesn’t break my whole app
like Nick said, but only some of my calls to
skip_before_filter which don’t stop the before_filters from
being executed anymore (The only failure I noticed in my app, was that
I couldn’t access actions that relied on skipping a before_filter that
was defined in ApplicationController – then again, my
SessionController – login/logout actions – relied on that very
behavior … and didn’t break :-) … I definitely assume that there
is no real problem with rails before_filters in general, but that this
also has something to do with problems related to constant loading
(i.e. camelize), still it’s funny that this seems to impact my app
only in this one scenario …

2) To get my app running in production mode though, I had to comment
out the following in treetop-1.2.2/lib/treetop.rb

# require “facets/stylize”
# require “polyglot”
# Polyglot.register([“treetop”, “tt”], Treetop)

I suppose this doesn’t lead to any problems, since these libs are only
necessary for generating the parser and not for executing it? The
thing here is that polyglot obviously offends rails’ production mode,
since both mongrel and thin both barf on me when I try to access any
actions.

To be more specific, if I require the .treetop file (as should be
possible thx to polyglot), then my server won’t start at all, telling
me friendly things like

../polyglot-0.2.0/lib/polyglot.rb:35:in
`load’: undefined local variable or method `load_error’ for
Polyglot:Module (NameError)

If I load the grammar via Treetop.load, the server starts up, but fails
on every action (the first failure is the same as the above, after
this all subsequent failures seem to be rooted in
load_missing_constant errors most probably due to the behavior that
Nick mentioned.

So for all that are experiencing similar problems:

My current workflow is to leave the require statements on my dev box
while I’m working on the grammar (knowing that I cannot access my app
properly during that time), and having them commented out in the
treetop installation on my production server (and also when I’m
accessing my app on my dev box)