Amit's Thoughts: 2003-08

From Amit’s Thoughts

Boost Libraries for C++ #

I've been wanting to use the Boost libraries for quite a while now. Unfortunately I always got stuck when trying to install them. Now there's a solution (for me, anyway) -- RedHat's Rawhide distribution includes boost, boost-devel, boost-python, boost-python-devel, and boost-doc. I found these RPMs on rpmfind.net.

Labels: howto

Yapps 2.1 #

I've been working on a major upgrade to Yapps. I had a lot of ideas in mind but I decided to first concentrate on usability. In particular, error messages from parsers tend to be bad. I'm not sure what I actually want to see as an error message though. In the past I tested Yapps by using good grammars and inputs and making sure the output was correct. I'm now starting to build up a set of bad grammars and inputs. I'm not yet sure how to evaluate the error messages in an automated way, so for now I'm manually checking them.

Writing good error messages is hard. You have to get into the right frame of mind -- that of your user -- and figure out what would be most helpful to the person trying to figure out how to fix the error.

Commercializing the Internet #

There’s got to be content on the Internet that people pay for, or else it’s not viable. The Internet needs to be a vehicle for mass-market advertising and for compelling content. We have to figure out the advertising part, because multiple streams of revenue are vital.

-- Merrill Brown, senior vice president of Real Networks

I just want to say to Mr. Brown: GO AWAY! Leave our Internet alone!

Yapps 3.0 #

[Update: I abandoned this project for several reasons, the main one being that I didn't have an immediate use for a parser system at the time.]

I've been working on a major upgrade to Yapps (Yet Another Python Parser System). Yapps was designed with two things in mind: it should be easy to use and it should be easy to understand output. I think I got those two pretty well. However, as I try to push the system to do more, I'm finding that easy to use and easy to understand output are coming into conflict:

• LL(1) recursive descent parsers are easy to understand. But they make it harder to write grammars, because you have to manually left-factor.
• Better error messages would make Yapps easier to use. But tracking the information needed for better error messages clutters up the parser, making it harder to understand.

So far, I've been leaning towards more ease of use and less readable parsers. Some more things I want to implement towards that goal:

• Abstract rules: a way to reuse common patterns used in writing rules.

  If you want to parse a comma-separated list of numbers in Yapps 2, you'd write a rule like this:

  rule number_list:
        {{ numbers = [] }}
        (
           number {{ numbers.append(number) }}
           ( ',' number {{ numbers.append(number) }}
           )*
        )?
        {{ return numbers }}
  

  If you then want to parse a colon-separated list of strings, you have to write a very similar rule with a different delimiter and pattern. When you need to parse a semicolon-separated list of statements, you may be wondering how you can avoid writing this pattern repeatedly.

  Abstract rules are rules that have named placeholders. For the above rule, you'd write

  rule delimiter_separated_list[delimiter][element]:
        {{ results = [] }}
        (
           element {{ results.append(element) }}
           ( delimiter element {{ results.append(element) }}
           )*
        )?
        {{ return results }}
  

  You'd then be able to invoke that rule with delimiter_separated_list[','][number], delimiter_separated_list[':'][string], and delimiter_separated_list[';'][statement]. I'm running out of delimiters, though. I've already used [ A B C ] for optional matches. I'll probably switch to the standard EBNF form for optional matches, (A B C)?, to free up brackets for something else.

• Inheritance of Parsers: the ability to reuse an existing parser's rules when defining a new parser.

  In Yapps 2, I find that I often copy and paste tokens and rules from one parser to another. The most common ones are strings, numbers, identifiers, and delimiter-separated lists. I'd like to be able to define these in one place and then inherit them in the parsers I'm writing.

  To implement this, Yapps needs to be able to decipher an existing parser's First and Follow sets. I can either do this via reflection at the Python class level or I can implement inheritance directly at the grammar level. Either way, parsers would be more useful if you could reuse them in larger parsers.

I've also been thinking about some more radical changes to Yapps:

• Backtracking: abandoning the LL(1) restriction.

  The Toy Parser Generator (don't be fooled by the name -- it's quite powerful!) supports full backtracking. You don't have to worry about whether your grammar is LL(1) or not; you can write it in a more natural style. However, I'm not sure how backtracking interacts with imperative actions. I've thought about using something like Haskell monads, but without decent support for blocks, it's hard to do this properly in Python. I might be able to delay all the actions by constructing a string at parse time and then using exec when the parser is complete. However, that has the disadvantage that the actions can't influence the parser. (On the other hand, without backtracking, the actions can't meaningfully influence the parser anyway.)

• Run Time Parser Generation: abandoning the model of generating a Python source file.

  One annoying feature of most parser generators is that they generate source code, which then has to be compiled/run in a separate process. It'd be much nicer to build a parser within a running program. With Python, this doesn't seem hard to do. Even in the most static of languages -- C++ -- you can generate parsers without creating a separate source file, using the Spirit parser library.

I'm not yet sure whether I'll implement the more radical changes.

Labels: project

Cancel your Discover Card #

Since cancelling my Discover Card, I've been receiving nasty phone calls from people at Discover trying to get me to come back. This latest caller was rude/condescending to my wife, then said he must talk to me. He then tried to convince me to keep my account open but not get a credit card. And he'd give me a 0% interest rate. He says I should keep the Discover Card in case of emergency. I keep telling him no, but he keeps pushing. Grr.

Discover, if you're listening, here's a tip: being rude and nasty to potential customers is a bad business strategy.

I'm not the only one who has had bad experiences with Discover Card's customer service folk. Read this letter and this letter and this letter (at PlanetFeedback.com).

Update [2004-08-01]: See this list of bad experiences with cancelling a Discover card.

Labels: review

Pick your battles #

Clay Shirky reminds us that whether we're for or against some new techonology, it doesn't really matter. It's a better use of your time to figure out how to deal with the new technology.

Learning languages #

Lojban is an interesting looking language. Also take a look at their phonetic rules.