Text to HTML: syntax #

In any discussion about a new markup language (like ReStructuredText), there will be some debate about syntax. Here’s what I’ve been reading:

There are lots of good ideas out there, and there are a lot of issues I hadn’t thought about. My own notes tend to be formatted so that I can easily edit them with XEmacs and filladapt-mode. Where I don’t have particular conventions already established (like named links), I would like to use what’s already out there.

For allowing HTML in my notes, I was originally thinking that I’d require XHTML, but this note about XHTML made me realize that all I wanted was a way to find closing tags, and not all of XHTML. I plan to require all tags to be closed (<X> ... </X>) or self-closing (<X />), but I do not plan to require other aspects of XHTML.

I envision three levels of parsing:

  • At the top level, a notes file is composed of several notes separated by some marker. The marker may contain some information. For my notes files, the marker is [bracketed text] on a line by itself, and the text inside brackets is the name of the note.
  • At the block level, there are four types of blocks:
    1. Lists begin with a bullet (*, 1., etc.).
    2. HTML blocks begin with an HTML block-level tag (<p>, <div>, etc.).
    3. Preformatted text begins with spaces for indentation. All subsequent lines indented at least that number of spaces will be part of the preformatted text block.
    4. Any other text (including HTML inline tags) begins a paragraph.
  • At the inline level, there are three types of spans:
    1. HTML spans begin with an HTML inline-level tag (<img>, <a>, <abbr>, etc.) and end at the corresponding closing tag.
    2. Markup spans begin with some markup character (*, `, \) and typically end with the same markup character. However, the rules are tricky because markup characters sometimes occur in contexts where they are not being used for markup. The \ character is used for escaping in contexts where the following character would otherwise be treated as a markup character. (This seems complicated.)
    3. Text spans begin with any text.

Indentation plays a key role in parsing preformatted and list blocks Implementing this may be trickier than I initially expected, especially because my notes were not written with these rules in mind.

ReStructuredText and Twiki have had a lot of good work put into them, and it’s silly to reinvent it, unless I’m trying to solve a different problem. I have to remind myself that my main goal is to parse my existing notes, and my secondary goal is to be able to write new notes easily. If the syntax of old and new documents is going to be different, then perhaps my approach should be to write a parser for my old notes that converts everything to my new note format, or perhaps I should manually convert everything.

– Amit – Saturday, August 28, 2004 – 2 comments

Text to HTML #

I’ve been looking through my old notes and have realized that like many other people, I follow certain conventions when writing text files. I’d like to publish some of these notes to a web site. The simplest thing to do would be to upload the notes as text. The second simplest thing to do would be to upload the notes as HTML, with a <pre> around the entire note. However, it would be cooler if I could take advantage of the conventions I've used to write something richer than text.

There are lots of systems that allow you to write text and produce HTML. I wrote one a few years ago to turn this text into this set of HTML pages. ReStructuredText turns text into HTML; see the sample page. Wikis turn text into HTML as well. Of these, the goals of structured text were the closest to my own: intuitive, simple, readable. However, none of these actually worked with my own text formatting conventions. My A* pages use { ... } for formatting. Wikis use lots of ` backquotes `. For example, see this set of rules, including “Use five single-quotes, or triples within doubles, for some other kind of emphasis.” ReStructuredText conventions are the closest to what I've used in the past.

My goal was to have a system that could read my existing notes for the past 15 years. I didn’t want to go back and edit all those notes to fit some formatting convention. In particular, these are some differences between my conventions and ReStructuredText:

  • I often use indentation for preformatted text. ReStructuredText uses indentation for indented paragraphs (using <blockquote>—ew!) and :: for preformatted text.
  • I use *bold*, /italics/; ReStructuredText uses **bold**, *italics*. I don’t think this would really matter though.
  • I don’t have a good convention for sections. In the last 5 years I’ve been using [header] on a line by itself, but I don’t have any way to have subsections. I have been writing short notes, not documents, so I never really needed subsections. ReStructuredText uses text with a line underneath or on top to mark section headings. It seems like a nice convention, but my existing notes don’t use it.
  • I don’t have a way to include images, because I am just writing simple notes. ReStructuredText has this: 
    .. image:: images/biohazard.png
   :height: 100
   :width: 200
   :scale: 50
   :alt: alternate text
    At this point, I started wondering why I would want to write ReStructuredText instead of HTML: <img src="images/biohazard.png" height=100 width=200 alt="alternate text">.
  • I have no way to include links with anchor text. I do include URLs and email addresses, often but not always surrounded by <...>. ReStructuredText uses ` and _ characters, with several nice ways to avoid writing URLs inline in the document body.

Looking over my notes, I’ve realized that this task is harder than it initially seemed. For example, my notes mix plain text with code/output. I’d like plain text to be rendered in proportional font and code/output to be rendered in monospace font. I don’t have a good way to solve this for my existing notes. For future notes, I could use ReStructuredText’s convention of ``text``.

I have two problems to solve: how do I read my existing notes, and what do I do for new notes? I’m not trying to write documents (I am comfortable writing HTML); I’m looking for something simpler for quick notes. The ReStructuredText format for images made me realize that it’s really a new markup language. Simple things like paragraphs and lists are close to what I’d write naturally, but most of the formatting conventions are not. Then it hit me: if I need a markup language anyway, why not use HTML?

The problem with HTML (especially for quick note taking) is that it’s cumbersome for the simple things: text, paragraphs, lists, code snippets. For less common things (images, tables, anchor text), it isn’t as much of a win to invent a new language. So my plan is to parse text with the most common text formatting conventions (paragraphs, lists, etc.), but allow any HTML tag to “escape” into HTML mode. That way if I want to do anything complicated, I can drop into HTML, but if I want to stay simple, I can write plain text.

Why doesn’t ReStructuredText do the same thing? StructuredTextNG includes a rule to let people use SGML, but it didn’t seem to get much attention. I suspect it’s because they do not want to assume that HTML is the output format (it might be PDF or XML) and the target audience is less comfortable with HTML. TWiki also has both structured text and HTML. The original Wiki does not, and there is a discussion about pros and cons.

It will take some time to play with this system before I know whether I like it. I might end up finding that I do want to avoid HTML altogether, or that I want to write entire documents in text. I am likely to find that some of my old notes are ambiguous and cannot be parsed easily. It may be easier to go through them and fix them up instead of writing a more complicated parser.

Update: [2011] It seems that Markdown has won, not only for me, but for lots of other people. It also allows embedding HTML for when Markdown isn't enough. As of 2011, Markdown is being used by an increasing number of tools and apps.

– Amit – Saturday, August 28, 2004 – 1 comment

Emacs keys in Windows apps #

The XKeymacs utility claims to implement Emacs keybindings in Windows apps. I haven’t tried it yet.

Labels: ,

– Amit – Saturday, June 12, 2004 – 2 comments

Giving to charities #

If you are considering giving to charities, consider giving the tax deduction to them too. (In the United States at least, donations to charities are considered “tax-deductible” and you do not have to pay taxes on that amount.) If you donate an item or cash worth x, and your tax rate is r (30% would mean r=0.3), then you’re going to get x*r back in your tax refund. Give that to the charity (in cash). You’ll get x*r2 back in your tax refund. Give that to the charity too. You’ll get x*r3 back. Give that to the charity too. Rinse and repeat.

What the charity will end up getting is x + x*r + x*r2 + x*r3 + .... Written another way, that's x * (1 + r + r2 + r3 + ...). This can be rewritten as x * (1 / (1 - r)).

So when you’re giving an item worth x to the charity, if you decide to give all the tax credits to them, they will end up with something worth x / (1 - r). Let’s say you’re donating an item worth $100 and your marginal tax rate is 33% (federal + California can easily be this high). You can give them $100/(1-0.333) = $150 if you go through the itemized tax deductions. So give them the original item plus the $50. That’s a big bonus for them (especially since it’s cash), and all it costs you is the time and hassle of keeping track of the receipts for tax purposes. If you had kept the tax credit, you would have $33, but if you give it to them, it’s worth $50.

– Amit – Sunday, May 30, 2004 – 1 comment

Zen Stories #

Last night I found a nice collection of Zen stories.

– Amit – Wednesday, May 26, 2004 – 0 comments
Subscribe to: Posts (Atom)