Thursday, February 23, 2012

I find that I often don’t know a structure or abstraction until I’ve worked with something more concrete. This happens in programming but it also happens when creating structured information.

While helping Wild Shadow Studios with Realm of the Mad God, we held our discussions in Google Wave for two years, until Google shut it down. Why did we use a tool everyone else thought was useless?

  • Group Chat: The 3–6 of us could chat about the game.
  • Asynchronous: We could continue conversations while not all of us were online.
  • Nesting: We could reply to individual points made in another message.
  • Attachments: We shared test binaries or screenshots.

It was nice for keeping everyone “on the same page”.

Wave didn’t work well for us when it came to adding structure. We had thousands of chat messages in the system, but when a bug or feature or idea came up, there wasn’t an easy way to transition into a task tracker. As a result, those kinds of things were too easily forgotten.

When I use tools like Asana or Trello or Cohuman, they want me to start with structured data, but they don’t handle the informal unstructured nature of IM or Wave. Where do those tasks come from? They came from somewhere, but that history isn’t preserved when you create a new task in a task tracking tool.

I want a tool that lets me hold unstructured conversations, then extract portions of conversations that led to a task, and move them into my task tracking tool. Unstructured comes first for me; structure is what I add later, once I understand what structure I want. Databases and most structured tools I use get this wrong — they ask me to decide the structure first. Spreadsheets, Wikis, Wave, and most unstructured tools also get this wrong — they let me easily work without structure, but don’t offer me a transition to something structured. JotSpot, before it was bought by Google, is the only tool I tried that had a smooth transition from unstructured to structured.

It’s also possible I’m cursed to like things that have a lot of potential, but can’t be understood in five minutes, and thus are doomed to niche status. I think Wave and JotSpot both fit into this category.



Erik Babel wrote at Friday, February 24, 2012 at 4:49:00 PM PST

Yeah, that would be cool! I think if wave had stayed around longer, we would have seen bots(?) that would track your content and create links to/from structured environment. Like recognizing what bugs you're talking about and storing them as comments in the bug tracker.

I think the reason wave flopped is because it was only browser based. Had we seen a good client that wasn't prone to extreme slowing down on big conversations it would have been less frustrating for the masses.

Alex Schroeder wrote at Saturday, February 25, 2012 at 10:38:00 AM PST

As a wiki developer I'm interested in how to move unstructured page content into a structure. Can you provide an example?

Here are three examples I can think of:

A page full of brainstorming, thread mode stuff, interjections. You'd like to select a region on the page, pick "Copy to new page…" from a context menu, type a new name and do it.

One a page, you'd like to tag a page without having to edit it.

You'd one page with a hierarchical list of links to server as an outline. On any page, you have a link that allows you to look at this outline and use checkboxes (or radiobuttons?) to select where in the outline this particular page belongs.

I use a similar solution for a wiki that essentially collects bookmarks:

(note the link to a bookmarklet and the source code at the bottom)

(note how the links on this page and the unordered lists on the linked pages define the outline that is shown by the add-link script)

What do you think—is this a way out of the impasse?

Amit wrote at Sunday, February 26, 2012 at 9:13:00 AM PST

Erik: I think browsers at the time weren't able to handle it but browsers now are reasonable and two years from now they might handle Wave easily. Perhaps Wave was just before its time.

Alex: I liked the JotSpot approach: annotate information on wiki pages with structure.

Look at the country pages on wikipedia for example. Every page has a box in the top right with name/value pairs.

In the JotSpot world, the country pages could start out unstructured, and then later someone could define a CountryInformation table. Over time, the country pages would be edited to include an instance of CountryInformation.

Data moved from the unstructured part of the page into the CountryInformation table would be available in queries. You'd be able to query all the CountryInformation tables across all pages, and build a sortable filterable list (e.g. database query result) of countries and key metrics.

The primary information resides on the individual country pages but parts can be extracted. There's a smooth transition, and immediate benefits for making the transition, but you don't have to decide up front what that structure is going to be.

I believe JotSpot also allowed multiple structure elements on a page. Unlike a database where related information is stored in separate tables, and is awkward to edit, this approach allowed you to keep all the information in one place and annotate bits of it with the type of structures they represented.

JotSpot was removed long ago. I wish I got a chance to play with some of these ideas.

Alok wrote at Sunday, February 26, 2012 at 9:42:00 AM PST

So let me put some structure into your key arguments in this post.

> I often don’t know a structure or abstraction until I’ve worked with something more concrete

> We had thousands of chat messages in the system, but when a bug or feature or idea came up, there wasn’t an easy way to transition into a task tracker.

> hold unstructured conversations, then extract portions of conversations that led to a task, and move them into my task tracking tool

So what I have done here is extract portions of your post and the task (user story) for what you want seems pretty clear.

Does this fall into the domain of NLP AI, probably does. And at the very least this task could use a lot of help from it. What else could help? Creating little meaningful structures in the unstructured conversations. Simple things like using an inline text formatting system (markdown, textile etc.). They surely can help in emphasizing portions of the conversation. Maybe create a mind map of thought trails from the sentences ...

But this is an interesting idea. And I am sure, the next Google will make a lot of breakthroughs here.

This Google is busy with Google+ and overriding user cookie preferences by forcing crap.

Laura wrote at Sunday, March 18, 2012 at 5:05:00 PM PDT

Great points about what is needed in online collaboration tools. I can definitely envision a tool that has two parts and links back and forth between the unstructured and the structured. There may even be some already, but often they tend to be either too complicated or too simple.

Andrew Okunev wrote at Saturday, March 24, 2012 at 7:34:00 AM PDT

Amit, you have almost literally described our vision behind ( Let me know if you would like to play with it (we are not open to the public yet).

Amit wrote at Saturday, March 24, 2012 at 12:55:00 PM PDT

Andrew, I can't quite tell how works but what I saw on the site looks neat. Thanks for the offer — at the moment I'm not working with a team but I may take you up on that offer in the future :)

Kev Jackson wrote at Wednesday, August 22, 2012 at 9:44:00 PM PDT

HipChat 'feels' similar to Wave in that it's starting place is unstructured chatting rather than a more structure issue-tracker.

It also supports extensions such as integration with build tools and github (which is handy for collaborative development)

Daerst wrote at Monday, September 3, 2012 at 11:34:00 PM PDT

I used trac[1] and some projects. It's like a wiki tailored for developers - with a project roadmap and ticket system you can use for features, bugs and so on. We had the "unstructured" conversations in the wiki and extracted "structured" information into tickets.

The teams were always quite small (6 people max) so this worked well. With more people you will have to deal with collisions when editing the wiki.

For a more real-time communication, I could think of using some EtherPad and just posting a link to it in the wiki. You can have your conversations there, apply some basic structure through various pads and use trac for tickets and keeping results of the conversations in the wiki.


Amit wrote at Monday, September 10, 2012 at 5:33:00 PM PDT

I've also found Rizzoma and Pivory. Both are somewhat Wave-like. Rizzoma seems more about unstructured chat turning into structured tasks and mind maps and wikis; Pivory seems more about unstructured chat turning into a discussion forum.

Amit wrote at Saturday, October 6, 2012 at 7:37:00 AM PDT

Also see Cosheet, which applies some of these ideas to spreadsheets.

BillSeitz wrote at Wednesday, December 5, 2012 at 4:29:00 AM PST

There are a few StructuredWiki engines out there, though I haven't used the structured aspect of them.

I think you'll often find yourself doing at least a little refactoring. The biggest effort in refactoring is non-automatable (NLP), so I think efforts to ease the technical side to be somewhat beside the point (once you learn to use multiple windows, which seems beyond many people).