Starbeamrainbowlabs

Stardust
Blog

Skyliner: Automated text document outlining

When editing large documents, it's often helpful to have a hierarchical "navigation view" of sorts. For a text document like the Markdown that I'm typing now for this blog post, it would consist of a list of headings in the document. For a Javascript or C♯ file, it might consist of classes, functions, and methods.

Either way, it's a helpful thing to have - but for some ridiculous reason as far as I know a generic text document outlining tool doesn't exist.

While GitHub's Atom (my code editor of choice) has an atom-ide has an outline view that works great, it only supports a limited number of languages (you have to have a plugin installed for every language), and sometimes it hangs and takes like 30 seconds plus to generate the outline.

To this end, I intend to remedy the situation with a new library I'm writing call Skyliner.

It's based on finite-state automata and regular expressions (also, Wikipedia and regexper), and it streams the input and generates an outline line-by-line. It provides both a command-line interface and a Javascript API (though the command-line interface currently consumes all input before generating any output, but the library is capable of streaming objects as it consumes the input). As of the time of typing, it supports the following languages:

While using regular expressions means that the output won't be perfect (especially in the case of XML/HTML), it does mean that adding support for a new language is as simple as defining a new finite-state automaton in a Javascript object. Adding support for Lua was a 10-15 minute job - including automated tests! Support for more languages is definitely on the way.

The combination of line-by-line parsing, regular expressions, and finite-state automata also means that it's much faster than Atom IDE, because it doesn't have to parse the entire document into an abstract syntax tree before generating the outline.

My goal here is to have good support as many languages as possible, rather than amazing support for just a small handful of languages. This isn't to say that I won't fix issues with existing languages as they come up, but the focus is really on ensuring that whenever I open a text document in Atom.

Once I've added support for a bunch of languages, written some documentation, and published it on npm, I'm going to try my hand it implementing my first plugin for GitHub's Atom. Apparently plugins for Atom aren't too difficult to port to Visual Studio Code, so maybe I'll take a look at doing that too (but I don't use Visual Studio Code much, so not a priority).

The code is open-source already (link below) and completely usable, but since it's an ongoing process you can expect another post on here in the future about my progress.

Skyliner on GitHub: https://github.com/sbrl/skyliner

If you'd be interested in some more detail about how it works, I'll be writing some documentation soon, which will appear in the above Git repository.

Tag Cloud

3d 3d printing account algorithms android announcement architecture archives arduino artificial intelligence artix assembly async audio automation backups bash batch blog bookmarklet booting bug hunting c sharp c++ challenge chrome os cluster code codepen coding conundrums coding conundrums evolved command line compilers compiling compression containerisation css dailyprogrammer data analysis debugging demystification distributed computing docker documentation downtime electronics email embedded systems encryption es6 features ethics event experiment external first impressions future game github github gist gitlab graphics hardware hardware meetup holiday holidays html html5 html5 canvas infrastructure interfaces internet interoperability io.js jabber jam javascript js bin labs learning library linux lora low level lua maintenance manjaro network networking nibriboard node.js operating systems own your code pepperminty wiki performance phd photos php pixelbot portable privacy problem solving programming problems project projects prolog protocol protocols pseudo 3d python reddit redis reference releases rendering resource review rust searching secrets security series list server software sorting source code control statistics storage svg talks technical terminal textures thoughts three thing game three.js tool tutorial tutorials twitter ubuntu university update updates upgrade version control virtual reality virtualisation visual web website windows windows 10 xmpp xslt

Archive

Art by Mythdael