wincent Wincent Colaiuta's weblog

« January 2008 | Archives index | March 2008 »

February 29, 2008

Add sample database configuration file (wincent.com, aedc881)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:29 PM

Remove default index.html (wincent.com, 1c84a1c)

We want to actually exercise the Rails stack, so we'll let the route fall through to a simple controller.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:06 PM

Add basic Rails test app (wincent.com, 9434674)

This will be a minimal test app for testing my deployment strategies (Git, Capistrano, nginx, Thin etc).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:05 PM

Bump version number for 0.6 release (wikitext, e7bd194)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:07 PM

Update benchmark notes with new baseline (wikitext, eac9f5f)

There have been quite a few changes since the last baseline so I wanted to draw a new line in the sand; also, importantly, this is the first point where we have full compatibility with RHEL 5.1.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:22 PM

Fix spelling error in benchmark notes (wikitext, 1a59fd3)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:22 PM

Let the compiler decide when to inline (wikitext, cf1128c)

This change was initially prompted because I was getting duplicate symbol errors when linking for functions which should have been inlined on RHEL 5.1.

My first trouble-shooting technique was disabling inlining to confirm that the linker errors went away (they did) but it turns out that there was a secondary benefit: the compiler is actually much smarter than me, because when I leave things up to the compiler instead of explicitly requesting inlining, the spec suite runs 10 to 20% faster. There is quite a bit of Ruby/RSpec overhead in the spec suite, so the speed improvement of the parser itself is probably significantly more.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:12 PM

Fix bug in "trim link target" function (wikitext, 44d9cec)

This was a bug exposed thanks to a "value computed is not used" warning emitted when building on RHEL 5.1 (but which doesn't appear on Mac OS X or RHEL 3). Although it was a bug, looks harmless enough: basically the leftmost character would be altered instead of the desired behaviour of moving the leftmost boundary. So the worst-case scenario would be an aesthetic glitch in the output, but it turns out that in the spec suite the glitch never appeared in practice.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:33 PM

RSTRING_PTR and RSTRING_LEN fixes for Ruby 1.8.5 compatibility (wikitext, 7e157f8)

Testing on RHEL 5.1, which comes with Ruby 1.8.5, indicates that 1.8.5 doesn't define the RSTRING_PTR and RSTRING_LEN macros, so define them if they're not already defined.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:28 PM

Fix unused variable warnings for GC variables (wikitext, 6f29a90)

These GC-related variables aren't used, but we don't want to be warned about that seeing as it is intentional (we just want them on the stack to prevent premature GC).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:16 PM

Fix "inline is not at beginning of declaration" warning (wikitext, e5a1e6e)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:10 PM

Turn on warnings (wikitext, 1f410fe)

I noticed that the compile on RHEL 5.1 was much chattier than on Mac OS X and RHEL 3; it turns out that RHEL 5.1 has a bunch of warnings enabled by default wheres on the other targets they are not and so I was getting warning-free builds.

Crank up the warnings so that all problems can be found and fixed, and we can have non-chatty builds on RHEL 5.1.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:07 PM

Return nil from the profiling_parse method (wikitext, cd6bf8e)

This silences a compiler warning.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:54 AM

Remove unused variable (wikitext, 2cff164)

While it is true that we often need to know what's at the top of the stack, we never actually used this variable. Seeing as we often look ahead we are constantly advancing through the token stream and so have to look up the top item dynamically on demand.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:51 AM

February 23, 2008

Bump version number post-release (wikitext, 0c91203)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:28 PM

Bump version number prior to release (wikitext, 3657402)

Now at 0.5.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:26 PM

Add one long-running spec to guard against GC and memory-related bugs (wikitext, c0cc9c3)

Add a spec that processes a string 100,000 times. This causes the spec suite to run in a matter of seconds rather than tenths of a second, but it is desirable to have at least one long-running spec in order to try and catch memory or GC-related bugs.

Without this it appears that the entire spec suite can run without ever triggering GC (or at least, without ever triggering GC enough to expose any memory-related flaws). I know this because in developing the Data_Wrap_Struct code I was seeing the spec suite pass but was seeing problems in the benchmarks because they ran for longer.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:15 PM

Add invalid_encoding test to benchmarks subdirectory (wikitext, 29af551)

This isn't so much a benchmark as a test, but I don't want to include it in the specs directory because it is such a long-running script.

The basic idea is to parse a large slab of text 100,000 times and make sure that there is no resource leakage even when exiting abnormally from the parse method.

You can't really test this within a spec because the spec suite doesn't run for long enough for resource leaks to really manifest themselves (on my humble machine the spec suite of 564 examples runs in under a third of a second). I could bloat the spec suite to make it run for longer, but that would slow down development and detecting resource leaks from within the specs wouldn't be all that straightforward anyway.

So the idea is that you run this invalid_encoding script and watch memory usage while it processes the same slab 100,000 times over. The slab is long enough that the parser is fully exercised (things like indentation, various syntax etc), and only at the end do we introduce some invalidly encoded UTF-8 which causes the parse to derail.

It's important to do these long-running tests because when I was initially figuring out how to use Data_Wrap_Struct (before I knew about the need to use the volatile keyword) I was seeing the full spec suite passing, but crashes when running the long-running benchmarks. This was because Ruby's GC was never being triggered during the spec run and so the problem never manifested itself.

On my machine this script reports 2.55MB RSIZE, 18.53MB VSIZE, 1.66MB private and 820KB shared, and those figures don't budge at all during the entire run.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:57 PM

Wrap str structs with Data_Wrap_Struct (wikitext, 4389475)

This is the companion commit to 7f506d3 which wrapped ary structs. The purpose of this commit is to ensure that str structs get freed in the event of an abnormal exit from the parse function (ie. via an exception).

Note that in order for this change to be made I had to change the initialization of the tabulation struct; it needs to be initialized in the parse function so that it can be added to the stack in the appropriate scope, and this in turn means that it needs to be pre-initialized rather than lazily initialized.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:34 PM

Extract str_free function into separate function (wikitext, 2bc8593)

Rather than an inline function we need a normal function so that we can pass its address to Data_Wrap_Struct.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:20 PM

Add GC_WRAP_ARY macro (wikitext, f4b4579)

Clean up Data_Wrap_Struct usage a little by encapsulating the pattern of "call Data_Wrap_Struct() and store return value on stack in a variable marked as volatile" in a macro.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:12 PM

Call Data_Wrap_Struct on ary structs (wikitext, 7f506d3)

There is a minuscule overhead introduced by this commit (the allocation of three Ruby objects per parse, which in turn means three more objects involved in Ruby's mark-and-sweep Garbage Collection scheme) which brings with it a robustness enhancement: if we exit the parse method irregularly (by an exception) the three ary structs will be properly freed rather than leaking.

This was never a big deal as the structs themselves were typically small (defaulting to 64 entries each) and irregular exits shouldn't really ever happen in practice (the only times when we would exit via an exception are when we run out of memory, in which case we're totally hosed anyway and process termination is just around the corner, and when we have invalidly encoded UTF-8 input; this latter scenario may be an issue if an attacker specifically tries to consume resources by repeatedly feeding in hand-crafted bad input, but I don't know whether such an attack would be practically feasible).

The key piece of the puzzle is that the object returned by Data_Wrap_Struct must be locatable on the C stack in order for Ruby to refrain from collecting it prematurely. Without the volatile keyword the compiler will see that the pointer is initialized and then never used again and is free to optimize it away to nothing, which then allows Ruby to go ahead and free the structure before its time. The volatile keyword here prevents this optimization, at least in GCC, and is apparently an "accepted hack" that is used in the Ruby source itself.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:55 PM

Extract ary_free() out into separate file (wikitext, 840b62d)

Make this a non-inline function so that its address can be passed to the Data_Wrap_Struct() function.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:22 PM

February 22, 2008

Inital forums implementation (wincent.com, d93f88d)

This is an initial, incomplete implementation of the forums. At this stage we have a basic forum and topic model (forums have many topics, topics have many comments) and controller actions and views for creating new forums and topics.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:27 AM

February 21, 2008

Update Rakefile to include new Rails additions with gem (wikitext, fafe7ba)

The Rails-specific functionality is now included with the gem, but note that it is not enabled by default. If you wish to include it you need to add either or both of these to your environment.rb:

require 'wikitext/string' require 'wikitext/rails'

In addition to the normal "require 'wikitext'".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:48 PM

Custom string methods for Rails (wikitext, c58b779)

These are the additions to the String class that I've been using in my Rails app. They consist of a "to_wikitext" method and a shorter "w" alias for it.

"Foo".w

Would evaluate to:

<p>Foo</p>\n

This also demonstrates the use of a simple pre-processing phase. In this case I'm looking for strings like "bug #12" and converting them to "[[bug/12|bug #12]]" before handing them over to the wikitext parser, which has the effect of turning text like "bug #12" into autolinks.

In reality you'd probably provide your own custom implementation seeing as your needs are likely to be different form mine, but this demonstrates the kind of thing that you would do.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:45 PM

Add Rails template handler (wikitext, cc6949b)

This allows you to create Rails view templates written in pure wikitext. It's intended for purely static content seeing as wikitext, by definition, can't have interpolated Ruby in it.

For example, given a controller "ThingsController" and an action "show_stuff", a template named "show_stuff.html.wikitext in the "app/views/things/" folder would be rendered using the wikitext parser.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:43 PM

February 20, 2008

Initial versions of Capistrano files (not yet customized) (wincent.com, 902eb8c)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:53 PM

Move database.yml into sample (wincent.com, fba3f9a)

Preparing for deployment, rename "database.yml" to "database.sample.yml".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:22 PM

February 19, 2008

Shuffle some migration numbers (wincent.com, 3e766e7)

Fill in gaps caused by deleted migrations.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:27 PM

Make full-text indexing faster by bypassing Active Record (wincent.com, 7976609)

Well, not bypassing it entirely as we still go through connection.insert, but still, this yields a 10-fold speed-up. It is, however, still too slow for my liking; several seconds to index a largish (40+ KB) article in development mode. Even though production will likely be much faster I still don't think this is good enough.

In all likelihood I'll defer indexing and do it in a background process.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:10 PM

Add preprocessing of wikitext (wincent.com, 8d444fd)

We automatically turn strings like "bug #12" and "issue #19" into links to the corresponding item on the issue tracker.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:04 PM

Add full-text search to Posts model (wincent.com, 8e8cbcf)

This commit includes tweaks to get my acts_as_searchable model working.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:50 PM

Extract common tag results code into a partial (wincent.com, b470ec6)

The "search" and "show" templates are very close to identical, only the preamble and postamble is different, so extract the common stuff into a partial.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:11 PM

Sanitize and limit tags in search to 10 (wincent.com, c90d6d5)

Remove duplicate tags in search query and limit the total number to 10 to guard against attacks on the database. It is highly unlikely that a search with more than 10 tags would return any results anyway, given that the average number of tags per item is probably well under five.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:50 PM

Add search form to tag search results page (wincent.com, d828803)

This allows the user to do another search after doing a previous one, and also provides a fallback for users who don't have JavaScript enabled and click on the "Search" link in the navbar.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:41 PM

Restrict visibility during tag searches (wincent.com, 470696b)

The super user can see everything, logged in users can see public records plus their own, and anonymous users can see only public records.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:40 PM

Tighten up post Atom feed generation (wincent.com, 49fce76)

Supply author -- it is required by the standard -- and use summary plus link-to-content rather than sticking the excerpt in the content.

This commit includes some additional links in the code comments to the related standards and summaries of them.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:10 PM

Bump version number post-release (wikitext, 25838d5)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:04 PM

Bump version number prior to release (wikitext, 20dc375)

Now at 0.4. We're essentially feature-complete as far as I can tell, so between here and 1.0 there should only be bug fixes and/or performance optimizations.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:03 PM

Update README prior to release (wikitext, 5abaac2)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:02 PM

Fix a couple of minor typos in the documentation (wikitext, 2cb477a)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:02 PM

Update documentation for new options (wikitext, f6f5a8c)

Add documentation for space_as_underscore and img_prefix.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:44 AM

More img specs (wikitext, 8853b7d)

Add image markup to the integration tests and demonstrate embedding of images in various types of inline span.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:33 AM

Additional specs for img_prefix attribute (wikitext, 2a1afb2)

These were previously tested in the context of img tag generation (see img_spec.rb). Now we test the same thing, but in isolation using attribute accessors.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:24 AM

Add ability to set and override image prefix (wikitext, 60c9b60)

By default a reference like {{foo.png}} will now create an img tag which sources "/images/foo.png". This can be overridden using the img_prefix attribute, or by passing an option when initializing the parser. See the specs for a demonstration of the various options.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:20 AM

Teach options processor to handle nil values (wikitext, 15a3470)

Previously when looking at the options hash we had no way of distinguishing but nil ("unset") and nil ("explicitly set to nil"). Now we explicitly check for the latter so that overriding is a little easier; the caller can set options to nil in order to suppress them.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:19 AM

Add specs for image markup (wikitext, 8358662)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:01 AM

Teach parser to interpret image markup (wikitext, 9f49894)

Given wikitext input like:

{{foo.png}}

will emit:

<img src="foo.png" alt="foo.png" />

The "alt" attribute is added because it is required for XHTML validity. In the future I may add a support for overriding the attribute content, perhaps with a syntax like this:

{{foo.png|a picture of foo}}

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:39 AM

Teach tokenize to tokenize new symbols (wikitext, fdf879d)

Tokenize IMG_START, IMG_END, LEFT_CURLY and RIGHT_CURLY, and update the specs accordingly.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:58 AM

Add LEFT_CURLY and RIGHT_CURLY symbols (wikitext, dd43eff)

In order to avoid ambiguity we have to separately tokenize single curly braces and exclude them from the standard PRINTABLE rule. Otherwise the greedy matching behaviour would match a run like "foo{{bar}}baz" as a single PRINTABLE token rather than a PRINTABLE, IMG_START, PRINTABLE, IMG_END, PRINTABLE sequence.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:52 AM

Add IMG_START and IMG_END token types (wikitext, 3ff145f)

These will be '{{' and '}}' and will be used for inserting images.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:50 AM

Note about multi-tag search options (wincent.com, e9b5ebd)

For now given "foo bar baz" we show only items which have all three tags. This seems the most logical default, but note here that we could also easily show items which match any of the tags, and rank them according to the number of tags which match.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:07 AM

Add "main_form" CSS class (wincent.com, 2ca2c9e)

For table-oriented forms, make the "main form" for the page much wider (95%) and have the enclosed form inputs scale along with it. This is particularly nice for entering large slabs of text, such as on the wiki or for weblog posts.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:06 AM

Validate permalink format for links (wincent.com, d522c0d)

Allow letters, numbers and hyphens only.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:05 AM

Show id in Links controller (wincent.com, 3af4297)

This is an admin-only view, and the permalink is optional, so it makes sense to show the id here in the index view.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:04 AM

Bump version number post-release (wikitext, 5e55ff4)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:40 AM

Bump version number prior to release (wikitext, ce01c88)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:37 AM

Ignore dot-so files on Linux (wikitext, 25c10e2)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:34 AM

Tighten up indent function (wikitext, 2df3c11)

This wasn't calculating the string end point correctly nor updating the stored length which meant that it was unnecessarily refilling the buffer every time; this is quite counterproductive seeing as the whole point of the reusable buffer was to avoid re-generating the same indentation string over and over.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:28 AM

February 18, 2008

Correct file mode on spec files (wikitext, 3870f90)

Two spec files were missing the executable bit.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:47 PM

Get rid of Mac OS X-specific header paths (wikitext, 34972c4)

While including "ruby/ruby.h" will work on Mac OS X, it won't on Linux so change it to "ruby.h" where it should work on both. "mkmf" will take care of specifying the appropriate include search path.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:40 PM

New "package" task for Rakefile (wikitext, a0d647c)

This clobbers everything, runs everything (make and specs), before doing the "gem" task.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:26 PM

Add tag search box to navigation bar (wincent.com, 94ad75f)

May later swap in full-text search box here, but for now go with tags as it is simpler.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:48 PM

Add tag search (wincent.com, 80753b1)

We offer two kinds of search. The first one isn't really "search" at all; basically, if the user navigates to "/tags/foo" then we show a list of everything (if anything) that's been tagged with "foo".

In the second mode we do a real search using a query string. For example, "/tags/search?q=foo" will display all items tagged with "foo"; "/tags/search?q=foo+bar" will display all items tagged with "foo" and with "bar" (merely being tagged with one of them is not enough). If the user searches for three tags, "foo bar baz", and there "baz" is not a known tag name then we show the results for "foo" and "bar", and display a flash advising that "baz" doesn't exist and was excluded from the results.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:46 PM

Group tag results according to taggable type (wincent.com, 7513d98)

So we show all articles with a tag, then all posts etc.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:14 PM

Basic tag display (wincent.com, 61b8b6f)

When showing a given tag, show all of its taggings in a table.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:37 PM

Prettify wiki URLs (wincent.com, 7a865f9)

Adopt MediaWiki convention of converting spaces to underscores in article titles. This means that we get cleaner-looking URLs in the browser's address/location bar (eg "foo_bar" instead of "foo%20bar") at the cost of having to disallow underscores in article titles.

I've added a corresponding validation. I prefer explicitly disallowing the underscore rather than accepting it and therefore permitting the ambiguity wherein you can create two articles, "foo bar" and "foo_bar", and you can only conveniently link to one of them.

I think this is an acceptable tradeoff because attractive URLs are important and space is exceedingly common in article titles whereas underscore is not.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:55 AM

Allow option overrides at intialization time (wikitext, f2658b6)

This is a convenience provided so that you can set your defaults at intialization time rather than following an "initialize-then-customize" pattern.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:50 AM

Bump version number post-release (wikitext, 38cef92)

Now at 0.2.1. The next release will be 0.3. The next (and perhaps last) feature that I want to add prior to 1.0 will be markup for inserting img tags.

The syntax will most likely be "{{foo.png}}", inspired by Creole, as it should be nice and easy to parse.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:16 AM

Bump version number prior to release (wikitext, 4d7123a)

Bump to 0.2.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:09 AM

Add specs for "space_to_underscore" mode (wikitext, b3d136f)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:08 AM

Fix trailing space removal (wikitext, 13e622d)

This was an off-by-one error that was harmless under normal operation because both code paths ended up returning the same length.

But when running under "space_to_underscore" mode it was enough to actually prevent the trailing space removal from working in the case where there was only one trailing space.

It worked when "space_to_underscore" was false only because each space was actually three characters (%20) so the test passed anyway.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:06 AM

Teach link encoding function to convert spaces to underscores (wikitext, b435829)

Only occurs when space_to_underscore is set to true.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:20 AM

Pass space_to_underscore through to link encoding function (wikitext, 4ff3468)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:18 AM

Revert "Implement space_to_underscore conversion" (wikitext, 6b5f819)

The right place to do this is not in fact the sanitization function but the link encoding function. Discovered writing specs for the function.

This reverts commit f340961.

Posted 10:15 AM

Implement space_to_underscore conversion (wikitext, f340961)

Here's the actual conversion from spaces to underscores inside the link sanitization function. It only takes place if we're not in rollback mode and the space_to_underscore instance variable is true.

Not that when not in rollback mode both leading and trailing spaces are stripped so the change has no effect in those parts of the string.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:07 AM

Invert sense of "trim" parameter in sanitization function (wikitext, e3b5ea0)

There are only two real call sites of this function, one in rollback mode and the other when emitting a valid link. Rename the parameter to "rollback" and invert the sense so that passing Qtrue indicates "in rollback mode" and Qfalse "not in rollback mode".

This in turn will help us when converting spaces to underscores when space_to_underscore is set to Qtrue.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:59 AM

Teach link sanitization function to take a parser struct parameter (wikitext, 4256e0a)

This allows us to easily pass in instance variables of interest, such as the new "space_to_underscore".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:53 AM

Expose space_to_underscore attribute in parser (wikitext, 1fc5b8b)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:47 AM

Add space_to_underscore to parser struct (wikitext, 6659357)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:46 AM

Add new instance variable, space_to_underscore (wikitext, adfcb69)

This instance variable will permit the creation of MediaWiki-style links, where [[foo bar]] is a like to "/wiki/foo_bar" rather than "/wiki/foo%20bar".

The benefit is prettier URLs; the downside is that [[foo bar]] and [[foo_bar]] now point to the same article. As this is a trade off it will default to off by default.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:43 AM

Add sample document and cheatsheet (wikitext, a7f20b7)

These are sample documents written using wikitext markup and talking about wikitext markup; as such they are a nice demonstration of what can be done.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:38 AM

Extract tag-related behaviour into shared spec (wincent.com, 4f817df)

Rather than testing the Taggable module directly, define a set of shared behaviours and test them in each model using "it_should_behave_like".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:25 AM

Add comments about polymorphic_url problems (wincent.com, 2e87120)

There should be a fairly easy workaround for this in the next release (post 2.0.2).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:24 AM

Extract "recent" finder into Post model (wincent.com, 1fa464b)

This method is used by both the normal (html) index as the Atom feed.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:23 AM

February 17, 2008

Set up Atom feed for posts (wincent.com, 02f8381)

We use a custom atom field helper seeing as the one that comes with Rails can't handle resources with a :controller override in the routes file (due to a limitation in the polymorphic_url method).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:02 PM

Ignore coverage subdirectory (wincent.com, bb4ed11)

This is where output gets written when running "rake spec:rcov".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:43 PM

Add specs for Post model (wincent.com, 7d22ac0)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:18 PM

Use non-greedy match when searching for a unique permalink (wincent.com, 9abb465)

This prevents "foo" from being confused with "foo-bar", "foo-bar-2" and so on. I am not sure whether REGEXP is a MySQL extension or not, but seeing as I am deploying with MySQL that shouldn't be a problem anyway.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:44 PM

Handle pathological case in post permalink generation (wincent.com, e4136b1)

Handle case where the suggested permalink evaporates away to nothing due to the presence of illegal charactes and nothing else. There are actually two eventualities here: one where the record has never been saved and doesn't even have an id to fall back to (here we use the word "post" as a starting point); the other where the record does have an id and we can use that.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:58 AM

February 16, 2008

Auto-generation of blog post permalinks (wincent.com, f71a66b)

It would be nice if we could normalize everything to ASCII but Iconv can't be trusted to do the same thing across platforms so instead opt for an even lossier approach: basically everything that's not ASCII is converted into a hyphen.

We perform one database query to see if there's a clash to make sure that we generate a unique permalink each time. This would be racy in a multi-user scenario but it's envisaged that I'll be the only one publishing articles so this is probably fine.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:31 PM

Remove localization code (wincent.com, 285fd68)

Rather than doing a half-baked job I prefer to remove it entirely.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:24 PM

Initial cut at "issues" controller index (wincent.com, ff28775)

Simplify the model by removing the supporting "statuses" class. There is no need for this kind of dynamic complexity, and not only that, we can use the same three basic hard-coded statuses (new, in-progress and closed) for all issue types (bug reports, feature requests, support tickets, feedback etc); there is really no call for fine-grained statuses like "closed (duplicate)", "worksforme" and the like, as they serve only to confuse users.

In order to get pagination and sorting working I had to teach both the paginator and the sortable module to pass on the relevant query parameters when generating URLs.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:48 PM

February 15, 2008

Remove commented out CSS (wincent.com, 61f798b)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:35 PM

Set up navbar (wincent.com, 6b3cfb4)

In 2.0 spirit we're going for the simplest navigation possible. Some of the CSS exhibits some mysterious behaviour (namely, I had to set a -0.35em left margin on the links in the list items in order to get the highlighting to correctly extend all the way to the preceding border; I have no idea why this hack was required) but it displays correctly in both Firefox and Safari.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:42 PM

Add pagination to weblog (wincent.com, b39a0af)

Simple first/prev/next/last pagination.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:25 PM

Add "public" field to comments migration (wincent.com, da3e4c5)

It is useful to be able to flag individual comments as public/private. By default, comments will be public, and will be visible as long as the "commentable" to which they are attached is public. But there are some cases where it is good to hide individual comments on an otherwise public "commentable"; for example, in the issue tracker most of the discussion may be carried out in public, but there may be individual security-related comments that should be kept private.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:19 AM

Renumber migrations to fill in gap (wincent.com, bfa4476)

This fills in the gap left by the removal of the "create revisions" migration.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:19 AM

Only show comments if accepted now or in the past (wincent.com, 5e48659)

There are actually three states we need to distinguish among here: posts where comments are accepted, posts where comments are not accepted but have been in the past, and posts where comments are not accepted now and never have been.

In the first two cases we'll show comment-related stuff in the views, but in the last case we won't show anything at all.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:16 AM

February 14, 2008

Make sure admin sees all comments in posts "show" template (wincent.com, 00d93e7)

Previously the admin would only see comments awaiting moderation. Now he sees all non-spam comments.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:24 PM

Inline moderation of post comments (wincent.com, ec346e2)

Add two buttons underneath each comment, but only when logged in as an admin user. The "spam" button marks a message as spam and then fades out the offending comment; the "ham" button clears the "awaiting moderation" attribute of the comment and fades out the "spam"/"ham" buttons.

At the top of the listing we show three comment counts: published comments, comments awaiting moderation, and spam comments. On the main listing page the comment count that we show is actually the sum of published comments and comments awaiting moderation.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:08 PM

Add comments to weblog (wincent.com, 5559ac9)

This required some nasty dynamic programming due to the nature of nested polymorphic associations. Basically if you want to avoid having to repeat basically the same comment form for all your "commentable" models (which would almost entirely defeat the purpose of having a polymorphic comments model in the first place) then you have to jump through some hoops at runtime to figure out which resource you are nested inside at runtime.

The other change which tags along for the ride here is per-page page title overrides.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:57 PM

Many cosmetic changes (wincent.com, 4d0860c)

These are tweaks to the CSS, additional graphics and juggling around some links in the articles and posts controllers.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:23 PM

Use 303 response for link redirection in links controller (wincent.com, 96a1203)

Use the "see other" response, seeing as these redirections are always permanent and should always be GET requests.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:22 PM

Various improvements to tagging and validation (wincent.com, 9c09d17)

Improve the design of the tagging system to make it more consistent over the "new" (create) and "edit" (update) templates and actions.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:02 AM

February 13, 2008

Add Posts model (wincent.com, ca2dca0)

This is actually the weblog model. It shares a lot of similarities with the wiki part of the site, so it was quite quick to come up with this initial implementation.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:04 PM

Bump version number post-release (wikitext, fd80e1c)

The released version was 0.1. As far as I can tell this is basically a feature-complete release (in terms of the features that I have envisioned for 1.0), so any subsequent releases will be bug-fix-only ones, probably following a numbering scheme like 0.2, 0.3 and so on.

The plan is to use the bundle in production, add more specs, correct any issues that might come up, and go to 1.0 once a reasonable interval has passed.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:16 PM

Minor updates to README prior to release (wikitext, 03b8fda)

These changes bring the README into line with the current behaviour of the bundle.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:10 PM

Add specs for articles routing (wincent.com, bc941e4)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:06 PM

Add specs for Articles helper (wincent.com, ab4b878)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:05 PM

Full spec suite for Article model (wincent.com, 90d6ef3)

This spec suite covers every line of code in the Article model.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:56 PM

Add attribute accessible declaration to Article model (wincent.com, f39c25c)

The whole thing is accessible, but add this anyway as basic good security practice (the model may be extended in the future).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:55 PM

Fix bug in Article redirect format validation regexp (wincent.com, 4dae4d8)

Missing parentheses meant that this would sometimes pass when it shouldn't have.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:54 PM

Fix spec breakage caused by removal of Revisions model (wincent.com, 1f9a20f)

These were some lingering files which should have been removed at the same time as the Revisions model.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:20 PM

Move login_as method into spec helper (wincent.com, 8e136a7)

This was originally in the sessions controller specs, but move it up into the spec helper, where it can be used by all controller specs.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:19 PM

Shorten body_html helper (wincent.com, e54a968)

Make it a simple one-liner.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:02 PM

Extract body_html method into Articles helper (wincent.com, 46d62b2)

This allows the "new", "edit" and "show" templates to all use the same method for showing the body text.

In the case of the "new" template being called for the first time there is no article and so we return an empty string; but when validation fails there could well be body content so we return it if available. This is used in the preview.

Same for the "edit" template, where we use the text in the preview.

Finally in the "show" template we use the method to display the body normally, not as a preview.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:12 AM

Extract ugly code into new Articles helper (wincent.com, 3773864)

The "link_to_remote" and "observe_field" calls are ugly and involve a lot of repeated options that appear in both the "new" and "edit" templates, so extract them into the Articles helper.

We also pre-populate the preview content in the edit template (but not in the "new" template because there we don't have any content when the template is first displayed; I may later modify this to handle the situation where there is content due to a failed validation and a re-rendering of the form).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:06 AM

Fix syntax error in new article template (wincent.com, 2d35085)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:00 AM

Show spinner graphic while updating preview (wincent.com, a7dd462)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:00 AM

Disable Haml indentation for text areas as well (wincent.com, fcb2803)

In "edit" forms Haml was indenting the text that appeared inside <textarea></textarea> spans, which basically mangled the wikitext, making it impossible to edit any article without ruining it.

So wrap the text areas in preserve() calls to prevent this. This results in some very ugly source code so I am keen to find a more elegant solution.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:44 AM

Use "post" rather than "get" for wiki article AJAX preview (wincent.com, 4f318b5)

"get" won't work for really large articles, so use "post" instead.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:38 AM

Start populating site-wide nav bar (wincent.com, 477b843)

Trying to keep the number of links in the navbar nice and low.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:47 AM

More high-level updates to Articles controller (wincent.com, 5c6888e)

A new "index" template which shows the most recently updated articles, and the edit template which I actually meant to include in an earlier commit.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:46 AM

Remove references to Revision model in example data (wincent.com, efa788e)

It's necessary to remove these references in order to get the console to work (because FixtureReplacement sees the data and expects to find the corresponding model).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:45 AM

Nicer CSS for PRE blocks (wincent.com, de69284)

Do Wikipedia style borders around PRE blocks to make them stand out better.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:44 AM

February 12, 2008

Finish off (wiki) articles controller (wincent.com, e27c63a)

Now we have an edit action, the show action handles redirects (and tries to catch cyclical redirects), and basic validation is performed.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:54 PM

Fix handling of permalinks with periods in them (wincent.com, 1a848c1)

Rails considers periods to be a route separator, which I think is very unfortunate. Fortunately, you can use the :requirements option in the routes file to override this on a per-resource basis. This means that wiki articles with periods in the title will work, as will tags.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:56 PM

Drop Revisions model (wincent.com, b6312eb)

My initial wiki implementation only had a "revisions" notion because I unthinkingly modelled it after other wikis. In reality I don't need it and it is an unecessary complication, so remove it. (This is not a community-driven wiki, nor one which allows anonymous edits, so the ability to go back in time is not as important).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:59 PM

Add live preview to wiki (wincent.com, 8140e20)

When creating a new wiki article provide a live AJAX preview. I've got this set to a fairly low frequency (checks for changes every 30 seconds). You always have the option of hitting the preview link to force an immediate update.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:15 AM

February 11, 2008

Remove redundant parameters to dedent method (wikitext, f29d562)

Given that 6 is the default dedent quantity there's no need to explicitly specify it.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:40 PM

Teach dedent helper to default to 6 spaces (wikitext, e0ba800)

Now that I've converted all the specs I see that in every single case I dedenting by 6 spaces. So teach the dedent helper to assume 6 spaces if passed only one parameter.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:38 PM

Update ul spec to use indentation helper (wikitext, ea4a291)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:33 PM

Update tokenization specs to use indent helper (wikitext, 93f8540)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:26 PM

Update regression specs to use indentation helper (wikitext, e02a70a)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:25 PM

Update pre specs to use indentation helper (wikitext, 221ca57)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:24 PM

Update integration spec to use helpers (wikitext, bbaa877)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:16 PM

Update indentation spec to use helpers (wikitext, bfa5709)

Here we make use of the dedent helper, and add a converse "indent" one to make the specs easier to maintain. As a demonstration of the usefulness of the indent method we add additional assertions for other indent widths.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:10 PM

Update blockquote specs to use "dedent" helper (wikitext, 3d507da)

While making this change I also discovered and fixed a couple of example blocks in which I was setting up test data but never making an assertion.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:05 PM

More blockquote specs (wikitext, 00e2719)

These demonstrate the use of the "dedent" helper introduced in 54a7a69.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:41 PM

Add "dedent" helper for specs (wikitext, 54a7a69)

I'm finding the multiline spec data hard to read because it breaks the flow of the indentation (where does one spec end and another begin? are we nested inside multiple "describe" blocks? etc).

So here I add a "dedent" helper which will allow me to indent the spec data for better readability while editing, and then dedent the data at runtime for actual checking.

As a safety valve I make sure that we're not asked to dedent more whitespace than we actually have.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:38 PM

More blockquote specs and one fix for nested pre blocks (wikitext, b6e1984)

Simple inversion mistake meant that we weren't correctly nesting pre blocks inside blockquote ones.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:30 AM

Basic blockquote implementation using real HTML tags (wikitext, 88da5f3)

Still need to write more specs to be sure I've covered all the edge cases, but the basic stuff seems to work.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:12 AM

February 10, 2008

Update list and header rules for new BLOCKQUOTE_START token (wikitext, 9e985ab)

These rules previously held if at the start of the line or immediately after a BLOCKQUOTE token; now that we have BLOCKQUOTE_START as well we need to add that to the lsit of conditions too.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:13 PM

Add tokenization of literal blockquote tags (wikitext, 8888a18)

This is the first step in adding support for literal <blockquote> and </blockquote> tags. The rationale for this is that when quoting large slabs of text it is much more convenient to use starting and ending tags rather than prefixing every single line with a greater-than sign.

Because blockquote and pre blocks are intimately interrelated I won't allow mixing the two types of syntax, as it could produce results too confusing for the user.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:04 PM

Add specs for treat_slash_as_special instance variable (wikitext, ca14d4c)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:01 PM

Add a bunch more PRE specs (wikitext, 48a3f27)

I'm checking as many weird corner cases and exceptions here as I can think of.

The basic behaviour is to allow a literal <pre> only at the very top level or inside a P block, as long as the latter is not inside a BLOCKQUOTE block (this is because I'll later be adding literal <blockquote></blockquote> support and I don't want to allow intermixing the two markup styles to avoid confusion). Any open span-level elements are automatically closed whenever such a <pre> block is entered, including rolling back any unfinished internal or external links.

If an unexpected <pre> or </pre> token appears anywhere else, echo it in escaped form to provide feedback to the user; I think this is better feedback than just autoclosing the previous spans/blocks and then starting a <pre> block anyway.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:51 PM

Fix bad emission of BLOCKQUOTE inside NO_WIKI spans (wikitext, f8384a1)

Here we were emitting a literally greater-than sign when we should have been producing a named entity.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:32 PM

Initial handling of PRE_START and PRE_END in the parser (wikitext, e2ee2bb)

This handles the basic case. Will be adding specs to explore the edge cases and make the necessary changes to handle them.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:08 PM

Rely on automated early return from rollback functions (wikitext, 1dacef0)

This allows us to cut about 30 lines of code and should improve maintainability.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:43 PM

Add early returns to the rollback functions (wikitext, 1c5c4a6)

This should make life easier for callers who may want to rollback but aren't sure: basically they can just rollback anyway and if there is nothing to be done the functions will return early.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:30 PM

Add PRE_START and PRE_END to token.c (wikitext, c9137ef)

Without this we can't see those new tokens in our token dumps during debugging.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:25 PM

Add tokenizing mode to "wikitext" tool (wikitext, 34c34cc)

In this mode the output from "tokenize" is pretty printed.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:22 PM

Enhance output using HighLine (wikitext, 4262958)

We use HighLine here to easily provide us with a bold prompt and prepend and append appropriately sized lines to the output so that it stands out in the "wikitext" tool.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:04 PM

Add "wikitext" commandline tool (wikitext, a757151)

This is a simple tool that allows you to play with wikitext from the command line. Type in text and hit Ctrl-D to process it; it will be translated into HTML and emitted to the standard output. Hit Ctrl+C to exit the session.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:47 PM

Add tokenization of PRE_START and PRE_END (wikitext, 940a83b)

As a convenience, it would be nice to offer the ability to use literal <pre></pre> tags to mark up PRE spans rather than just using the existing mechanism of prefixing each line with a space. This will be particularly useful when pasting in long code samples and the like.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:06 PM

Note about routes.rb modifications in Edge Rails (wincent.com, a945990)

I prefer to stick to released versions so I can't use this just yet, but make a note about the new ":as" parameter that can be used in the routes file (beats using :collection).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:02 PM

Go back to using Haml "preserve" method (wincent.com, c031ee8)

Will soon follow up with a patch to undo the ugliness from inside the "render" method. Clearly a hack, and I'd love to be rid of it, but will have to be that way for now.

The main issue with it is that it involves applying two totally redundant transformations: one to insert newline escapes and another to remove them. Combined, these cancel each other out and the total effect is a no-op, but we're forced to use them to prevent Haml from mangling the PRE blocks (we gain correctly formatted PRE blocks in the browser but lose our nice source formatting and get back a horrible uninterrupted blob of text instead).

This time-wasting transform is especially a shame because of all the effort I've put in to making the wikitext translator fast, so I'd still like to find an elegant way to teach Haml how to pass designated blocks of text without touching the formatting in any way.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:47 PM

Revert "Teach Haml to respect preformatted text" (wincent.com, add2e6c)

I'm going to follow Nathan's advice here and just monkey patch the "render" method rather than meddling with Haml's internals. For the timebeing seems the lesser of two evils.

This reverts commit 0eb62734c9e3b266f2138fd51047eaf3ea736374.

Posted 6:44 PM

Add "regressions" spec file (wikitext, 6138c84)

This will be a general "catch all" file in which I put bug test cases to make sure that we don't get regressions in the future for any fixed issues.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:40 PM

Documentation: remove source code comments covered elsewhere (wikitext, 6574117)

Remove redundant RDoc comments from the main wikitext module source file as these are now covered in the separate RDoc file in the doc subdirectory.

At the same time update the definition of "treat_slashes_as_special" to reflect the new, stricter definition of what classifies as a special link.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:31 PM

Add specs for internal special links (wikitext, 5dafcee)

These specs test "special" links in the context of internal link markup.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:26 PM

New link encoding specs (wikitext, 306ccde)

These specs confirm that out special link detection works rigorously.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:19 PM

Add public method for testing special link encoding (wikitext, d3415db)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:14 PM

Hook up "special link" treatment (wikitext, e36dc7e)

The encoding function previously learnt how to identify and flag special links. Here we teach the parser how to act upon seeing such a flag.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:09 PM

Make "special link" matching a little stricter (wikitext, 9012ad4)

Expect "special links" to conform to a narrower syntax, /\A[a-z]+\/\d+\z/, to minimize the risk of false positives.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:06 PM

Perform prescan to identify special links (wikitext, e298dd6)

When encoding a link target perform a quick prescan to see if this is a "special" link or not. At this stage, "special" links are those which contain only letters, numbers and slashes. The idea is that whereas a link like "[[foo bar]]" is interpreted as a wiki article link, one like "[[bug/12]]" is interpreted as a link still internal to the site but external to the wiki, specifically to the bug-tracking part of the site which would be accessed with a path like "/bug/12" (as opposed to the wiki article which would have a path like "/wiki/foo%20bar").

Seeing as the prescan does constitute extra work, only perform it if the treat_slash_as_special instance variable is set to true. On the other hand, when the prescan does identify a special link it aborts the encoding right there and less work is done in that case.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:54 PM

Remove local "autolink" variable (wikitext, cacaf11)

Use the equivalent member in the parser structure instead.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:31 PM

Teach _Wikitext_parser_encode_link_target to take a parser parameter (wikitext, 605aba1)

Instead of passing a string object, pass a pointer to a parser struct instead. This is groundwork for teaching _Wikitext_parser_encode_link_target how to handle links when "treat slash as special" is in effect.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:15 PM

Remove unnecessary regular expression machines from Ragel machine (wikitext, 3de5048)

I just learned that the "i" (case-insensitive) modifier can be applied equally to standard concatenation literals as it can to regular expressions. Simplify the grammar (fewer escape sequences) and make life easier on the syntax highlighter by replacing the regular expression machines with concatentation literals. The generated code is identical before and after.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:54 PM

Add support for HTTPS URIs (wikitext, 06a4b58)

This was a bit of an oversight, as I always meant to incorporate it alongside HTTP URIs.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:51 PM

Add tokenization of mailto URIs (wikitext, c70f28a)

We already had tokenization of email addresses, which in combination with autolinking gave us clickable email address links.

Now we have tokenization of "mailto" URIs, which means that we now also have the option of using the standard external link syntax, like we would with any other URI, to provide the link text rather than just making the address itself clickable.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:42 PM

February 9, 2008

Add specs for questionable or invalid links (wikitext, b00e7b1)

These specs show how the transformer responds in the face of odd or questionable input (strange characters in link text, for example).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:19 PM

Handle URIs in internal link text (wikitext, 256c0e9)

A URI in the link text part of an internal link is invalid, so roll it back. (Note that external links are differnt: there there is no harm in allowig URIs in the link text. It is only a problem in the internal link case because it could confuse the routing if we allowed URI characters in article titles.)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:18 PM

Clean-up rollback functions (wikitext, 1367c53)

We can cut out about 33 lines of code by taking advantage of the following invariant: as soon as we see a link start marker we know that we'll either be emitting a link, or the text that results from rolling back a failed link. This means that instead of checking to see if we need to start a paragraph in the rollback functions we can just start one unconditionally as soon as we see the link start markers.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:43 PM

Fix unterminated external links nested inside LI blocks (wikitext, 50350ef)

This is the companion fix (and specs) to commit 2e23cfe, this time handling the external link case.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:36 PM

Fix for unterminated links in LI blocks (wikitext, 2e23cfe)

Another class of bad input; this is the spec that tests for it and the modifications required to make it pass.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:34 PM

Fix handling of unclosed span-level elements in lists (wikitext, 2a996a7)

This commit adds some specs which uncovered the problem as well as the rather simple fix.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:31 PM

Add specs to catch regressions (wikitext, 01ada5a)

Add several more specs related to recently discovered flaws to make sure the issues don't creep back in again.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:06 PM

Get integration specs passing (fix OL/UL nesting in BLOCKQUOTE) (wikitext, ae0ffab)

When starting a brand new list may need to pop back to (but not including) the nearest BLOCKQUOTE; this fixes the case where a new list is started inside a BLOCKQUOTE on the line immediately after an open paragraph. It is necessary to pop the paragraph in order for things to be correctly formatted.

With this change all the specs are back passing again.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:05 PM

Fix TT handling inside BLOCKQUOTE blocks (wikitext, 52ad5ee)

This was a corner case which was failing: a BLOCKQUOTE block with some text on one line (which starts a nested P block) followed by a new line (the P block continues) with a TT span.

We fix this by cleaning up the handling of excess element popping.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:50 PM

Remove special case for PRE in the "pop excess elements" function (wikitext, 504bb6e)

By explicitly popping back to the closest BLOCKQUOTE block (if any) we can avoid the need to supply special case code in the "pop excess elements" function.

This is better because it keeps the logic for the PRE case more localized.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:11 PM

Minor cleanup and fix for blockquote/pre interaction (wikitext, 20db47c)

This cleanup includes a fix for some odd interaction in one case: a blockquote block containing paragraphs alternating with nested pre blocks.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:05 PM

Fix breakage in PRE specs (wikitext, 4be3597)

This is required to prevent us from emitting an initial CRLF upon entering a PRE block for the first time.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:38 PM

Tweak newline emission of pre blocks (wikitext, b364c14)

Defer emission of the CRLF until we've seen a bit more. This fixes some edge cases (specs forthcoming).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:37 PM

Fix minor wart with pre block closing (wikitext, 172a51c)

Under some circumstances (unnested pre block followed by two consecutive newlines) we would emit a block that looked like this:

<pre>foo </pre>

Rather than like this:

<pre>foo</pre>

While the former displayed fine in the browser, the latter looks better in the HTML source.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:39 PM

Use custom string struct to store line endings (wikitext, bf8a30f)

This shaves off a sliver of time on longer inputs because it allows us to replace a number of calls to rb_str_append with the more efficient rb_str_cat.

Before:

longer slab of ASCII text 13.510000 0.060000

After:

longer slab of ASCII text 13.450000 0.050000

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:27 PM

Speed up indentation by caching spaces string (wikitext, 149ec37)

We lazily create a string struct on demand the first time it is needed, and as larger indent levels are required we grow the buffer and populate it with spaces. We never shrink the buffer and so never have to re-populate it. It is freed only at the end of the parsing operation.

So this will have little or no effect in really short snippets of wikitext (one liners where there is no indentation at all) but on longer slabs with lots of nesting it should afford a nice boost. A typical use case (examples with more nesting would see a larger improvement):

Before:

longer slab of ASCII text 14.260000 0.040000

After:

longer slab of ASCII text 13.510000 0.060000

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:09 PM

Update benchmark notes (wikitext, e9d67ef)

Since I last added benchmark information to the notes a few features have been added which slow down processing, so we need a new baseline before I begin the next round of optimizations.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:46 PM

Add custom string struct implementation (wikitext, 4f06bdd)

This is actually the result of some work on an experimental topic branch which didn't work out. Now that we have the parser struct in place, however, it should be possible to gradually make more and more use of the efficient custom string struct instead of instantiating full-blown Ruby Strings. Object instantiation is one of the most costly operations in the translator, and it brings with it a corresponding cost in garbage collection, so I'm hoping that this will lead to a nice speed boost.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:39 PM

Add indentation specs (wikitext, bf79f03)

Test user overrides of base indentation. The intention here is for incorporation in Haml templates (or any other templating system that takes care to produce beautifully-formatted HTML). The user can supply a base indentation level that matches the expected indentation level in the enclosing Haml template; in this way the wikitext-generated portion won't break the nice indentation of the template as a whole.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:28 PM

Normalize negative indents to 0 (wikitext, 4f88966)

If the user tries to specify a custom base indent of less than zero, normalize it to zero to prevent later problems.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:27 PM

February 8, 2008

Update integration tests for indentation (wikitext, 15637bb)

With this all specs are passing again.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:10 PM

Fix list indentation (wikitext, af34180)

Quick a tricky one, this one, because the LI token is the only one which varies its behaviour depending on the context.

For example, when it is just a standard list item containing some text we want it to behave like a P token (ie. when we dedent we don't want to emit any spaces). On the other hand, if the LI contains a nested list then it takes on block-like properties and we do want to emit spaces when dedenting (ie. like blockquote).

We implementing this by introducing another imaginary token, NESTED_LIST, which is present when a nested list is present and guides the dedentation process accordingly.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 9:05 PM

Fix pre indentation (wikitext, 7428f7d)

Adjust the specs and add a missing call to the indent function when opening a pre span.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:04 PM

Adjust internal link specs for indentation (wikitext, 724126d)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:59 PM

Adjust header specs for indentation (wikitext, f976e2d)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:58 PM

Adjust external links specs for indentation (wikitext, 63b004e)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:58 PM

Initial activation of indenting code (wikitext, ee665c4)

At the moment this breaks some of the specs but I wanted to get it in the repository in any case as it looks to be very nearly complete.

Will follow this up with edits to the specs to make them pass, as there are now many expected results that will need to be modified to include the indentation.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:56 PM

Hook up indentation method (wikitext, a7bc5e6)

At the moment doesn't do anything but it's ready to go.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:34 PM

Fix autolinking spec (missing variable) (wikitext, 340e5a5)

In a recent refactoring (splitting an example into multiple ones) I missed out carrying across this variable.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:34 PM

Add specs to test external_link_class overrides (wikitext, a69f859)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:32 PM

Simplify _Wikitext_rollback_failed_external_link (wikitext, 05f860e)

Now this function too only takes one parameter and can't be made any simpler. While making this change I noticed that the "mailto_class" instance variable wasn't being used, so this commit incorporates the necessary modifications to remedy that.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:29 PM

Simplify _Wikitext_rollback_failed_link (wikitext, 5b73a71)

This function is now as far as it can go, with the parameter count reduced to one.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:14 PM

Update code comments for function descriptions (wikitext, f85b8b6)

In light of the recent and ongoing refactoring, clarify some of the function descriptions in code comments to avoid possible confusion.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:13 PM

More function simplification (wikitext, dd54a50)

Here we are four more members to the parser struct to avoid having to pass so many parameters.

This allows us to simplify the "pop from stack", "pop from stack up to", "pop excess elements", "rollback failed link" and "rollback failed internal link" functions considerably; the parameter counts for these functions have gone from 4, 6, 6, 8 and 5 to 2, 4, 1, 3 and 2 respectively, with the possibility of further simplificationin the future.

A nice side effect of this change is that the calling parser code can be simplified in many places too; I as able to trim off a number of repeated lines that were no longer necessary, or which could be moved out into the functions.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:48 PM

Simplify rollback of external links (wikitext, c7ffbac)

Roll a couple more fields into the parser struct and use that to reduce the number of arguments passed to the rollback function for external links.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:37 PM

Make the "start para if needed" function into a one-param function (wikitext, a9ad57d)

The clean-up for this function has now been taken as far as it can go as we're down to a single parameter.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:27 PM

Roll pending_crlf into parser struct (wikitext, 1aab4ce)

This is a nice clean-up that gets rid of another parameter to the "start para if necessary" function.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:20 PM

Introduce parser_t struct (wikitext, f55c418)

This will make passing parameters around much easier because we'll be able to pass a single pointer rather than a raft of variables. As a starting point, I've converted the "start para if necessary" function to accept a pointer to the new struct as one of its parameters.

Once fully completed this will be a substantial clean-up and a big enhancement for maintainability.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:14 PM

Maintain internal indentation level variable (wikitext, ad3271a)

At this stage we don't do any indentation but we do pass the variable around in preparation for when we do.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:00 PM

Emit trailing newline after opening blockquote block (wikitext, d718a4b)

This is in preparation for more indentation changes. Seeing as you can nest things inside a blockquote and those things can change the nesting, it's necessary to start the block content on a new line.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:39 PM

Add specs for nesting headers inside blockquote blocks (wikitext, dc41edb)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:31 PM

Minor cleanup in list handling (wikitext, 0a8fe9f)

I suspect I am going to need to add some common code here prior to emitting, so pull this into an if block of its own.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:30 PM

Teach parser to extract "indent" option (wikitext, c032589)

Don't actually do anything with it yet, but it's there ready to be used.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:16 PM

Teach Wikitext::Parser#parse to accept an options hash (wikitext, 73fb12a)

This is actually the way it used to be but I removed it seeing as I never used any options. But now I plan to add support for using a preset indentation so this is a necessary prerequisite.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:58 PM

Fix tokenization bug (failure on curly bracket) (wikitext, c523b5e)

The printable machine wasn't matching the full set of ASCII because I was overlooking one code point. Add that code point and specs to make sure this bug never regresses.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:53 PM

Use new preserving hack (wincent.com, 1be5ca4)

Teach the app to use the new whitespace preservation hack for Haml.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:16 PM

Teach Haml to respect preformatted text (wincent.com, 0eb6273)

This is not a pretty solution but it works for the time-being. Basically we insert markers wherever we want the input text to be passed through literally.

This is necessary because of the way template nesting works. For example, if you suppress indenting in a "show" template, the contents of that rendered template will eventually get used by the enclosing application layout, and at that point you'll get additional indenting added back in.

So the only way to communicate back to higher levels that you want your formatting preserved is to insert marker tags and act upon them accordingly.

Note that this is different than what the "find_and_preserve" method does; that merely looks for pre blocks and filters them through preserve before passing them on.

In the meantime I am going to continue looking for a more elegant solution.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:12 PM

Add notes on Rails routing bug (wincent.com, df1be01)

I haven't yet found the cause of this one but it seems that Rails doesn't like wiki articles with numbers or periods in the names (not sure which yet).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:43 AM

Workaround Haml whitespace damage of pre blocks (wincent.com, ec72ab3)

Haml provides a preserve() helper for this which I am reluctantly using. Downsides: turns large slabs of transformed wikitext into indecipherable, hard-to-debug, ugly slabs; and makes another pass over the entire input stream, which runs against the primary design goal of the wikitext translator (speed).

I'm currently investigating an alternative workaround, the seed of which you can see in the application helper. Basically I want to suppress all indentation inside the given block; if I can come up with a workable solution will submit a patch to Haml.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:40 AM

Add initial validations for Article model (wincent.com, 5881ce0)

At this stage we just validate the presence of the title.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:38 AM

Add tagging to wiki (wincent.com, de6322a)

You can now specify new tags when creating a wiki article, and they are shown when the article is shown.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:38 AM

Move scaled_tag helper function into application helper (wincent.com, 57f734a)

Move the scaled_tag helper function from the tags helper to the main applicaton helper so that all views can benefit from it.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:36 AM

Fix pretty tag URLs (wincent.com, 339a076)

These weren't working due to a misnamed param key.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:36 AM

First cut at wiki functionality (wincent.com, b451c58)

This is a basic first cut at wiki functionality. At the moment we have a form for creating new articles; on submitting the form we create the article model and attach a first revision model as well with the initial content of the body.

Articles are located by their titles, and our routes map to the "wiki" namespace rather than the "article" one. This means that our paths are like "/wiki/foo" rather than "/article/1" or even "/article/foo".

If you try to access an article which doesn't exist and you're logged in as an admin user then you are taken directly to the "new" form with the title pre-populated so you can create the missing article if you wish.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:02 AM

February 7, 2008

Special case record_not_found (wincent.com, d35ee64)

In the default case (when there is no override) the uri parameter will actually be an instance of ActiveRecord::RecordNotFound, so watch out for that and intercept it; letting it through causes an exception to be raised.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:13 PM

Move login form into a partial (wincent.com, 8c98144)

This will allow us to display the login form on any page as well as on its own dedicated page.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:55 PM

Show tagging count in tool-tip in tag cloud (wincent.com, 2e0f600)

Hovering the mouse over a tag name in the tag cloud now shows a tool-tip with the total number of taggings for that tag.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:44 PM

Normalize tag name to lowercase upon storage (wincent.com, 75a0af2)

This is for consistency. Note that the case-insensitivity of the uniqueness constraints means that it was already impossible to create two tags with names like "Foo" and "foo", but this change rules out different but inconsistent tag names like "Tennis" and "football" (they would be "tennis" and "football" instead).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:41 PM

Minor rewrapping for readability (wincent.com, ba69361)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:22 PM

Add validation of tag name attribute (wincent.com, 4cdb738)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:15 PM

Special styling for links (wincent.com, 45e3ce0)

This is for horizontal rows of links, to make them stand out a little more.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:48 PM

Change tag cloud formatting (wincent.com, 0ca64e0)

Changes to the controller to order tags by name, and to the CSS to display them in a more cloud-like fashion (drop list, just put all the items in a div, and center the cloud within the div).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:35 PM

Size tag font depending on popularity (wincent.com, b724d1a)

According to "Web 2.0" doctrine you need tags and you need to change the font size to indicate the popularity of a tag.

This commit introduces the resizing in the range of 1.0em (for impopular tags) through to 2.0em (for the most popular).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:15 PM

Make separate administrator namespace (wincent.com, 13a0aec)

In general I prefer in-place editing where possible but there are some cases where it is useful to have an administrator-only view of a model. For example, consider the Tag model, where it is nice to have a read-only "cloud" view for normal users, while for admin users we can provide a straightforward table view with in-place editor fields and sortable table columns.

Not sure why, but at some point my "admin_only" and "logged_in_only" helpers broke, so I've simplified them by removing the "simple_concat" method and now they're working again.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:59 PM

Tags interface (wincent.com, 37ee0e1)

This is the beginning of the tagging interface. Note that because I want to sort tags by the number of times they're used I had to add a counter_cache column to the tags table.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:11 PM

Teach acts_as_sortable to accept default sort order parameter (wincent.com, 813baad)

This can then be used to, for example, set a default sort order in the index action of the tags controller.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:09 PM

Update translations specs (wincent.com, 08270ea)

Tweak the default RSpec specs to get this passing.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:19 PM

Add show method to comments controller (wincent.com, 35495cb)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:18 PM

Modify links specs to get them passing (wincent.com, 252308d)

Temporarily disable the RSpec-provided defaults and make other tweaks necessary to get all the link specs passing. At this point we can start running autotest again because everything passes.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:17 PM

Migration tweaks (wincent.com, 44d306e)

Set some null constraints on some of the model migrations to tighten things up.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:13 PM

Fix sessions controller specs (wincent.com, e89c44d)

This was a tricky one, who to simulate a user login in the specs. The problem is that the testing machinery assumes one request per spec, so you can't do the obvious easy solution of doing a real login and then following it up with other specs.

The next most obvious option is just calling the "current_user=" method but that won't work either. That method sets stuff in cookies and the session and those aren't set up yet prior to doing a post in your specs.

So in the end we just reach in and set the "@current_user" instance variable directly. Note that we have to intercept the call to the "login_before" before filter otherwise it will proceed to clear the instance variable as soon as we initiate an actual request.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:29 PM

Whitespace fix: trim unnecessary empty line (wincent.com, e0608c3)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:28 PM

Update example data to get specs passing (wincent.com, 0a11fad)

This includes an update to the Status model specs, overriding the default specs provided by RSpec.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:51 AM

Re-run all specs whenever schema changes (wincent.com, 9511ecd)

Another autotest tweak which will cause all specs to run after any migration or other database change.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:49 AM

Fix locale specs (wincent.com, aa385f3)

Again, minimal changes to get them running.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:37 AM

Minimal changes to get Email specs passing (wincent.com, 1bffd19)

Again, just make the minimal changes to get the RSpec-supplied default specs working.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:35 AM

Run specs whenever example_data.rb changes (wincent.com, a673655)

Add an .autotest file so that whenever the FixtureReplacement sample data changes the entire spec suite will be run automatically.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:33 AM

Fix Comment specs (wincent.com, b522ae6)

The default specs that RSpec provides weren't working so make the minimal changes required to get them going.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:31 AM

Drop redundant authentication specs (wincent.com, 1545453)

The authentication code is so intimately intertwined with the User model and all the rest that it doesn't make much sense to try and test it independently.

Remove the specs which redundantly re-test stuff that is already tested elsewhere. This makes one less site to have to worry about updating when refactoring.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:25 AM

Whitespace fix (wincent.com, c9914a6)

Remove unneeded empty line.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:23 AM

Drop HTML caching from revisions table (wincent.com, 5313bbb)

Now we just store the wikitext markup in the database and generate HTML on demand.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:22 AM

Fix typo in String additions file (wincent.com, 02bf922)

Just a code comment fix.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:22 AM

Add routing for wiki (wincent.com, 1e83226)

Wiki articles are modelled using the Article model, but we want paths like "/wiki/foo" rather than "/article/foo", so set up the routes accordingly.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:21 AM

Simplify locale/translation routing (wincent.com, ea50092)

I can use the new :has_many option here to make the routing simpler.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:20 AM

Whitespace fix for routes file (wincent.com, 002b836)

Add missing newline at end of file to quieten diff output.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:20 AM

Add convenience methods to String class for translating wikitext markup (wincent.com, 8a32b3d)

I was originally planning on storing a copy of the wikitext source in the database and also a copy of the generated HTML too. But it turns out that the wikitext translator is even faster than I had hoped so I can just store the wikitext markup and convert it into HTML on the fly.

This commit adds a couple of helper methods to the String class for doing this conveniently.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 10:17 AM

February 6, 2008

More internal link specs (wikitext, c92f50a)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 7:13 PM

Add trimming function for link targets (wikitext, d4ff4f5)

Unfortunately I found a case where the early scan-time suppression of leading and trailing whitespace gets tripped up by faulty input; I had forgotten that newlines and end-of-files do not auto-close still-open link spans, and so in those cases you actually do a rollback and by then you've already eaten your whitespace.

So this commit adds a trimming function which is called only when the link is fully parsed, confirmed valid, and ready to be emitted.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 6:01 PM

Trim trailing whitespace from internal link text (wikitext, fdf390a)

This follows up on the preceding commit that trimmed leading whitespace.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:38 PM

Suppress whitespace immediately after the separator in internal links (wikitext, 968db92)

If we've seen the internal link start marker, the link target, and the separator, we know we're guaranteed to have a valid link (even if the user doesnt close it we will auto-close it). In this case it is safe to suppress the emission of whitespace rather than letting it through and sanitizing afterwards (like we have to do with spaces that come before the separator).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:28 PM

Minor update to README (testing) (wikitext, d8dfb10)

Mention testing the extension.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:47 PM

Many more specs for internal link edge cases (wikitext, c4cbd3d)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:36 PM

Fix "non_space" tracking when sanitizing links (wikitext, c0bccec)

This is the same bug that could occur during reallocation as was fixed for the link encoder in commit 0b85588.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:32 PM

Fix whitespace trimming when reallocating in the encoder (wikitext, 0b85588)

Must update the "non_space" pointer when reallocating the buffer in the link encoder, otherwise we can crash.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:30 PM

Don't trim whitespace when rolling back (wikitext, 77476d9)

This fixes the issue mentioned in the last commit.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:29 PM

Teach sanitizer to trim trailing whitespace (wikitext, 1207027)

The only problem here is when we rollback a failed link we might end up trimming space from it; will look into a fix for that.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:50 PM

Eat trailing space while encoding link targets (wikitext, a77ecb5)

Similar commit for link sanitization to follow.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 3:20 PM

Eat leading whitespace during link encoding (wikitext, 3b20c0b)

As mentioned in e8b3b86, we need to eat whitespace here.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:59 PM

Eat leading whitespace in link targets (wikitext, e8b3b86)

We do this during the sanitization phase because its only at that point that we know we've got valid input and are actually going to emit a link. If we did it during parsing but prior to confirming that we had a valid link then we might end up throwing away a space that we'll later need.

A follow-up link will make the same change for link encoding.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:54 PM

More autolinking specs (wikitext, 256d5c2)

Emails should not be converted into hyperlinks if autolinking is turned off.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:53 PM

Fix crasher with email addresses inside nowiki and pre spans (wikitext, 47c718c)

Fix crasher discovered while adding autolinking specs for email addresses.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:34 PM

Basic autolinking for email addresses (wikitext, 4f5854e)

This handles the basic case; the edge cases still need to be handled (for example, what happens when an email address is part of an internal link text, internal link target, or external link text); I'm going to write specs for these cases before proceding with the implementation.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:27 PM

Tokenize email addresses (wikitext, 9089f82)

This is a prerequisite before we can look at autolinking email addresses in the parser.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:17 PM

Update Token "types" method (wikitext, 853c4a3)

This for the newly added STRONG_START, STRONG_END, EM_START and EM_END tokens.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 2:16 PM

Add ary_clear method and use it (wikitext, 9b411a0)

A bit of a micro-optimization: add an ary_clear method equivalent to the Array#clear method. Although the speed-up should be intangible it does make the resulting code look a little bit neater.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:35 PM

Use ALLOC_N instead of malloc (wikitext, ebe7aea)

There was a lone malloc call in the parser which is best replaced with ALLOC_N (the latter will automatically invoke the Ruby garbage collector if allocation fails, and will raise an exception if allocation fails again after that).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:19 PM

Update strong and em specs (wikitext, 5d24e81)

These now cover a number of edge cases with invalid input.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:14 PM

Update parser to handle literal em and strong HTML tags (wikitext, e9a600d)

This is a basic implementation that handles most well-formed input. There are a number of known edge cases that still need to be handled, and more specs to test them out.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:58 PM

Tokenize em and strong HTML tags (wikitext, 2baf13a)

This is useful for cases where you either type HTML tags instead of wikitext markup without thinking, or where you explicitly prefer to use explicit tags for the purposes of disambiguation (ie. does "'''''" mean "strong em" or "em strong"?). By default, the wikitext extension assumes strong em because it is an on-the-fly transformer and can't look ahead to see what the likely ordering is, but sometimes you want the other order and using HTML tags is one way of imposing it (the other is to insert a zero-length nowiki span, or insert whitespace, between the opening em and strong tags).

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:08 PM

Add specs for case insensitive tokenization of TT tokens (wikitext, 93ceb30)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 12:08 PM

Update links in documentation (wikitext, 9e6ee97)

Include links to Git repository browser and RubyForge project page.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:56 AM

Rakefile updates for RubyForge project (wikitext, 9d9131a)

Add Rakefile task for uploading RDoc documentation to RubyForge website.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 11:38 AM

February 5, 2008

Initial cut at documentation (wikitext, 57fec1c)

Still much that could be added but this is basic coverage.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:09 PM

Minor code comment cleanup (wikitext, c77cf30)

Minor cleanup done during initial write-up of documentation.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 8:06 PM

Add RDoc target to Rakefile (wikitext, 579b99d)

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:31 PM

Rename link encoding and sanitization methods for consistency (wikitext, 395d75a)

Now all the public methods in Wikitext::Parser begin with "Wikitext_parser".

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 5:30 PM

Include "ext/depend" in Gem (wikitext, f47230b)

This needed in order for "gem install" to work.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:48 PM

Updates to Rakefile for RubyGems 1.0 (wikitext, f753682)

autorequire is deprecated now and if you want warning-free builds you have no choice but to specify a rubyforge_project value.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 4:41 PM

Add profiling_parse method (wikitext, c712074)

Just a wrapper method to make profiling a bit easier. The goal is to reduce the overhead of the "times" method during profiling.

Signed-off-by: Wincent Colaiuta <win@wincent.com>

Posted 1:17 PM

Replace more rb_str_append calls with rb_str_cat (wikitext, a5954dc)

This time for token text, and it yields an even bigger speed-up. Before:

short slab of ASCII text 2.380000 0.010000 2.390000 ( 2.461861) short slab of UTF-8 text 4.860000 0.000000 4.860000 ( 5.016289)

After:

short slab of ASCII text 1.5700