Nukumi2 0.5 released
Nukumi2 0.5 is released.
This is the second release of Nukumi2 since exactly this day last
year. Nukumi2.5 (or, for technical reasons, Nukumi2 0.5) is primarily
a maintenance release fixing some small bugs; the biggest change is
the switch from Needle to Dissident for dependency injection. This
change will require you to do a minor change in your config.rb.
Nukumi2.5 is one of the bug-fixing releases before Nukumi3, which will
never exist. Instead, Nukumi2.5 will slowly converge to
Nukumi2.5029078750, which is the second Feigenbaum constant.
What’s new in Nukumi2 0.5?
- Atom 1.0 support (thanks to Sebastian Vuorinen)
- Using Dissident for dependency injection instead of Needle
- Works with Ruby 1.8.3+
This release does not incorporate the Kashmir templating engine yet,
just because I lack the time to rewrite the templates.
You can get Nukumi2 0.5 at:
http://chneukirchen.org/releases/nukumi2-0.5.0.tar.gz
Merry blogging and a happy new year. :-)
NP: Bob Dylan—I Shall be Released
Moving to Atom 1.0
Thanks to a patch by Sebastian Vuorinen,
Nukumi2 now supports Atom
1.0 feeds. I changed the feeds of chris
blogs and
Anarchaia to Atom 1.0. If your feed reader
can’t cope with it, please get a different one, use the RSS2 feed, or
use the old Atom 0.3 feeds which have the extension .atom0.3. I
don’t want to keep them around for too long. (I’ll check the logs for
this.)
If you want to publish Atom 1.0 too, darcs pull now.
Thanks a lot, Sebastian.
NP: Bob Dylan—It Takes A Lot To Laugh, It Takes A Train To Cry
Thoughts on Vooly
I’ve been using Vooly for ten days now,
but not completely as I had it in mind: I wrote entries for
Anarchaia using it, inbetween
ordinary HTML.
Now, how does this work, actually? Anarchaia is, if you look closely,
just a list of certain things, just as
- Links
- Flickr images
- Quotes
- IRC quotes
- Lyrics
Each of these get their own Vooly tag, for example links are structured like
this:
<<link << URL >>
<< INTRO >> DESCRIPTION >>
Or, as a real example:
<<link << http://www.picotux.com/ >>
<< picotux >>, the world's smallest Linux computer.>>
(IRC-)Quotes and Lyrics behave a bit differently, as there is special
non-Vooly syntax used. I do that because it’s easier to write and
feels more natural too:
<<quote God not only plays dice in physics but also in pure
mathematics. Mathematical truth is sometimes nothing more than a
perfect coin toss.
--- Gregory Chaitin, A Random Walk in Arithmetic>>
Everything after the --- will get converted to the source.
As you (hopefully) can see, writing repetitive HTML with “macros” like
this is much easier than doing it manually (or with XML/XSLT, even!).
Now, let’s see how this actually is implemented: Since Anarchaia is
powered by Nukumi2, it’s just a matter
of another plugin for it, called AugmentedHTML. This plugin is trivial:
class AugmentedHTML
def initialize(string)
@string = string
end
def to_html
v = Vooly::Slurp.new(@string, false)
data = v.slurp.with_mapping! Decorator::MAPPING
data.to_s
end
end
AugmentedHTML uses Vooly::Slurp, a tree-like interface (I prefer
this term over DOM) to Vooly documents. It provides a nice method
with_mapping! that calls constructors given in the mapping.
For example, for the <<link>>, this looks like:
class Link < Decorator
TEMPLATE = Kashmir.new(<<EOF)
<p class="link">
<a href="^href">^title</a>^description
</p>
EOF
def initialize(href, title, description='')
@href = href
@title = title
@description = description
end
def to_s
TEMPLATE.expand { |e|
e.href @href
e.title @title
e.description @description
}
end
end
You can see, I’m exercising all the cool technologies I built in the
last months. :-) The important part is the initializer, look how the
elements of the document get mapped onto parameters.
Then, we simply call to_s recursively on the Slurp, and voilà,
there is our HTML!
Now, this works pretty good, but there are some issues with Vooly with
respect to using it that way. One, perhaps minor thing, are syntactic
constructs like
<<foo <em>cool</em>>>
This will not work as expected, since the parser will (and has to!)
match on the first >> to close the tag. Therefore, I decided
to make Vooly drop the very last space of tag content, so you simply
write
<<foo <em>cool</em> >>
And it’s not ambiguous anymore.
The more difficult thing is that I can’t use (my beloved) BlueCloth
anymore, since BlueCloth (and Markdown in general?) will interpret
<foo as unclosed HTML,
and ignore that block till it finds the matching <.
This is very unfortunate, but I can’t figure a way how to avoid it.
Also, AugmentedHTML must run before BlueCloth, since BlueCloth
will also try to escape the “superfluous” angle brackets! I wonder if
I should maybe change Vooly brackets to {{ and }}, but these are
icky to type (at least on German keyboards), and don’t look and feel
as near as cool as << and >> do…
NP: Bright Eyes—Easy/Lucky/Free
Tagged Entries
Today I extended Nukumi2 to use Technorati
Tags for each entry’s
categories. This means you can easily find my entries about Ruby by
looking at the “Ruby” Tag Site.
Thanks a lot to _why who mentioned
this on RedHandled.
NP: Interpol—Obstacle 1
Ruby related news
There have happened some quite interesting Ruby things:
rand.rb has been officially
released. (It used to be in RPA for several weeks by now.)
Jamis Buck has a cool story
about building transcription software that uses Ruby, Ogg Vorbis and
vim.
Nukumi2 now has documentation. At least a little bit to get you
started. :-) I’m quite confident I get a 0.1 out this year…
NP: Pearl Jam & Neil Young—Act of Love
Nukumi2 documentation
The Nukumi2 codebase is doing very well, and the code would be fairly
ready for release. But there is not a single line of documentation
written by now, and lots of things are still specified informally if
at all.
At least basic documentation is clearly needed before release of 0.1.
(Un)fortunately, I’m a bit sick right now, so I couldn’t code
(anything useful) anyway; I hope I can spend some time writing
documentation, maybe an introduction to the core concepts and a
step-by-step guide to your own blog.
Documentation will be written of course using Nukumi2 itself, to
demonstrate it’s hackability and flexiblility.
Then, I need to figure how to package it… I use lots of “vendor”
libraries written by myself, but located in other repositories; I
think I can work around that with some bits of rake hacking.
(Hey, I package using recursive calls to Rakefiles interspersed
with calls to darcs dist… :-))
In case anyone wants to contribute artwork, such as a default template
or a button/logo, please mail chneukirchen@yahoo.de.
NP: Elliot Smith—Memory Lane
Nukumi2 transition done!
Finally, this blog is converted to Nukumi2!
(Should you ever feel the need to look at the old one, try
http://kronavita.de/chris/old-blog.)
Old permalinks will point to the old blog.
As a side-effect, you now can also subscribe to certain categories of
this blog. To do that, simply change the .html of the URL to .rss
respective .atom (preferred). Everything else should be the same
for the usual visitor (well, except the new, lighter design…).
The first public release (Will be 0.2, I think) will need some more time,
tough… maybe for Christmas. :-)
NP: Die Toten Hosen—Ich bin die Sehnsucht in dir
Nukumi2 goes live!
Over the weekend, I’ve ported this blog over to Nukumi2. You can try
the new version at http://kronavita.de/chris/blog2. Please don’t
update any bookmarks, as the new version will be replacing the old.
If you find any errors, bugs or other things (validation errors!), drop
me a mail.
Update: Feeds compared with relative links suck, I need to look
into how to do that.
Update 2: They suck more than I thought. I’m using absolute links
for now…
NP: Tom Waits—Grapefruit Moon
Learning Regexp Optimization the Hard Way
Playing a bit with Nukumi2, I discovered a very interesting thing: If
I disable BlueCloth, my pages suddenly render a lot slower.
“This ain’t possible”, I stumbled. BlueCloth is fairly fast, but
still the slowest component of the rendering stage. “What the hell is
going on?”
I timed the rendering of a few entries by themselves, they were all
faster than with BlueCloth. Well, all except one. Yesterday’s. I
trimmed it a bit, and finally found out that if I removed that part,
it would run at full speed again:
class Class
def attr_inject(attribute)
(@__attributes_to_inject ||= []) << attribute
end
end
I ran a profiler (ruby -r profile), and it turned out String#scan
was taking quite long… Where is it used? In RubyPants, look at the code:
def tokenize
tag_soup = /([^<]*)(<[^>]*>)/
tokens = []
prev_end = 0
scan(tag_soup) {
tokens << [:text, $1] if $1 != ""
tokens << [:tag, $2]
prev_end = $~.end(0)
}
if prev_end < size
tokens << [:text, self[prev_end..-1]]
end
tokens
end
Ok, it’s a part of the HTML tokenizer… The only reason scan can be
slow is the regular expression, let’s have a closer look:
tag_soup = /([^<]*)(<[^>]*>)/
Well, it looks alright: First match any preceding text and then a tag.
After toying around for fifteen minutes with that regexp, trying
various tweaks (e.g. making it non-greedy), I built a proper test
case. That string took very long, ~6 seconds to parse it 100 times:
<hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh...
Even more h to come. I remembered a scary issue about
regular expressions: Under certain circumstances, backtracking can make
Regexps run in exponential time (various string sizes showed that)
because they try every possible match.
It really looked like that was the case… But where the hell does it
backtrack? There are no alternations, and the match should run
linearly. Well, it should. But it didn’t. It turned out that it
really tries every possible match, because the match isn’t anchored.
D’oh! Of course, there is no anchor. Luckily, I remembered \G from
back when I used Perl. man perlre says:
\G Match only at pos() (e.g. at the end-of-match position
of prior m//g)
Hey, cool. That should be exactly what we need. Make that line simply
tag_soup = /\G([^<]*)(<[^>]*>)/
and off it goes! Blazingly fast. :-)
Who would have thought two characters can make such a big difference?
(Of course, I’d have never discovered that “bug” if I only ran it on
correct input (as >> is not valid HTML), so test your code with
“invalid” data too!)
NP: Sum 41—We’re All To Blame
RubyPants
Today, I ported the Perl library SmartyPants to Ruby. SmartyPants
calls itself an “web publishing utility that translates plain ASCII
punctuation characters into “smart” typographic punctuation HTML
entities.” By the way, it’s written by the author of
Markdown (which I use
for a long time
now),
John Gruber.
Porting was quite easy, as I had a Perl and a Python implementation
available (playing “Rosetta Stone” is fun :-)), I only got stuck at
some Perl-specific Regexp stuff, but that was worked around easily.
Before RubyPants, I used some very primitive regexen to make em-dashs
(--- gets —) and and ellipses (... gets …). However, I never
got around converting quotes ("foo" gets “foo”) because that is
quite difficult. Fortunately, SmartyPants handles most cases, and the
very special cases can easily get corrected manually.
Grab RubyPants 0.1 while it’s still hot!:
rubypants.rb
(Proper release comming soon)
NP: Mark Knopfler—Boom Like That
Nukumi2 news
Ok, I’ve got a fairly good working spike of
Nukumi2.
However, I won’t release any code as it’s really ugly and I don’t
anyone want to see it. (lots of globals and complicated regexen
around…)
However, I’m now confident that I can start writing Nukumi2 the
Right Way. The design is correct, the navigation works. Static rendering
too.
NP: Bob Dylan—Shooting Star
News on Nukumi2
I’ve starting to spike Nukumi2 and doing quite good process. I’m now
sure that my categorization code (dubbed Topical, to be described
later) will work and provide the main navigation. Unfortunately, I
did a rather stupid thing defining the Topical “query” syntax that
turns out to end up in invalid URIs (WEBrick has problems with //).
Oh well, gotta change that and update all the unit tests…
Also, my application container will need a rewrite of some parts; just
that kind of stuff that happens when you write libraries without
having an application in mind. ;-)
NP: Velvet Revolver—Headspace
Nukumi2 Roadmap
In the last few weeks, I have been writing some Ruby libraries that
will ultimately empower Nukumi2, my next-generation blogware. In particular,
these, and their current state, are:
bungee: The application container that will hold Nukumi2 (and other
software, I hope). It will allow to use Nukumi2 using CGI or as a
Webrick servlet, but can also do static page generation. bungee is
currently early beta, but the design works.
topical: A dynamic and flexible categorization framework. I only
started that yesterday, but I can tell it works. :-)
tangerine: Nukumi2 won’t make use of XSLT (at least, not by
default). tangerine will be used to define the page templates. It
already works really nicely, but hasn’t been used a lot yet.
dana: Later versions of Nukumi2 will use possibly dana, a
pre-alpha, so-far proof-of-concept object database, mainly for
caching purposes.
These are other Ruby libraries, not written by me, that I am likely to
use:
needle is a rubyish
dependency-injection framework. I will almost surely want to use it
(at least 33% of the Nukumi1 code would have been superfluous if I
had known about dependency-injection back then…).
WEBrick: This Ruby-based webserver will be
the preferred way to access Nukumi2, however, CGI and static pages
will be supported too.
BlueCloth: BlueCloth has been
proven to be very useful and will continue to stay the default
markup language. Other Clothes will stay to be easy to add, though.
Now, why do we (or I, at least) need Nukumi2 at all?
I’m really dissatisfied with the current code base. Every feature I
want to add—and that should be easy—is harder than the one
before. I don’t understand the main processing loop anymore (shame
on me). Besides, I hope (nah, I’m sure…) that my coding qualities
have improved, and that refactoring won’t cut it in this case.
There is much more Ruby blogware around now that inspires me. When I
started coding Nukumi, all I knew of was an early Hobix, and various
blosxom ports. Now, I have looked at recent Hobix versions,
Blogtari!, Rublog, etc. etc. Nukumi2 won’t be without reason a new
major version.
There are various features that I’d like to have: dynamic page
generation (this will change a lot), better categorization
features…
It sounds like fun. :-)
I’ll start writing a spike this week; I estimate about 2 weeks for it.
(I shouldn’t estimate in my blog entries, I know.) Then, I’ll redo it
using test-driven development. I hope to get a 0.1 out this year.
NP: Green Day—American Idiot
(American voters, it will depend on you whether I have to play that
song again on November 2nd.)
Nukumi 0.5 released
After a long time without releases, I release Nukumi 0.5 today.
New features since the last release include:
- New Formatters
- External Formatters
- Fixed URLs
- Hiding pages
- Multiple keys
- Support for Categories
- Support for Actions
- Emacs support
- Improved feeds
See the README for more information.
Download: nukumi-0.5-patch-57.tar.gz
NP: Neil Young & Pearl Jam—I Shall Be Released
Nukumi as a CMS
Using a small plugin, I have been able to make Nukumi generate static
pages with a fixed URL. This can be used to empower a whole site
using Nukumi, even the parts that don’t relate to the blog.
Therefore, I’ve also moved the more useful parts of my personal
page to Nukumi.
There is now a intro page at http://www.kronavita.de/chris to
provide a menu for accessing the subpages.
NP: Johnny Cash—25 Minutes to Go
