chris blogs: November 2004

30nov2004 · Nukumi2 documentation

The Nukumi2 codebase is doing very well, and the code would be fairly ready for release. But there is not a single line of documentation written by now, and lots of things are still specified informally if at all.

At least basic documentation is clearly needed before release of 0.1. (Un)fortunately, I’m a bit sick right now, so I couldn’t code (anything useful) anyway; I hope I can spend some time writing documentation, maybe an introduction to the core concepts and a step-by-step guide to your own blog.

Documentation will be written of course using Nukumi2 itself, to demonstrate it’s hackability and flexiblility.

Then, I need to figure how to package it… I use lots of “vendor” libraries written by myself, but located in other repositories; I think I can work around that with some bits of rake hacking. (Hey, I package using recursive calls to Rakefiles interspersed with calls to darcs dist… :-))

In case anyone wants to contribute artwork, such as a default template or a button/logo, please mail chneukirchen@yahoo.de.

NP: Elliot Smith—Memory Lane

27nov2004 · LoC statistics for darcs

Today, I did a nice hack using Darcs to make a Lines of Code statistic.

Darcs provides a command—darcs trackdown—that gets run on every revision of the repository until the given command returns true. Usually, you’d use that to “trackdown” a certain bug by running the testcase on every revision.

However, trackdown also provides a nice way to simply run an command over each revision. I ran this on the darcs repository:

darcs trackdown 'find . -type f -print | xargs wc -l |
      tail -1 | tee -a /tmp/darcsloc; false'

Note the false, so it will always “fail”.

Now twiddle with that data a bit, and you can quite easily get a diagram like:

Lines Of Code of the Darcs repository

(The “vertical lines” are time skews of the Darcs changelog, as patches only have a “record time”, not a “merge time”…)

NP: Neil Young—My My, Hey Hey

25nov2004 · Nukumi2 transition done!

Finally, this blog is converted to Nukumi2!

(Should you ever feel the need to look at the old one, try http://kronavita.de/chris/old-blog.)

Old permalinks will point to the old blog.

As a side-effect, you now can also subscribe to certain categories of this blog. To do that, simply change the .html of the URL to .rss respective .atom (preferred). Everything else should be the same for the usual visitor (well, except the new, lighter design…).

The first public release (Will be 0.2, I think) will need some more time, tough… maybe for Christmas. :-)

NP: Die Toten Hosen—Ich bin die Sehnsucht in dir

24nov2004 · Freud in der Physik?

Was mag sich der Autor wohl gedacht haben, als er das verfasst hat? (Dorn-Bader, Physik Oberstufe, Schrödel 1983, S. 53):

Wir haben nicht bedacht, dass im schwerefreien Raum das Wasser einen geraden Strahl längs der neuen Abspritzgeschwindigkeit v0 schräg nach oben bilden würde. In diese Richtung müssen wir nun deshalb auch die bislang waagerechte Latte drehen. Nun geht der “Laufsteg von Bettina” schräg nach oben. Da die Stäbchen an der Latte gelenkig befestigt sind, bleiben sie vertikal. Sie markieren jetzt die von der schrägen Latte aus gesehenen Fallwege und werden unten naß — wie es sein soll. […] Ihr Scheitel liegt jetzt schräg über der Abspritzdüse.

Nein, was haben wir gelacht… :-)

Final move to Nukumi2 tomorrow!

NP: Tom Waits—Bad Liver And A Broken Heart

24nov2004 · Nukumi2

Nukumi2 is a very flexible and open framework for building static and dynamic content websites. It was mainly created to allow blogging and has features to ease this, but in general every kind of mainly content providing website can be created with it. For example, it empowers this site.

Also see blog posts about Nukumi2.

Features

  • Multiple output flavors (default templates are fully validated):
    • XHTML
    • RSS2
    • Atom
  • Dynamic (Webrick) or static (file generation) publishing.
  • Search-engine friendly URIs
  • Emacs mode

What’s new in Nukumi2 0.5?

  • Atom 1.0 support
  • Using Dissident for dependency injection instead of Needle
  • Works with Ruby 1.8.3+

This release does not incorporate the Kashmir templating engine yet, just because I lack the time to rewrite the templates.

The switch to Dissident means you need to change your config.rb file, please read the documentation (especially “A Step-by-step Guide to Your Own Blog/Designing the TopicTree”) about the new syntax of the register block.

Planned features

  • Prev/Next linking
  • Better detection of new entries (no more active pings)
  • More backends:
    • XMLRPC API
    • Database
    • Version controlled
  • CGI/FastCGI interface
  • Multiple users
  • Comments

Dependencies

In order to run Nukumi2, you need (all available via RPA):

  • Ruby 1.8.1 or later
  • BlueCloth

rake is needed for Nukumi2 development.

History

  • December 30, 2005: Second public version 0.5 released.

  • December 30, 2004: First public version 0.1 released.

Getting it

You can download rand.rb at
http://chneukirchen.org/releases/nukumi2-0.5.0.tar.gz.
md5sum: c3650d4d1adc73b16cbf302385be495b

Upstream source

You can get latest development releases using darcs by executing:

darcs get http://chneukirchen.org/repos/nukumi2.5

darcs send is the preferred way to send patches, but mailing diffs is fine too.

Thanks to…

  • Sebastian Vuorinen for implementing Atom 1.0 support.

Authors

Please mail bugs, feature requests or patches to the mail addresses above or use IRC to contact the developers.

Copyright

Copyright © 2004, 2005 Christian Neukirchen
This work is licensed under the terms of the GNU General Public License.

23nov2004 · Lange keine Quotes mehr...

…doch da sind sie wieder!

Der Mixer ist ein durcheinandergerät.

Der Kot wird kodiert und das Leder lädiert.

Sie gingen ans Wasser um die Pferde zu tranken.Was?? Zu ertränken?

…Alkohol aus dubiosen Quellen…Aus Frankreich?!

Was ist eine Mutante?Das Gegenteil vom Muonkel?

Hattest du schon Kontakt mit Herr Wulz?Ja, aber nur im Gespräch!

NP: Tom Petty, George Harrison, Neil Young, Eric Clapton, Bob Dylan, Roger McGuinn—My Back Pages
Brilliant song!

21nov2004 · Nukumi2 goes live!

Over the weekend, I’ve ported this blog over to Nukumi2. You can try the new version at http://kronavita.de/chris/blog2. Please don’t update any bookmarks, as the new version will be replacing the old.

If you find any errors, bugs or other things (validation errors!), drop me a mail.

Update: Feeds compared with relative links suck, I need to look into how to do that.

Update 2: They suck more than I thought. I’m using absolute links for now…

NP: Tom Waits—Grapefruit Moon

19nov2004 · Needle Extras

Jamis Buck announced Needle-Extras 1.0.0 today on ruby-talk.

It’s a “collection of additional services that may be used with the Needle dependency injection container for Ruby”.

If you want to do anything that goes further than the most basic use of Needle, I strongly recommend you to get it and look at the sources. You can learn lots from them.

So far, Needle-Extras consists of:

  • AttrInject, “an implementation of dependency injection that uses declared interfaces to determine dependencies.” (Yeah, see “attr_inject: A step towards interface injection”. My idea. :-))

    I think it is an good example of how to implement a Needle::Pipeline::Element.

  • Multicast, a service that “allows you to easily broadcast messages to a specified set of objects. It is, in essence, a kind of observer pattern, with the observers being given to the multicaster when it is created. Events are then sent to the observers by invoking methods on the multicaster.”

    Read this if you want to define services that behave as if they were in the core Needle library.

  • RequireLibrary, “a mini-framework for registering service libraries with Needle so that they can be imported into other projects with a minimum of headache.”

    This shows how a general-purpose Needle utility can be done.

I have definitely some use for all of them…

NP: Phish—While My Guitar Gently Weeps

17nov2004 · Learning Regexp Optimization the Hard Way

Playing a bit with Nukumi2, I discovered a very interesting thing: If I disable BlueCloth, my pages suddenly render a lot slower. “This ain’t possible”, I stumbled. BlueCloth is fairly fast, but still the slowest component of the rendering stage. “What the hell is going on?”

I timed the rendering of a few entries by themselves, they were all faster than with BlueCloth. Well, all except one. Yesterday’s. I trimmed it a bit, and finally found out that if I removed that part, it would run at full speed again:

class Class
  def attr_inject(attribute)
    (@__attributes_to_inject ||= []) << attribute
  end
end

I ran a profiler (ruby -r profile), and it turned out String#scan was taking quite long… Where is it used? In RubyPants, look at the code:

def tokenize
  tag_soup = /([^<]*)(<[^>]*>)/

  tokens = []

  prev_end = 0
  scan(tag_soup) {
    tokens << [:text, $1]  if $1 != ""
    tokens << [:tag, $2]

    prev_end = $~.end(0)
  }

  if prev_end < size
    tokens << [:text, self[prev_end..-1]]
  end

  tokens
end

Ok, it’s a part of the HTML tokenizer… The only reason scan can be slow is the regular expression, let’s have a closer look:

tag_soup = /([^<]*)(<[^>]*>)/

Well, it looks alright: First match any preceding text and then a tag. After toying around for fifteen minutes with that regexp, trying various tweaks (e.g. making it non-greedy), I built a proper test case. That string took very long, ~6 seconds to parse it 100 times:

&lt;hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh...

Even more h to come. I remembered a scary issue about regular expressions: Under certain circumstances, backtracking can make Regexps run in exponential time (various string sizes showed that) because they try every possible match.

It really looked like that was the case… But where the hell does it backtrack? There are no alternations, and the match should run linearly. Well, it should. But it didn’t. It turned out that it really tries every possible match, because the match isn’t anchored.

D’oh! Of course, there is no anchor. Luckily, I remembered \G from back when I used Perl. man perlre says:

       \G  Match only at pos() (e.g. at the end-of-match position
           of prior m//g)

Hey, cool. That should be exactly what we need. Make that line simply

  tag_soup = /\G([^<]*)(<[^>]*>)/

and off it goes! Blazingly fast. :-)

Who would have thought two characters can make such a big difference?

(Of course, I’d have never discovered that “bug” if I only ran it on correct input (as >> is not valid HTML), so test your code with “invalid” data too!)

NP: Sum 41—We’re All To Blame

15nov2004 · attr_inject: A step towards interface injection

During the needlefication of Nukumi2, I came across a lot of service definitions like this:

class Blog
  attr_accessor :backends
  attr_accessor :page
  attr_accessor :view
  attr_accessor :topictree
  # ...
end

blog {
  b = Nukumi2::Blog.new
  b.backends = backends
  b.config = config
  b.page = page
  b.topictree = topictree
  b
}

Needing to define all these accessors was a thorn in my side. Although I would need some attr_readers for them, writing is not a thing I want to allow.

The use of attr_reader, attr_writer and attr_accessor is not just a shortcut in Ruby; it also has a semantical meaning: You are allowed to read this attribute, to write this attribute or to do both. RDoc, for example, will base it’s documentation on them, there will be a special section Attributes in the class documentation.

Therefore, I whipped up a new kind of “accessor”—attr_inject—that allows to declare an attribute for dependency injection. Now, the class will look like:

class Blog
  attr_inject :backends
  attr_inject :page
  attr_inject :view
  attr_inject :topictree

  attr_reader :topictree

  # ...
end

It only opens the things for the “general public” that are actually wanted. However, having these declarations, we can now simply do:

blog {
  Nukumi2::Blog.new.inject_attributes(this_container)
}

Pretty nice, heh? :-)

For now, I just have some very simple code; some additions will be needed to make use of all possibilities. It would be nice to allow users to override attribute mappings (for now, attributes and service names need to match):

blog {
  Nukumi2::Blog.new.inject_attributes(this_container,
                     :backends => my_special_backend)
}

Allowing the use of inject_attributes exactly once would be good too.

By the way, all that’s needed for attr_inject is just these 15 lines:

class Class
  def attr_inject(attribute)
    (@__attributes_to_inject ||= []) << attribute
  end
end

class Object
  def inject_attributes(registry)
    (self.class.instance_variable_get(:@__attributes_to_inject) ||
        []).each { |attribute|
      instance_variable_set "@#{attribute}", registry[attribute]
    }
    self
  end
end

For “serious” use, this code will need more checks and features, of course. Another problem is that attr_inject is not in the standard library and therefore will break code that may be used with and without Needle. This can easily be avoided by defining attr_inject to do nothing unless it’s already defined, though…

NP: Bob Dylan—Is Your Love in Vain?

13nov2004 · RubyPants 0.2 released

As stated on ruby-talk and “Progress of Nukumi2”, I released RubyPants 0.2 yesterday.

You can get it at http://kronavita.de/chris/data/rubypants-0.2.tar.gz.

RubyPants is a Ruby port of the smart-quotes library SmartyPants.

The original SmartyPants is a free web publishing plug-in for Movable Type, Blosxom, and BBEdit that easily translates plain ASCII punctuation characters into “smart” typographic punctuation HTML entities.

RubyPants is a general-purpose library that uses an API compatible to Red- and BlueCloth to perform these punctuation translations.

RubyPants 0.2 is also available via RPA:

rpa install rubypants

Have fun.

Nukumi2 is making nice progress. Today I added RSS and Atom feed support, plus Needlefication.

NP: The Brian Jonestown Massacre—Short Wave

11nov2004 · Musidex

Since I couldn’t find anything in my playlist anymore (Actually, this is not true. xmms’ J works excellent for finding single songs, but grouping and getting stuff together is hard), I decided storing my MP3/OGG-metadata in a TopicTree. Unfortunately, some of my MP3 are not tagged, so I needed to do some ugly heuristics to find name and title…

Ideally, I’d run something like this

musidex --random /artist/bobdylan /year/1/9/8 >playlist.m3u

and get a randomized playlist of all stuff by Bob Dylan he did in the 80’s. I tried running xmms on a playlist coming out of a FIFO, but that didn’t work. For real dynamic playlists, I’d probably need a HTTP server (WEBrick to the rescue!).

Of course, a nice frontend would rock too…

NP: Pearl Jam—Light years

08nov2004 · Sidney in Ägypten

Zum Ende der Herbstferien mal wieder Quotes:

Haha, ich dachte grad’, Seattle wäre Sidney, aber Sidney liegt ja in Ägypten!

Busenknettag

Die macht immer “mmmm” wenn sie zur Tür reinkommt.Vielleicht muss sie noch schlucken?

Mal wieder etwas langsamer schalten…

Apropos Schalten: Andi, herzlichen Glückwunsch zur bestandenen (auf Anhieb! :D) Führerscheinprüfung!

BTW, Darcs 1.0.0 has been released!

NP: Billy Talent—Try Honesty

07nov2004 · Progress of Nukumi2

Nukumi2 is doing very well, I’m actually writing on code that will be in the first public release 0.1 codename “godspeed”. :-) I hope I’ll publish this release this month, but I can’t promise it.

Not mentioned new features include:

  • Clotheslines: You can now chain multiple clothes together, respective any code that follows the StarCloth API (which is StarCloth.new(text).to_html). My sample blog runs currently on this clothesline:

    BlueCloth RubyPants NiceHR
    

    Which is essentially the same as:

    result = NiceHR.new(RubyPants.new(
               BlueCloth.new(text).to_html).to_html).to_html
    
  • RubyPants will be there, a proper standalone release will appear this week. (John Gruber already linked to it, thanks a lot ;-))

  • NiceHR, a sample cloth that will wrap <hr /> into a <div class="separator"> for better taming with CSS. It’s primarily an example on how to use REXML for tasks that were done with XSLT in Nukumi1.

NP: Bob Dylan—I Want You (At Budokan)

04nov2004 · RMS @ ulm.ccc

Whee, I’m leaving this evening to Ulm to see Richard M. Stallman in real life! He’ll be giving a speech about the dangers of Software Patents there. Thanks in advance to the CCC Ulm for organizing this.

Pretty excited…

NP: Interpol—Slow Hands

03nov2004 · RubyPants

Today, I ported the Perl library SmartyPants to Ruby. SmartyPants calls itself an “web publishing utility that translates plain ASCII punctuation characters into “smart” typographic punctuation HTML entities.” By the way, it’s written by the author of Markdown (which I use for a long time now), John Gruber.

Porting was quite easy, as I had a Perl and a Python implementation available (playing “Rosetta Stone” is fun :-)), I only got stuck at some Perl-specific Regexp stuff, but that was worked around easily.

Before RubyPants, I used some very primitive regexen to make em-dashs (--- gets —) and and ellipses (... gets …). However, I never got around converting quotes ("foo" gets “foo”) because that is quite difficult. Fortunately, SmartyPants handles most cases, and the very special cases can easily get corrected manually.

Grab RubyPants 0.1 while it’s still hot!: rubypants.rb (Proper release comming soon)

NP: Mark Knopfler—Boom Like That

03nov2004 · Election 2004

I’m quite disappointed on the election results. (And I have to say that an amazingly big number of people outed as Bush voters—people I’d never have expected to :-/)

The next four years are likely to be “fun”.

Come writers and critics
Who prophesize with your pen
And keep your eyes wide
The chance won’t come again
And don’t speak too soon
For the wheel’s still in spin
And there’s no tellin’ who
That it’s namin’.
For the loser now
Will be later to win
For the times they are a-changin’.

NP: Bob Dylan—The Times They Are A-Changing

Copyright © 2004–2016