chris blogs: May 2008

22may2008 · Heboris

Drop your productivity as fast as the tetrominoes fall!

If you think Tetris is a simple game, let me refer you to TetrisConcept, a wiki which aims “to compile every Tetris detail known to mankind.” Due to tricks like t-spins and other tactical rotations, the game becomes rather delicate.

Reading Tetris the Grand Master: A gameplay essay is a good way to learn about this.

Heboris Unofficial expansion is the best and most flexible version of Tetris I ever saw. It supports nine different rotation systems and a magnitude of different modes (especially really fast ones), while almost everything can be configured.

It was a bit tricky to get running on OS X, so here are the instructions:

  • Acquire This version works fine on Windows.

  • Acquire the OS X binary.

  • From MacPorts, install libsdl-framework, libsdl_mixer-framework, and libsdl_image-framework.

  • Copy from the latter .zip into the former’s exe/ directory.

  • Edit heboris.ini and change it to “english = 1;” to replace most of the japanese.

Run and enjoy! By default, the cursor keys move and Z/Y, X, C are the action buttons (No idea where Hold is, rebind it to V maybe).

If you want to play against CPU, you need to define key bindings, enable the CPU for the second player in the Option menu and then configure the second player using his keys—the CPU will just play for it.

NP: Liars—Plaster Casts of Everything

21may2008 · Review: Practical REST on Rails 2 Projects

Practical REST on Rails 2 Projects
by Ben Scofield.
Apress, Berkeley 2008.
284 pages.

[Full disclosure: I have received a copy of the book in exchange for this review.]

The book, targeted at intermediate and advanced Rails users, starts with a chapter “Why REST?” that tries to explain how REST helps interconnect applications and reflects the structure of the web. As examples for such connectivity a few mashups are presented. The author states that APIs can make web applications much more useful and interesting, but costly to develop. He contrasts REST with XML-RPC and SOAP. Finally, the basics of REST are outlined: It is a client/server architecture based on stateless requests that describe and transform resource representations. The author argues that REST makes it easier to develop clients and servers and extend these applications in the future, last but not the least because REST is implemented well in Rails 2.

After demonstrating the use of (deprecated) ActiveWebService, the author shows how these ideas are expressed nowadays in Rails 2, outlining the history of simply_restful. He continues by explaining the new styles of URI routing that also dispatch on the HTTP method like map.resources/map.resource/nesting and map.namespace. The author also addresses Rails’ support for multiple output formats and new helpers related to routing. Scaffolding is discussed and will be used in the book. At last, there is a mention of ActiveResource to use RESTful Rails applications together.

The third chapter develops the main application of the book dubbed MovieList. It is used throughout the rest of the book. MovieList, a site that informs about movies and their releases, lets users express their interests and displays when new movies with the same actors are released. The code is not developed test-first, but the downloadable code contains a test suite. Occasionally the code is pretty weird, for example it defines setters that are merely called for their side-effects using #update_attributes. In some places, explicit iteration over ActiveRecords would have been solved better by doing it in the database. Also, the generated HTML is partly unsemantic and hard to scrape (which is not that bad if the data can be reached by the API, you may argue). The author explicitly defines notifications and interests for the movies to belong to the logged-in singleton user (they reside at /user/interests, not /users/:id/interests), supposedly so one cannot see other people’s interests. In later chapters, he decides however to at least revert this for notifications—it would have been better to properly design it in first place, as it actually is a nice feature and more RESTful anyway (can you speak of “current users” in a stateless request, really?) and show how to protect the page for users that don’t want to allow it to be seen. The author mentions at the end of the chapter that a “great deal of planning, testing and other work has gone undescribed”; wouldn’t it have been useful to have these parts in the book as well? The actual architectural concerns of REST applications are not really mentioned in the book.

The next four chapters deal with accessing the MovieList applications from other technology. Chapter 4 uses JavaScript to provide a widget users can embed on their homepage and shows how to do full-fledged access to the application using AJAX after extending it to support JSON. Chapter 5 shows how to access the site using the PHP framework Squidoo. During this, a feature is added to allow users to display the movie releases within a time frame. This is the code used to parse the relative time, and I’m not kidding:

raw_time = params[:time] || '1 month'
time = eval("#{raw_time.sub(/ /, '.')}.from_now")

How this gaping remote code injection hole passed any kind of technical review is a miracle to me. Ironically, the next section is called “Injection Flaws”, and addresses SQL injection and so-called “HTML injection”, which actually is passing anything you want as parameters. The author then decides to “fix” it by checking the time parameter in the PHP script calling the Rails application. Duh.

Chapter 6 builds an client for the iPhone, optimized to its interface constraints: a small screen, popup keyboard and lower bandwidth. It uses the commonly used approach of defining a new Rails format that is triggered by a special subdomain or by user agent sniffing. The chapter shows how to use iUI to make the interface look native, too.

Chapter 7 shows how to embed the application to Facebook either by using iframes or the FBML. I have no idea how the contents of this chapter are related to REST, especially since the FBML approach actually calls everything using POST.

Chapter 8 is called “Dealing with success” and is about making the application faster and more robust. Apart from the classic caching approaches (which work very well in REST due to the idempotency of GET, but see below) and foreshadows of denormalization, it contains a few general hints on Rails and database performance. It also addresses throttling access to the API by using API keys and setting up auditing to monitor the site.

Finally, Chapter 9 tries to place “Rails in the enterprise” and explains the chances, but also the problems of REST and Rails in the enterprise. It contains a small example of how to create a RESTful interface for a SOAP backend.

Conclusion: Generally, I found the book lacking. Instead of shifting focus to the design and architecture of real-world REST applications and showing up the patterns that can be used to help development, the book shows how to combine a simple CRUD application with half a dozen of buzzword loaded Web 2.0 stuff. The semantics of REST are only half-heartedly addressed (a third of page 13 discusses what the HTTP methods mean), the idempotency of GET merely assumed (it’s actually in a parenthesized half-sentence on page 72), and the actual means of applying REST (proper status codes, correct/multiple content types) are not made explicit. Instead of wasting over 15 pages on screenshots unrelated(!) to the application and another 2 pages on showing a WSDL that is very much useless, the reader would have had more benefit from a table of HTTP status codes and content types. Also, writing a REST client in Ruby is not addressed by means other than (the limited, non-general) ActiveResource.

I think this sounded too negative, please bear with me: The book is okay. It’s well written, and if you like a whirlwind tour of Web 2.0 things one can do it’s a good read. It’s just not really about REST, or at least not what I’d expect of a book about REST.

Rating: 3.5 of 5 points.

NP: Dandi Wind—Hostages

17may2008 · Workshop on Self-sustaining Systems 2008

These are my personal notes from the Workshop on Self-sustaining Systems (S3).

[A quick note beforehand: The descriptions make use of a metric called “lines of code” (LOC), which widely is regarded as being insufficient for measuring program size. Due to lack of better means, I decided to mention them nevertheless. Take them with a grain of salt.]

The conference started after lunch, but we got there earlier so we had the chance to socialize with the HPI people already.

Ian Piumarta gave the first invited talk on his Late-bound lambda object architecture, wherein he decided to fight the fact that software tends to become too complex by finding out how much you can say with how little. He started by contrasting the three cubic miles of paper that is the US case law to Maxwell’s four tiny equations which are able to fully explain all electromagnetic phenomena.

Moving to software, he showed the source code line counts of some popular projects, and we could see, for example, that has more lines of code than the entire FreeBSD operating system. Comparing lines of code with the size of literary categories, he decided that 20kLOC, which roughly resemble a 400 page roman, are the maximum size of a program a programmer has the chance of fully understanding.

He continued explaining how it is possible to get much done in such a limited amount of code by showing off how simple things like shape rendering, which only consists of a few elementary transformations, is enough to render fonts and essentially provide a full graphical interaction system.

He gave a quick tour of other interesting ideas to be further investigated, such as multiple subjective perspectives, which result from the realization that objects have several roles, or regarding computation as “fields” (re)acting on particles, a concept which makes it easy to specify complex interactions between many objects.

Since predicting anything is difficult, he argues that it highly important to be as dynamic as possible, by using the fewest minimal abstractions that allow for any possible feature/paradigm to be added to the language/system.

His design combines functions and objects into the foundation of his system, COLA. Functions, represented as s-expressions provide the proto-behavior, while proto-structure expresses form by providing objects that messages are sent to. They provide the dichotomic base, since without function, form cannot be animated while function has no representation without form. This results in a self-describing structure.

(Actually, after the talk he told me they are trying to unify functions and objects into one thing, I’m very curious about the result.)

His talk was larded with memorable insight such as looking at GTK+ and noticing “C is highly deficient and wasteful” or encouraging the students to look at the “old” papers where technology was severely limited because “sometimes progress is behind you”.

The next talk was by Christophe Rhodes, who spoke on SBCL – A sanely bootstrappable Common Lisp. After a short overview of Common Lisp, he summarized the history of SBCL, which started as a fork of CMUCL in 1998. The main reason of the fork was that CMUCL was horrible to bootstrap and it actually wasn’t possible to compile CMUCL without already having the last version of CMUCL before that.

Christophe Rhodes emphasized the importance of a system being able to rebuilt itself from a “blueprint” since it makes the result a lot clearer, easier and predictable. He showed an example of a bug in a core data structure which was easily fixed by changing the structure and recompiling the system—without the ability to rebuilt from scratch, magic hackery would have been required to modify the existing structures to be compatible enough until they could replace themselves.

SBCL nowadays can be compiled using half a dozen of different CL implementations and does, once built, not any dependencies on the build environment.

This unusual level of self-sustainability has many benefits: it is more fun, enables quicker turnaround, makes the system more future-proof, and doesn’t limit improving the core of the system to hackers with magic abilities. Instead, every developer can work on the code because in the end it’s just another big Lisp program. Because of this, more people can help with the system and maintain it, thereby having more control over the destiny of the system as a whole.

Charlotte Heerzel was the next, presenting Reflection for the masses, in which she showed how to implement 3-Lisp using CLOS. She noticed that programming languages are made powerful by abstractions, but there are cases where one wants to get rid of them, for example if you need access to the current continuation.

Reflexive languages, on the other hand, allow the programmer to control internalization, normalization, and externalization within the language.

She showed lots of code, implementing a small Lisp interpreter in CPS and then adding structural reflection by exposing the internal data structures as abstract data types and behavioral reflection by introducing reflective-lambda, which has access to the current continuation, environment and code. For example, this allows to implement when as a first-class function.

The first day ended with two social events: first, there was a boat trip around Potsdam, where I had the great luck to sit on a table next to Richard P. Gabriel and Pascal Constanza. We had a long interesting discussion with them about the lack of (helpful?) limitations in programming, what designers really do, how Java became popular, the danger of the obvious and demonising copy and paste programming. I learned one thing about how rpg decides which poems to publish, which I cannot keep back:

Richard P. Gabriel writes a poem each day, and once a year, he needs to select six of them for publishing (six seems to be the usual amount the publisher wants). So how does he do it? Using a computer, he randomly picks sets of six, until all of them don’t suck. Then he sits down and revises them.

We crystallized this as the essence of design: to choose the things that don’t suck.

Afterwards, we had a dinner buffet on a restaurant boat until night.

The next day started with Daniel H. H. Ingalls (say it in German!) demonstrating The Lively Kernel – a self-supporting system on a web pagex. He admitted that web programming is complicated, but implementing Morphic in JavaScript seemed like an easy thing to do. While he explicitly mentioned that he is not proud of the code, he thinks it’s easy enough and works well.

The whole Lively Kernel is rendered using dynamically generated SVG and doesn’t include any (visible) HTML at all. All drawn stuff are vector objects. In good Smalltalky manner, the system uses MVC extensively.

After a short demonstration with some turtle graphics and live-code editing, he pointed out some of the non-obvious features of it: For example, to support multiple people working on it, they introduced a change-set format for JavaScript which allows to check in modified parts of the system into version control. Also, they have a pretty sophisticated system for profiling by dynamic method rewriting to insert measuring code.

The whole Lively system at the moment is a mere 10kLOC of JavaScript and already includes many details such as rich-text-boxes.

On the topic of security, Dan Ingalls proclaimed that his security philosophy was like this: “Make it work, then make it secure, and I hope someone else does step 2.”

Carl Friedrich Bolz presented a joint-work of eight Pythonists and Squeakers titled Back to the future in one week – Implementing a Smalltalk VM in PyPy which they started in October during a five day sprint.

PyPy, initially a Python implementation in Python, but now moving towards a general compiler tool-chain, enables one to write highly flexible language implementations because most things are late-bound. Therefore, garbage collection, object layout and the threading system can be exchanged easily and allow for lots of experimentation. PyPy aims to autogenerate dynamic compilers from interpreters written in a reduced set of Python called RPython with nevertheless allows full-fledged compile time metaprogramming.

They already implemented all Squeak bytecodes and most of the primitives. The resulting system, SPy, can load unmodified Squeak images and run simple benchmarks. It is roughly 10x slower than Squeak itself, and they plan to support the graphical builtins in the future as well.

The team continued hacking on SPy in a sprint in Berlin just after the S3.

Guillermo Adrián Molina next introduced Huemul, a Smalltalk implementation that directly generates native code. It is a very small system of only 4.5kLOC since it doesn’t try to do everything but instead reuse existing code. For example, he uses libc, pthreads, setjmp/longjmp, GTK+ for the UI and OpenGL. The system is MIT-licensed and inspite of lacking real optimizations already pretty fast: It runs roughly 832 million bytecodes/s and therefore is comparable to the performance of commercial Smalltalk systems.

Huemul looks very interesting and certainly is a thing that deserves more attention.

Are bytecodes an atavism?, Theo d’Hondt wondered when he noticed that people are fixated on virtual machines: They think it’s the only way to write a fast system.

However, interpreters are the simplest way to express the semantics of a language.

After a glimpse of the history of bytecode from BCPL over Pascal-P to Smalltalk, Self and Java, he presented Pico, a tiny language implemented using CPS that is smaller than Scheme that was roughly as fast as PLT Scheme (as of 2004)

Based on his experience with Pico, he decided to write PicoScheme, which is written in a subset of C and works by compiling s-expressions into an abstract syntax tree that then is interpreted. Soon he realized that “Scheme isn’t all that simple to implement”, but now he has a very promising implementation of the most parts of Scheme (nothing really relevant is missing) that is as fast as PLT by now. It also is very compact: the GC only has 150 LOC.

After the talk, he promised to open PicoScheme to the public in the future.

The last talk was called On Sustaining Self by Richard P. Gabriel. I only can recommend to take any chance to hear him speaking because he’s doing excellent presentations—I’m unable to do his audio-visual impressions justice, so please excuse the rough sketches: The talk started off with playing Hogni Lisberg’s cover of “All along the watchtower” while Noble and Biddle’s eternal words of their Manifesto in “Notes on Post-Modern Programming” appeared on the screen. “There must be some way out of here!”

The ultimate goal of all computer science is the program. The performance of programs was once the noblest function of computer science, and computer science was indispensable to great programs. Today, programming and computer science exist in complacent isolation, and can only be rescued by the conscious co-operation and collaboration of all programmers.

There are three ways to build self-sustaining systems according to Gabriel: First, Designed Perfection, which is what most people try to do today; it is very efficient, but entropy will get you. Second, Instinctual Adaption, which is resilient and flexible. Third, Learning, which is costly, but gives the best results in a highly dynamic environment.

A poem assembles on the screen; to brilliant guitar music there appears:


and it morphs into:


He proclaims:

Abstraction ignores the relevant,
therefore it requires ignorance!

He does a case-study on Levittown, which overcame the limitations and plannedness of itself and turned into a non-designed suburb.

He presented a case of artificial evolution showing the FPGA evolution experiment that generates very effective chip designs that nobody really understands how they work.

Finally he enters an dialogue with himself asking “How are cities designed?” At first look, they look modular, but they are so full of interconnected dependencies that they are impossible to modualize really. Also, the nature of the city is not planned.

In the end, the recognizance: Design is an illusion.

(I highly recommend you to watch the video when it is online.)

Summary: Attending the conference has been a great pleasure. Although being rather short and small (only one-and-a-half days and maybe 50 attendees), there were top-notch invited talks and many important people (and also many unknown, but friendly, clever and interesting ones!) around that everyone simply could talk to. The social events were well organized (free dinner, free beer) and actually allowed to socialize.

I seriously hope S3 can turn into a periodic conference because I’d really like to attend it again.

[Let’s also mention the not-that-good-stuff, just for the sake of completeness and so you see there’s not much to dislike: introductionary marketing speeches; the “Workshop” in the name misrepresents the conference; occasional WiFi failure.]

NP: Bob Dylan—I Believe In You

13may2008 · Off to Potsdam: Attending S3

Tomorrow I take the train to Potsdam to attend the Workshop on Self-sustaining Systems (S3), which means I get the chance of meeting rpg and many other people that worked on Lisp, Self and related cool stuff in real life.

The stuff I’m working on got not finished by far, but maybe I can write down enough on the train to explain it to interested parties.

If you want to hook up, don’t hesitate to contact me. I’m there until Saturday morning.

Anarchaia and chris blogs will resume publishing Sunday, May 18.

NP: Manu Chao—Politik Kills

Copyright © 2004–2016