chris blogs

It’s been over a year since the last installment of the series.

1. Sifting through others’ .zshrc, one occasionally finds aliases like:

   alias ls=' ls'
   alias cd=' cd'
   

   … and so on. The reason for this is simple, but not obvious: with setopt HIST_IGNORE_SPACE, these commands will be ignored, even if you don’t start these commands with a space.

2. We already talked about brace expansion like {1..5} which expands to 1 2 3 4 5. But did you know you also can do {1..10..2}, which expands to 1 3 5 7 9 and {005..14..2} which expands to 005 007 009 011 013? Oh, and {10..1} works as well as {10..1..2}. Now you can throw seq(1) away!

3. Using C-r and C-s to search history is well known, but the default search is a bit limited. Use these lines to enable search by globs, e.g. gcc*foo.c:

   bindkey "^R" history-incremental-pattern-search-backward
   bindkey "^S" history-incremental-pattern-search-forward
   

4. One nice trick if you often suspend vi by C-z:

   foreground-vi() {
     fg %vi
   }
   zle -N foreground-vi
   bindkey '^Z' foreground-vi
   

   This will make C-z on the command line resume vi again, so you can toggle between them easily. Even if you typed something already!

5. zsh has lots of documentation, but finding what you want to know can be difficult. The manpage zshall(1) contains everything, and this function will make it easy to search in:

   zman() {
     PAGER="less -g -s '+/^       "$1"'" man zshall
   }
   

   Try zman fc or zman HIST_IGNORE_SPACE! (Use n if the first match is not what you were looking for.)

6. Recently, I’ve become an avid user of the directory stack, but not really for its intended usage; instead, I use it together with the next trick. Here’s how you can persist the dirstack across sessions:

   DIRSTACKSIZE=9
   DIRSTACKFILE=~/.zdirs
   if [[ -f $DIRSTACKFILE ]] && [[ $#dirstack -eq 0 ]]; then
     dirstack=( ${(f)"$(< $DIRSTACKFILE)"} )
     [[ -d $dirstack[1] ]] && cd $dirstack[1] && cd $OLDPWD
   fi
   chpwd() {
     print -l $PWD ${(u)dirstack} >$DIRSTACKFILE
   }
   

   First, we limit the dirstack to nine entries, load them from .zdirs if possible, and then we save them again on every directory change.

   For a long time, I used to have something similar that only saved $OLDPWD, so I could open a new shell and cd - and be back where I last changed to. But now I use this, and AUTO_PUSHD, together with the next trick.

7. Every zsh user knows that you can use dirs to display the dirstack, and cd -N to go to the N-th element.

   But did you know zsh will show the dirstack on cd -TAB? It’s awesome, and does all the directory jumping I need.

   % cd -TAB
   1 -- /home/chris/mess/current
   2 -- /home/chris/mess/current/mdnsd
   3 -- /home/chris/mess/current/mdnsd/libutil
   4 -- /home/chris
   5 -- /home/chris/src/aewm-1.2.7/clients
   6 -- /home/chris/mess/2011/47/fspanel-0.7
   7 -- /home/chris/mess/2011/47
   8 -- /home/chris/src/mcwm
   

8. Which words end with ‘tent’? Of course you can do grep tent$ /usr/share/dict/words, but did you know you can do look _tent and press TAB (_ is where the cursor is)?

9. This fantastic zsh trick is from Julius Plenz: complete words from tmux pane.

10. Perhaps you know zmv already, but it can be a bit nasty. E.g. to rename all *.lis files to *.txt, the manual recommends:

    zmv '(*).lis' '$1.txt'
    

    However, with the awesome -W mode, you can write this instead:

    zmv -W '*.lis' '*.txt'
    

    If you are not sure what happens, use the dry-run mode first (-n).

    That concludes this, now hopefully yearly, installment. Perhaps you’ll find even more new more things in my recently cleaned up .zshrc. Enjoy your Z shell!

    NP: EMA—Milkman

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: The Indelicates—Gethesemane

Do you read the manpages of tools you use everyday? You should.

For example, just yesterday, I found this gem in the sudo manpage:

            ...
            for some reason, sudo is unable to update a file with its
            edited version, the user will receive a warning and the
            edited copy will remain in a temporary file.

-f          Force execution of the command, even if the user doesn't
            fulfill the sudoers policy.  This is useful for fixing
            up botched policy files (e.g. when visudo was not used).
            Note that the user still needs authenticate himself with
            a password or another authentication mechanism.

-g group    Normally, sudo runs a command with the primary group set to
            the one specified by the password database for the user the
            command is being run as (by default, root).  The -g (group)
            ...

sudo -f!? What the fuck?

I quickly checked the sources, and it turns out that this feature needs to be enabled during compliation with the --enable-force flag.

I also noticed a small glitch in the implementation: it is not possible to use sudo -f -i or sudo -f -s, but you can workaround that by using sudo -f su - and sudo -f su.

As far as I can tell, Arch and Debian don’t have this feature enabled, and neither does Gentoo, which not even provides a use-flag for it. It is, however, turned on by default on Ubuntu (after all, they make heavy use of sudo), RHEL, Fedora Core (since version 12), and openSUSE (and thus, probably, also in Canterbury).

NP: The Brian Jonestown Massacre—Their Satanic Majesties’ Second Request

It’s been almost three years since the last installment, so here is the next dollop of tips:

1. =(command) expands to a tempfile with the output of command that is deleted after the line has finished. In effect, the same as <(command) but allows applications to seek. E.g.:

   xpdf =(zcat foo.pdf.gz)
   

2. !-history-expansion is nice, but can be confusing if you have a command line with many ! that should be left alone. Either quote the ! with single quotes or write !" at the beginning of the line (yes, that " is left unclosed):

   % !" echo Hey there! Wow!!
   Hey there! Wow!!
   

3. An application of modifiers is !:t, which results into the basename of the last argument. Very useful when working with URLs, for example. You’ll never have to strip the path manually again:

   % wget ftp://ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p330.tar.gz
   % tar xzvf !:t
   

4. When playing with parameter expansion flags, it often is annoying having to use variables for immediate values:

   % foo=bar.c; echo ${foo:a:u}
   /HOME/CHRIS/BAR.C
   

   Instead of the ugly solution

   % echo ${$(echo bar.c):a:u}
   

   better use this:

   % echo ${${:-bar.c}:a:u}
   

   Here, ${:-bar.c} is an instance of the well-known ${FOO:-BAR} default substition operator.

5. To run a command several times, use repeat. Useful for benchmarks, e.g.:

   % repeat 3 time sleep 1
   sleep 1  0.00s user 0.00s system 0% cpu 1.002 total
   sleep 1  0.00s user 0.00s system 0% cpu 1.005 total
   sleep 1  0.00s user 0.00s system 0% cpu 1.002 total
   

6. Use glob modifiers to sort glob expansions. Helpful are: (om) (sort by modification time) or (n) (sort numerically):

   % pdfjoin chapter*.pdf(n) -o all.pdf
   

7. Another useful glob modifier is P, for example to prefix a flag:

   % tar czvf foo.tar.gz * *.tmp(P:--exclude:)
   

   (yes, tar can exclude patterns, but some other tools can’t, and zsh does patterns better anyway.)

8. Some ZLE hacks I use. To override default completion in various ways:

   # Force file name completion on C-x TAB, Shift-TAB.
   zle -C complete-files complete-word _generic
   zstyle ':completion:complete-files:*' completer _files
   bindkey "^X^I" complete-files
   bindkey "^[[Z" complete-files
   
   
   # Force menu on C-x RET.
   zle -C complete-first complete-word _generic
   zstyle ':completion:complete-first:*' menu yes
   bindkey "^X^M" complete-first
   

9. A function to make adding flags or prefixing arguments easier:

   # Move to where the arguments belong.
   after-first-word() {
     zle beginning-of-line
     zle forward-word
   }
   zle -N after-first-word
   bindkey "^X1" after-first-word
   

10. Complete with words in the history (like Emacs dabbrev):

    # Complete in history with M-/, M-,
    zstyle ':completion:history-words:*' list no 
    zstyle ':completion:history-words:*' menu yes
    zstyle ':completion:history-words:*' remove-all-dups yes
    bindkey "\e/" _history-complete-older
    bindkey "\e," _history-complete-newer
    

    Of course, all things are mentioned in the comprehensive manual, or the great User’s Guide to the Z-Shell which I wholeheartedly recommend. But one needs to find them. :)

    NP: Aimee Mann—Freeway

I lost my “administrative hymen” last month when a server of mine was rooted due to the recent Exim exploit (CVE-2010-4345). This is a post-mortem of the incident.

For shame, it took me almost a month to even detect the break-in, when I wondered why I didn’t get any mail from that address.

The same intruder cracked many machines and installed rootkits on them. He was not very professional: he left traces in the “panic log” of exim when he tried to download a configuration file with wget like this:

wget ... >exim.conf

instead of

wget -O exim.conf ...

… which of course writes the wget status messages into the file, too, which results in fatal parse errors:

# cat /var/log/exim4/paniclog
2010-12-16 01:09:54 string too large in smtp_notquit_exit()
2010-12-16 02:27:17 string too large in smtp_notquit_exit()
2010-12-16 05:12:11 string too large in smtp_notquit_exit()
2010-12-16 14:49:27 string too large in smtp_notquit_exit()
2010-12-16 18:25:20 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:25:20 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:45:03 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:54:13 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf
2010-12-16 18:55:02 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf
2010-12-16 19:55:02 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf

The installation of the rootkit also changed timestamps:

He added a key to root’s .ssh/authorized_keys and started another sshd on a non-standard port (59997). He did not detect that this port was blocked out-bound with iptables.

The exploit circulated around December 7, the patch was released December 10, and I got hacked December 16. And I actually read the news but thought I was not affected. Sigh.

Clearly, this should not happen. I admit the following mistakes:

• The system ran a old (2007), un-updated version of GRML that I installed in a hurry with grml2hd without spending too much time thinking about the details.

• It was not updated often because things used to break, being based in Debian unstable, and since it had many packages installed, updating often was a hassle.

• It came with loads of stuff that I didn’t need, but they cluttered up logwatch and other intrusion detection tools.

• As a result of this, I didn’t forward Cron messages to a mailbox I actually read.

• It came with Exim preinstalled, so I used that for the single mailbox I had on that machine.

  (Not that this would have saved much; the bug was also in the last version before the security update, and since the intruder (accidentally) shut down the mail system, I would not have gotten any messages about the break-in.)

  Lessons learned for the future:

• Set up minimal, but meaningful logging, so it will be read.

• Send administrative messages to external hosts.

• Update the system regularily. I now installed Debian squeeze, which is going to be stable “really soon now”, and then will serve its purpose for the next few years with comparatively few updates.

• Do a minimal install. This time I used grml-debootstrap, which only installed a core Debian system, on which I added exactly the packages I need.

• Use Postfix instead of Exim. Postfix not only is vastly easier to set up than exim (at least on Debian, Postfix default configuration fits on a single screen, while Exim has a configuration directory filled with almost 2000 lines of text), but also has a better security record. And I use Postfix on other servers too, so I will be more alert if I read about it being unsecure.

  Considerations for the future that I probably won’t follow:

• Make lots of partitions and sprinkle noexec, nosuid mount options. (Complicates setup and disk space planning.)

• Setup remote syslog. (I don’t have the resources for this, not worth it for a single machine.)

• Switch to a distro that doesn’t package antique stuff. (Which?)

• Use SELinux or such stuff.

  NP: Leonard Cohen—Avalanche

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: Trembling Bells—Seven Years A Teardrop

The 12 Days of Unix…

On the twelfth day I left it, my Unix gave to me:
    Twelve boards a-blowing;
    Eleven chips a-smoking;
    Ten ports a-jamming;
    Nine floppies frying;
    Eight gettys dying;
    Seven blown partitions;
    Six bad controllers;
    Five core dumps;
    Four bad blocks;
    Three heads crashed;
    Two faulty tapes;
    And a burnt-out V.D.T.

… and many other holiday textfiles.

NP: Jeff Buckley—Lilac Wine

Der Tag, vor dem der große Christ
zur Welt geboren worden ist,
war hart und wüst und ohne Vernunft.
Seine Eltern, ohne Unterkunft,
fürchteten sich vor seiner Geburt,
die gegen Abend erwartet wurd,
denn seine Geburt fiel in die kalte Zeit.
Aber sie verlief zur Zufriedenheit.
Der Stall, den sie doch noch gefunden hatten,
war warm und mit Moos zwischen seinen Latten,
und mit Kreide war auf die Tür gemalt,
dass der Stall bewohnt war und bezahlt.
So wurde es doch noch eine gute Nacht,
auch das Heu war wärmer, als sie gedacht.
Ochs und Esel waren dabei,
damit alles in der Ordnung sei.
Eine Krippe gab einen kleinen Tisch,
und der Hausknecht brachte heimlich einen Fisch.
(Denn es musste bei der Geburt des großen Christ
alles heimlich gehen und mit List.)
Doch der Fisch war ausgezeichnet und reichte durchaus
und Maria lachte ihren Mann wegen seiner Besorgnis aus
denn am Abend legte sich sogar der Wind,
und war nicht mehr so kalt, wie die Winde sonst sind.
Aber bei Nacht war es fast wie ein Föhn,
Und der Stall war warm und das Kind war sehr schön.
Und es fehlte schon fast gar nichts mehr,
da kamen auch schon die Dreikönig daher!
Maria und Joseph waren zufrieden sehr.
Sie legten sich sehr zufrieden zum Ruhn.
Mehr konnte die Welt für den Christ nicht tun.

— Bertolt Brecht, Die gute Nacht

NP: Jimi Hendrix—Testify

Wenn der erste Advent so früh liegt wie dieses Jahr, ist es natürlich besonders schwer, schon heute die diesjährigen Adventskalender zu verlinken.

Allerdings rechne ich fest mit:

• dem Mathekalender der DFG,
• dem Perl Advent Calendar,

• und 24ways.

  Mehr eventuell zum 2. Advent…

  Auch Glühwein gabs heuer schon, z.B. gestern beim Shutdown von AllColoursAreBeautiful.

  NP: The Brian Jonestown Massacre—Hyperventilation

I have been using these two almost trivial scripts for a few months now and they have become rather essential for my workflow.

clip2firefox:

#!/bin/sh
# clip2firefox - open selected text as firefox address
firefox "$(xclip -out)"

clip2goog:

#!/bin/zsh
# clip2goog - google for selected text

selurl() {
  setopt localoptions extendedglob
  sel=$(xclip -out)
  input=( ${(s::)sel} )
  print ${(j::)input/(#b)([^A-Za-z0-9_.!~*\'()-])/%$(([##16]#match))}
}

firefox "http://google.com/search?q=$(selurl)"

I give them shortcuts with xbindkeys:

"clip2goog"
  Mod4 + g

"clip2firefox"
  Mod4 + o

… and instantly I can open any URL or google for every selection I made.

When I first read about these scripts, I thought they weren’t a big deal, but they are insanely useful, believe me.

NP: The Brian Jonestown Massacre—The Be Song