chris blogs

12mar2013 · 10 fresh zsh tricks you may not know...

Time for a new instance of your favorite installment of this blog!

  1. You probably know M-. to insert the last argument of the previous line. Sometimes, you want to insert a different argument. There are a few options: Use history expansion, e.g. !:-2 for the third word on the line before (use TAB to expand it if you are not sure), or use M-. with a prefix argument: M-2 M-.

    Much nicer however is:

    autoload -Uz copy-earlier-word
    zle -N copy-earlier-word
    bindkey "^[m" copy-earlier-word
    

    Then, M-m will copy the last word of the current line, then the second last word, etc. But with M-. you can go back in lines too! Thus:

    % echo a b c
    % echo 1 2 3
    % echo <M-.><M-.><M-m>
    % echo b
    

    Man, I wish I knew that earlier!

  2. Sometimes, you want to combine a few previously typed lines into one:

    % vi foo.c
    % gcc -o foo foo.c
    % gdb --args foo -v
    

    Repeating these three commands all the time gets annoying (even if you know about C-o, it’s messy if you run other commands in between).

    So, lets combine them into one line. Are you moving your hand to your mouse for copy&pasting? Bzzzt!

    We can use history expansion as well:

    % !-3<TAB>; !-2<TAB>; !-1<TAB>
    % vi foo.c; gcc -o foo foo.c; gdb --args foo -v
    

    If you don’t see the numbers easily, you could do !vi<TAB> and so on. I always wanted to use C-r for this, however, thus I defined:

    autoload -Uz narrow-to-region
    function _history-incremental-preserving-pattern-search-backward
    {
      local state
      MARK=CURSOR  # magick, else multiple ^R don't work
      narrow-to-region -p "$LBUFFER${BUFFER:+>>}" -P "${BUFFER:+<<}$RBUFFER" -S state
      zle end-of-history
      zle history-incremental-pattern-search-backward
      narrow-to-region -R state
    }
    zle -N _history-incremental-preserving-pattern-search-backward
    bindkey "^R" _history-incremental-preserving-pattern-search-backward
    bindkey -M isearch "^R" history-incremental-pattern-search-backward
    bindkey "^S" history-incremental-pattern-search-forward
    

    Now C-r will work in a recursive way. It looks like this:

    % <C-r>vi<RET>;
    % vi foo.c; <C-r>gcc
    % vi foo.c; >>gcc -o foo foo.c<< <RET>;<C-r>gdb
    % vi foo.c; gcc -o foo foo.c; >>gdb --args foo -v<< <RET>
    % vi foo.c; gcc -o foo foo.c; gdb --args foo -v
    

    Since C-r is not very useful if you already typed something, I feel this redefinition is quite neat.

  3. I have defined $WORDCHARS to exclude / since I usually don’t want C-w or M-DEL to remove whole paths. But sometimes I do, thus:

    function _backward_kill_default_word() {
      WORDCHARS='*?_-.[]~=/&;!#$%^(){}<>' zle backward-kill-word
    }
    zle -N backward-kill-default-word _backward_kill_default_word
    bindkey '\e=' backward-kill-default-word   # = is next to backspace
    
  4. You probably heard of prompts like : juno ~/src/zsh ; (or just ; if you are hardcore-minimalist) which have the benefit that you can copy the whole line in your terminal emulator and just paste it to run it again:

    : juno ~/src/zsh ; grep -r bindkey .
    : juno ~/src/zsh ; : juno ~/src/zsh ; grep -r bindkey .
    

    The : will nicely gobble up (almost) everything until the ;. But the paste keeps repeating the prompt, which is ugly, and I don’t like to have the : and ; in my prompt really. That’s why I had the IMHO ingenious idea to set my prompt to:

    juno src/zsh% 
    

    … which actually is:

    juno src/zsh%␣
    

    That is, the last character is a Unicode non-breaking space (U+00A0). Which will look like a plain space and behave like one. Except that we can bindkey it to clear the input buffer:

    nbsp=$'\u00A0'
    bindkey -s $nbsp '^u'
    

    Now, pasting the prompt will make it remove itself. Great!

  5. Growing up with bash, I never was fond of zsh’s menu-select widget. I want TAB to complete as much as possible, and if I ever press TAB again, it should display the completions and just let me type on.

    However, sometimes, I’d like to use the menu-select widget, e.g. if the files have prefixes that make selection hard (Maildirs, anyone?) or consist of weird special chars only.

    It took me quite long to figure out how to enable menu-select for certain widgets only. The main problem was that the option NO_ALWAYS_LAST_PROMPT disables the menu widget. Thus, we have to unset it locally:

    zle -C complete-menu menu-select _generic
    _complete_menu() {
      setopt localoptions alwayslastprompt
      zle complete-menu
    }
    zle -N _complete_menu
    bindkey '^F' _complete_menu
    bindkey -M menuselect '^F' accept-and-infer-next-history
    bindkey -M menuselect '/'  accept-and-infer-next-history
    bindkey -M menuselect '^?' undo
    bindkey -M menuselect ' ' accept-and-hold
    bindkey -M menuselect '*' history-incremental-search-forward
    

    The latter keybindings make it a convenient file selector and browser (using / and DEL).

  6. One thing that zsh lacks is the ability to start an interactive shell session from inside a shell script (e.g. rc(1) can do that). Sometimes you want to spawn a shell that runs a command, but is interactive after the program finished (e.g. when launched from urxvt or tmux).

    Luckily, I found zshi. I have defined it as a script to allow execution from everywhere. It needs some support from the .zshrc:

    if [[ $1 == eval ]]; then
      shift
      ICMD="$@"
      set --
      zle-line-init() {
        BUFFER="$ICMD"
        zle accept-line
        zle -D zle-line-init
      }
      zle -N zle-line-init
    fi
    

    That solution is a bit more complex, but it allows you to press Up (or run r) to execute the command again.

  7. If you are watching series, you want to get to the next episode:

    % mplayer foobar-S01-E23.mkv
    % <Up><C-x a>
    % mplayer foobar-S01-E24.mkv
    

    zsh includes incarg, but it only works if you put the cursor on the number. This solution increments the last number, anywhere, and knows about zero padding:

    _increase_number() {
      local -a match mbegin mend
      [[ $LBUFFER =~ '([0-9]+)[^0-9]*$' ]] &&
        LBUFFER[mbegin,mend]=$(printf %0${#match[1]}d $((10#$match+${NUMERIC:-1})))
    }
    zle -N increase-number _increase_number
    bindkey '^Xa' increase-number
    bindkey -s '^Xx' '^[-^Xa'
    
  8. I use Emacs keybindings, but sometimes I wish I had vi’s command mode. Luckily, it’s just a C-x C-v away in the default configuration! Heck, you may even go ahead and do:

    bindkey '^[' vi-cmd-mode
    

    … and i will put you back into Emacs mode again.

  9. A great anti-feature of history expansion is when it fails:

    % a carefully constructed command line !?gcc !?vim !?quux
    zsh: no such event: gcc !<Up>
    % a carefully constructed command line im !?quux
    

    And your history expanders are gone. Not so with this snippet:

    function _recover_line_or_else() {
      if [[ -z $BUFFER && $CONTEXT = start && $zsh_eval_context = shfunc
            && -n $ZLE_LINE_ABORTED
            && $ZLE_LINE_ABORTED != $history[$((HISTCMD-1))] ]]; then
        LBUFFER+=$ZLE_LINE_ABORTED
        unset ZLE_LINE_ABORTED
      else
        zle .$WIDGET
      fi
    }
    zle -N up-line-or-history _recover_line_or_else
    function _zle_line_finish() {
      ZLE_LINE_ABORTED=$BUFFER
    }
    zle -N zle-line-finish _zle_line_finish
    

    This will keep the last line in all cases, allowing you to fix it:

    % a carefully constructed command line !?gcc !?vim !?quux
    zsh: no such event: gcc !<Up>
    % a carefully constructed command line !?gcc !?vim !?quux
    
  10. Renaming long file names sucks. Many use graphical file managers for it. I use imv (interactive mv):

    imv() {
      local src dst
      for src; do
        [[ -e $src ]] || { print -u2 "$src does not exist"; continue }
        dst=$src
        vared dst
        [[ $src != $dst ]] && mkdir -p $dst:h && mv -n $src $dst
      done
    }
    

    It will even create the target directory if it doesn’t exist.

  11. Bonus item: This is more for fun than serious use. An updating clock in your prompt:

    _prompt_and_resched() { sched +1 _prompt_and_resched; zle && zle reset-prompt }
    _prompt_and_resched
    PS1="%D{%H:%M:%S} $PS1"
    

    As usual, these things and many others are integrated in my .zshrc. Enjoy your Z shell!

    NP: Silly—EKG

06jan2013 · A grab bag of Git tricks

Since its release I’ve been a fan of Git. (I still can remember downloading the initial version.) The thing I like most is that it can be extended and customized in an unixy way. Over time, I have collected some scripts and tricks that I would like to present to a wider audience. Git information online abounds (I especially recommend Mark J. Dominus in-depth posts on Git), thus I will only show stuff I haven’t seen elsewhere.

git news

Let’s start with a simple alias which you can simply add to your .gitconfig:

[alias]
        news = log -p HEAD@{1}..HEAD@{0}

I am tracking quite a lot of open source projects by cloning them into ~/src and running git pull on them occasionally. Next, I run git news and see only the commits (with diff) that have arrived since the last pull.

Of course it is a very simplistic alias and it probably won’t do what you want if you actually change the HEAD yourself—e.g. by committing. (A more robust version could, for example, parse the output of git reflog and search for the last pull.) On the other hand, as it is, it also can be useful for showing what came in with a merge. I also use it for repositories where I git cvsimport into, with the same benefits.

git comma

Admittedly, I’m a fan of dirty working trees, which is why—when I don’t use magit or finely-grained git add -p/git commit -p already—I commit whole files at once like git commit foo.c bar.c.

One thing that has always annoyed me is that I cannot git commit files unknown to Git, enforcing an explicit git add step only for these new files! One day I took the plunge and wrote git-comma (a portmanteau of commit and add) which gives its best to behave exactly like git commit except for adding the yet-unknown files beforehand. This was a bit more tricky than I expected because I wanted it to work correctly even in the face of partially staged files, thus a stupid git add on all arguments would not work (also, you only want to add explicitly named files, not whole directories and so on). Finally, git comma tries to clean up properly if you decide to abort the commit, unstaging the files again.

(IMO, this should be a flag or configuration option for git commit.)

git attic

A newer script, but a very useful one, is git attic, whose namesake perhaps gives you a shiver down the spine, being reminded of this CVS quirk.

Yet, CVS’ manner with deleted files—moving them into a folder called Attic—had one benefit which cannot be denied: it was easy to see what had been removed and to access the contents again.

Of course, Git has no problem with file removal, but having a look at the old contents can be laborious.

Thus I wrote git-attic, which presents you a nice list of files together with their deletion date:

% git attic
2012-08-14 441e782^:Etc/ChangeLog-5.0
2012-05-31 0793393^:Completion/Unix/Command/_systemctl
2012-01-31 6a364de^:Test/Y04compgen.ztst
2012-01-31 6a364de^:Test/compgentest
2011-08-18 f0eaa57^:Completion/Zsh/Command/_schedtool
...

The output is designed to be copy’n’pasted: Pass the second field to git show to display the file contents, or just select the hash without ^ to see the commit where removal happened.

(By default, I don’t detect renames, since I want to see which paths don’t exist anymore. If you are looking for “lost” content, feel free to pass -M to the script to detect renames and only show truly deleted files.)

A minimalist, yet powerful zsh prompt

As an avid zsh user for years, I have been using a simple but powerful shell prompt which looks like hecate src/zsh% for years (since 2010-02-11 actually, thanks to homegit, see below.) and ridiculed experiments to make the zsh prompt a kitchen sink. However, my Git usage grew and I started occasionally mixing up branches.

Thus I decided to grin and bear it and wondered how to make a minimalist nevertheless useful Git-enhanced prompt. One feature of my prompt was that it only shows the last few segments of the current working directory (usually 2, which is enough for me unless I need to work in some javaesque file labyrinth). One day I decided to integrate the current Git branch into these path segments. Now, my prompt looks like this:

hecate src/zsh@master% cd Doc

… and it actually sticks to the repository root:

hecate zsh@master/Doc% cd Zsh

When the level gets too deep, the branch and repository moves to the front:

hecate zsh@master Doc/Zsh%

The depth is still configurable:

hecate zsh@master Doc/Zsh% NDIRS=4
hecate src/zsh@master/Doc/Zsh%

I’ve quite come to like this presentation. Additionally, it also works with detached heads (useful when rebasing):

hecate src/zsh@master/Doc/Zsh% git checkout HEAD~42
...
hecate src/zsh@master~42/Doc/Zsh%

For free, you get some feedback when bisecting:

hecate ~/src/zsh@master% git bisect bad
hecate ~/src/zsh@bisect/bad% git bisect good HEAD~42
hecate ~/src/zsh@bisect/bad~21% git bisect good
hecate ~/src/zsh@bisect/bad~5% git bisect reset
hecate ~/src/zsh@master%

This is the code in all its glory:

# gitpwd - print %~, limited to $NDIR segments, with inline git branch
NDIRS=2
gitpwd() {
  local -a segs splitprefix; local prefix gitbranch
  segs=("${(Oas:/:)${(D)PWD}}")

  if gitprefix=$(git rev-parse --show-prefix 2>/dev/null); then
    splitprefix=("${(s:/:)gitprefix}")
    branch=$(git name-rev --name-only HEAD 2>/dev/null)
    if (( $#splitprefix > NDIRS )); then
      print -n "${segs[$#splitprefix]}@$branch "
    else
      segs[$#splitprefix]+=@$branch
    fi
  fi

  print "${(j:/:)${(@Oa)segs[1,NDIRS]}}"
}

Perhaps it turned out to be a bit more challenging than expected. ;) Integration into the prompt is trivial, however:

function cnprompt6 {
  case "$TERM" in
    xterm*|rxvt*)
      precmd() {  print -Pn "\e]0;%m: %~\a" }
      preexec() { printf "\e]0;$HOST: %s\a" $1 };;
  esac
  setopt PROMPT_SUBST
  PS1='%B%m%(?.. %??)%(1j. %j&.)%b $(gitpwd)%B%(!.%F{red}.%F{yellow})%#${SSH_CLIENT:+%#} %b'
  RPROMPT=''
}

cnprompt6

homegit

For the last five years I have used Git to manage my dotfiles and I use the repository on a plethora of machines.

I found the following zsh alias to be the simplest and best method to use Git for this purpose:

alias homegit="GIT_DIR=~/prj/dotfiles/.git GIT_WORK_TREE=~ git"

Why not a function? Because an alias will make zsh autocomplete homegit just like it completes git already, without any additional work.

Why not a ~/.git? I decided against it because I didn’t want to accidentally commit stuff from any subdirectory and feared a git clean could wipe my sweet home directory.

The homegit approach works very well for me and I have not felt a need for more complex solutions which symlink dotfiles or copy them around.

Note that the git-* scripts presented here can be called transparently from homegit as well, e.g. with homegit attic. And since $GIT_DIR is set in the environment, the scripts can just call git and will just work correctly!

411 commits as of now tell me I perhaps should scale back customizing stuff all the time, but it can be very helpful indeed to see how things changed over time. Also, tracking changes other programs make to your files (and being able to revert them) is totally worth it.

git trail

One of the newest additions to my Git zoo is git trail, a tool I wanted for years, really. With many branches, it’s easy to get confused about what branched off where and what actually is part of this topic branch and whether this topic branch has been merged but then forgotten or…

Perhaps you feel my pain. Perhaps you tried git show-branch once to get an overview of such a mess, but I feel it’s easier to see stereographic projections of a T-Rex in its output than the state of your branches.

Thus I wrote git-trail, which shows how to reach commits in the current branch from other branches. Since we don’t have enough local branches to make it interesting, lets show remote branches too (-r):

hecate tmp/rack@master% git trail -r
2013-01-04 7e1f081 master
2013-01-04 7e1f081 remotes/origin/HEAD
2013-01-04 1e75faa remotes/origin/hijack~2
2013-01-04 1e75faa remotes/origin/master~1
2012-11-03 1824547 remotes/origin/unstandard_uri_escape~1
2012-03-18 7d7977f remotes/origin/rack-1.4~77
2011-05-22 a50dda5 remotes/origin/rack-1.3~99
2010-06-15 dc6b54e remotes/origin/rack-1.2~38
2010-01-03 e6ebd83 remotes/origin/rack-1.1~23
2009-04-25 d221938 remotes/origin/rack-1.0~24
2009-01-05 7fed4c7 remotes/origin/rack-0.9~15
2008-08-09 e9f9f27 remotes/origin/rack-0.4~6

What you see is the first common commit between every branch and the current branch, together with the commit date. If the branch is listed without suffixes, it is completely included. Else, you effectively see how the branch diverges. For example, in rack-1.4, there have been 77 patches since branching from master. The feature branch hijack consists of two commits. Lets look at the view from that feature branch:

hecate tmp/rack@master% git trail -r origin/hijack
2013-01-04 8a311fb remotes/origin/hijack
2013-01-04 1e75faa master~1
2013-01-04 1e75faa remotes/origin/HEAD~1
2012-11-03 1824547 remotes/origin/unstandard_uri_escape~1
2012-03-18 7d7977f remotes/origin/rack-1.4~77
2011-05-22 a50dda5 remotes/origin/rack-1.3~99
2010-06-15 dc6b54e remotes/origin/rack-1.2~38
2010-01-03 e6ebd83 remotes/origin/rack-1.1~23
2009-04-25 d221938 remotes/origin/rack-1.0~24
2009-01-05 7fed4c7 remotes/origin/rack-0.9~15
2008-08-09 e9f9f27 remotes/origin/rack-0.4~6

We see that there have been commits to master since hijack was branched, and we should perhaps rebase hijack if we wanted to submit it.

Let’s say we simply merged it into master:

hecate tmp/rack@master% git merge origin/hijack
...
hecate tmp/rack@master% git trail -r
2013-01-06 68de794 master
2013-01-04 8a311fb remotes/origin/hijack
2013-01-04 7e1f081 remotes/origin/HEAD
2012-11-03 1824547 remotes/origin/unstandard_uri_escape~1
2012-03-18 7d7977f remotes/origin/rack-1.4~77
...

Now hijack appears undecorated: it is completely contained in the current branch history.

Let’s say we work on the other feature branch next, unstandard_uri_escape:

hecate tmp/rack@master% git checkout unstandard_uri_escape
hecate tmp/rack@unstandard_uri_escape% git trail
2012-11-03 decaa23 unstandard_uri_escape
2012-11-03 1824547 master~10^2~1

We can now rebase it to make it a proper child of master:

hecate tmp/rack@unstandard_uri_escape% git rebase master
hecate tmp/rack@unstandard_uri_escape% git trail
2013-01-06 92b40fa unstandard_uri_escape
2013-01-06 c30da33 master

And then master can be fast-forwarded:

hecate tmp/rack@unstandard_uri_escape% git checkout master
hecate tmp/rack@master% git trail
2013-01-06 c30da33 master
2013-01-06 c30da33 unstandard_uri_escape~1
hecate tmp/rack@master% git merge unstandard_uri_escape 
Updating c30da33..92b40fa
Fast-forward
...
hecate tmp/rack@master% git trail
2013-01-06 92b40fa master
2013-01-06 92b40fa unstandard_uri_escape

I hope this exposed how git trail helps me to keep track of dealing with branches.

git neck

The perfect match for git-trail is git-neck, which show commits from the HEAD until the first branching point… that should explain the name.

So, what is the “neck” of our master branch as above?

hecate tmp/rack@master% git neck -r
92b40fa Add a decoder that supports ECMA unicode uris
c30da33 Merge remote-tracking branch 'origin/hijack'
7e1f081 Merge pull request #480 from udzura/master
3edd1e8 Add a rackup option for one-liner rack app server
6d41179 Extract Builder.new_from_string from Builder.parse_file

Likewise, let’s have a look at that remote feature branch sticking around:

% git neck -r origin/unstandard_uri_escape
decaa23 Add a decoder that supports ECMA unicode uris

It was just a single commit. We can also look at the neck of an old release branch:

hecate tmp/rack@master% git neck -r origin/rack-0.4
92f79ea Make Rack::Lint::InputWrapper delegate size method to underlying IO object.
e33cc65 Update to version 0.4
ab9a95e Fix packaging script
1ccdf73 Update README
1b56583 Document REQUEST_METHOD future changes
f0977a8 Disarm and document Content-Length checking in Rack::Lint for 0.4

And we see the 6 commits that are only in rack-0.4.

If you remember the situation before merging the feature branches:

hecate tmp/r2@master% git trail -r
2013-01-04 7e1f081 master
2013-01-04 7e1f081 remotes/origin/HEAD
2013-01-04 1e75faa remotes/origin/hijack~2
2013-01-04 1e75faa remotes/origin/master~1
2012-11-03 1824547 remotes/origin/unstandard_uri_escape~1

Here, the neck is the part until master forked off:

hecate tmp/r2@master% git neck -r
7e1f081 Merge pull request #480 from udzura/master
3edd1e8 Add a rackup option for one-liner rack app server
6d41179 Extract Builder.new_from_string from Builder.parse_file

git neck is most useful if you are working in a feature branch which no other branch forks off, because then the neck goes until where you forked it.

Using git diff without Git

At last, another small trick: git diff works between any two files (or directories), even if you don’t use Git at all to track them. But you gain some advantages over regular diff, like --word-diff, --color or --stat without having additional tools beyond Git installed.

Also, you can use git diff --binary to generate efficient binary deltas which you can apply again provided you have the unpatched file. (Possibly you need to edit the patch to make both filenames the same, so git apply finds everything.)

NP: Sophie Hunger—What it is

24dec2012 · Merry Christmas!

Just Believe

(comic by Jim Benton)

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: Toxoplasma—Gut und Böse

16aug2012 · 25, 031, 0x19

Jeden Abend werfe ich
eine Zukunft hinter mich,
die sich niemals mehr erhebt
denn sie hat im Geist gelebt.
Neue Bilder werden, wachsen;
Welten drehn um neue Achsen,
werden, sterben, lieben, schaffen.
Die Vergangenheiten klaffen.
Tobend, wirbelnd stürzt die Zeit
in die Gruft. — Das Leben schreit!

— Erich Mühsam

NP: Walter Mossmann—Lied für meine radikalen Freunde

17feb2012 · 10 new zsh tricks you may not know...

It’s been over a year since the last installment of the series.

  1. Sifting through others’ .zshrc, one occasionally finds aliases like:

    alias ls=' ls'
    alias cd=' cd'
    

    … and so on. The reason for this is simple, but not obvious: with setopt HIST_IGNORE_SPACE, these commands will be ignored, even if you don’t start these commands with a space.

  2. We already talked about brace expansion like {1..5} which expands to 1 2 3 4 5. But did you know you also can do {1..10..2}, which expands to 1 3 5 7 9 and {005..14..2} which expands to 005 007 009 011 013? Oh, and {10..1} works as well as {10..1..2}. Now you can throw seq(1) away!

  3. Using C-r and C-s to search history is well known, but the default search is a bit limited. Use these lines to enable search by globs, e.g. gcc*foo.c:

    bindkey "^R" history-incremental-pattern-search-backward
    bindkey "^S" history-incremental-pattern-search-forward
    
  4. One nice trick if you often suspend vi by C-z:

    foreground-vi() {
      fg %vi
    }
    zle -N foreground-vi
    bindkey '^Z' foreground-vi
    

    This will make C-z on the command line resume vi again, so you can toggle between them easily. Even if you typed something already!

  5. zsh has lots of documentation, but finding what you want to know can be difficult. The manpage zshall(1) contains everything, and this function will make it easy to search in:

    zman() {
      PAGER="less -g -s '+/^       "$1"'" man zshall
    }
    

    Try zman fc or zman HIST_IGNORE_SPACE! (Use n if the first match is not what you were looking for.)

  6. Recently, I’ve become an avid user of the directory stack, but not really for its intended usage; instead, I use it together with the next trick. Here’s how you can persist the dirstack across sessions:

    DIRSTACKSIZE=9
    DIRSTACKFILE=~/.zdirs
    if [[ -f $DIRSTACKFILE ]] && [[ $#dirstack -eq 0 ]]; then
      dirstack=( ${(f)"$(< $DIRSTACKFILE)"} )
      [[ -d $dirstack[1] ]] && cd $dirstack[1] && cd $OLDPWD
    fi
    chpwd() {
      print -l $PWD ${(u)dirstack} >$DIRSTACKFILE
    }
    

    First, we limit the dirstack to nine entries, load them from .zdirs if possible, and then we save them again on every directory change.

    For a long time, I used to have something similar that only saved $OLDPWD, so I could open a new shell and cd - and be back where I last changed to. But now I use this, and AUTO_PUSHD, together with the next trick.

  7. Every zsh user knows that you can use dirs to display the dirstack, and cd -N to go to the N-th element.

    But did you know zsh will show the dirstack on cd -TAB? It’s awesome, and does all the directory jumping I need.

    % cd -TAB
    1 -- /home/chris/mess/current
    2 -- /home/chris/mess/current/mdnsd
    3 -- /home/chris/mess/current/mdnsd/libutil
    4 -- /home/chris
    5 -- /home/chris/src/aewm-1.2.7/clients
    6 -- /home/chris/mess/2011/47/fspanel-0.7
    7 -- /home/chris/mess/2011/47
    8 -- /home/chris/src/mcwm
    
  8. Which words end with ‘tent’? Of course you can do grep tent$ /usr/share/dict/words, but did you know you can do look _tent and press TAB (_ is where the cursor is)?

  9. This fantastic zsh trick is from Julius Plenz: complete words from tmux pane.

  10. Perhaps you know zmv already, but it can be a bit nasty. E.g. to rename all *.lis files to *.txt, the manual recommends:

    zmv '(*).lis' '$1.txt'
    

    However, with the awesome -W mode, you can write this instead:

    zmv -W '*.lis' '*.txt'
    

    If you are not sure what happens, use the dry-run mode first (-n).

    That concludes this, now hopefully yearly, installment. Perhaps you’ll find even more new more things in my recently cleaned up .zshrc. Enjoy your Z shell!

    NP: EMA—Milkman

24dec2011 · Merry Christmas!

Occupy North Pole

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: The Indelicates—Gethesemane

01apr2011 · sudo -f

Do you read the manpages of tools you use everyday? You should.

For example, just yesterday, I found this gem in the sudo manpage:

            ...
            for some reason, sudo is unable to update a file with its
            edited version, the user will receive a warning and the
            edited copy will remain in a temporary file.

-f          Force execution of the command, even if the user doesn't
            fulfill the sudoers policy.  This is useful for fixing
            up botched policy files (e.g. when visudo was not used).
            Note that the user still needs authenticate himself with
            a password or another authentication mechanism.

-g group    Normally, sudo runs a command with the primary group set to
            the one specified by the password database for the user the
            command is being run as (by default, root).  The -g (group)
            ...

sudo -f!? What the fuck?

I quickly checked the sources, and it turns out that this feature needs to be enabled during compliation with the --enable-force flag.

I also noticed a small glitch in the implementation: it is not possible to use sudo -f -i or sudo -f -s, but you can workaround that by using sudo -f su - and sudo -f su.

As far as I can tell, Arch and Debian don’t have this feature enabled, and neither does Gentoo, which not even provides a use-flag for it. It is, however, turned on by default on Ubuntu (after all, they make heavy use of sudo), RHEL, Fedora Core (since version 12), and openSUSE (and thus, probably, also in Canterbury).

NP: The Brian Jonestown Massacre—Their Satanic Majesties’ Second Request

14feb2011 · 10 more zsh tricks you may not know...

It’s been almost three years since the last installment, so here is the next dollop of tips:

  1. =(command) expands to a tempfile with the output of command that is deleted after the line has finished. In effect, the same as <(command) but allows applications to seek. E.g.:

    xpdf =(zcat foo.pdf.gz)
    
  2. !-history-expansion is nice, but can be confusing if you have a command line with many ! that should be left alone. Either quote the ! with single quotes or write !" at the beginning of the line (yes, that " is left unclosed):

    % !" echo Hey there! Wow!!
    Hey there! Wow!!
    
  3. An application of modifiers is !:t, which results into the basename of the last argument. Very useful when working with URLs, for example. You’ll never have to strip the path manually again:

    % wget ftp://ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p330.tar.gz
    % tar xzvf !:t
    
  4. When playing with parameter expansion flags, it often is annoying having to use variables for immediate values:

    % foo=bar.c; echo ${foo:a:u}
    /HOME/CHRIS/BAR.C
    

    Instead of the ugly solution

    % echo ${$(echo bar.c):a:u}
    

    better use this:

    % echo ${${:-bar.c}:a:u}
    

    Here, ${:-bar.c} is an instance of the well-known ${FOO:-BAR} default substition operator.

  5. To run a command several times, use repeat. Useful for benchmarks, e.g.:

    % repeat 3 time sleep 1
    sleep 1  0.00s user 0.00s system 0% cpu 1.002 total
    sleep 1  0.00s user 0.00s system 0% cpu 1.005 total
    sleep 1  0.00s user 0.00s system 0% cpu 1.002 total
    
  6. Use glob modifiers to sort glob expansions. Helpful are: (om) (sort by modification time) or (n) (sort numerically):

    % pdfjoin chapter*.pdf(n) -o all.pdf
    
  7. Another useful glob modifier is P, for example to prefix a flag:

    % tar czvf foo.tar.gz * *.tmp(P:--exclude:)
    

    (yes, tar can exclude patterns, but some other tools can’t, and zsh does patterns better anyway.)

  8. Some ZLE hacks I use. To override default completion in various ways:

    # Force file name completion on C-x TAB, Shift-TAB.
    zle -C complete-files complete-word _generic
    zstyle ':completion:complete-files:*' completer _files
    bindkey "^X^I" complete-files
    bindkey "^[[Z" complete-files
    
    
    # Force menu on C-x RET.
    zle -C complete-first complete-word _generic
    zstyle ':completion:complete-first:*' menu yes
    bindkey "^X^M" complete-first
    
  9. A function to make adding flags or prefixing arguments easier:

    # Move to where the arguments belong.
    after-first-word() {
      zle beginning-of-line
      zle forward-word
    }
    zle -N after-first-word
    bindkey "^X1" after-first-word
    
  10. Complete with words in the history (like Emacs dabbrev):

    # Complete in history with M-/, M-,
    zstyle ':completion:history-words:*' list no 
    zstyle ':completion:history-words:*' menu yes
    zstyle ':completion:history-words:*' remove-all-dups yes
    bindkey "\e/" _history-complete-older
    bindkey "\e," _history-complete-newer
    

    Of course, all things are mentioned in the comprehensive manual, or the great User’s Guide to the Z-Shell which I wholeheartedly recommend. But one needs to find them. :)

    NP: Aimee Mann—Freeway

16jan2011 · Got root?

I lost my “administrative hymen” last month when a server of mine was rooted due to the recent Exim exploit (CVE-2010-4345). This is a post-mortem of the incident.

For shame, it took me almost a month to even detect the break-in, when I wondered why I didn’t get any mail from that address.

The same intruder cracked many machines and installed rootkits on them. He was not very professional: he left traces in the “panic log” of exim when he tried to download a configuration file with wget like this:

wget ... >exim.conf

instead of

wget -O exim.conf ...

… which of course writes the wget status messages into the file, too, which results in fatal parse errors:

# cat /var/log/exim4/paniclog
2010-12-16 01:09:54 string too large in smtp_notquit_exit()
2010-12-16 02:27:17 string too large in smtp_notquit_exit()
2010-12-16 05:12:11 string too large in smtp_notquit_exit()
2010-12-16 14:49:27 string too large in smtp_notquit_exit()
2010-12-16 18:25:20 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:25:20 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:45:03 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:25:20--  http://62.141.42.28/exim4.conf
2010-12-16 18:54:13 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf
2010-12-16 18:55:02 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf
2010-12-16 19:55:02 Exim configuration error in line 1 of /etc/exim4/exim4.conf:
  option setting expected: --2010-12-16 18:54:12--  http://62.141.42.28/exim4.conf

The installation of the rootkit also changed timestamps:

He added a key to root’s .ssh/authorized_keys and started another sshd on a non-standard port (59997). He did not detect that this port was blocked out-bound with iptables.

The exploit circulated around December 7, the patch was released December 10, and I got hacked December 16. And I actually read the news but thought I was not affected. Sigh.

Clearly, this should not happen. I admit the following mistakes:

  • The system ran a old (2007), un-updated version of GRML that I installed in a hurry with grml2hd without spending too much time thinking about the details.

  • It was not updated often because things used to break, being based in Debian unstable, and since it had many packages installed, updating often was a hassle.

  • It came with loads of stuff that I didn’t need, but they cluttered up logwatch and other intrusion detection tools.

  • As a result of this, I didn’t forward Cron messages to a mailbox I actually read.

  • It came with Exim preinstalled, so I used that for the single mailbox I had on that machine.

    (Not that this would have saved much; the bug was also in the last version before the security update, and since the intruder (accidentally) shut down the mail system, I would not have gotten any messages about the break-in.)

    Lessons learned for the future:

  • Set up minimal, but meaningful logging, so it will be read.

  • Send administrative messages to external hosts.

  • Update the system regularily. I now installed Debian squeeze, which is going to be stable “really soon now”, and then will serve its purpose for the next few years with comparatively few updates.

  • Do a minimal install. This time I used grml-debootstrap, which only installed a core Debian system, on which I added exactly the packages I need.

  • Use Postfix instead of Exim. Postfix not only is vastly easier to set up than exim (at least on Debian, Postfix default configuration fits on a single screen, while Exim has a configuration directory filled with almost 2000 lines of text), but also has a better security record. And I use Postfix on other servers too, so I will be more alert if I read about it being unsecure.

    Considerations for the future that I probably won’t follow:

  • Make lots of partitions and sprinkle noexec, nosuid mount options. (Complicates setup and disk space planning.)

  • Setup remote syslog. (I don’t have the resources for this, not worth it for a single machine.)

  • Switch to a distro that doesn’t package antique stuff. (Which?)

  • Use SELinux or such stuff.

    NP: Leonard Cohen—Avalanche

24dec2010 · Merry Christmas!

Santa glugging

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: Trembling Bells—Seven Years A Teardrop

Copyright © 2004–2013