chris blogs

27feb2017 · A time-proven zsh prompt

I’ve been using below shell prompt since 2013 and only slightly tweaked it over time. The most significant change was probably displaying the Git branch.

The basic idea of my prompt is to not show redundant or obvious information. This allows the prompt to be short, yet useful.

By default, the prompt displays the hostname, shortened directory, and a % to signify a zsh. The hostname is bold to make it stand out when you are scrolling, and the sigil is colored to mark the beginning of the command. It looks like this:

juno ~% ./mycommand -x

Long directory names are truncated in the middle:

juno /tmp/dirwithare…gname%

In rare cases, only showing two levels of hierarchy may be confusing, so you can set $NDIRS to something higher, e.g. 4:

juno deeply/nested/dir/structure%

When the previous command failed, the prompt also displays the exit status of the previous command:

juno 42? ~%

When there are background jobs running, the prompt shows how many there are:

juno 1& ~%

Note how the status and job display use the associated ASCII symbols.

When we are in a Git repository, the current branch is displayed inline as part of the base directory (when possible), or as a prefix, together with the repo name. By design, in the most common cases this keeps the prompt very short:

juno prj/rack@master%
juno rack@master/doc%
juno rack@master doc/Rack%

When the prompt detects a SSH session, the prompt sigil is doubled, so we are a bit more careful there:

hecate prj/lr%%

When the shell runs as root, the sigil is red (I don’t usually run zsh as root):

juno /etc#

That’s it, essentially. Apart from the Git integration, it’s really straight-forward. Not visible above is trick 4 to simplify pasting of old lines, and how it updates the title of terminal emulators to hostname: dir respectively hostname: current-command (which needs quite complicated quoting).

The whole thing is defined in the PROMPT section of my .zshrc.

NP: Light Bearer—Aggressor & Usurper

02jan2017 · zz: a smart and efficient directory changer

A nice feature I’ve become used to in the last year is a so-called “smart directory changer” that keeps track of the directories you change into, and then lets you jump to popular ones quickly, using fragments of the path to find the right location.

There is quite some prior art in this, such as autojump, fasd or z, but I could not resist building my own implementation of it, optimized for zsh.

As far as I can see, my zz directory changer is the only one with a “pay-as-you-go” performance impact, i.e., not every directory change is slowed down, but only every use of the smart matching functonality.

The idea is pretty easy: we add a chpwd hook to zsh to keep track of directory changes, and log for each change a line looking like “0 $epochtime 1 $path” into a file ~/.zz. This is an operation with effectively constant cost on a Unix system.

chpwd_zz() {
  print -P '0\t%D{%s}\t1\t%~' >>~/.zz
}
chpwd_functions=( ${(kM)functions:#chpwd?*} )

The actual jumping function is called zz:

zz() {

How does the matching work? It’s an adaption of the z algorithm: The lines of ~/.zz are tallied by directory and last-used time stamp, so for example the lines

0 1483225200 1 ~/src
0 1483225201 1 ~/tmp
0 1483225202 1 ~/src
0 1483225203 1 ~/tmp
0 1483225204 1 ~/src

would turn into

6 1483225204 3 ~/src
4 1483225203 2 ~/tmp

Also, the initial number, the effective score of the directory, is computed: We take the relative age of the directory (that is, seconds since we went there), and boost or dampen the results: the frequency is multiplied by 4 for directories not older than 1 hour, doubled for directories we went into today, halved for directories we went into this week, and divided by 4 else.

  awk -v ${(%):-now=%D{%s}} <~/.zz '
    function r(t,f) {
      age = now - t
      return (age<3600) ? f*4 : (age<86400) ? f*2 : (age<604800) ? f/2 : f/4
    }
    { f[$4]+=$3; if ($2>l[$4]) l[$4]=$2 }
    END { for(i in f) printf("%d\t%d\t%d\t%s\n",r(l[i],f[i]),l[i],f[i],i) }' |

By design, this tallied file can be appended again with new lines originating from chpwd, and recomputed whenever needed.

The output of this tally is then sorted by age, truncated to 9000 lines, then sorted by score. (My ~/.zz is only 350 lines, however.)

      sort -k2 -n -r | sed 9000q | sort -n -r -o ~/.zz

With this precomputed tally (which is generated in linear time), finding the best match is easy. It is the first string that matches all arguments:

  if (( $# )); then
    local p=$(awk 'NR != FNR { exit }  # exit after first file argument
                   { for (i = 3; i < ARGC; i++) if ($4 !~ ARGV[i]) next
                     print $4; exit }' ~/.zz ~/.zz "$@")

If nothing was found, we bail with exit code 1. If zz is used interactively, it changes into the best match, else the best match is just printed. This allows using things like cp foo.mkv $(zz mov).

    [[ $p ]] || return 1
    local op=print
    [[ -t 1 ]] && op=cd
    if [[ -d ${~p} ]]; then
      $op ${~p}
    else

If we found a directory that doesn’t exist anymore, we clean up the ~/.zz file, and try it all over.

      # clean nonexisting paths and retry
      while read -r line; do
        [[ -d ${~${line#*$'\t'*$'\t'*$'\t'}} ]] && print -r $line
      done <~/.zz | sort -n -r -o ~/.zz
      zz "$@"
    fi

With no arguments, zz simply prints the top ten directories.

  else
    sed 10q ~/.zz
  fi
}

I actually shortcut zz to z and add a leading space to not store z calls into history:

alias z=' zz'

The full code (possibly updated) can be found as usual in my .zshrc.

I use lots of shell hacks, but zz definitely is among my most successful ones.

NP: Leonard Cohen—Leaving The Table

24dec2016 · Merry Christmas!

Comic © Liz Climo

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch
Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: Against Me!—Haunting, Haunted, Haunts

15jan2016 · Dear Github

These kind of posts seem popular these days.

My top six of features GitHub is missing:

  1. Searching for text in commit messages. Fixed 2017-01-04. About 2/3 of the repos I clone, I solely clone to run git log --grep.
  2. Searching in the wiki. Fixed 2016-08-08.
  3. Archive tarballs with submodule checkouts included; else submodule usage is totally pointless.
  4. Marking issues private to committers. Useful both for embargoed security issues and to keep out an angry mob.
  5. Being able to disable pull requests. For projects that use Github mainly as a mirror.
  6. IPv6 support. It’s 2016, damnit.

Sincerely,
chris2

NP: Revolte Springen—Hinter den Barrikaden

24dec2015 · Merry Christmas!

Consumers' crèche

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

NP: Elende Bande—Uns das Leben

19feb2015 · Six hacks for less(1)

Recently I got around to configuring less, and I collected these few tricks:

  1. Sometimes I look at lists with less, and then do things step-by-step, keeping the current action at the top of the page. This works nicely until you end up at the last page of the file, and then can’t scroll down. You lose track of where you are at and get confused.

    It would be much nicer scrolling down, and filling up the buffer with ~ after the end of file, just as if you had searched in the pager.

    Actually, with ESC-SPC, you can move a full page down, filling up the buffer with ~. Toying around a bit, you’ll find out that you can override the “page length” with a prefix, i.e. 1 ESC-SPC will move down one line only!

    However, this is still inconvenient to type all the time, thus let’s define a keybinding. For this, create a file ~/.lesskey where we will put the key definitions. This file then will be compiled using lesskey(1) and generate a binary configuration file ~/.less. (I guess you can be lucky that m4 is not involved in this mess…)

    One problem is actually binding the key. You can easily bind the cursor down key (\kd) to forw-screen-force, but how do you pass 1? The canonical hack is to use the noaction action, which will behave just like you’ve typed the keys after it. Thus, we write:

    #command
    \kd noaction 1\e\40
    j noaction 1\e\40
    

    (By the way, that #command comment is important to tell lesskey you are defining key commands.)

    Finally, scrolling bliss!

    Actually, scrap that.

    The badly underdocumented key J (and K) will scroll how I want, but you only read about that in the example inside lesskey(1). Therefore, we can just do:

    #command
    \kd forw-line-force
    j forw-line-force
    

    These keybindings are there since at least 1997 and I’ve never found them before…

  2. While we are redefining keys, I’ve always found it a bit clumsy to read multiple files, having to type :n and :p. Using [ and ] is much more convenient (at least on a US keyboard), and by default these keys do things of questionable utility.

    #command
    [ prev-file
    ] next-file
    
  3. Did you ever wish to give feedback from less? Like have a script output some info, and you decide how to go on? Since less always exits with status 0 usually, this I thought this was tricky to do, but the quit action actually can return an arbitrary exit code, encoded as a character.

    I bound Q and :cq (like in vim) to exit with status 1:

    #command
    Q quit \1
    :cq quit \1
    

    Now you can do stuff like look at all files and have them deleted when you press Q instead of q to exit:

    for f in *; do less $f || rm $f; done
    
  4. I use less a lot to look at patches, git log output, and ocassionally mailboxes. The D command as defined below will move to the next line starting with diff or commit or From␣.

    #command
    D noaction j/\^diff|commit|From \n\eu
    

    It will also “type” ESC-u to hide the highlighting. Now I can simply press D to jump to the next chunk of interest.

  5. To return to where you started from after a search or going to the end of file, type ''. Typing '' again will go back, so this is also nice to toggle between two search results.

  6. Back in the old days of X11R2(?) there was a tool called xless, which was exactly that: a pager like less that ran in its own X11 window. It’s quite useful. We can recreate this by combining a X11 terminal emulator and plain less with a small zsh snippet:

    xless() {
      {
        exec {stdin}<&0 {stderr}>&2
        exec urxvt -e sh -c "less ${(j: :)${(qq)@}} </dev/fd/$stdin 2>/dev/fd/$stderr"
      } &!
    }
    

    Watch the trick how we pass the stdin/stderr file descriptors and the file arguments!

    Now you can just run command-spitting-out-loads | xless and the output will be shown in a new terminal and not lock your shell.

    NP: Feine Sahne Fischfilet—Dreieinhalb Meter Lichtgestalt

17feb2015 · 10 fancy zsh tricks you may not know...

Wow, almost two years have passed since the latest installment of our favorite clickbait zsh tricks series.

  1. When editing long lines in the zle line editor, sometimes you want to move “by physical line”, that is, to the character in the terminal line below (like gj and gk in vim).

    We can fake that feature by finding out the terminal width and moving charwise:

    _physical_up_line()   { zle backward-char -n $COLUMNS }
    _physical_down_line() { zle forward-char  -n $COLUMNS }
    zle -N physical-up-line _physical_up_line
    zle -N physical-down-line _physical_down_line
    bindkey "\e\e[A" physical-up-line
    bindkey "\e\e[B" physical-down-line
    

    Now, ESC-up and ESC-down will move by physical line.

  2. Sometimes it’s nice to do things in random order. Many tools such as image viewers, music or media players have a “shuffle” mode, but when they don’t, you can help yourself with this small trick:

    SHUF='oe|REPLY=${(l:5::0:)RANDOM}${(l:5::0:)RANDOM}${(l:5::0:)RANDOM}|'
    

    Just append ($SHUF) to any glob, and get the matches shuffled:

    % touch a b c d
    % echo *($SHUF)
    d c a b
    % echo *($SHUF)
    c a d b
    

    Note that this shuffle is slightly biased, but it should not matter in practice. In doubt, use shuf or sort -R or something else…

  3. Are you getting sick of typing cd ../../.. all the time? Why not type up 3?

    up() {
      local op=print
      [[ -t 1 ]] && op=cd
      case "$1" in
        '') up 1;;
        -*|+*) $op ~$1;;
        <->) $op $(printf '../%.0s' {1..$1});;
        *) local -a seg; seg=(${(s:/:)PWD%/*})
           local n=${(j:/:)seg[1,(I)$1*]}
           if [[ -n $n ]]; then
             $op /$n
           else
             print -u2 up: could not find prefix $1 in $PWD
             return 1
           fi
      esac
    }
    

    With this helper function, you can do a lot more actually: Say you are in ~/src/zsh/Src/Builtins and want to go to ~/src/zsh. Just say up zsh. Or even just up z.

    And as a bonus, if you capture the output of up, it will print the directory you want, and not change to it. So you can do:

    mv foo.c $(up zsh)
    
  4. Previous tricks (#6/#7) introduced the dirstack and how to navigate it. But why type cd -<TAB> and figure out the directory you want to go to when you simply can type cd ~[zsh] and go to the first directory in the dirstack matching zsh? For this, we define the zsh dynamic directory function:

    _mydirstack() {
      local -a lines list
      for d in $dirstack; do
        lines+="$(($#lines+1)) -- $d"
        list+="$#lines"
      done
      _wanted -V directory-stack expl 'directory stack' \
        compadd "$@" -ld lines -S']/' -Q -a list
    }
    zsh_directory_name() {
      case $1 in
        c) _mydirstack;;
        n) case $2 in
             <0-9>) reply=($dirstack[$2]);;
             *) reply=($dirstack[(r)*$2*]);;
           esac;;
        d) false;;
      esac
    }
    

    The first function is just the completion, so cd ~[<TAB> will work as well.

  5. Did you ever want to move a file with spaces in the name, and mixed up argument order?

    % mv last-will.tex My\ Last\ Will.rtf
    

    Pressing ESC-t (transpose-words) between the file names will do the wrong thing by default:

    % mv My last-will.tex\ Last\ Will.rtf
    

    Luckily, we can teach transpose-words to understand shell syntax:

    autoload -Uz transpose-words-match
    zstyle ':zle:transpose-words' word-style shell
    zle -N transpose-words transpose-words-match
    

    Voila:

    % mv My\ Last\ Will.rtf last-will.tex
    
  6. If you are an avid Emacs user like me, you’ll find this function useful. It enters the directory the currently active Emacs file resides in:

    cde() {
      cd ${(Q)~$(emacsclient -e '(with-current-buffer
                                   (window-buffer (selected-window))
                                   default-directory) ')}
    }
    

    You need the emacs-server functionality enabled for this to work.

  7. I’m working on many different systems and try to keep a portable .zshrc between those. One problem used to be setting $PATH portably, because there is quite some difference among systems. I now let zsh figure out what belongs to $PATH:

    export PATH
    path=(
      ~/bin
      ~/.gem/ruby/*/bin(Nn[-1])
      ~/.opam/current/bin
      ~/.cabal/bin
      ~/.go/bin
      /usr/local/bin
      /usr/local/sbin
      /usr/bin
      /usr/sbin
      /sbin
      /bin
      /usr/games
      /usr/games/bin
    )
    path=( ${(u)^path:A}(N-/) )
    

    The last line will normalize all paths, and remove duplicates and nonexisting directories. Also, notice how I pick up the latest Ruby version to find the Gem bin dir by sorting them numerically.

  8. One of the hardest things is to set the xterm title “correctly”, because most people do it wrong in some way, and then it will break when you have literal tabs or percent signs or tildes in your command line. Here is what I currently use:

    case "$TERM" in
      xterm*|rxvt*)
        precmd() {  print -Pn "\e]0;%m: %~\a" }
        preexec() { print -n "\e]0;$HOST: ${(q)1//(#m)[$'\000-\037\177-']/${(q)MATCH}}\a" }
    esac
    
  9. For a cheap, but secure password generator, you can use this:

    zpass() {
      LC_ALL=C tr -dc '0-9A-Za-z_@#%*,.:?!~' < /dev/urandom | head -c${1:-10}
      echo
    }
    
  10. Sometimes it’s interesting to find a file residing in some directory “above” (e.g. Makefile, .git and similar). We can glob these by repeating ../ using the #-operator (You have EXTENDED_GLOB enabled, right?). This will result in all matches, so let’s first sort them by directory depth:

    % pwd
    /home/chris/src/zsh/Src/Builtins
    % print -l (../)#Makefile.in(Od)
    ../Makefile.in
    ../../Makefile.in
    

    Now we can pick the first one, and also make the file name absolute:

    % print (../)#Makefile.in(Od[1]:A) 
    /home/chris/src/zsh/Makefile.in
    

    I knew the #-operator, but it never occurred to me to use it this way before.

    Until next time!

    NP: Pierced Arrows—On Our Way

24dec2014 · Merry Christmas!

Consumers' crèche

Frohe Weihnachten, ein schönes Fest, und einen guten Rutsch ins neue Jahr wünscht euch Christian Neukirchen

Merry Christmas and a Happy New Year!

Bitte lesen: Liebeserklärung an die Vielfalt - eine Weihnachtsbotschaft.

NP: Against Me!—Holy Shit

03dec2014 · Recovering a Git repository from filesystem corruption

Recently I had to fix a Git repository where something unfortunate happened: probably due to accessing a NTFS partition that was still mounted in a hibernated alternative operating system, several files became corrupted (and actually had their contents exchanged with different files on the disk!).

git fsck discovered corrupted blobs, which we tried to recover first, when we detected their content does not make sense at all. These blobs were irrevocably lost, but we still wanted to get out the rest of the Git history alive.

Usually, applying the following technique is not necessary, because you can either just clone again from your Git upstream or recover the repository from the last backup—but both did not exist in this case.

The corrupted blobs actually all belonged to a single commit that happened a few months ago. The solution was thus to remove this commit from the history, keeping all other trees intact (of course, commit ids would change, but the content won’t).

I first tried to do this with git rebase, but it is of course the wrong tool, since it will try to remove the change of the defect in all following history.

Finally, I had a use-case for git filter-branch. To make it short, we can filter out the defect commit using:

git filter-branch --commit-filter \
  '[ $GIT_COMMIT = badbadbadbad ] && skip_commit "$@" ||
                                     git commit-tree "$@"'

This will rewrite all commits after badbadbadbad, but not touch their actual content.

git fsck still was not happy, we thus made a clean copy using

git clone --no-local --no-hardlinks mybrokengit myfixedgit

Now git fsck reported no errors and all other revisions were still ok. (Also, the blobs have been packed, so the next data corruption will be more fatal… ;))

I cannot think of any other version control system where a repair like this would have been possible. Thanks, Git!

NP: Against Me!—Exhaustion & Disgust

16mar2014 · Review: Learning Shell Scripting with Zsh

Learning Shell Scripting with Zsh
by Gastón Festari.
Packt Publishing, Birmingham 2014.
132 pages.

[Full disclosure: I have received a digital copy of the book in exchange for this review.]

The book is titled Learning Shell Scripting with Zsh: Your one-stop guide to reading, writing, and debugging simple and complex Z shell scripts which could not be more wrong. A far more appropriate title would have been: Getting started with Zsh: Your guide to interactive use and first steps of customization.

A first glance through the table of contents proves me right: There are chapters on “Getting started”, “Alias and History”, “Advanced Editing”, “Globbing”, “Completion”, and “Tips and Tricks”, but no explicit mention of writing shell scripts for purposes other than customization of an interactive shell.

I’m not completely sure of the target audience: The book assumes basic familiarity with (Bourne) shell already, half-heartedly explains pipes or redirections as well as how to define shell functions, but e.g. there is no mention of how to pass or parse arguments. After a single example of a very simple completion function in Chapter 5 (with explanation), the reader is assumed know enough for writing his own completions. I highly doubt that.

I found the book riddled with small mistakes, imprecise to wrong explanations and plain terribly sloppy typesetting of the code examples, which hurts particularly in a book about shell where syntax is lax but proper newlines matter. Examples are often not very well chosen and occasionally confusing even to me, who is very familiar with the topic.

The over-emphatic style with its needless rambling and cheeky language does not save the book but probably annoys the reader even more. (I hope not to ever read “You know, because widescreen.” in a book again.) Without, a lot more actual content and perhaps a real introduction to shell usage would have fit into these compact 118 pages.

The book finishes with a recommendation to read From Bash To Z Shell next. All I can recommend is: better skip this book completely and start with that one if you are interested in a good book about zsh.

Rating: 2 of 5 points.

NP: Against Me!—White Crosses

Copyright © 2004–2016