<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Gist</title><link href="https://neo1.codeberg.page/" rel="alternate"></link><link href="https://neo1.codeberg.page/feeds/all.atom.xml" rel="self"></link><id>https://neo1.codeberg.page/</id><updated>2026-10-20T00:00:00-05:00</updated><entry><title>calibre css file</title><link href="https://neo1.codeberg.page/calibre-css-file.html" rel="alternate"></link><published>2026-10-20T00:00:00-05:00</published><updated>2026-10-20T00:00:00-05:00</updated><author><name>Thomas A. Anderson</name></author><id>tag:neo1.codeberg.page,2026-10-20:/calibre-css-file.html</id><summary type="html">&lt;h2&gt;Motivation&lt;/h2&gt;
&lt;p&gt;Calibre viewer can be configured with a custom css style.  This is done in&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;Calibre&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Preferences&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Styles&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Which presents you with a css file to edit using the calibre file editor.  The
editor is plain and limiting.  It is &lt;strong&gt;much&lt;/strong&gt; more convenient to use a proper
editor to edit …&lt;/p&gt;</summary><content type="html">&lt;h2&gt;Motivation&lt;/h2&gt;
&lt;p&gt;Calibre viewer can be configured with a custom css style.  This is done in&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;Calibre&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Preferences&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Styles&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Which presents you with a css file to edit using the calibre file editor.  The
editor is plain and limiting.  It is &lt;strong&gt;much&lt;/strong&gt; more convenient to use a proper
editor to edit a local file for this purpose.  Unfortunately, calibre does not
allow this as an option.   The following thread &lt;a href="https://www.mobileread.com/forums/showthread.php?t=51500"&gt;Custom CSS Stylesheets for
Calibre's Viewer&lt;/a&gt; on
&lt;a href="https://www.mobileread.com"&gt;www.mobileread.com&lt;/a&gt; contains lots of useful css code, but
using the default calibre file editor is extremely painful, not least of which
is because you have to open and close it every time you wish to make an edit.&lt;/p&gt;
&lt;h2&gt;Solution&lt;/h2&gt;
&lt;p&gt;The file &lt;a href="https://codeberg.org/neo1/misc/src/branch/master/calibre-css/calibre-css.sh"&gt;calibre-css.sh&lt;/a&gt;
contains a shell script to update the calibre css stylesheet using a css file of your choice.
Edit the file &lt;code&gt;calibre-css.sh&lt;/code&gt; and set
&lt;code&gt;CSS_FILE&lt;/code&gt; and &lt;code&gt;CALIBRE_CONFIG_DIRECTORY&lt;/code&gt; accordingly. &lt;code&gt;CSS_FILE&lt;/code&gt; is the local css file you wish to use.
&lt;code&gt;CALIBRE_CONFIG_DIRECTORY&lt;/code&gt; is the default calibre configuration location, this is set to system default.&lt;/p&gt;
&lt;p&gt;Now you can run &lt;code&gt;calibre-css.sh&lt;/code&gt; to inject your css stylesheet file into calibre.  NOTE: your calibre viewer should not be running when you do this, since we are changing the viewers stylesheet.&lt;/p&gt;
&lt;p&gt;All the work in &lt;code&gt;calibre-css.sh&lt;/code&gt; is done by a one liner, the last line of the file,  which is explained below:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;jq --arg val &amp;quot;$(cat &amp;quot;$CSS_FILE&amp;quot;)&amp;quot; &amp;#39;.&amp;quot;session_data&amp;quot;.&amp;quot;user_stylesheet&amp;quot; = $val&amp;#39; &amp;quot;$config_file&amp;quot; &amp;gt;&amp;quot;$tmp&amp;quot; &amp;amp;&amp;amp; mv &amp;quot;$tmp&amp;quot; &amp;quot;$config_file&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;--arg val "$(cat "$CSS_FILE")"&lt;/code&gt; : sets the variable val to the contents of &lt;code&gt;$CSS_FILE&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;'."session_data"."user_stylesheet" = $val'&lt;/code&gt; : &lt;code&gt;jq&lt;/code&gt; program to run.  It assigns the variable "session_data"."user_stylesheet" to the value of the variable val, i.e. the contents of &lt;code&gt;$CSS_FILE&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;"$config_file"&lt;/code&gt; : is the source file&lt;/p&gt;
&lt;p&gt;&lt;code&gt;&amp;gt; "$tmp"&lt;/code&gt; : send the output of &lt;code&gt;jq&lt;/code&gt; to a temporary file&lt;/p&gt;
&lt;p&gt;&lt;code&gt;&amp;amp;&amp;amp; mv "$tmp" "$config_file"&lt;/code&gt; : and if all goes well replace the original with the temporary file&lt;/p&gt;
&lt;p&gt;In closing
&lt;a href="https://codeberg.org/neo1/misc/src/branch/master/calibre-css/calibre.css"&gt;calibre.css&lt;/a&gt;
is the style sheet I use.  It's an amalgamation of various sources, that mimics the solarized color scheme.&lt;/p&gt;</content><category term="w3m, url, dmenu, fzf"></category></entry><entry><title>The problem of url routing.</title><link href="https://neo1.codeberg.page/the-problem-of-url-routing.html" rel="alternate"></link><published>2026-10-19T00:00:00-05:00</published><updated>2026-10-19T00:00:00-05:00</updated><author><name>Thomas A. Anderson</name></author><id>tag:neo1.codeberg.page,2026-10-19:/the-problem-of-url-routing.html</id><summary type="html">&lt;h2&gt;Motivation&lt;/h2&gt;
&lt;p&gt;One eventually stumbles into the problem of routing a url to it's intended
program.  The operating system configures some defaults, but they are often
inadequate. The typical solution is to copy the url into the system clipboard
and then paste it into the desired program.  This is better but …&lt;/p&gt;</summary><content type="html">&lt;h2&gt;Motivation&lt;/h2&gt;
&lt;p&gt;One eventually stumbles into the problem of routing a url to it's intended
program.  The operating system configures some defaults, but they are often
inadequate. The typical solution is to copy the url into the system clipboard
and then paste it into the desired program.  This is better but still limiting.
A more sophisticated solution is to use a clipboard manager to route the copied
url.  Two such tools are &lt;code&gt;parcellite&lt;/code&gt; and &lt;code&gt;xclip&lt;/code&gt;, and they work well. My
preferred solution is &lt;code&gt;dmenu&lt;/code&gt; and a shell script, which is what this post
is about.  This solution is leaner and offers more control, at the expense of
doing some shell scripting.&lt;/p&gt;
&lt;h2&gt;dmenu&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;dmenu&lt;/code&gt; is a menu tool from suckless tools. It gives the user a menu and echoes
the choice.  A shell script can grab and process the choice echoed by &lt;code&gt;dmenu&lt;/code&gt;,
in effect routing a url to the program chosen by the user.  One can do this by
creating a script &lt;code&gt;browser-launcher.sh&lt;/code&gt; and making it the default browser.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="ch"&gt;#!/bin/sh&lt;/span&gt;
&lt;span class="c1"&gt;# browser-launcher.sh&lt;/span&gt;

&lt;span class="c1"&gt;# if no url provided, use the one in the clipboard&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;!&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;[&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;then&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nb"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;--&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;xclip&lt;span class="w"&gt; &lt;/span&gt;-o&lt;span class="w"&gt; &lt;/span&gt;-selection&lt;span class="w"&gt; &lt;/span&gt;clipboard&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;

&lt;span class="k"&gt;fi&lt;/span&gt;

&lt;span class="nv"&gt;browser&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;dmenu&lt;span class="w"&gt; &lt;/span&gt;-l&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;-p&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Open with:&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;&amp;lt;&amp;lt;-EOF&lt;/span&gt;
&lt;span class="s"&gt;        w3m&lt;/span&gt;
&lt;span class="s"&gt;        feh&lt;/span&gt;
&lt;span class="s"&gt;        firefox&lt;/span&gt;
&lt;span class="s"&gt;        mkebook&lt;/span&gt;
&lt;span class="s"&gt;        yt-dlp&lt;/span&gt;
&lt;span class="s"&gt;        chromium&lt;/span&gt;
&lt;span class="s"&gt;    EOF&lt;/span&gt;
&lt;span class="k"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# $1 contains the url to route&lt;/span&gt;
&lt;span class="c1"&gt;# $browser contains the menu item chosen by the user, i.e. the program to send the url to&lt;/span&gt;

&lt;span class="c1"&gt;# send $1 to $browser program the background&lt;/span&gt;
&lt;span class="o"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;case&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$browser&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;in&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;w3m&lt;span class="o"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;           &lt;/span&gt;&lt;span class="c1"&gt;# ${1%%\?*} is necessary to strip parameter options, or w3m exits prematurely&lt;/span&gt;
&lt;span class="w"&gt;           &lt;/span&gt;urxvt&lt;span class="w"&gt; &lt;/span&gt;-e&lt;span class="w"&gt; &lt;/span&gt;sh&lt;span class="w"&gt; &lt;/span&gt;-c&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;w3m &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;1&lt;/span&gt;&lt;span class="p"&gt;%%&lt;/span&gt;&lt;span class="se"&gt;\?&lt;/span&gt;&lt;span class="p"&gt;*&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;feh&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;feh&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;firefox&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;firefox&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;mkebook&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;mkebook&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;yt-dlp&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;yt-dlp&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;chromium&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;chromium&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;*&lt;span class="o"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="c1"&gt;# default option&lt;/span&gt;
&lt;span class="w"&gt;         &lt;/span&gt;&lt;span class="nb"&gt;exit&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;0&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;esac&lt;/span&gt;
&lt;span class="o"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;&amp;amp;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;To make &lt;code&gt;browser-launcher.sh&lt;/code&gt; the default browser copy the file
&lt;code&gt;/usr/share/applications/firefox-esr.desktop&lt;/code&gt; to
&lt;code&gt;$HOME/.local/share/applications/browser-launcher.desktop&lt;/code&gt;.  Edit the new file
changing the line starting with &lt;code&gt;Exec&lt;/code&gt; to &lt;code&gt;Exec=browser-launcher.sh %u&lt;/code&gt;.  This
assumes that browser-launcher.sh is in the &lt;code&gt;$PATH&lt;/code&gt; Then we make it the default
browser by&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;xdg-settings&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;default-web-browser&lt;span class="w"&gt; &lt;/span&gt;browser-launcher.desktop
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;One can also setup a hotkey to launch &lt;code&gt;browser-launcher.sh&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The file &lt;code&gt;browser-launcher.sh&lt;/code&gt; in the example above is simple and is meant as
a template to demonstrate proof of concept.  Since it's a shell script one can
do some sophisticated processing.  This is where the approach proves superior
to a clipboard manager.  Below is a (slightly) more complicated &lt;code&gt;case&lt;/code&gt;.  We process an &lt;code&gt;x.com&lt;/code&gt; url to extract
the user name and launch &lt;code&gt;firefox&lt;/code&gt; with the new url pointing to the users &lt;code&gt;threadreader.com&lt;/code&gt; page.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;firefox-threadreader&lt;span class="o"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="p"&gt;#http://&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="p"&gt;#https://&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="p"&gt;#*/&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;url&lt;/span&gt;&lt;span class="p"&gt;%%/*&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;firefox&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;https://threadreaderapp.com/user/&lt;/span&gt;&lt;span class="nv"&gt;$url&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="p"&gt;;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;fzf&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;fzf&lt;/code&gt; offers a similar capability to &lt;code&gt;dmenu&lt;/code&gt; with the benefit of fuzzy searching through menu items.
The limitation is the menu can only be deployed in the terminal which is limiting.&lt;/p&gt;</content><category term="w3m, url, dmenu, fzf"></category></entry><entry><title>Comparing awk, R and python</title><link href="https://neo1.codeberg.page/comparing-awk-r-and-python.html" rel="alternate"></link><published>2026-08-21T00:00:00-05:00</published><updated>2026-08-21T00:00:00-05:00</updated><author><name>Thomas A. Anderson</name></author><id>tag:neo1.codeberg.page,2026-08-21:/comparing-awk-r-and-python.html</id><summary type="html">&lt;h2&gt;Motivation.&lt;/h2&gt;
&lt;p&gt;Now and then one wants to know the amount of RAM used by a particular program.
With multiprocessing this is harder than it looks.  There's even a cli tool
&lt;a href="https://www.selenic.com/smem/"&gt;smem&lt;/a&gt;. It's a nice tool written in Python.
Making it painfully slow for interactive cli use. The proper way to …&lt;/p&gt;</summary><content type="html">&lt;h2&gt;Motivation.&lt;/h2&gt;
&lt;p&gt;Now and then one wants to know the amount of RAM used by a particular program.
With multiprocessing this is harder than it looks.  There's even a cli tool
&lt;a href="https://www.selenic.com/smem/"&gt;smem&lt;/a&gt;. It's a nice tool written in Python.
Making it painfully slow for interactive cli use. The proper way to do this is
with cgroups.  This assumes you know ahead of time the program you want to
monitor and you have cgroups available and setup.&lt;/p&gt;
&lt;h2&gt;Solution&lt;/h2&gt;
&lt;p&gt;To that end lets script this, in the process we compare awk, R, and python.  The command &lt;code&gt;pgrep -f command_name&lt;/code&gt;
will spit out the desired process PID's, which we pass on to &lt;code&gt;ps&lt;/code&gt;.  Like so:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# shell function&lt;/span&gt;
smem&lt;span class="o"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;ps&lt;span class="w"&gt; &lt;/span&gt;-o&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;uss,rss,pss&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;pgrep&lt;span class="w"&gt; &lt;/span&gt;-f&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="k"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;So for example &lt;code&gt;smem firefox&lt;/code&gt; will spit out a table listing USS, RSS and PSS RAM use of each firefox thread.
All that's left is to sum each column to get the totals and we're done.
Note &lt;code&gt;pgrep&lt;/code&gt; matches a regex, so you might want to verify what you are matching. A better alternative would be&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;smem&lt;span class="o"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nb"&gt;local&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;MATCH&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;pgrep&lt;span class="w"&gt; &lt;/span&gt;-f&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="k"&gt;)&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;ps&lt;span class="w"&gt; &lt;/span&gt;-o&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;command&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$MATCH&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;ps&lt;span class="w"&gt; &lt;/span&gt;-o&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;uss,rss,pss&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$MATCH&lt;/span&gt;
&lt;span class="o"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The simplest way to sum the columns of the table is to write
a &lt;a href="https://codeberg.org/neo1/misc/src/branch/master/smem"&gt;script&lt;/a&gt;.  The scripts
are written in awk, R and Python.  They all do the same thing.  They sum every column of
a table.  For example:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;A&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;cat&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;&amp;lt;&amp;lt;EOF&lt;/span&gt;
&lt;span class="s"&gt;A B&lt;/span&gt;
&lt;span class="s"&gt;1 2&lt;/span&gt;
&lt;span class="s"&gt;3 4&lt;/span&gt;
&lt;span class="s"&gt;EOF&lt;/span&gt;
&lt;span class="k"&gt;)&lt;/span&gt;

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$A&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk
&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;          &lt;/span&gt;B
&lt;span class="w"&gt;         &lt;/span&gt;&lt;span class="m"&gt;4&lt;/span&gt;&lt;span class="w"&gt;          &lt;/span&gt;&lt;span class="m"&gt;6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;table_total.awk&lt;/code&gt; takes two optional arguments, the first one is input units, and the second one is output unit format.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$A&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1000&lt;/span&gt;

&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;          &lt;/span&gt;B
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="m"&gt;4000&lt;/span&gt;&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="m"&gt;6000&lt;/span&gt;

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$A&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1000&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;si

&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;          &lt;/span&gt;B
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="m"&gt;4&lt;/span&gt;.0k&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="m"&gt;6&lt;/span&gt;.0k
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The final command (in &lt;code&gt;smem.sh&lt;/code&gt;) becomes:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# Display output in binary units&lt;/span&gt;
smem_awk&lt;span class="o"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;ps&lt;span class="w"&gt; &lt;/span&gt;-o&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;uss,rss,pss&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;pgrep&lt;span class="w"&gt; &lt;/span&gt;-f&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$1&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1024&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;iec

&lt;span class="o"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# Example: chromium with no tabs open uses 308M of unique RAM, but 1.3G of non-unique RAM.&lt;/span&gt;
$&lt;span class="w"&gt; &lt;/span&gt;smem_awk&lt;span class="w"&gt; &lt;/span&gt;chrome

&lt;span class="w"&gt;       &lt;/span&gt;USS&lt;span class="w"&gt;        &lt;/span&gt;RSS&lt;span class="w"&gt;        &lt;/span&gt;PSS
&lt;span class="w"&gt;      &lt;/span&gt;308M&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;.3G&lt;span class="w"&gt;       &lt;/span&gt;525M
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The Python script is short and elegant.  The R script is also short, but ugly.
The awk script is lengthy.  Lets compare performance.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;T&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;cat&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;&amp;lt;&amp;lt;EOF&lt;/span&gt;
&lt;span class="s"&gt;USS   RSS   PSS&lt;/span&gt;
&lt;span class="s"&gt; 3008  7740  3297&lt;/span&gt;
&lt;span class="s"&gt; 1680  6188  1904&lt;/span&gt;
&lt;span class="s"&gt;   44  4164   134&lt;/span&gt;
&lt;span class="s"&gt; 2184  3816  2472&lt;/span&gt;
&lt;span class="s"&gt;   52  4288   147&lt;/span&gt;
&lt;span class="s"&gt;EOF&lt;/span&gt;
&lt;span class="k"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# side note: table_total.awk is calling numfmt which seems a bit off in some cases.&lt;/span&gt;
&lt;span class="c1"&gt;# Example, converting 25.6Mi gives 26Mi.&lt;/span&gt;
$&lt;span class="w"&gt; &lt;/span&gt;numfmt&lt;span class="w"&gt; &lt;/span&gt;--from-unit&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class="m"&gt;20&lt;/span&gt;&lt;span class="k"&gt;))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;--to&lt;span class="o"&gt;=&lt;/span&gt;iec-i&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;25&lt;/span&gt;.6
26Mi

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1024&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;iec-i&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$T&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;       &lt;/span&gt;USS&lt;span class="w"&gt;        &lt;/span&gt;RSS&lt;span class="w"&gt;        &lt;/span&gt;PSS
&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="m"&gt;6&lt;/span&gt;.9Mi&lt;span class="w"&gt;       &lt;/span&gt;26Mi&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="m"&gt;7&lt;/span&gt;.8Mi

real&lt;span class="w"&gt;    &lt;/span&gt;0m0.007s
user&lt;span class="w"&gt;    &lt;/span&gt;0m0.006s
sys&lt;span class="w"&gt; &lt;/span&gt;0m0.001s

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.R&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1024&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;iec&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$T&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;USS&lt;span class="w"&gt;      &lt;/span&gt;RSS&lt;span class="w"&gt;      &lt;/span&gt;PSS
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;6&lt;/span&gt;.8&lt;span class="w"&gt; &lt;/span&gt;MiB&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;25&lt;/span&gt;.6&lt;span class="w"&gt; &lt;/span&gt;MiB&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;7&lt;/span&gt;.8&lt;span class="w"&gt; &lt;/span&gt;MiB

real&lt;span class="w"&gt;    &lt;/span&gt;0m0.211s
user&lt;span class="w"&gt;    &lt;/span&gt;0m0.551s
sys&lt;span class="w"&gt; &lt;/span&gt;0m0.032s

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.py&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;1024&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;iec&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;&lt;span class="nv"&gt;$T&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;       &lt;/span&gt;USS&lt;span class="w"&gt;         &lt;/span&gt;RSS&lt;span class="w"&gt;         &lt;/span&gt;PSS
&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;6&lt;/span&gt;.8&lt;span class="w"&gt; &lt;/span&gt;MiB&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="m"&gt;25&lt;/span&gt;.6&lt;span class="w"&gt; &lt;/span&gt;MiB&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="m"&gt;7&lt;/span&gt;.8&lt;span class="w"&gt; &lt;/span&gt;MiB

real&lt;span class="w"&gt;    &lt;/span&gt;0m0.451s
user&lt;span class="w"&gt;    &lt;/span&gt;0m0.781s
sys&lt;span class="w"&gt; &lt;/span&gt;0m0.068s
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Python is painfully slow at almost half a second.  R is a bit slow but still snappy.  Awk is blazing fast.
Guess which I'll be using.&lt;/p&gt;
&lt;p&gt;To be fair this is an apples to oranges comparison.  The reason the Python version
so clean, concise and simple is because it's loading some powerful libraries to
do the work.  Were the Python code written in the same manner as awk it would
have a different performance profile.  It would also be longer and uglier.&lt;/p&gt;
&lt;h2&gt;What happens at scale?&lt;/h2&gt;
&lt;p&gt;What if our table is massive? Spoiler: Python comes out significantly ahead.&lt;/p&gt;
&lt;p&gt;To that effect lets write a script to generate a table of random integers between 0 and 100:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# a 3 by 4 table&lt;/span&gt;
$&lt;span class="w"&gt; &lt;/span&gt;table_gen.py&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;3&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;4&lt;/span&gt;
&lt;span class="w"&gt; &lt;/span&gt;A&lt;span class="w"&gt;  &lt;/span&gt;B&lt;span class="w"&gt;  &lt;/span&gt;C&lt;span class="w"&gt;  &lt;/span&gt;D
&lt;span class="m"&gt;59&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;42&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;41&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;65&lt;/span&gt;
&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;7&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;2&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;18&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;80&lt;/span&gt;
&lt;span class="m"&gt;26&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;67&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;67&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;17&lt;/span&gt;

&lt;span class="c1"&gt;# table with 2 million rows and 5 columns&lt;/span&gt;
$&lt;span class="w"&gt; &lt;/span&gt;table_gen.py&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;2000000&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&amp;gt;&lt;span class="w"&gt; &lt;/span&gt;table.txt

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&amp;lt;&lt;span class="w"&gt; &lt;/span&gt;table.txt
&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;          &lt;/span&gt;B&lt;span class="w"&gt;          &lt;/span&gt;C&lt;span class="w"&gt;          &lt;/span&gt;D&lt;span class="w"&gt;          &lt;/span&gt;E
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;98951126&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;99034755&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98973663&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98991072&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98951685&lt;/span&gt;

real&lt;span class="w"&gt;    &lt;/span&gt;0m2.844s
user&lt;span class="w"&gt;    &lt;/span&gt;0m2.836s
sys&lt;span class="w"&gt;     &lt;/span&gt;0m0.008s

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.R&lt;span class="w"&gt; &lt;/span&gt;&amp;lt;&lt;span class="w"&gt; &lt;/span&gt;table.txt
&lt;span class="w"&gt;        &lt;/span&gt;A&lt;span class="w"&gt;        &lt;/span&gt;B&lt;span class="w"&gt;        &lt;/span&gt;C&lt;span class="w"&gt;        &lt;/span&gt;D&lt;span class="w"&gt;        &lt;/span&gt;E
&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;98951126&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;99034755&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;98973663&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;98991072&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="m"&gt;98951685&lt;/span&gt;

real&lt;span class="w"&gt;    &lt;/span&gt;0m2.241s
user&lt;span class="w"&gt;    &lt;/span&gt;0m2.519s
sys&lt;span class="w"&gt;     &lt;/span&gt;0m0.161s

$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.py&lt;span class="w"&gt; &lt;/span&gt;&amp;lt;&lt;span class="w"&gt; &lt;/span&gt;table.txt
&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;           &lt;/span&gt;B&lt;span class="w"&gt;           &lt;/span&gt;C&lt;span class="w"&gt;           &lt;/span&gt;D&lt;span class="w"&gt;           &lt;/span&gt;E
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;98951126&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="m"&gt;99034755&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="m"&gt;98973663&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="m"&gt;98991072&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="m"&gt;98951685&lt;/span&gt;

real&lt;span class="w"&gt;    &lt;/span&gt;0m1.180s
user&lt;span class="w"&gt;    &lt;/span&gt;0m1.283s
sys&lt;span class="w"&gt;     &lt;/span&gt;0m0.340s
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The file &lt;code&gt;table.txt&lt;/code&gt; only takes up 30M.  Once you get into even larger tables python is a clear winner, but the startup cost of python is much too high for a quick interactive system tool.  (At least the way I implemented it.)&lt;/p&gt;
&lt;p&gt;We could also process the table in parallel across different cpu cores.  With 4 cores, using gnu &lt;code&gt;parallel&lt;/code&gt;,&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;parallel&lt;span class="w"&gt; &lt;/span&gt;--pipe-part&lt;span class="w"&gt; &lt;/span&gt;-a&lt;span class="w"&gt; &lt;/span&gt;table.txt&lt;span class="w"&gt; &lt;/span&gt;--block&lt;span class="w"&gt; &lt;/span&gt;-1&lt;span class="w"&gt; &lt;/span&gt;table_total.awk&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk
&lt;span class="w"&gt;         &lt;/span&gt;A&lt;span class="w"&gt;          &lt;/span&gt;B&lt;span class="w"&gt;          &lt;/span&gt;C&lt;span class="w"&gt;          &lt;/span&gt;D&lt;span class="w"&gt;          &lt;/span&gt;E
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;98951126&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;99034755&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98973663&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98991072&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98951685&lt;/span&gt;

real&lt;span class="w"&gt;    &lt;/span&gt;0m0.866s
user&lt;span class="w"&gt;    &lt;/span&gt;0m2.989s
sys&lt;span class="w"&gt;     &lt;/span&gt;0m0.077s
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The Python and R scripts would need to be modified to handle missing headers to do this properly.  Note that while the real time is lower than the Python script, the user time is much higher due to utilizing multiple cpu cores.  In other words the Python script is much more efficient, and parallelizing should yield a greater benefit.
Due to a quirk in awk (namely char + digit = digit, i.e 'A' + 2 = 2), we can do&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;parallel&lt;span class="w"&gt; &lt;/span&gt;--pipe-part&lt;span class="w"&gt; &lt;/span&gt;-a&lt;span class="w"&gt; &lt;/span&gt;table.txt&lt;span class="w"&gt; &lt;/span&gt;--block&lt;span class="w"&gt; &lt;/span&gt;-1&lt;span class="w"&gt; &lt;/span&gt;table_total.py&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;table_total.awk
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="m"&gt;98951126&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;99034755&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98973663&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98991072&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="m"&gt;98951685&lt;/span&gt;

real&lt;span class="w"&gt;    &lt;/span&gt;0m0.948s
user&lt;span class="w"&gt;    &lt;/span&gt;0m2.852s
sys&lt;span class="w"&gt;     &lt;/span&gt;0m0.426s
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Note the missing header in the output.&lt;/p&gt;
&lt;p&gt;This is not faster than parallelizing awk alone, because of Python startup
cost.  But if we were to scale up the table size yet again, we should see
performance benefits.&lt;/p&gt;</content><category term="awk, R, python"></category></entry></feed>