Main Page
Notes on technology
As with the other notes on this site, take anything below with a pinch of salt. It's not my considered opinion, just something I jotted down some time in the past 3 years. Buried in here are the workarounds to various problems I've run into in the past; with any luck putting the page online will help anybody else in the same situation.
No, these aren't in any particular order; I'm far too lazy for that.
I've been meaning for some time to properly learn the zsh shell - seems so much more powerful than bash:
show files with:
< filename
repeat last command with
!!
[ok, so easier to press the up arrow, but you can do funkier things on the basis of this]
easier directory expansion
/v/w/n [TAB]
expands to
/var/www/notes
[assuming there are no other similarly-named directories it could expand to]
looking into programs to keep track of what work I'm doing
Problems:
- runs in ncurses - can't see any easy way to automate it
- not sure how actively it's being developed
- written in c, so hard for me to fiddle with
- poor documentation
-
tags:
locally:
unison placea placeb
test server:
unison -testServer a.tmp ssh://remotehost/a.tmp
remote user:
unison a.tmp ssh://uname@remotehost/a.tmp
remote user, with unison in odd place:
unison a.tmp ssh://uname@remotehost/a.tmp -servercmd /full/path/to/unison
http://www.ubuntuforums.org/archive/index.php/t-86329.html - lots of pointers to other discussions.
danish-howto
tldp: international keyboad
writing python/tk widgets to handle polyglot input
alias asdf "loadkeys /usr/lib/kbd/keytables/dvorak.map"
alias aoeu "loadkeys /usr/lib/kbd/keytables/us.map"
Setting ubuntu up for multilingual input methods
go to system->preferences->keyboard
click on the layouts tab. Add all the languages you like
Now, you need to choose a keyboard shortcut to switch between them. Click on the 'layout options' tab. Under 'Group Shift/Lock behavior', you will see many checkboxes for keys to switch between input methods. Check one (or
Ubuntu lets you enter text in many scripts, enabling you to write in almost any language. Most of these input methods are included in a default installation of Ubuntu, so to use them you only need to adjust some GNOME settings. However a few languages (notably Chinese, Korean and Japanese) require you to install additional software.
Group 1: Keyboard layout
This method applies to:
All European languages
Arabic
Cyrillic languages (e.g. Russian)
Some Indian languages
Text in these languages/scripts can be entered by configuring the 'keyboard properties' in Ubuntu.
Go to Syste
documentation:
lists
wiki
issues
scim
documentation
debugging
scim - see separate entry
documentation
find guidelines
booting:
ntpupdate
dhcp
laptop stuff
look at apm for toshiba
network-magic
inet.d
path: /user@machine:/path/tofile
prettify todo and reading lists, put them in blosxom sidebar
set up unison cron job
deal with enlightenment sound issue
properly upgrade kernel
get new hard drive (from Rich?)
apt-get upgrade
mutt is so complicated that I've given up on it - possibly it is powerful, but I think thunderbird can do as much, and do it without requiring so much time learning what to do.
ctrl -m/n - new message
- expand all threads
\ - collapse all threads
ctrl-g - find
ctrl-t get new messages
ctrl-l - forward
f - next message
n - next unread message
f6 - next mail pain
ctrl-k - go to search bar
4 - label 'todo'
5 - label 'later'
m - mark as (un)read
j - mark as jung
is very good, and easy to use
I'm using custom1 for msn name, and custom2 for 'need to contact about...'. custom3 will be 'how well I know them, on a scale of 1-100, but using 10s only for the moment. 4 is livejournal
tags:
This is a place to note down conventions that let me automatically search through what I'm writing - so I can remember it, and write scripts that take it all into account
TODO: write some scripts to analyse my notes
markdown syntax
'tags:
[only used in some entries, slight formatting variations (e.g some lines begin '#tags', others just 'tags', spaces or commas as separators). Tag 'private' to mark things that shouldn't be published. I'm going to use 'public' to tag things that I am OK to put up
todo - lots of dissertation-related entries marked like this
google - not a consistently used tag, but I tend to write this when I'm marking things to search for
things taken from computer languages: =!, :: (BNF)
forms of quotation [author: year,page]
I think there are some FIXME's in there as well, somewhere. But I've been using them even less consistently than everything else.
tags: todo
Going by Susan Crawford's notes, one of the most interesting topics at the Aug 2005 Foo Camp was stuff growing out of voip.
Installing subway on debian or ubuntu
[permalink]
The python web framework subway has a helpful install page at http://www.gosubway.org/install/
Unfortunately, this chokes at a couple of points if you install in Debian or ubuntu, and it isn't immediately obvious how to fix it.
Firstly, you need to have python 2.4 installed (it my case, it was already). you'll also need python-dev
you'll also need to install the 'stateserver binary' mentioned on the subway install page.
surprised I couldn't find a general notes entry for unsorted tech stuff - so I'm creating one now.
Decided that most of the blogs in my reader are dead or moribund, so I've been through and added a chunk of a-listers. Hopefully they'll point me to some more interesting stuff shortly.
mp3s from the 'future of web apps' conference are available here - something for work.
a python/tkinter program to tell me when to get back to work
version 0.1: pop up a window. schedule via crontab
tags:
going through my notebooks, i come across the following list. No idea how I reached it, but the books look interesting:
- Benedict Anderson, imagined communities
- R Putnam, bowling alone
- N. Townsend, the package deal
- B Wellman, networks in the global village
- C Taylor, sources of the self
- E Gellner, nationalism
- F Dumont, les ideologies
- more heat than light
blosxom+emacs+apache+markdown
This is the most important, and I think the most unusual, part of my set-up. It's a way of taking notes which are easy to write, easy to view, easy to search, and easy to incorporate into other projects.
I use emacs to write notes in more-or-less plain-text format, which are stored in a dedicated directory hierarchy (currently ~/notes).
Blosxom (which is essentially a blogging tool) takes these notes, and presents them as a web-page. It's theoretically a blog, but there are pages for each category, and from my point of view it just becomes an easier way to look at directories of text files.
I say more-or-less plain text because I use the 'markdown' plugin with blosxom. This lets me add formatting marks to my text that are intuitive, easy to type, and don't make it hard to read the file in plain text - and it converts them into html. So I mark headings with a hash, bold words with asterisks either side, and indent/blockquotes with a > -all pretty much how I would mark them in an email.
The big advantage of this is that it leaves all my notes in plain text on the computer. That lets me manually search them (with grep, or whatever). It also leaves me confident that I'll be able to transfer them into any system I start using in the future. Anything that can't import plain text is doomed to failure anyway. The final advantage is that I can quite easily write shell scripts to alter my notes. Blosxom too is very easy to adjust, and has a decent plugin architecture. So I can keep on tweaking it so that everything appears the way I want it. For example the markdown heading format annoyed me because you had to type more hashes to get a smaller heading (i.e. '#' was converted to
and '######' was converted to ). Reversing that arrangement took me involved rewriting exactly one line of code.
polipo
Polipo is a web proxy, designed for a single user. I use it to cache every page I look at, so I can revisit them when I'm away from a net connection. This is pretty handy when I'm researching and writing, because often I just want to check one niggly fact.
My only issue with it is that it doesn't cope very well with connecting and disconnecting from the net. If I just pull out the ethernet cable, it'll refuse to serve any pages. I have to restart it with>
polipo offlineProxy=true
and then when I reconnect, I have to use
polipo offlineProxy=true
...which isn't ideal from the pov of simplicity
How to find an interesting irc channel?
[permalink]
silly me for trying to upgrade from woody to sarge! Lots of problems, but hopefully I'll get most of them fixed this evening:
- the X configuration in which I'd somehow stopped the monitor going funny, was overwritten
- console-cyrillic is causing problems again
- it didn't run lilo, or tell me that I had to before rebooting
- I did find a useful hardware detection utility, discover
- had to run apt-get dist-upgrade many, many times. Which is silly
- the kernel upgrade, which was the main reason I was sargeifying myself, didn't happen
even Howard Rheingold is taking an interest in RWT
login box - why can't I use tab
Jim Moore and others have created an investment fund targetted entirely at RSS (i.e. blogs, news aggregation, etc). Is it really that big an area?
Reading list - computerish things
[permalink]
- danah boyd's papers, especially her MA thesis
- Granovetter's paper on weak ties: commentary here and here. JSTOR pdf paper here or more prettily here
- Duncan Watts: 'six degrees: the science of a connected age'
- paper on informal corporate networks uncovered by analysis of groups intensively emailing each other
- TrustMail - filtering based on whuffie
- Azure 2000 manual (is online, don't have link)
Sources to read regularly
close file in emacs: C-x k
optparse - documented here and here
import optik as optparse #give false name to module. optik is the module under its original name, optparse when it's in the standard library.
Regular Expressions
if you can, use string functions instead: index, find, count, replace, split. But they're case-sensitive, among other problems
my meta-language:
C=dummy character
import re
$ - end of string
^ - start of string
\b - word boundary
C? - optional C
C{0,6} - 0-6 Cs in a row
\d - any number 0-9
\D - any non-digit
\d+ - one or more of \d
\d* - zero or more of \d
(\d) - anything in brackets is remembered
re.sub('before', 'after', string)
as raw text (so no need for double-backslashing)
re.sub(r'\bROAD$', RD.', s)
^C?C?C?$ - one-to three Cs
^M?M?M?(CM|D?C?C?C?|CD)$
Unit Testing
write test cases as classes, which subclass TestCase class in unittest module
assertEqual - fail if two values aren't equal
test toRoman for out-of-range input:
def testNegative(self):
"""toRoman should fail with negative input"""
self.assertRaises(roman.OutOfRangeError, roman.toRoman, -1)
Looking for a decent addressbook, etc, system.
Requirements:
- linux-based
- works with mutt
- lightweight (should be able to run in bg without me noticing)
- open data format (preferably text files or xml that's easy to fiddle with)
- well-developed and well-supported (don't want a program that I turn out to be the only user of)
- has web frontend (so I can put up on ohuiginn.net)
- easy to synchronise between ohuiginn.net and thule
- has debian package
the shortlist
gnome-pim
Blackbook
is defunct
ldap
mozilla address book
kaddressbook
may be really slow without kde
evolution adress book
Jpilot
simple, lightweight
bbdb
lbdb
gnomecal
addresses
abook
text-based
wonder if I could write a web frontend?
tags:
Given all the talk about irc backchannels in conferences and college lectures, I'm wondering if anyone is talking about using them in parliament, or in other areas of government?
UK pundits and politicians never tire of worrying about the declining role of parliament. The UK parliament is useful, because of its powers to hold the executive to account. The problem is that 90% of the time it's too damn boring, and that politicians have better things to do than sit round waiting for the one important exchange of the day.
This seems like a perfect case for backchannels. Attendance is high, but most of the people there aren't too concerned about what is actually being discussed - they're waiting for a later debate or a vote, or turning up to demonstrate their interest, or hoping for the chance to put a question. The speeches are a mirror image. Politicians often have to say something to prove they're concerned, or to meet obligations to report to parliament, or even to filibuster. There isn't much need to worry about backchannels making speakers uncomfortable: nothing on IRC compares to the childish, abusive heckling that goes on in parliament.
All this seems like a perfect case for getting politicians to use IRC in the House. The only problems that leap to mind are:
- There are probably all kinds of procedural problems, and lack of wifi
- Most politicians probably aren't familiar with irc
- Politicians are likely to feel inhibited in what they say, for fear that somebody will leak irc logs to the press
None of these seem to be insurmountable. So, I wonder:
- who else has been thinking about this? All I can find is this blog entry about texting in the Japanese parliament
- What advantages/disadvantages have I missed?
- Is this something that could eventually happen, or just a pleasant fantasy?
the copy-paste thing again. alleged solution for xemacs is
(setq interprogram-cut-function 'own-clipboard)
(setq interprogram-paste-function 'get-clipboard)
some very rough notes from NotCon:
[permalink]
there is no politics of the net; if there is, don't assume it will be progressive. corps and lobbyists are GOOD at using the net
does the underlying net infrastructure matter to non-technies (IP addresses, IANA, IPv6 vs. trusted networks)
nerd determinism and nerd fatalism both bad.
If Napster was Alexandria, current properly-distributed p2p is the unburnable library
DRM: is it inherently non-end2end? can it work technologically, or just legally?
group politics (incl. blog politics) bad. Celebrity culture bad. lack of critical depth in net sociology - could do with more of the wanky critical theory. Digital divide is real, despite being ignored among the microrepublics of the alpha geeks. Net is like real-world frontiers in the denial of aborigine culture: i.e. pre-net politics is far more important than net politics.
what % of netizens do you need not using DRM to bring it down?
archive.org
for Harvard, it costs $2 to loan and return a book. Cheaper to print-on-demand.
What libraries do is make out-of-print books avaiable
archive.org bookmobile costs $15k, including vehicle. now have nfp spinoff, 'anywhere books', in Uganda
night of the living dead is now public-domain
linktv's 'mosaic' compiles middle eastern TV broadcasts. more generally, archive.org has been recording (but not broadcasting) many TV feeds for a while now.
hard drives are cheaper than tapes - they use racks of linux boxen with 4x300G hard drives. 1 petabyte costs $2m. hard drive failure is 6%pa
main archive is on teh san andreas fault - are trying to expand/copy to other locations: Amsterdam, Alexandria.
Recently, somebody (David Weinberger?) stepped into the 'what are blogs' debate, with the comment that they are now well-enough established as a medium for us to stop thinking about them in terms of diaries, or jorunalism, or any of their other precursors. 'horseless carriage' was a useful explanation at first, but once it became an entity in its own right, the term 'car' made more sense. The 'car' was inherently meaningful, whereas the 'horseless carriage' was an attempt at explanation by analogy. So we should stop worrying whether 'blog' means 'paperless diary', or whatever, and think of the blog in itself.
There's another example of this process in Inwood's biography of Hooke. I saw it this morning, or last night, but I can't find or remember it any more. So this entry was a bit pointless. Damn
Nick Douglas turns up an impressive rant on #joiito
[slightly edited for clarity]
i'm just spouting out questions i'd be interested in
should i count the podcast idea as dead?
explorations of social dynamics - that's not as well-covered
again, i have a real itch for discussing MMORPGs
also forums
blogs
like if we discussed where mmorpgs will go in the next 5 years
then maybe, say, biomods
what's the future of biomods?
or where everyone goes to get good wallpapers
who changes their settings all the time, who rarely does so
how much is gamer culture infiltrating tech culture?
how divided are the techs from the tech-nots?
hacks from hacknots?
is it good or bad to draw old-school geeks into new-school geekery?
why do pdas still suck?
why aren't more people making movies and sharing them?
why aren't homemade movies showing up in theaters?
what's everyone's favorite music to play while working? - "motivational music" but less dorky
what are the 10 best tech films? 10 most realistic? 10 most optimistic?
why are bloggers still so self-righteous?
why can't we break a million visits a day to any one blog?
why is tv still so huge?
how do we get MTV culture kids onto the net?
do we want to?
what will web sites look like 2-5 years from now? do we have to put up with technorati green for another half decade?
is google still good?
is yahoo still for the middle-aged and teenaged?
when will macs stop kicking ass?
how do we stop people from downloading so much fucking spyware?
why don't more cities have blanket wifi?
why would blanket wifi help more than 2% of us?
is it pretentious to still love megaman?
is digg.com the future of the internet?
who can afford all these fucking gadgets?
ok, i think that's all i have
i could always write up a doc with 100 more
your job is to delete the 80 crap ones
and then find 5 guys who REALLY REALLY want to talk about this in a way that more than one little subsection of geekery can understand
2d is easy, 3d is hard
electron microspy requires preparation and to work in a vacuum
SPMs: scanning probe microscopes
::AFM atomic force microscope
STM scaning tunelling microscope
SNOM/NSOM near-field scanning optical microscope
Taken from a Cientifica whitepaper
'soft lithography' = nanoprinting. use micro/nanostructured forms as molds
so far hard to build precise multilayer structures - so can't easily replace existing circuit-building techniques
hmmmm...not much detail in that report, now is there?
from cientifica public crap whitepaper
'85: buckminsterfullerene (buckyball) discovered by Smaller, Curl, Kroto.
buckyballs are C60 arranged in interlocking hexagons and pentagons
C60 is still best, though variants with (C[tens]-C[hundreds]) have been dicovered. Carbon nanotubes are related
the BC250m Permian extinction caused by something full of buckyballs.
individual molecueles are v. strong, but they are only weakly attached to other molecules --> good lubricants. May also have applications in the tyre industry.
nb electronic properties of buckyballs ('interesting')
peapods: fullerenes inside nanotubes. this allows interesting electical effects.
buckyballs are large enough to have the potential to act as bases for drugs, eg for HIV
may superconduct at higher temperatures than liquid nitrogen, but this is experimentally dubious. fullerenes are known to be superconducting at well under a degree above absolute zero
C60 is the largest material on which the wave/particle duality has been demonstrated.
endohedral fullerenes:buckyballs with other atoms trapped inside them. Notation X@C60, where X is the trapped atom(s). The caged atom is close enough to give electrons to the fullerene.
functionalisation: attach other molecules to the 'pincusion' of a fullerene molecule
Since I no longer have a hermes account, I'm sending my email to my 3ix account (mail.ohuiginn.net), and I'm going to try learning mutt to read it.
You need three components to get email working: mutt itself (to read the mail), getmail (to download it from a pop or imap server) and msmtp (to send it via smtp). It's more usual to use fetchmail and sendmail to receive and send, respectively, but these are larger programs, probably more than I need for a single-user machine. All of these are part of debian stable:
apt-get install mutt getmail msmtp
getting pop working the easy way
First, there's the quick and dirty way to read a pop folder
g #open folder
pop://mail.ohuiginn.net #connect to pop server
[change username to daniel@ohuiginn.net, enter password]
That only works if you're happy to read
getting pop working properly
The alternative is to configure getmail to download pop into a directory, and then read it locally with mutt. This way, I minimise storage on the remote server (where I only have 50mb), and it looks easier to configure.
But getmail configuration was really horrible, and had me hating getmail, and linux in general. I even went to the trouble of half-installing fetchmail, but that looked even worse.
Anyway, having installed the getmail package, create a .getmail directory in your home directory, and start editing a file getmailrc inside that
After much trial and error, I copied and guessed the following getmailrc. Half the getmail documentation seems to be out of date, and incompatible with the current program. Either that, or I was being even more incompetent than I thought. Here's the file:
[default]
verbose = 1
readall = 0 # only get new messages
delete = 1 # Delete mail after retrieval
message_log = ~/.getmail/getmail.log # Log message retrieval and delivery
timeout = 240 # TCP timeout value; increase if on a poor connection or
max_message_size = 0 # Default is 0, which means no limit.
[daniel-ohuiginn.net] # Simple configuration for a single-user POP3 mailbox
server = mail.ohuiginn.net
username = daniel@ohuiginn.net
password = [*********]
use_apop = 0 # Use APOP authentication for this account instead of cleartext POP
postmaster = ~/mail/ohuiginn/ # Default delivery to this Maildir
[lists-ohuiginn]
# insert more sections like the one above here - all will be checked
The really nasty line in here is:
postmaster = ~/mail/ohuiginn/
This is the directory you want the mail put into. It has to be a 'Maildir', but the getmail docs won't tell you what that means. A Maildir is one with subdirectories new, cur and tmp. It's easy enough to sort that out yourself
mkdir ~/mymaildir
cd ~/mymaildir
mkdir new cur tmp
That should sort you out for getmail configuration. Run it manually to test, by typing getmail. If that works, you can add it as a cron job (something I haven't got round to yet).
In mutt, you can open this folder by typing g (go to folder), and then browsing the directory structure
msmtp
Once you've apt-get'ed msmtp, configuration isn't too hard. Just create .msmtprc in your home directory, and insert something like the following:
account ohuiginn
host mail.ohuiginn.net
from dan@ohuiginn.net
user daniel@ohuiginn.net
password [*******]
If you're storing your password in that file, better make sure it isn't world-readable:
chmod 700 ~/.msmtprc
Now, configure mutt to send mail via msmtp (it seems to assume sendmail by default. Insert the following into the configuration file (.muttrc in your home directory):
set sendmail="/usr/bin/msmtp"
set use_from=yes
set realname="Dan O'Huiginn"
set from=daniel@ohuiginn.net
set envelope_from=yes
If there are multiple smtp accounts defined in your .msmtprc, mutt will choose one based on matching the 'from' field in the lines above.
mutt itself
I've not yet done much configuration of mutt. It all happens in .muttrc, and there are many, many options. I've copied one from dotfiles.com, which gives me pine-like keyboard shortcuts. That's good enough for the moment - default mutt shortcuts are really counterintuitive, especially if you've been living with pine for four years.
Setting up Mozilla to use unifont
[permalink]
[I have no idea if this is the correct or best way to get unicode to display properly on Mozilla, but it's working for me so far]
Mozilla 1.7.5 on Debian Sarge doesn't display huge chunks of unicode by default, including arabic, chinese, hindi, and some diacritical marks. The problem seems to be that it doesn't ship with decent international fonts, at least not int he us-english version.
So first download and install unifont:
- go to http://czyborra.com/unifont/. Download unifont.hex.
- check if you have hex2bdf and bdftopcf on your system - locate hex2bdf; locate bdftopcf. If you don't have them, download them (from the page above) and make them executable (chmod u+x hex2bdf)
- follow the instructions on the unicode howto to install:
$ gunzip unifont.hex.gz
$ hex2bdf < unifont.hex > unifont.bdf
$ bdftopcf -o unifont.pcf unifont.bdf
$ gzip -9 unifont.pcf
# cp unifont.pcf.gz /usr/X11R6/lib/X11/fonts/misc
# cd /usr/X11R6/lib/X11/fonts/misc
# mkfontdir
# xset fp rehash
Now you have installed unifont, which will display large amounts of unicode. Now, to get Mozilla using it. Open the browser, and type about:config into the address bar. A huge list of options appears. cut it down to the relevent ones by typing 'unicode' into the filter box.
You'll see several lines beginning something like:
font.name.[something.x-unicode
You need to set each of these in turn to use unifont, rather than whatever they had before. Right-click the entry in the 'value' column on the right, and choose 'modify'. Replace the content with gnu-unifont-iso10646-1
Do this for each unicode font.name entry. Now restart, and see if unicode is working. Try the BBC foreign language services (bbc arabic, bbc chinese, bbc hindi), or google for a unicode test page.
For more information:
Google scholar and academic databases make it possible to track references to texts. Would it be possible to use the number of references to highlight the most important parts of the text - that is, the parts most quoted. You could then produce some kind of precis, missing out the bits nobody else bothers about.
As a trial, it would probably be best to start with something very widely quoted - shakepeare or the bible, for example. With the bible it might even be possible to simply do a google search for each verse.
So, choose a book, and for each verse do a search for:
the text of the verse
the reference in several forms ( Mark 1:12, Mark 1.12, etc)
count them up, make a total for the verse
for refences to an entire verse (Mark 6), divide the reference-power equally among the verses.
then find some way of visually displaying it. one way might be something like the tag-clouds that are so popular at the moment: each reference increases the font-size slightly. Then you'll be able to see what is paid attention to. E.g. 'god is love' would become absolutely immense. but more impressive would be how the passage describing homosexuality as an abomination would be bigger than the rest of that book put together.
For the verse parts of Shakespeare (the sonnets, perhaps?) you could do it simply by puting the line into quotes and feeding it to google. then 'shall I compare thee to a summers day' would come out top. In fact, this might be a manageable place to start.
If I could make this work with books, it might be posible to measure the extent of the bias towards early parts of a book.
elements of the basic line-by-line program would be:
read in every line (perhaps just into an array of lists)
labelling lines, comments, etc. get given standard values, not fed to google
for each line, count number of references
store the data:
use pickle and some custom data class
class sonnetline:
def __init__:
linenumber=0
line="blank text"
googlecount=0
output it:
initially, just tweak the amount of size-increase in the font until it looks ok
center the sonnets - that's the easiest way to read text of different sizes
perhaps also change the text boldness - ie. vary the colour from light grey up to solid black
Last modified: Sun Nov 5 06:02:17 CET 2006