How my five-year-old understands my job
An XKCD comic describes how my five-year-old (and some adults) understands my job:
http://www.xkcd.com/722/
nginx and textpattern
I run a lean combination of os, webserver and cms on my VPS- NetBSD, Nginx and Textpattern- and it seems to be a fairly uncommon one, judging from the few results in google. That’s not a bad thing, but it does mean rolling your own.
To get a current build of ngix, it’s necessary to build it yourself or out of pkgsrc/wip. Since nginx keeps cgi and php at arms length, I use spawn-fastcgi to run a pool of php interpreters. This is also trivial to build. I slapped together a basic spawn-fcgi rc file and start both nginx and the pool like any other service.
My core nginx.conf looks like this:
user nginx nginx; worker_processes 2; events { worker_connections 512; } http { include /usr/pkg/etc/nginx/mime.types; default_type application/octet-stream;log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';access_log /var/log/nginx/access.log main; error_log /var/log/nginx/error.log notice; rewrite_log on; # rewrite shows up in error.log at level 'notice', # turn this off unless debugging to avoid bloated logssendfile on; tcp_nopush on; tcp_nodelay off;keepalive_timeout 45 20; keepalive_requests 25;gzip on; gzip_http_version 1.0; gzip_comp_level 2; gzip_min_length 1100; gzip_buffers 4 8k; # gzip always compresses "text/html", others that benefit... gzip_types text/plain text/xml text/css text/javascript application/x-javascript application/xml application/xml+rss;server { listen 80; server_name localhost; location / { root share/examples/nginx/html; index index.html index.htm; } }include /usr/pkg/etc/nginx/virtual-othersite.conf; }
Configuration of nginx is straight-forward if you accept “messy” urls (i.e. http://www.example.com/foo/index.php) but everyone prefers “clean” urls and that gets into rewrite rules. I haven’t found any for Textpattern (that work with clean urls, anyway) so I came up with my own. The virtual site configuration contains these rewrite rules for textpattern and the fastcgi handling:
server { listen 80; server_name www.othersite.com; root /var/www/othersite.com; index index.html index.php;location / { # rewrite all requests to the maint page # if it exists... # if (-f $document_root/maintenance.html) { rewrite ^(.*)$ /maintenance.html last; break; }# rewrites for textpattern locations... somewhat brittle, relies # on knowledge of the args # if (!-e $request_filename) { rewrite ^/(favicon.ico|robots.txt) @404 break; rewrite ^/archive(.*)$ /index.php?s=archive$1 last; rewrite ^/about.*$ /index.php?s=about last; # enable the next rule during setup, then disable... # rewrite ^/textpattern/setup/(.*)$ /textpattern/setup/index.php$1 last; rewrite ^/textpattern(.*)$ /textpattern/index.php$1 last; rewrite ^/ /index.php; } }location ~ \.php$ { try_files $uri @404; fastcgi_pass 127.0.0.1:9000; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; }location @404 { return 404; break; }# cache static files location ~ ^/(images|javascript|stylesheet)/ { expires 1d; } }
These rules appear to work with the exception of in-line commenting, which redirects to the article. Pop-up comments, since they use a full url, work as expected. The 404 handling is to short-circuit having the fastcgi processes handle common known bad requests.
Some things are almost universal
I’ve been vacationing and using various internet providers with my netbook (much easier than worrying about internet cafes and dealing with one time pads for passwords). Almost universally, the routers and access points at the hotels and apartments have default passwords and minimal settings to achieve some functionality beyond inefficiently turning electricity into heat. Individuals and companies specializing in providing network service seem to do slightly better job (though in one case the AP wasn’t secured after clearing the Sputnik agent).
Email packrat
I’m an email packrat. I have organized, compressed archives of nearly everything I’ve received or sent since 1999. In duplicate. I qualify “nearly” because in 2006 I switched to maildir after corrupting an mbox and out of necessity began culling several mailing lists. And the only reason it only goes back ten years is that I can’t read some of the tapes and can’t find a second copy on a different medium or simply don’t have the operating system any more (OS/2, for example). But that isn’t the point. The point is that I found an email I sent six years ago in less time than it took to write this entry and I did it using little more than grep (I really must sit down and index all of that with mairix).
To my family and friends running Windows...
I regularly get questions about the virii, worms, trojans and other malware infecting Windows. Lately a very convincing and hard to clean piece of malware is making the rounds so I decided to write up several previous discussions to get all this in one place.
First, let’s be clear: I don’t manage or run Windows systems at work anymore and have not in several years. I spend most of my time in Unix; I’m an occasional user of a managed corporate Windows deployment and someone else worries about all this. When I do need to run Windows at home, it runs on a Mac inside a VMWare Fusion virtual machine or under FreeBSD or Linux using Virtual Box. I take snapshots of the virtual machine and restore it when I’m done. If that made no sense then detailing what I do for myself is probably not going to work for you, but I’ll try to offer some advice on cleaning up and avoiding some of these problems.
The procedure for cleaning up is simple, in theory:
- Take the machine off the network, preventing reinfection and the spread of the existing infection.
- Get the host to a state where important files (documents, pictures, music, etc.) can be backed up. If you do not have a backup device, improvise by burning to CD or DVD or use a USB stick.
- Use the original installation or recovery disk from the vendor to “nuke and pave” (wipe clean and reinstall) the box returning it to an original pristine state.
- Restore your data.
- Prevent reoccurrence.
In practice, it’s not so simple. Skim the longer explanation below. Don’t panic. If it sounds like more than you can do, spend the $250 for your local computer shop or the Geek Squad (Best Buy) to handle it.
Before getting into the malware, some comments on network hygiene. If the box is connected directly to the internet via a cable modem, it’s probably infected whether you know it or not. See this article Unpatched PC Survival Time Just 16 Minutes which was true in 2004 and still applies today. If you have DSL or FiOS with a company-supplied router (not a bridge) or have a router or wireless base that you supplied (like one of these from Linksys or Belkin among several makers) then you might have a small barrier between your machines and whatever comes knocking from the outside world. If you don’t have one, get one before you begin or you’re wasting your time. Be sure to update to the latest firmware and change the default password.
Next a comment on the behavior of malware, if the box is infected with one piece of malware it probably has more. Disconnect it from the network and leave it off until you can deal with it for a few hours uninterrupted. Many pieces of malware scan the local network or present fake services or subvert them (for example, DHCP which is used to dynamically assign an IP address) to infect other hosts. These can spread over a wireless network, too. Trojans often require someone to do something- accept a pop-up, run a game, load a program, play a video- as a means of getting their foot in the door. Worms and self-propagating nasty things don’t need help. Plan to spend time checking and cleaning any other computers on your home network.
Using a different, uninfected machine download fresh copies of Avast! (anti-virus), Ad-Aware (malware cleaner), Malwarebytes (another malware cleaner), and Spybot Search & Destroy (yet another malware cleaner) and burn them to a CD. The free versions are fine for this purpose. Be sure to only use the official releases linked above as there are a bunch of fakes circulating with trojans in them. Last download the complete Microsoft IE and Windows updates and burn those to a CD. You’ll need all of these to remove the initial infection or reinstall.
If you have a full-system backup, stop. Your backup is probably infected too. If you’ve made an ad hoc backup using a CD, DVD or memory stick assume it is infected but go ahead and make a second one now. Trying to save your applications and settings is usually a lost cause so focus on the data. If the machine is somewhat usable (ex, pop-ups, fake blue screen, etc.), try to install the new copies of the programs above then run them to clean out the system. If this fails or you can not install and clean you are at serious risk of losing your data. As a last ditch effort where you can not get one-on-one professional help, you might try creating a boot disk on another machine and copying off the files by hand or installing a second minimal copy of the original operating system in a second directory or on a different harddisk. If your machine is recent and can boot from USB, there are commercial and non-commercial (ex. Bart’s PE Builder) ways to build a bootable Windows image on a flash drive but if you can do that, you probably don’t need my help and would not be in this mess. Understand that even if you succeed in retaking the machine, the software on it is not trustworthy. The point is that you should be able to back up your data.
At this point, if it’s not obvious where I’m heading, I recommend wiping the machine entirely and reinstalling from scratch before you connect to the network. Apply the patches you downloaded above for Windows and Internet Explorer. Now that you have a clean machine, install the anti-virus and anti-spyware mentioned above. Do not restore your files from backup yet! All updates need to be done before putting the box back on the network or you’ll simply get reinfected. You should have that hardware device (router, access point, etc.) I mentioned earlier between you and the cable or DSL.
Do not restore your backups and reinstall your apps. Seriously. Scan them with the anti-virus and anti-spyware first. If you’ve taken full system backups, you are going to restore only the data (MyDocuments, etc.). Do not restore the whole system and overwrite what you just cleaned up. Restore this subset of the backup to a new folder and scan it. If you work with an ad hoc backup, copy the data to a new folder and scan it. Next move on to applications. Only install ones for which you have the original disk or trust the download site. Do not trust C|Net, download.com or any of the other sites that track software they didn’t write. Only install from the original vendor or author. Do not install any “free” games, screen-savers, plugins or video codecs. It’s okay to install Adobe Flash and Shockwave and the Apple Quicktime Player. When you are done, take a full system backup and create a “Restore Point” using the Microsoft System Restore tool. This will be the reference copy if you ever need to do it again.
Finally, to prevent reinfection you need to change habits:
- Don’t run or double-click anything you download that isn’t vetted first. This means no solitaire games, toolbars, screensavers, browser plugins or other gadgets. It also means no email attachments and documents. Be paranoid.
- Don’t click on pop-ups or click “Okay” until you know what the message means.
- Don’t use file-sharing networks and peer-to-peer software. It encourages you to run software you haven’t vetted and to open files from untrusted sources.
- Install anti-virus and keep it updated. Pay for the subscription.
- Install anti-spyware, keep it updated and run it regularly. Pay for the subscription.
- Use a white-list only firewall. Only permit the traffic you want to and from your PC. Microsoft ships one built in and I’ve heard that Comodo is good and Norton is acceptable. Pay for the subscription. Either way, pay attention to the pop-ups from it.
- Use a different browser. I suggest Firefox instead of Internet Explorer. It’s generally better and presents a smaller target.
- Disable or restrict ActiveX, Javascript, Flash, Java and every other plugin. For Firefox users this is best achieved using the NoScript Plugin and only permitting trusted sites to execute scripts.
- Run a filtering proxy and configure your browsers to use it. The open source Privoxy can help you screen out junk. Download Privoxy for Win32 from sourceforge.
- Don’t use Outlook or Outlook Express for email. Their previewing of attachments is a opportunity for infection. Try Thunderbird instead.
- Don’t use AOL chat or any other branded chat clients. If you have to use a chat client try Pidgin.
- Take regular full backups. An external hard drive and the commercial Novastor or the open source Areca will do.
- Make regular ad hoc backups of your data. Burning to DVD is an easy and reliable method.
- Make a recovery disk, ideally a recovery USB stick, and keep it updated.
I know no one is going to follow that list until they get burned for the second time.
Yard sale find
I don’t know why I picked it up. Maybe because there was just enough power left in the batteries to run a couple of calculations and the self-test before they died, but for $2 I bought a circa-1983 HP 12C financial calculator. It’s missing the owner’s manual, the vinyl sleeve is in tatters and the quick reference lost its spiral binding long ago but the calculator itself is in excellent shape.
$9 worth of button cell batteries later, I have a working RPN calculator. It’s a neat little device and HP still sells it in two models- one of which switches between RPN and Algebraic modes- and supplies (pdf) manuals and guides:
- HP 12C Summary
- HP 12C Solutions
- HP 12C Users Guide
- HP 12C Quick Start
- HP 12C training modules [4.2MB zip]
I don’t know what I’m going to do with it but I’ll leave on my work desk and see if I use it. I usually have a Lisp session open in a terminal or a newer, faster computing device nearby, but it’s quite fast for what it does compared to slapping together a one-off and checking the result.
Dropping wanderlust, back to mutt
I’ve given wanderlust, a mail user agent for Emacs, a solid six month try as my only mail reader after a prior year of dabbling with it. It’s good but not great. I found no improvement over mutt with Emacs as my editor. That combination just lets me read mail most of the time instead of swearing and grovelling through the code.
Cherokee web server
In the past I’ve used NginX for a development web server but I stumbled across the Cherokee http server recently which fits my criteria of small, fast, lightweight, trivial to build on unix-like systems and easy to configure.
Whatever the project’s intentions, the result is perfect for my style of semi-disconnected development on my laptop with an instance per work dir listening on the loopback and independently serving different components. It has a combined rsize under 2MB for the workers and the controller meaning it’s dwarfed by the reset of my tools (like Firefox and Emacs) and it’s trivial to configure thanks an included web-based admin tool.
Palm Pre SDK?
Maria jumped onto the early-adopter bandwagon for the first time recently and I had a chance to play with the Palm Pre smartphone. It’s a nifty little device. Not so nifty that I’m shelling out cash for one- yet. I carry a blackberry for work and still have hard feelings (and a pile of broken Pilots and old Visors in a box somewhere) on Palm’s failure to hold onto and grow out of the PDA market that it defined- but interesting enough that when she described what the built in apps can do I had some ideas and went looking for the SDK.
Surprise. It’s still in “early adopter” mode and you have to register and wait until your name comes up in whatever scheme they’re using to choose developers. That’s the sort of thing you do when the device is not yet widely available and the tools and documentation are raw. Except “early adopters” for SDK’s expect it to be raw. They’re developers, remember, used to incremental releases and half done documentation. It’s the sort of thing that happens when marketing is afraid that you’ll get one, and only one, try at launching before the bad reviews sink it. I looked, there’s next to nothing in the Palm application store and it’s going to stay that way until Palm gets the SDK out into people’s hands and I don’t mean by someone leaking it (heh, read last night that it was leaked) so it finds its way into torrents.
clever c trick
I had a few days of vacation visting my folks and started a little hobby hacking in the evenings- the details of which are grist for other, longer posts- but the set up is that I’m working on an FFI (foreign function interface) and have a variety of structs that I want to pass back and forth. I don’t want to handle each struct with unique getter/setters or use hairy if/else or switch/case statements because it’s tedious to write and hard to maintain and I’m not sure where I’m going with this code. I worked out a generic dispatch function with a jump table and having the C preprocessor and compiler build most of it for me. To do that I needed to get the offset of the structure member and its size and came up with this preprocessor macro:
#define _off_sz(type,memb) offsetof(type,memb), sizeof(((type *)0)->memb)
I’d never seen it before and thought myself quite clever with the cast
zero since I can do this:
#include <stddef.h>
#include <stdlib.h>
...
typedef struct tag_vtable_rec {
pointer (*fp)();
int argc;
size_t offset;
size_t size;
} vtable;
...
vtable ffi_foo_vt[] = {
{ some_func, 2, _off_sz(SomeStruct, SomeMember) },
{ other_func, 1, _off_sz(SomeStruct, OtherMember) },
...
};
...
It turns out that this technique of casting zero and using `sizeof()` is old and well-known. I have found it argued over in `comp.lang.c` at least as early as 2002 and Nigel Jones wrote a macro almost exactly like mine (and properly casts the `sizeof()` result to `size_t`, which I’ll fix) in 2004 for Embedded Systems Magazine.
Ross Lonstein
