Hosting webtrees on NearlyFreeSpeech

Brief notes on setting up webtrees (PHP, MySQL/MariaDB) genealogy software on NearlyFreeSpeech.net Web Hosting

• Email did not work out of the box (required for user signup, etc): NFS requires sites use sendmail. The default option webtrees uses is "sendmail -bs" which implies an SMTP config, instead use "-t" (see doco)
• If you have a domain name managed by NFS, you can host any standard type of site from a subdomain very easily when you create it and it will add the DNS record. Then just configure webtrees to use that, eg `base_url="http://webtrees.example.com"` in `data/config.ini.php`
• Set up HTTPS with Let's Encrypt; NFS provides a CLI script `tls-setup.sh` - the first time I ran it I got an error accessing the `.well-known/acme-challenge/...` file it creates, but the 2nd time it worked fine. Then it auto-upgrades connections to HTTPS, which meant static files didn't load, so I updated the base_url to https in the config file as above. 
• Pretty URLs: A slight deviation from the documentation. NFS uses Apache 2.4 and apparently without the mod_access_compat module for backwards compatibility (fix found here). The contents of data/.htaccess should be changed to "Require all granted". For the root .htaccess file, I used "RewriteBase /" because webtrees is hosted at the root, and otherwise doesn't work once rewrite_urls="1" is set in data/config.ini.php 
• Changing the session timeout: default is 7200s (2 hours) but apparently depends on number of page loads, so if you have a quiet site it may not log you out?... don't know why the cookies don't just expire. 
• Upgrading: As of July 2022 I have successfully completed 2 upgrades without any above changes being reverted, but it's still possible. Just follow the release instructions.

Importing data from other tools:

• GEDCOM files can be imported (use standard v5.5.1) - if they are small they can be uploaded from the webpage, but beyond some size (maybe 50MB?) they will take a while and then fail to upload, so you have to copy the file to the `data/` folder on the server. 
• When you start a large import (I had a 755,000 individual file), it shows you progress on the screen, but for some reason it only progresses while you have a tab open - you can't sleep the computer, but you can just keep it in the background. If the computer does go to sleep, if you just browse back to the tree in the Control Panel, it will show the import status and continue. 
• I had a file that failed to import because of a bad character in a description; the file was valid UTF-8 but inserting this seemed to confuse the SQL statement with "SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF2\xAC\xA0\xB3 S...' for column `webtrees`.`wt_individuals`.`i_gedcom`..." It appeared a bad copy paste caused this; I opened the GEDCOM file directly in a text editor, found the entry from the value it was inserting, and removed some weird characters, to be fixed up later. 

Workaround to delete a large tree:

I needed to delete the aforementioned large tree after debugging the bad characters, and to import new changes, but the website delete button didn't give any feedback and eventually the database locked up, as it continued to try to delete in the background. I stopped the DB process gracefully via the web interface, with no ill effects except people complained the site was down, and the tree remained untouched. 

Obviously this is beyond the scope of a normal NFS MariaDB instance, but I had been able to observe the delete statements running via phpMyAdmin, yet they were still there afterwards, so it must be running in a transaction. I looked up the source to determine the SQL statements, and ran each or a few at a time (so not in a transaction, but still in order), and it actually worked perfectly and much faster. 

First, determine the tree ID, a.k.a. `gedcom_id` that you want to delete, from the wt_gedcom table. 

Then run these statements for that ID, eg 2 (sorry about the lack of code formatting): 

delete from wt_gedcom_chunk where gedcom_id = 2;

delete from wt_individuals where i_file = 2;

delete from wt_families where f_file = 2;

delete from wt_sources where s_file = 2;

delete from wt_other where o_file = 2;

delete from wt_places where p_file = 2;

delete from wt_placelinks where pl_file = 2;

delete from wt_name where n_file = 2; -- big

delete from wt_dates where d_file = 2; -- big

delete from wt_change where gedcom_id = 2;

delete from wt_link where l_file = 2; -- big one 2.2 mil rows

delete from wt_media_file where m_file = 2;

delete from wt_media where m_file = 2;

delete wt_block_setting from wt_block_setting join wt_block on wt_block.block_id = wt_block_setting.block_id where gedcom_id = 2

delete from wt_block where gedcom_id = 2;

delete from wt_user_gedcom_setting where gedcom_id = 2;

delete from wt_gedcom_setting where gedcom_id = 2;

delete from wt_module_privacy where gedcom_id = 2;

delete from wt_hit_counter where gedcom_id = 2;

delete from wt_default_resn where gedcom_id = 2;

delete from wt_log where gedcom_id = 2;

delete from wt_gedcom where gedcom_id = 2;

These may change over time if tables are added or changed. 

Of course, you should take a database backup first. Adapted from the NFS documentation, running this on the web host will create a gzipped SQL backup of database "mywebtrees" with a datetime stamp:

mysqldump --user=yourusername --host=mywebtrees.db mywebtrees -p | gzip > /home/private/backup-webtrees`date "+%F_%H-%M-%S"`.sql.gz

It would be prudent to copy this to your own backup lest NFS lose it, same as your media files. 

FreeNAS: Share an arbitrary path via SMB

 By default, FreeNAS/TrueNAS won't let you share a folder that's not part of its managed volumes, giving the error "The path must reside within a volume mount point"which probably means ZFS pools/datasets only. 

This is probably for a good reason, and doing otherwise implies you're doing something outside the scope of the services it is meant to provide, and you could break stuff; this certainly won't be covered by support. Caveat emptor. 

My use case is I have some old USB hard drives I attach to the NAS box so they can be accessed from jails like Plex, but aren't important enough to use precious redundant storage. They're mounted read-only into /mnt, via init commands, and it would be nice to be able to access them directly over the network too, but FreeNAS doesn't want to know about it. 

So, onto breaking stuff. There's a way to trick it: bait and switch. It's really just user interface validation. 

* Create a folder at the path you want to share 

* Set up your share as you like, access the empty folder to test it works

* Delete the folder and replace it with a symlink! 

* Restart the SMB service (if it won't bother others): it seems to lose the path, but on restart it shares as you'd expect! Maybe after a while it would work without restarting. 

I can't vouch for permissions working as expected, especially around write, as I'm only accessing read-only volumes with read-only users in this instance. Mapping SMB users to *nix users is kind of madness anyway. 

I had also tried:

* creating a symlink in my existing volume pool, but it didn't show up in the path browser for creating shares, and had the same error when I entered the path directly. 

* creating a symlink into an existing share: for whatever reason it said I didn't have permissions for the share, though all the files and folders are mounted fully readable, but as a different user. It's possible there's a group permissions tweak to be made somewhere, or it may be SMB refusing to follow symlinks, especially across volumes/devices. 

Update July 2022:

I used this same trick on TrueNAS 12.0 and it still worked. I now have a 10TB external USB3 drive for bulk, non-critical storage and I can copy from the share at > 100MB/s. However there are some caveats. 

It's formatted as ext4 so it will work with Linux and FreeBSD, as TrueNAS still doesn't support writing to NTFS. I first tried exFAT so it would work with Windows too but TrueNAS doesn't appear to support that at all. It only supports ext2fs, and even then writing is quite slow - about 3-5MB/s. 

The tricky part is that this time I want to be able to write to it too, even if only at 3MB/s - mainly just to move things around. However, I get an error:

Error 0x8007054F: An internal error occurred. 

So something is wrong with the permissions, mapping those to ext2, or something. I'll update if I find a solution. 

Further update: I did not find a solution, instead I decided to accelerate plans to abandon FreeNAS/TrueNAS and move to a Linux-based system with full Docker and external drive support. 

I chose OpenMediaVault which is based on Debian, has a lot of the same monitoring that I needed, such as S.M.A.R.T and UPS. Through plugins it also support ZFS so I can mount my zpool - but eventually I will be decommissioning it and shifting to a file system pool such as mergerfs, or just SnapRAID for important files. 

Fix Windows 10 Store error 0x80073CF9

With the release of Windows Subsystem for Linux 2, I wanted to try the new Windows Terminal, so I went to install it via Windows Store. But I kept getting this error Code 0x80073CF9 and none of the Microsoft solutions/Troubleshooters helped. It would download exactly 65 kB and then fail a few seconds later, every time, after many attempts and restarts. 

I hadn't used the Store much but other things had installed fine, like Ubuntu. This is not the only way to install this app, as it's open source, you could build and install it yourself, but why bother? Plus I might want to use Store for something else and didn't want a broken system. 

Long story short, it appears that the Terminal installer requires encryption to be supported (although it's free and open source!?), and somehow mine was not enabled. Even after doing a registry hack, when I restarted it was disabled again. I had to run these two commands, which were hard to track down, to get it to finally stick, and then I could install. 

Instructions: 

• Open Windows Powershell as Administrator (right-click menu) 
• Run: Set-ItemProperty HKLM:SYSTEM\CurrentControlSet\Control\FileSystem NtfsDisableEncryption -Value 0
• Run: fsutil behavior set disableencryption 0
• Restart
• Check that it's set properly (0) with: Get-ItemProperty HKLM:SYSTEM\CurrentControlSet\Control\FileSystem NtfsDisableEncryption

Now you should be able to install Windows Terminal and other apps throwing the same error. 

Links I used, which may help if this isn't your solution: 

• https://support.microsoft.com/en-ph/help/4027498/microsoft-store-fix-problems-with-apps
• https://answers.microsoft.com/en-us/windows/forum/all/windows-10-store-fails-with-0x80073cf9/b8ec2724-b087-4599-8651-0b35760962d6
• https://www.thewindowsclub.com/fix-windows-10-store-error-0x80073cf9/#:~:text=Error%20code%200x80073CF9%20denotes%20Severity,do%20to%20fix%20the%20problem.&text=Run%20an%20SFC%20scan%20as,to%20repair%20corrupt%20system%20files.&text=At%20times%2C%20downloads%20don't%20work%20right%20with%20wireless%20connections.
• This comment on the above got me going in the right direction, but was only half right: http://disq.us/p/273p2ke  
• https://www.thewindowsclub.com/error-0x80073cf9-this-app-couldnt-be-installed/
• Closer: https://github.com/microsoft/terminal/issues/4761
• Hooray! https://www.technipages.com/disable-windows-file-encryption-efs 

Add HTTPS to FreeNAS Nextcloud plugin

Using Let's Encrypt and certbot with auto-renewal.

I installed the Nextcloud plugin some time ago in FreeNAS and found it useful, but it had no support for HTTPS at all. Initially I only used it internally so that was okay-ish, but wanted to roam, and even internally certain integrations just won't work at all without it (WebDAV, see my previous post). So I had a working install and didn't want to have to redo it all, I just wanted to add HTTPS. Of course, it is rare to find a straightforward, and complete answer on the Internet to a specific circumstance, even one that seems like it should be common.

If I wasn't time-poor, or was starting from scratch and was aware of this limitation I'd probably have used this automation script or this guide to hardening instead, or even better, a Docker container with configuration as code, but for that to work I'd have to get Docker working again...

My instructions are partly from https://www.ixsystems.com/community/threads/nextcloud-lets-encrypt-nginx.72643/ but I did NOT allow SSH direct to the nextcloud host, I go through the FreeNAS host using jexec. You could also use the shell in the UI but it's painful.

Prerequisites:

• A domain name which you can point to your FreeNAS host, perhaps with a subdomain using a CNAME record. 
• Control over port forwarding to FreeNAS - you'll need to enable port 80 for certbot to work (though I am unsure if it needs to be to the Nextcloud instance, actually) 
• Basic command line knowledge including how to connect to the specific jail 

Brief steps:

• Connect to your nextcloud host command line - if you don't know how to do that, this guide is too brief for you, sorry :) 
• Allow installing packages: vi /usr/local/etc/pkg/repos/FreeBSD.conf and change no to yes. (Recommended to change it back later)
• You may want to install a more preferable editor than vi at this point. 
• Install certbot: pkg install py27-certbot
• certbot certonly --webroot -w /usr/local/www/nextcloud -d your-domain-name.com
• certbot renew --dry-run
• crontab -e and add this to regularly check and renew if necessary: 
• 0 0,12 * * * /usr/local/bin/python2.7 -c 'import random; import time; time.sleep(random.random() * 3600)' && /usr/local/bin/certbot renew --quiet
• Now you have a certificate and all the automation to keep it up to date!
• Next we add HTTPS listener and redirect from HTTP - this was missing from other instructions I saw. 
• edit /usr/local/etc/nginx/conf.d/nextcloud.conf
• There will just be a server entry for port 80. Break it up like this:
• server {
•   listen 80;
•   server_name _;
• 
  
•   return 301 https://$host$request_uri;
• }
• 
  
• server {
•   listen 443 ssl;
•   ssl on;
•   ssl_certificate "/usr/local/etc/letsencrypt/live/your-domain-name/fullchain.pem";
•   ssl_certificate_key "/usr/local/etc/letsencrypt/live/your-domain-name/privkey.pem";
• 
  
•   server_name _;
• 
  
• service nginx configtest
• If the above is ok: service nginx reload
• https://scan.nextcloud.com/ to check your security 
• Turn off the package repos again to reduce attack surface. 

It's probably a good idea to upgrade Nextcloud while you're there:

$ pkg update
$ pkg upgrade nextcloud-php71
$ cd /usr/local/www/nextcloud
$ su -m www -c "php ./occ upgrade"

Restricted SMB Share writing files as specific user/group

I have a FreeNAS server and had a need to make SMB (Windows) shares which would write files as a particular user on the server, but be available to only certain unix users. 

Specifically, I installed Nextcloud and wanted to be able to access the files and upload in bulk instead of via the web interface, or the integrated Explorer client which is convenient but only works if you want to synchronise files locally (ie keep a copy) and are happy to move everything there. 

Nextcloud writes files as user/group www/www so I had to have the SMB share write files as that user, but be accessible to my user or group. I realise that isn't the best security or auditing model, and I might have achieved the same with a complex group config between FreeNAS and the jail, but it seemed unnecessary for my home server. 

I also created the group nextcloud_files to define the FreeNAS users that could access this share. 

In short, these advanced auxiliary SMB settings worked:

• valid users = @nextcloud_files
• force user = www
• force group = www

I was able to mount the share as the Windows user and copy data into it, and it appeared as the www user. 

Additionally, I tried to set it so that the shares would only be visible to the users who could access them, with either or both of these options:

• hide unreadable = yes
• access based share enum = Yes

But that doesn't seem to work and probably requires user-level settings on FreeNAS which is not available in the web interface, and I don't like invisible manual configuration. 

Nextcloud note: it's not a good idea to side-load files into the storage, although they do appear in the web frontend, it hasn't attributed disk usage properly and I'm not sure if this means certain metadata won't be tracked. There just doesn't seem to be a nice way to pre-load it with GBs of photos.

To update the database, there is a server-side command to re-scan all the files, ie: 

root@nextcloud:/ # sudo -u www php /usr/local/www/nextcloud/occ files:scan joel
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see http://php.net/manual/en/book.pcntl.php
Starting scan for user 1 out of 1 (joel)
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 305     | 21041 | 00:21:53     |
+---------+-------+--------------+

This is not ideal. 

Turns out there is also a WebDAV address you can alternatively map instead of a windows share, which would solve this problem! 

... but Windows doesn't like it, no matter whether it's internal, external, port included or not. Of course. 

UPDATE: I solved this and described it in another post

Fix crackling sound with X-Fi on Ubuntu/Mint pulseaudio

May I never lose this link

https://guh.me/solving-creative-sound-blaster-x-fi-titanium-crackling-slash-distortion-on-linux

The trick is disabling PulseAudio’s timer-based scheduling. Fire up the terminal, then type:

su -c "nano /etc/pulse/default.pa"

Look for the line that contains load-module module-udev-detect, and append tsched=0 to it. Now it should look like this:

load-module module-udev-detect tsched=0

Save the buffer and exit the editor, then restart PulseAudio with the following commands:

pulseaudio -k
pulseaudio --start

Run a Raspberry Pi and USB hub off one power brick

Annoyed by having to take up so many power plug spaces for supposedly low power devices? 

What's going on here?

• Raspberry Pi needs minimum 700mA 
• Power goes into USB hub (2A 5V DC) 
• Hub provides Pi with power via micro-USB (white)
• Pi's USB connects to hub for devices 
• It works, trust me :) just don't use lots of USB devices that need power. 

Ignore the bit at the bottom with the ribbon cable; it's a Nordic radio attached to the GPIO pins for a wireless node project. 

This whole thing is then hung on the wall. Blu-tak holds the hub on the front, and the enclosure has slots on the back to slide it onto screw heads. After a while the hub very slowly slides down and then hangs from the cable; I think I'll have to add tape.

Solar power monitoring on Raspberry Pi/Linux

I've had solar panels installed since 2012 after my wife and I bought our first home. I have always been interested in monitoring our power usage as well as solar power production and trying to optimise/minimise use. It was relatively easy to get data from our Power-One Aurora 2kW photovoltaic inverter, and upload that via a Raspberry Pi (v2) to PVoutput.org.

A full day's live output after it was working again

This worked fairly well with only minor hiccups from mid-2014 through to early 2016 when the Pi started having regular issues, and eventually stopped booting altogether in August. The SD card basically just wore out, I used ddrescue to recover it with < 100 kB missing but apparently those were important bytes. I have finally gotten around to recovering it, having to reverse engineer my own setup, so I am documenting it here. I will also cover how to make it more resilient.

I stalled on monitoring the power usage for the house, but recently discovered some progress in this area, which can be a whole other post.

You will need

This guide is for my setup as follows, yours may differ:

• Aurora PV inverters with either
• an internal RS485 serial port connection (may void warranty? do your own research) 
• an external USB port (larger models, probably indoor only) - though I have not used one
• Relevant hardware to connect
• in my case I used a long Cat 5 ethernet cable to carry the signals from the inverter
• an RS485 to USB adapter
• Raspberry Pi or other Linux-based host you're going to connect it to, ideally low power
• With an 8GB+ SD card, a good one, not no-name brand
• Ideally one that does wear-leveling, but I was unable to find any reliable info
• With Raspbian with desktop installed, configured and updated
• You aren't afraid to install packages and compile things
• Posting data to PVoutput.org or wherever, but this part is all software and much easier, plus my setup also logs locally. 

If you wanted to use a Windows host, there are actually more software options for Aurora inverters such as PVBeanCounter, but I don't want to run anything 24/7 that runs Windows.

Caveats

• The ONLY ready-to-go software I could find which supported Aurora, ran on Linux, and posted to PVoutput.org is called Aurora Monitor which:
• hasn't been updated in years, but seems rock-solid anyway
• is open source (GPLv3)
• requires a desktop! Although all I wanted was to log the data, it has a GUI, so I have to run the desktop which makes this a little more complicated and inefficient :( 
• however you don't need to connect a monitor permanently 
• also logs its data to disk, at a rate of about 350 kB per day, so make sure you have spare space or setup log rotation. 
• The above means you need a desktop Linux distro, I used Raspbian
• ... and the current Raspbian with Desktop is 4.7GB so you need an 8GB SD card
• Don't put the Pi anywhere that's hard to get to as you may occasionally need to reboot it or repair the SD card
• RS485 (multidrop serial) is not the same as regular RS232 (single) serial. You DO need the right USB device, which basically converts it to RS232. On the plus side, if you have multiple inverters, you can daisy chain them on the same line, but I think the unit I have only supports one connection. 
• Mine is an FTDI idVendor=0403, idProduct=6001 "FT232R USB UART" 
• The first one I bought was the cheapest possible and didn't last long, get FTDI

Steps

Read all these instructions before you start to make sure you aren't missing or misunderstanding anything!

Connect cable to RS485 USB adapter

Here's a stock photo of the same type of adapter I have, for example search ebay for "USB To TTL RS485 Serial Adapter FTDI FT232RL". It simply has screw terminals for the wires. Keep track of which is which for Rx/Tx.

For more photos of my setup look further down.

Connect to PV Inverter

WARNINGS!

• You must have an electrician do this! 
• This may void your warranty! 
• Follow the manufacturer's instructions to turn off ALL power to the inverter at the meter box and then PV array! 
• Make sure any entrances to the device are sealed up waterproof and insect-proof!
• Leave the other end of the cable disconnected from anything and with no wires touching while you do this end! 
• Do your own research! Don't blame me if any of this breaks or costs you money!
• If you don't understand exactly what this is about and what all this means, just stop now! 

This photo shows the screw-terminals for the serial connection. 

1. Run an appropriately weather shielded cable with enough wires from wherever your Pi is going to live, in through the holes in the bottom (replace or retrofit the hole cover as needed)
2. Ensure it has proper strain relief (simple way: tie a loose knot around a small non-conductive object) so it won't pull out and has enough wire to easily reach, but not fill up the unit
3. Wire them up, ensure any spare wires are cut off and won't touch anything
4. The terminator switch is used for multidrop cables, should be enabled only on the last unit. I think it wasn't changed. 
5. Close up the box
6. Power on the unit following correct process (power off in reverse) 
7. Connect up the other end to your RS486 to USB adapter and ensure it works (may have to fiddle around with rx vs tx, etc) - exercise left up to reader
8. Power off again
9. Waterproof the cable entrance
10. Power back on

Compiling Aurora Monitor

These assume you are familiar with Raspberry Pi, Linux (Raspbian), and have that set up already.

1. Download Aurora Monitor to the Pi and unzip it 
2. It has binaries for 32 bit and 64 bit but you need to compile it for ARM
1. run "sudo apt-get install build-essential libwxgtk2.8-0 libwxgtk2.8-dev"
2. run "cd /src"
3. run "make"
4. wait about 15 minutes (no, really)
5. "auroramon" binary is built, you can move it wherever you want
3. Run desktop if you haven't already
4. Run auroramon and ensure it works, but of course it won't have data.

Set up software

This part can only be done when the sun is out and the unit is producing power.

For more help and screenshots see http://auroramonitor.sourceforge.net/ but here is a quick rundown:

1. Plug in your USB adapter and configure the program:
1. Serial port to use (check "ls /dev/tty*" it's probably ttyUSB0 if there's only one) 
2. Location (lat/long) for automatic adjustment to sunrise/sunset and azimuth
3. Solar panel direction and angle if known (I don't)
4. Etc etc 
2. Click Inverter Status in the menu to ensure it's connected, you should also see some numbers in the boxes at the bottom 
3. After a few minutes there should be some data points on the chart! 
4. Uploading to PVoutput.org: get a free account and go to Account Settings to get your API key
5. Wait a few minutes to see your live data appear on your chart online! 

Resiliency

• Ensure the desktop starts on boot: run "sudo raspi-config" and go through the menus
• Make auroramonitor start with the desktop: https://raspberrypi.stackexchange.com/a/45554/4226
• For the SD card: optionally, reduce the root partition size a bit to leave room for remapped bad sectors (not sure if this will actually work); I left 256 MB at the end, using GParted on my PC 

After 3 years my USB adapter was getting a bit dusty, so to protect it from dust, insects or damage I put it in a plastic enclosure:

See my awesome screw-terminal wire protection?

Classy container

Due to the frosted plastic it has a cool glow!

Potential improvements

• Back up the Pi regularly so you don't get in the same situation I did, making recovery harder
• If you want things to be tidier, and you don't mind sacrificing your Pi and also making debugging harder, just run the Pi inside your meter box with Wifi. You will need an electrician to install a power point though. 
• Ensure you don't expose any electronics, in Australia we have big spiders and geckos that love my meter box 
• A dedicated Arduino with RS485 adapter and ESP8266 wifi would be cheaper, entirely sufficient and maybe even more resilient if all you want to do is send data to PVoutput.org or anywhere else 
• Link this setup with emonCMS (openenergymonitor.org) and/or home-automation.io 
• If all your host does is log Solar, it might as well shut down overnight, you could use a shutdown script after dark (don't just power it off!!) and an external timer to turn it off and back on before dawn. 

Just putting this here so I never lose it... 

Spider-man brain tumor truck nemesis FAPPO!

Hidden trick to programming ATTiny with USBasp

Just a quick note, this might have been obvious to other more experienced users, but I thought I had bought some dead ATTiny85 chips on ebay. The first two had worked fine, and then the next three wouldn't respond. Then I got some brand new ATTiny84 chips and couldn't get any of them to work, so I knew there must be a trick.

The short answer it turns out, is that you need to short J3 on the USBasp. Mine didn't even have headers on it, so I had to solder on some pins. This makes it work for 1MHz chips, which takes 8x longer to write. Once the reprogrammable fuses are set for 8MHz you can remove this jumper and write much faster. This is most easily done by writing a program with the Arduino IDE or even just using "Burn Bootloader" which just writes an empty bootloader script and sets the fuses.

The chips should come from the factory preset to 1MHz and all the other defaults, but I guess the first two chips I tried had already been used, as I didn't need the jumper? I worried that the other chips might be dead, or had their fuses set so you couldn't write to them, and looked into devices for fixing them. I decided it was cheaper to just get more chips than such a device. So now I have a surplus!

There was no description of J3 on the USBasp silkscreen and there had been no instructions with the device. I had wondered what it did but assumed it was for special cases or to do with reflashing the device itself (as is J2).

I found this trick buried as a small note in an instructable, http://www.instructables.com/id/The-Idiots-Guide-to-Programming-AVRs-on-the-Chea/step4/Setting-up-your-not-so-micro-computer/

Hope this helps someone like me!

Migrating to SSD: a faster and more reliable method

I recently bought an SSD and discovered there are several posts about specifically migrating a Windows 7 install from HDD to SSD, which it turns out is a little more tricky than just one HDD to a larger one. The drive has to be "aligned" to sector boundaries like a HDD is, for maximum speed, TRIM needs to be enabled, the system needs to know it's an SSD, the boot sector needs to be restored, etc. These are things that Windows 7 will set up automatically when fresh installing, but when migrating, they have to be set up manually. So they go through a number of steps to make this happen.

I found that most of those options were already set correctly, but unfortunately it didn't seem to help, I couldn't get booting. The Windows installer couldn't find the drive to repair. So I came up with a simpler method which is sure to set up your SSD correctly and therefore get you up and running faster.

This is the short answer, if you're already familiar with this process:
* install Windows 7 on the SSD, letting the installer partition it
* clone your old system partition over the new SSD system partition
* done. (disconnect your old hard drive before booting from SSD the first time)

The longer version, with more general migration advice, follows.

For reference, these are the articles I used:
http://arga.wordpress.com/2010/04/24/how-to-properly-clone-a-windows-7-system-partition-with-clonezilla/
http://www.howtogeek.com/97242/how-to-migrate-windows-7-to-a-solid-state-drive/

Prepare:
* your favourite cloning utility on USB or CD (I used clonezilla on a live USB stick, but it isn't the most user-friendly if you're not familiar with hard drive partitioning)
* install the SSD in your computer, ensure you can access it - try formatting it and throwing some files on it. If a laptop, you'll have to use an external USB connector and swap drives after cloning.
* get your Windows 7 Install media ready


Method:

1. Make a fresh windows 7 install onto the SSD - it takes about 15 minutes. Delete the partitions on it and let Windows take over the partitioning, unless you have a particular scheme in mind. Ensure that you can boot to the desktop. 
1. Optionally clone the fresh install to an image in case you want it back some time later - but cloning probably takes as long as a fresh install anyway.
2. Clone your old partition onto the SSD's partition, overwriting the install you just did. Don't touch the boot sector or partition table.
3. Shut down, disconnect the old hard drive, so you don't boot the old one, and also so windows doesn't get confused about drive letters. You can reconnect it later, after booting the cloned partition once, if you have other partitions or whatever that you want to recover, or to wipe it, etc. 
4. Ensure BIOS is booting from SSD
5. Boot into windows on your SSD!
6. Ensure the system drive letter is C (if you boot with your old hard drive connected, it will keep that as C and the one you just booted off will get assigned another letter - which gets very confusing, and breaks some programs)
1. Optionally shut down and reconnect the old hard drive, in case you want more partitions, or to format and re-use it. 

So if all goes well, that should only take about an hour, including prepping a usb stick to boot clonezilla. 15 minutes to install fresh, 15-20 minutes to clone, plus a bit of time for rebooting, disconnecting and reconnecting hard drives, tweaking BIOS etc.

Let me know if you try this and if it works, or if not why not.

Electric placemats

My fiancee suggested a project for me - make placemats that would keep plates warm throughout dinner, and therefore keep the meal palatable (she really doesn't like cold food).

I see two ways to do this - induction heating, and a ceramic thermal mass with heating. Leaving aside the issues common to both, ie how much power would be needed, how to control the temperature, and running power to them on the kitchen table (or whether batteries are even feasible), let's examine the options.

Induction heating

There are some fancy stoves that do this. From memory, it basically electromagnetically induces a current in a metal body with AC, which heats up from all the eddy currents. This requires pots and pans which are suitable for induction, but on the other hand, it can be more safe because you can put a plastic container or your hand on the stove and nothing gets hot, as long as it isn't already hot from heating something before.
However, there are some downsides. Few plates are made of metal, so you'd need special plates, or rather a plate and platemat set, for every person eating. Cutlery is also metal, so it would probably heat up too - you wouldn't want to leave your cutlery on the plate for a few minutes, then pick it up again burning hot! Finally, you have to run AC to your placemats, which means thinner cables than DC, but more dangerous current to have around drinks and knives.


Heated ceramic

I imagine a ceramic insulator/thermal mass around nichrome wire, or tungsten heating element such as used in electric frypans. The base would need to be a good thermal insulator, possibly wood, to stop it from burning the table, which might make it rather thick. You could use AC or DC power. You can use any plates that can stand being warmed.


Let's summarise:

Induction heating - pros:
Safer heating mechanism
Heats faster
Thinner cables
Thinner placemat?

Induction heating - cons:
Requires plate + placemat set
Cutlery is heated too
More dangerous AC

Heated ceramic - pros:
Can use any suitable plates
Choice of AC or DC (DC would have thicker cables, but is safer)


Heated ceramic - cons:
Probably thicker placemats due to insulation
Takes more time to cool off
Heavier?


I'm not sure which of these approaches would use more power, or be more costly to produce. Then there is heating control, but if I solve my electric frypan heating problem first, I can use that solution.

I'm going to ask Hackerspace Brisbane for their thoughts. 

A better electric frypan heater control

Cooking is based on applying heat to food, ideally in a consistent, controlled way. Electric frypans suck at consistent heating. This is because they use a cheap thermistor, and dump all their power into the element until it reaches the set temperature, and then *turn off* completely, so your steak stops sizzling, until the temp drops enough and it turns the element back on again.

For that matter, some electric stoves also suffer the same problem, but because of their thermal mass the temperature doesn't drop so much.

Gas stoves are highly regarded by cooks, because you can finely control the amount of heating, and it doesn't turn itself on and off randomly at inconvenient times! But gas stoves are more complicated, expensive, require maintenance (replacing gas bottles), and some people are scared of using gas.

Why should electric heating work so badly?? Don't we control electricity? Isn't it supposed to make things more flexible?

Well there is a simple, well-known way to control electric power for motors, heating, etc: Pulse Width Modulation. I don't know why frypan makers don't use it, except that it's a bit more complicated and maybe more expensive. But you vary the % of time you are heating, and you can do it very fast. It changes the control mechanism from desired temperature to % of power to output. And you can do this pretty easily with a 555 timer IC, make the dial control the pulse width, and pass the AC current with a bigass MOSFET, SCR, or relay - solid state if possible, or there will be lots of clicking.

My friend Dylan points out that changing the dial to increase the power just means the % is increased, which is different from the existing functionality where it will dump all power until it reaches that temperature. In short, this means it will take a lot longer to reach the desired increase in temp (or a drop). I'm not sure how to approach this problem. Perhaps a hybrid thermistor and PWM solution, where some fuzzy logic figures out what PWM value to use to maintain the temperature, and nudges it up and down if the temp changes. That immediately brings to mind the idea of using a microcontroller like AVR (eg Arduino), instead of 555 Timer and so on, but I'd like to avoid that, at least in the final product. Still, it could be an interesting project!

Find MechWarrior: Living Legends servers

Unfortunately, not a lot of people regularly play this fantastic total conversion for Crysis and Crysis Wars. Whenever they release an update, a bunch of people flood to the servers to try it out, then drop off. This is especially bothersome in Australia, because you can't get a decent number of people (> 5) going on a local server. Thankfully the lag compensation is quite good because I am able to play on US and EU servers with ~230ms ping.

That said, you usually have to drop into the game just to see if anyone, anywhere, is playing. I found a simple way to find out how many people were playing: GameTracker. That might seem obvious, but it doesn't allow searching by mod. But it turns out that all the maps in MWLL start with TC_ or TSA_, and you can search by that.

2011-06-27 Update: As of the 0.5 release, there are now also Arena maps, which all start with SA_. So in GameTracker, you can find them *and* the TSA maps with the same search string. URLs below are updated. 

This results in the following two URLs to find all the servers in Crysis Wars:

http://www.gametracker.com/search/warhead/?search_by=map&query=TC_&sort=c_numplayers&order=DESC
http://www.gametracker.com/search/warhead/?search_by=map&query=SA_&sort=c_numplayers&order=DESC

There are almost no hits for Crysis, but here they are anyway:

http://www.gametracker.com/search/crysis/?search_by=map&query=TC_
http://www.gametracker.com/search/crysis/?search_by=map&query=SA_

Next step: extract this data from the website and re-present it. Or do a Javascript hack over the results. Write a script to monitor favourite servers for players and notify. Or any number of things. But this will allow me to poll the servers and jump in when I have a moment :)