Keeping Multiple Development Machines Synchronized with Homeboy

Most of us developer types have at least two machines we use routinely, and managing that can be a chore. Specifically, I usually want to do the following every time I sit down at a machine to hack on something:

  • Keep config files like .bash_profile and .gitconfig  synchronized. (This requires scripting since Dropbox ignores hidden files.)
  • Patch all OS-level libraries using the native package management system on OSX, Ubuntu and CentOS.
  • Update my database daemons and other system services.
  • Upgrade development libraries.
  • Merge in ‘git pull origin master’ project source code for all my clones.

Doing all this every time I hop to a different machines was chore, so I wrote a few BASH scripts to help. More importantly, I recently packaged them into a public GitHub and released it! Homeboy is a set of small, plain BASH scripts. After following the simply installation instructions, you just run homeboy every morning:

$ homeboy

This will run the updates you specify, though I’ve only included the stuff I use regularly: namely brew (OSX), rvm (OSX and Linux), apt (Ubuntu), yum (Red Hat/CentOS) etc. I ask that you submit pull requests to add support for updating Perl, Python, MacPorts etc. The synchronization mechanism works by zipping the specified list of files into a .zip in a synchronized directory managed by Dropbox, SugarSync etc. “Pushing” your current set of files to Dropbox is done via:

$ homeboy-push

After pushing, the next time `homeboy’ is run on any configured machine, the .zip file will be unzipped into your home directory. It’s really not complicated, but saves time by having to make the same change a bunch of times across different machines and platforms, and having subtle differences.

When using the git options, homeboy assumes you have a single directory where all your clones are kept, such as ~/Developer/git. Every subdirectory that looks like a git clone will have ‘git pull origin master’ run inside it.

Pretty silly stuff, right? But hey, all I do is run `homeboy’ every morning I plan on doing development work on a machine, and sip on a cup coffee while everything is brought up to date. 🙂

Please help test and submit pull requests!

TGen News Dashboard Widget For Mac OS X

 

Download now!

The Translational Genomics Research Institute (TGen) is a non-profit 501(c)(3) organization focused on developing clinically relevant medical diagnostics and smarter treatments related to genomic profiling. I’ve had the pleasure of working with the institute for over a year on a project related to breast cancer, and thought I’d share a widget I wrote keep you up to date on TGen’s latest news items.

The TGen News: Dashboard Widget for Mac OS X v1.1 gives you an always up-to-date set of headlines, pulled straight from the TGen news feed.

If you enjoy this widget, please donate a few dollars to the TGen Foundation and do your part to support the many causes that TGen pursues. Enjoy!

From the TGen website:

At TGen, investigators are pushing the limits of cutting-edge research and technology to discover the genetic cause of disease. Experiments that were impossible and impractical only a few years ago are now conducted every day.

Discovery fuels TGen’s translational research and lies at the heart of our scientific investigations. TGen’s research divisions are designed to foster a wide range of genetic discoveries. These divisions draw heavily upon TGen’s scientific platforms to expedite findings. TGen’s labs are staffed by teams of researchers focused on making genomic discoveries in common diseases and disorders in the areas of oncology, neurogenomics and metabolic disease.

Note: This free product is provided by Preston Lee, and is neither officially endorsed nor supported by TGen.

LaTeX Broken On MacPort “texlive” Update

I spent about 4 hours today troubleshooting a stupid, stupid issue with the “texlive +full” package as provided by MacPorts, which is required to run certain GUI LaTeX editors such as TeXShop. Several days ago I ran an innocuous-looking…

preston$ sudo port sync
preston$ sudo port upgrade outdated

…and amongst many things that updated, texlive was one of them. Unfortunately, running tools such as the command-line “latex” command or GUIs such as TeXShop started giving me the following error:

This is pdfTeX, Version 3.1415926-1.40.11 (TeX Live 2010/MacPorts 2010_0)
restricted \write18 enabled.
—! /opt/local/var/db/texmf/web2c/pdftex/pdflatex.fmt doesn’t match pdftex.pool
(Fatal format file error; I’m stymied)

After much fruitless searching and hair pulling, I stumbled upon this ticket that notes the solution as:

preston$ sudo fmtutil-sys –all

After running this, LaTeX seems to work properly again. Sheesh.

(Note to product developers: I would personally pay at least $50 for a simple, clean, stand-alone OS X app that does LaTeX really well. TeXShop and TextMate both do a good job, but I’d really like a 5-star “just works” editor that doesn’t install a bunch of system-level stuff like MacTeX. Just sayin’. 🙂 )

Upgrading From Ruby 1.9.1 to Ruby 1.9.2

I’ve spent half the day so far inbeded in the furious stressful upgrade process of a handful of Ubuntu Linux 10.04 and Mac OS X Snow Leopard systems from Ruby 1.9.1 to Ruby 1.9.2. I haven’t even gotten to the Rails 3.0.0 stuff yet: just the baseline Ruby installation. I’ve gone through the upgrade process on both types of systems so far and the base issues have been the same. Here’s a common issue that many people are running into:

preston$ gem1.9

/opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems/source_index.rb:68:in `installed_spec_directories’: undefined method `path’ for Gem:Module (NoMethodError)
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems/source_index.rb:58:in `from_installed_gems’
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems.rb:883:in `source_index’
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems/gem_path_searcher.rb:81:in `init_gemspecs’
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems/gem_path_searcher.rb:13:in `initialize’
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems.rb:841:in `new’
from /opt/local/lib/ruby1.9/site_ruby/1.9.1/rubygems.rb:841:in `block in searcher’
from <internal:prelude>:10:in `synchronize’
<…and so on…>

Assuming you’re upgrading from a previous Ruby installation, note that the “site_ruby” directories are no longer used, and will eff up your 1.9.2 installation if you fail to delete them after the install. On OS X, run:

sudo rm -rf /opt/local/lib/ruby1.9/site_ruby/

On Ubuntu Linux 10.04, run:

rm -rf /usr/local/lib/ruby/site_ruby/

…to correct this issue. All note that you may see errors such as this:

root@li92-132:~# rake –version

/usr/local/lib/ruby/1.9.1/rubygems.rb:340:in `bin_path’: can’t find executable rake for rake-0.8.7 (Gem::Exception)

from /usr/local/bin/rake:19:in `<main>’

…despite have a rake gem installed. Apparently 1.9.2 comes with a version of rake internally, but is unable to find it for some reason relating to the rake.gemspec file. Remove the file to fix this issue. On Ubuntu Linux 10.04, run:
rm /usr/local/lib/ruby/gems/1.9.1/specifications/rake.gemspec
Notice the “1.9.1” path of the PATH. Yeah.. it’s weird. But for compatibility reasons your 1.9.2 installation will continue to use a path with 1.9.1. To quote the Ruby 1.9.2 FAQ page:
The standard library is installed in /usr/local/lib/ruby/1.9.1
This version number is “library compatibility version”. Ruby 1.9.2 is mostly compatible with the 1.9.1, so its library is installed in the directory.
I’m sure there’s a wonderful technical reason for this, but it’s still misleading and confusing as hell. I ended up manually deleting a bunch of stuff I shouldn’t have because I thought I was innocently “cleaning up” after the old version. Whatever. Additional suggestions:
  • Just to keep things clean, you may also want to remove your old Ruby 1.8.x builds. (I recommend doing so unless you have older apps that haven’t moved to 1.9.x yet.)
  • Phusion Passenger seems to work fine on Ubuntu 10.04 with the latest version of Apache 2 as of this writing, though don’t forget to recompile, reinstall, reconfigure and restart apache2 when you do so.
  • Check if you still need rack v1.0.1 installed (for older Rails app) before nuking everything. 🙁
I need a beer!

3D OSX Applications With Ruby-Processing Screencast

A two-part screencast series demonstating two different 3D ruby-processing applications. A slide presentation from 2009 created for the Phoenix Ruby Group is also attached as bonus material. Enjoy!

Part 1: Starfield

Demo of a 3D starfield simulation written in pure Ruby, running on the JRuby runtime as a nicely packaged .app program for Mac OS X. (Runs on Snow Leopard and Leopard.)

Part 2: Twiverse

3D Twitter client written in pure Ruby, running on the JRuby runtime as a nicely packaged .app program for Mac OS X. (Runs on Snow Leopard and Leopard.)

Slide Show

Recovering A Corrupt OpenLDAP Database On OSX Server

Recovering A Corrupt OpenLDAP Database On OSX Server
Last night we noticed some services provided by an OSX Leopard Server instance were not working correctly. The iChat, AFP and Web services were not authenticating. In Server Admin.app, the “Overview” tab of the Open Directory service reported…
LDAP Server is: Not Running
Password Server is: Running
Kerberos is: Not Running
Looking at the server error logs through Console.app, the following was occuring every 10 seconds..
com.apple.launchd[1] (org.openldap.slapd[27382]) Exited with exit code: 1
com.apple.launchd[1] (org.openldap.slapd) Throttling respawn: Will start in 10 seconds
The slapd daemon appeared not to be starting. Jumping to the command line, I tested the configuration using the `slapd -Tt` command.
core:openldap admin$ sudo /usr/libexec/slapd -Tt
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
bdb(dc=openrain,dc=com): PANIC: fatal region error detected; run recovery
bdb_db_open: Database cannot be opened, err -30978. Restore from backup!
bdb(dc=openrain,dc=com): DB_ENV->lock_id_free interface requires an environment configured for the locking subsystem
backend_startup_one: bi_db_open failed! (-30978)
slap_startup failed (test would succeed using the -u switch)
http://discussions.apple.com/message.jspa?messageID=9548971
With a little research, I concluded that..
The OpenLDAP database had been corrupted, and..
The `slapd_db_recover` tool (as present on some Linux installations) is instead named `db_recover`. Ah!
After carefully backing up the /var/db/openldap folder, I ran the recovery tool and re-tested the configuration..
core:openldap admin$ sudo db_recover -h /var/db/openldap/openldap-data/
core:openldap admin$ sudo /usr/libexec/slapd -Tt
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
config file testing succeeded
The errors in Console.app stopped, and the Server Admin.app panel started reporting..
LDAP Server is: Not Running
Password Server is: Running
Kerberos is: Not Running
I had to restart the AFP, iChat and Web services on the machine to get everything working again, but all seems well now.
Last night we noticed some services provided by an OSX Leopard Server instance were not working correctly. The iChat, AFP and Web services were not authenticating. In Server Admin.app, the “Overview” tab of the Open Directory service reported…
LDAP Server is: Not Running
Password Server is: Running
Kerberos is: Not Running
Looking at the server error logs through Console.app, the following was occuring every 10 seconds..
com.apple.launchd[1] (org.openldap.slapd[27382]) Exited with exit code: 1
com.apple.launchd[1] (org.openldap.slapd) Throttling respawn: Will start in 10 seconds
The slapd daemon appeared not to be starting. Jumping to the command line, I tested the configuration using the `slapd -Tt` command.
core:openldap admin$ sudo /usr/libexec/slapd -Tt
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
bdb(dc=openrain,dc=com): PANIC: fatal region error detected; run recovery
bdb_db_open: Database cannot be opened, err -30978. Restore from backup!
bdb(dc=openrain,dc=com): DB_ENV->lock_id_free interface requires an environment configured for the locking subsystem
backend_startup_one: bi_db_open failed! (-30978)
slap_startup failed (test would succeed using the -u switch)
With a little research, I concluded that..
  1. The OpenLDAP database had been corrupted, and..
  2. The `slapd_db_recover` tool (as present on some Linux installations) is instead named `db_recover`. Ah!
After carefully backing up the /var/db/openldap folder, I ran the recovery tool and re-tested the configuration..
core:openldap admin$ sudo db_recover -h /var/db/openldap/openldap-data/
core:openldap admin$ sudo /usr/libexec/slapd -Tt
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
overlay_config(): warning, overlay “dynid” already in list
config file testing succeeded
The errors in Console.app stopped, and the Server Admin.app panel started reporting..
LDAP Server is: Running
Password Server is: Running
Kerberos is: Running
I had to restart the AFP, iChat and Web services on the machine to get everything working again, but all seems well now.

iCal Domain Account Errors For New Events

Over the past couple weeks I’ve had issues getting my OSX 10.5 iCal client to continue working properly with our centralized CalDav server. I stopped being able to invite other domain users to my events as well as reserve “locations”, despite all my personal (non-domain) calendars continuing to work properly. I noted these iCal errors in Console.app…
*** -[NSConcreteTextStorage attributesAtIndex:longestEffectiveRange:inRange:]: Range or index out of bounds
CalDAVOperationQueue tried to dequeue operation <CalDAVScanDropBoxQueueableOperation: 0x174eb6a0> but it was not at the front of the queue.
When I tried to delete my domain account within iCal’s preferences, the application hung. When restarted I could no longer bring up the preference dialog and saw this error repeated in Console…
*** -[NSURL initWithString:relativeToURL:]: nil string parameter
Apparently deleting everything in ~/Library/Calendars and starting fresh is one solution. I have years worth of notes and interesting tidbits that I need to keep, however, so simply deleting all my data was not an option. With some educated guesswork, trial and error, I discovered that the following steps seems to make everything work without apparent data loss or corruption..
Quit iCal.
Go to ~/Library/Calendars and backup the entire directory, just in case.
Delete all “Calendar Cache” files as well as any directory ending in “.caldav”
Start iCal. It may give you a progress dialog about “Upgrading Calendars”. I think this means it’s rebuilding the cache file.
Go to “iCal -> Preferences…” and delete/readd your domain account.
Wait for the domain account to resync and you should be go to go.
Hope this helps!

appleOver the past couple weeks I’ve had issues getting my OSX 10.5 iCal client to continue working properly with our centralized CalDav server. I stopped being able to invite other domain users to my events as well as reserve “locations”, despite all my personal (non-domain) calendars continuing to work properly. I noted these iCal errors in Console.app…

*** -[NSConcreteTextStorage attributesAtIndex:longestEffectiveRange:inRange:]: Range or index out of bounds

CalDAVOperationQueue tried to dequeue operation <CalDAVScanDropBoxQueueableOperation: 0x174eb6a0> but it was not at the front of the queue.

When I tried to delete my domain account within iCal’s preferences, the application hung. When restarted, I could no longer bring up the preference dialog and saw this error repeated in Console…

*** -[NSURL initWithString:relativeToURL:]: nil string parameter

Apparently deleting everything in ~/Library/Calendars and starting fresh is the easiest solution. I have years worth of notes and interesting tidbits that I need to keep, however, so simply deleting all my data was not an option. With some educated guesswork, trial and error, I discovered that the following steps seems to make everything work again without apparent data loss or corruption..

  1. Quit iCal.
  2. Go to ~/Library/Calendars and backup the entire directory, just in case.
  3. Delete all “Calendar Cache” files as well as any directory ending in “.caldav”.
  4. Start iCal.
  5. It may give you a progress dialog about “Upgrading Calendars”. I think this means it’s rebuilding the cache file.
  6. Go to “iCal -> Preferences…” and delete/readd your domain account.
  7. Wait for the domain account to resync and you should be go to go.

Hope this helps!

Installing The MySQL Ruby Gem When Using The Official MySQL .dmg

If you installed MySQL using the official .dmg package provided by MySQL AB/Sun and are trying to install the native MySQL gem, you’ll need to pass a bunch of flags to the gem install process to get the thing to compile. Try this..

sudo gem install mysql — –with-mysql-config=/usr/local/mysql/bin/mysql_config

..if you still have issues (check your /opt/local/lib/ruby/gems/<ruby_ver>/gems/mysql-<gem_ver>/mkmf.log file), try this instead..

sudo gem install mysql — –with-mysql-dir=/usr/local/mysql –with-mysql-include=/usr/local/mysql/include/ –with-mysql-lib=/usr/local/mysql/lib/ –with-mysql-config=/usr/local/mysql/bin/mysql_config

Run `sudo gem install mysql` and definitely recheck the log file again as the gem install process sometimes reports false positives even if native components fail to install.

Virtual OS X Server Screenshots

Apple now allow you to virtualize OS X Server instances. While your virtualization options are limited, it’s very easy to set up on your existing OS X Server.

This is an virtualized OS X Leopard Server guest running in Parallels Server on a host OS X Leopard Server. You can see that the guest system is treated similarly to other Windows and Linux VMs in the Parallels Management Console.

Note that a distinct serial number/license seems to be required. The serial number for the host machine will not validate (I thought Apple was going to allow one VM instance???), so to use the sweet service configuration tools available in Server Admin, it appears you’ll need a separate license for now.

 

Redmine w/OS X OpenLDAP, Parallels Server and JumpBox

OpenRain used a slew of crappy Trac sites for issue tracking until we switched to Redmine several days ago. The decision came because..

  • Redmine can authenticate off LDAP with trivial configuration.
  • Redmine has multi-project support out-of-the-box.
  • Redmine has some nifty Gantt chart and calendaring schwag and is generally better.
  • Parallels Server (for OS X) is finally available.
  • JumpBox has a beta Redmine VM image available.

If you’ve got an existing LDAP infrastructure, the whole shebang shouldn’t take more than an hour or two to set up.

  1. Install Parallels Server on your OS X Leopard server.
  2. Download the Redmine JumpBox. Generate a new MAC address and boot it. Do the one-page configuration thingy in your browser.
  3. Log into Redmine and create a new “Authentication Mode” set to LDAP. If you’re using the default OpenLDAP schema that ships with Leopard server, enter the attributes like so..redmine.png
  4. All your users should now be able to log into your Redmine JumpBox using their LDAP credentials! You’ll have to set up your projects, ACLs etc. within Redmine, but that’s some pretty hot shizzle to get running in such a small timeframe.

Mad props to Redmine, Parallels, JumpBox and Apple for further simplifying my business.