Ruby Supercomputing: Using The GPU For Massive Performance Speedup

Confreaks just released a Ruby/JRuby GPU presentation of mine from Mountain West Ruby Conference 2011 in Salt Lake City, Utah. Slightly stale, but all still very relevant to HPC today!

http://confreaks.tv/videos/mwrc2011-ruby-supercomputing-using-the-gpu-for-massive-performance-speedup

“My Videotag” Linkbucks WordPress Hack

FYI to all the WordPress fans.. If you recently noticed that your links are getting hijacked by URLs pointing to “linkbucks.com”, try deleting your my-videotag plugin. I found this nastiness somehow embedded in the source code:

$ find . -name ‘*.php’ |xargs grep -e linkbuck
./wp-content/plugins/my-videotag/my-videotag.php: return ‘<a href=”http://77aafbe6.linkbucks.com/url/’ . $matches[2] . ‘//’ . $matches[3] . ‘”‘ . $matches[1] . $matches[4] . ‘ target=”_blank”>’ . $matches[5] . ‘</a>’;

Ack! We’ll be deleting the my-videotag plugin now. 🙂

Prolog for Ruby (ruby-prolog) Updated, Hits v1.0.1

After a long period of inactivity I’ve updated the F/OSS ruby-prolog gem! It’s been updated for ruby 2.0.0, bundler, and minitest, and released as v1.0.1!

ruby-prolog allows you to solve complex logic problems on the fly using a dynamic, Prolog-like DSL. Basic use is encompassed by stating basic facts using your data, defining rules, and then asking questions. Why is this cool? Because ruby-prolog allows you to leave your normal object-oriented vortex on demand and step into the alternate reality of declarative languages.

With ruby-prolog:

  • There are no classes.
  • There are no functions.
  • There are no variables.
  • There are no control flow statements.

You can use all these wonder things — it’s still Ruby after all — but they’re not needed, and mainly useful for getting data and results into/out of the interpreter. Prolog still tends to be favored heavily in artificial intelligence and theorem proving applications and is still relevant to computer science curricula as well, so I hope this updated release proves useful to the Ruby community.

Check out a simple ACL enforcement example or, for the daring, the ruby-prolog solution to the Tower’s of Hanoi problem!

 

Don’t Upgrade vCenter Server Appliance From 5.0 to 5.1

Thinking of upgrading your vCenter Server Appliance from a 5.0 image to the 5.1 image using the official VMware upgrade process covered in the documentationDon’t do it. It fails somehow. Every time.

Screen Shot 2013-06-27 at 3.08.32 PM

 

Unexpected error during the upgrade process.

To attempt to upgrade again, restore the old vCenter Server Appliance and external database from backup/snapshot, deploy the new vCenter Server Appliance again and start the upgrade process from the beginning.

Yeah.. you read the right. Now both your old and new appliance images are b0rked, and you’ll need to restore from the vCenter 5.0 backup that you definitely made before attempting the upgrade. In private emails, VMware has acknowledged the issues and has advised to stick with 5.0 and hold off on upgrading the vCenter Server appliance until the next major release since the upgrade/migration tools are such horrible crap. Time to sit back and wait it out!

 

Update: The built-in update tool in 5.0 that allows you to apply point release upgrades is also fairly broken. (See the red “Failed to install..” message in the background?)

Screen Shot 2013-06-27 at 3.26.14 PM

Vudu Disc-To-Digital Review

For those of you that just received this promo email…

vudu_promotion

I decided to give it a try with stack of old DVDs. The promo pricing is totally reasonable, but I doubt it’ll last. (50% off when you “convert” 10+ discs, then $2 off your total.) Your existing DVD/BD titles are $2 to convert, or $5 for upgrading a DVD to HDX. For the content I successfully converted, HDX quality is great. Some big caveats to considered before jumping in, though..

  • The Vudu To Go client necessary for the disc matching and verification process is buggy. On my Windows 8 Pro laptop — the only machine I have with a BD player — Vudu To Go would check for BD titles but outright refuse to check normal DVDs, displaying only a nasty error message. I had to use a secondary OSX machine (with a non-BD DVD player) to check DVD titles. I didn’t have any issues with my shopping cart when using two machines, but this was really inconvenient. I would think Vudu To Go on Windows 8 Pro would have the bugs worked out by now.
  • The disc-to-digital disc matching mechanism seemed to mis-match about 1 in 15 titles, such as my retail BD version of “Full Metal Jacket”.
  • Of my properly-recognized discs, Vudu only had rights to convert ~40% of the discs I tried. Understandable, but still a pretty low hit rate for users, especially for anyone like me that would really like to completely toss all discs in the trash, and don’t even have a DVD/BD player hooked up anymore.
  • For HDX quality, both machine and display *must* support HDCP. For example, “owning” Super 8 in HDX would only play in SD when web streaming due to my non-HDCP monitor.
  • Studios, of course, still place restrictions on your watching abilities, even though you’ve verified the disc. For example, I “own” 80’s comedy “Singles” in HDX, but “[t]his title is viewable on PC in SD only.” Lots of stupid crap like that.
  • Web streaming requires flash and uses a non-trivial amount of CPU. On my new brand new Dell Latitude 10 ST2 Win8 RT tablet, it’s totally unusable.
I hope Amazon launches an equivalent, because I’m already committed to an Amazon content library and really, really, really don’t want to keep another vendor. If you only have a few stacks of decent titles just taking up space, though, it’s worth considering!

Capistrano Hanging During .tgz Upload

I just spent an annoying amount of time troubleshooting a new cap-based deploy.rb for a Rails app to a new CentOS 6.4 server. The deploy.rb worked perfectly on other systems, but something about the configuration was causing `cap deploy’ to hang during the following:

servers: [“myapp.example.com”]
** sftp upload /var/folders/0j/vmvt2by53m901wtmfv8xfjw40000gn/T/20130530225953.tar.gz -> /tmp/20130530225953.tar.gz

Long story short, when manually sftp’ing to the server, I’d get:

Received message too long 458961005

Ah ha! After taking a cue from this dude, I edited the .bashrc to redirect any output to /dev/null. I could then successufully sftp to the server manually, and `cap deploy’ magically worked as expected. The offending line was actually rvm, which of course insists on print a color-coded message when you `rvm use 2.0.0′ (or whatever). Changing the offending line to:

rvm use 2.0.0 >> /dev/null

..did the trick.

Keeping Multiple Development Machines Synchronized with Homeboy

Most of us developer types have at least two machines we use routinely, and managing that can be a chore. Specifically, I usually want to do the following every time I sit down at a machine to hack on something:

  • Keep config files like .bash_profile and .gitconfig  synchronized. (This requires scripting since Dropbox ignores hidden files.)
  • Patch all OS-level libraries using the native package management system on OSX, Ubuntu and CentOS.
  • Update my database daemons and other system services.
  • Upgrade development libraries.
  • Merge in ‘git pull origin master’ project source code for all my clones.

Doing all this every time I hop to a different machines was chore, so I wrote a few BASH scripts to help. More importantly, I recently packaged them into a public GitHub and released it! Homeboy is a set of small, plain BASH scripts. After following the simply installation instructions, you just run homeboy every morning:

$ homeboy

This will run the updates you specify, though I’ve only included the stuff I use regularly: namely brew (OSX), rvm (OSX and Linux), apt (Ubuntu), yum (Red Hat/CentOS) etc. I ask that you submit pull requests to add support for updating Perl, Python, MacPorts etc. The synchronization mechanism works by zipping the specified list of files into a .zip in a synchronized directory managed by Dropbox, SugarSync etc. “Pushing” your current set of files to Dropbox is done via:

$ homeboy-push

After pushing, the next time `homeboy’ is run on any configured machine, the .zip file will be unzipped into your home directory. It’s really not complicated, but saves time by having to make the same change a bunch of times across different machines and platforms, and having subtle differences.

When using the git options, homeboy assumes you have a single directory where all your clones are kept, such as ~/Developer/git. Every subdirectory that looks like a git clone will have ‘git pull origin master’ run inside it.

Pretty silly stuff, right? But hey, all I do is run `homeboy’ every morning I plan on doing development work on a machine, and sip on a cup coffee while everything is brought up to date. 🙂

Please help test and submit pull requests!

Ubuntu 12.10 to 13.04 Server Upgrade Error

Are you Googl’ing around trying to figure out why you’re getting this error when trying to `do-release-upgrade’ your Ubuntu 12.10 system, even though you’re pretty much up to date?

root@mia:~# do-release-upgrade
Checking for a new Ubuntu release
Traceback (most recent call last):
File “/usr/bin/do-release-upgrade”, line 145, in <module>
fetcher.run_options += [“–mode=%s” % options.mode,
AttributeError: type object ‘DistUpgradeFetcherCore’ has no attribute ‘run_options’
Error in sys.excepthook:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/apport_python_hook.py”, line 137, in apport_excepthook
os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o640), ‘wb’) as f:
OSError: [Errno 2] No such file or directory: ‘/var/crash/_usr_bin_do-release-upgrade.0.crash’

Original exception was:
Traceback (most recent call last):
File “/usr/bin/do-release-upgrade”, line 145, in <module>
fetcher.run_options += [“–mode=%s” % options.mode,
AttributeError: type object ‘DistUpgradeFetcherCore’ has no attribute ‘run_options’
root@mia:~#
deb http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted

I just burned a couple hours trying to figure out why I only got this error on one specific server. It turns out that it’s a bug in the version of  the ubuntu-release-upgrader-core package, and you must be pulling updates from a “quantal-updates” repository to get the fixed package when you `apt-get upgrade’. To fix this, edit your /etc/apt/sources.list and make sure you have the following two lines:

deb http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted
deb-src http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted

After making sure you have these lines, get yourself updated again and re-try the release upgrade script:

apt-get update
apt-get upgrade
do-release-upgrade

…and you should be rollin’ towards Raring!

Proxy Authentication Against Devise-Based Ruby Applications

(Quick GitHub link!)

My team at TGen has a Rails application using a typical devise + cancan installation for authentication and authorization, respectively, and a related Apache HTTPD proxy server we needed to authenticate against active accounts in the webapp before passing traffic to their destination, which is a single host. We tried mod_authn_dbd for a while, but the combination of Devise’s default adapive bcrypt password encryption that is not supported by mod_authn_dbd by default, a complex set of SQL statements to process the authorization directly, and that none of the existing modules seem to be able to do what we want, made it a messy ordeal.

Instead, we created devise-proxy: a simple proxy server written as Rack Middleware that you can deploy using Passenger within Apache or nginx. You simply provide devise-proxy with a config.yml file defining the hostname and port of the devise webapp to use as an authentication web service, and hostname and port of a single server to use as a forwarding target.

Clients connecting to the proxy use the email/password for their proxy username/password credentials. All authenticated clients will be forwarded to your provided host/port, but using the path in the client’s original HTTP request. Clients failing to authenticate will receive a 403 and the forwarding host will never be hit.

gem install devise-proxy # to install

git clone git://github.com/preston/devise-proxy.git # to clone!

devise-proxy is released under a BSD license via the GitHub project, here. Good luck!