Blog

  • Vudu Disc-To-Digital Review

    For those of you that just received this promo email…

    vudu_promotion

    I decided to give it a try with stack of old DVDs. The promo pricing is totally reasonable, but I doubt it’ll last. (50% off when you “convert” 10+ discs, then $2 off your total.) Your existing DVD/BD titles are $2 to convert, or $5 for upgrading a DVD to HDX. For the content I successfully converted, HDX quality is great. Some big caveats to considered before jumping in, though..

    • The Vudu To Go client necessary for the disc matching and verification process is buggy. On my Windows 8 Pro laptop — the only machine I have with a BD player — Vudu To Go would check for BD titles but outright refuse to check normal DVDs, displaying only a nasty error message. I had to use a secondary OSX machine (with a non-BD DVD player) to check DVD titles. I didn’t have any issues with my shopping cart when using two machines, but this was really inconvenient. I would think Vudu To Go on Windows 8 Pro would have the bugs worked out by now.
    • The disc-to-digital disc matching mechanism seemed to mis-match about 1 in 15 titles, such as my retail BD version of “Full Metal Jacket”.
    • Of my properly-recognized discs, Vudu only had rights to convert ~40% of the discs I tried. Understandable, but still a pretty low hit rate for users, especially for anyone like me that would really like to completely toss all discs in the trash, and don’t even have a DVD/BD player hooked up anymore.
    • For HDX quality, both machine and display *must* support HDCP. For example, “owning” Super 8 in HDX would only play in SD when web streaming due to my non-HDCP monitor.
    • Studios, of course, still place restrictions on your watching abilities, even though you’ve verified the disc. For example, I “own” 80’s comedy “Singles” in HDX, but “[t]his title is viewable on PC in SD only.” Lots of stupid crap like that.
    • Web streaming requires flash and uses a non-trivial amount of CPU. On my new brand new Dell Latitude 10 ST2 Win8 RT tablet, it’s totally unusable.
    I hope Amazon launches an equivalent, because I’m already committed to an Amazon content library and really, really, really don’t want to keep another vendor. If you only have a few stacks of decent titles just taking up space, though, it’s worth considering!
  • Capistrano Hanging During .tgz Upload

    I just spent an annoying amount of time troubleshooting a new cap-based deploy.rb for a Rails app to a new CentOS 6.4 server. The deploy.rb worked perfectly on other systems, but something about the configuration was causing `cap deploy’ to hang during the following:

    servers: [“myapp.example.com”]
    ** sftp upload /var/folders/0j/vmvt2by53m901wtmfv8xfjw40000gn/T/20130530225953.tar.gz -> /tmp/20130530225953.tar.gz

    Long story short, when manually sftp’ing to the server, I’d get:

    Received message too long 458961005

    Ah ha! After taking a cue from this dude, I edited the .bashrc to redirect any output to /dev/null. I could then successufully sftp to the server manually, and `cap deploy’ magically worked as expected. The offending line was actually rvm, which of course insists on print a color-coded message when you `rvm use 2.0.0′ (or whatever). Changing the offending line to:

    rvm use 2.0.0 >> /dev/null

    ..did the trick.

  • Keeping Multiple Development Machines Synchronized with Homeboy

    Most of us developer types have at least two machines we use routinely, and managing that can be a chore. Specifically, I usually want to do the following every time I sit down at a machine to hack on something:

    • Keep config files like .bash_profile and .gitconfig  synchronized. (This requires scripting since Dropbox ignores hidden files.)
    • Patch all OS-level libraries using the native package management system on OSX, Ubuntu and CentOS.
    • Update my database daemons and other system services.
    • Upgrade development libraries.
    • Merge in ‘git pull origin master’ project source code for all my clones.

    Doing all this every time I hop to a different machines was chore, so I wrote a few BASH scripts to help. More importantly, I recently packaged them into a public GitHub and released it! Homeboy is a set of small, plain BASH scripts. After following the simply installation instructions, you just run homeboy every morning:

    $ homeboy

    This will run the updates you specify, though I’ve only included the stuff I use regularly: namely brew (OSX), rvm (OSX and Linux), apt (Ubuntu), yum (Red Hat/CentOS) etc. I ask that you submit pull requests to add support for updating Perl, Python, MacPorts etc. The synchronization mechanism works by zipping the specified list of files into a .zip in a synchronized directory managed by Dropbox, SugarSync etc. “Pushing” your current set of files to Dropbox is done via:

    $ homeboy-push

    After pushing, the next time `homeboy’ is run on any configured machine, the .zip file will be unzipped into your home directory. It’s really not complicated, but saves time by having to make the same change a bunch of times across different machines and platforms, and having subtle differences.

    When using the git options, homeboy assumes you have a single directory where all your clones are kept, such as ~/Developer/git. Every subdirectory that looks like a git clone will have ‘git pull origin master’ run inside it.

    Pretty silly stuff, right? But hey, all I do is run `homeboy’ every morning I plan on doing development work on a machine, and sip on a cup coffee while everything is brought up to date. 🙂

    Please help test and submit pull requests!

  • Ubuntu 12.10 to 13.04 Server Upgrade Error

    Are you Googl’ing around trying to figure out why you’re getting this error when trying to `do-release-upgrade’ your Ubuntu 12.10 system, even though you’re pretty much up to date?

    root@mia:~# do-release-upgrade
    Checking for a new Ubuntu release
    Traceback (most recent call last):
    File “/usr/bin/do-release-upgrade”, line 145, in <module>
    fetcher.run_options += [“–mode=%s” % options.mode,
    AttributeError: type object ‘DistUpgradeFetcherCore’ has no attribute ‘run_options’
    Error in sys.excepthook:
    Traceback (most recent call last):
    File “/usr/lib/python3/dist-packages/apport_python_hook.py”, line 137, in apport_excepthook
    os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o640), ‘wb’) as f:
    OSError: [Errno 2] No such file or directory: ‘/var/crash/_usr_bin_do-release-upgrade.0.crash’

    Original exception was:
    Traceback (most recent call last):
    File “/usr/bin/do-release-upgrade”, line 145, in <module>
    fetcher.run_options += [“–mode=%s” % options.mode,
    AttributeError: type object ‘DistUpgradeFetcherCore’ has no attribute ‘run_options’
    root@mia:~#
    deb http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted
    deb-src http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted

    I just burned a couple hours trying to figure out why I only got this error on one specific server. It turns out that it’s a bug in the version of  the ubuntu-release-upgrader-core package, and you must be pulling updates from a “quantal-updates” repository to get the fixed package when you `apt-get upgrade’. To fix this, edit your /etc/apt/sources.list and make sure you have the following two lines:

    deb http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted
    deb-src http://gb.archive.ubuntu.com/ubuntu/ quantal-updates main restricted

    After making sure you have these lines, get yourself updated again and re-try the release upgrade script:

    apt-get update
    apt-get upgrade
    do-release-upgrade

    …and you should be rollin’ towards Raring!

  • Proxy Authentication Against Devise-Based Ruby Applications

    (Quick GitHub link!)

    My team at TGen has a Rails application using a typical devise + cancan installation for authentication and authorization, respectively, and a related Apache HTTPD proxy server we needed to authenticate against active accounts in the webapp before passing traffic to their destination, which is a single host. We tried mod_authn_dbd for a while, but the combination of Devise’s default adapive bcrypt password encryption that is not supported by mod_authn_dbd by default, a complex set of SQL statements to process the authorization directly, and that none of the existing modules seem to be able to do what we want, made it a messy ordeal.

    Instead, we created devise-proxy: a simple proxy server written as Rack Middleware that you can deploy using Passenger within Apache or nginx. You simply provide devise-proxy with a config.yml file defining the hostname and port of the devise webapp to use as an authentication web service, and hostname and port of a single server to use as a forwarding target.

    Clients connecting to the proxy use the email/password for their proxy username/password credentials. All authenticated clients will be forwarded to your provided host/port, but using the path in the client’s original HTTP request. Clients failing to authenticate will receive a 403 and the forwarding host will never be hit.

    gem install devise-proxy # to install

    git clone git://github.com/preston/devise-proxy.git # to clone!

    devise-proxy is released under a BSD license via the GitHub project, here. Good luck!

  • How To Custom Brand The OpenStack “Horizon” Dashboard

    I’m deploying OpenStack “Essex” on Ubuntu Server 12.04, and have the openstack-dashboard package installed to provide the web-based “Horizon” GUI component newly added for the Essex release. Canonical also provides an openstack-dashboard-ubuntu-theme package that brands the Python-based Django GUI. Despite that the last major Canonical-maintained packages based on the OpenStack “Diablo” release in Ubuntu 11.10 did not include an administrative GUI, Horizon — as a standalone component — has been very stable for a mainstream debut. In the future, though, I’d like to see a quick and easy way to change the default branding to use your own logo, colors, and titles using only the GUI’s administrative screens.

    The horizon documents briefly mention branding customization to give you a head start, but you probably want more specific steps. Here’s my custom-branded Horizon dashboard with custom colors, logo, and site title:

    Once you know where to make the appropriate changes, it’s super simple. Step-by-step:

    1. Create a graphical logo with a transparent background. The text “TGen Cloud” in this example is actually rendered via .png files of multiple sizes I created with a graphics program. I used a 200×27 for the logged-in banner graphic, and 365×50 for the login screen graphic.
    2. Set the HTML title (shown at the top of the browser window) by adding the following line to /etc/openstack-dashboard/local_settings.py
      SITE_BRANDING = "Example, Inc. Cloud"
    3. Upload your new graphic files to:
      /usr/share/openstack-dashboard/openstack_dashboard/static/dashboard/img/
    4. Create a new CSS stylesheet — we’ll call ours custom.css — in the directory:
      /usr/share/openstack-dashboard/openstack_dashboard/static/dashboard/css/
    5. Edit your CSS file using the following as a starting point for customization, which simply overrides the Ubuntu customizations made in the ubuntu.css file. Change the colors and image file names as appropriate, though the relative directory paths should be the same.
      /*
      * New theme colors for dashboard that override the defaults:
      *  dark blue: #355796 / rgb(53, 87, 150)
      *  light blue: #BAD3E1 / rgb(186, 211, 225)
      *
      * By Preston Lee <plee@tgen.org>
      */
      h1.brand {
      background: #355796 repeat-x top left;
      border-bottom: 2px solid #BAD3E1;
      }
      
      h1.brand a {
      background: url(../img/my_cloud_logo_small.png) top left no-repeat;
      }
      
      #splash .login {
      background: #355796 url(../img/my_cloud_logo_medium.png) no-repeat center 35px;
      }
      
      #splash .login .modal-header {
      border-top: 1px solid #BAD3E1;
      }
      
      .btn-primary {
      background-image: none !important;
      background-color: #355796 !important;
      border: none !important;
      box-shadow: none;
      }
      
      .btn-primary:hover,
      .btn-primary:active {
      border: none;
      box-shadow: none;
      background-color: #BAD3E1 !important;
      text-decoration: none;
      }
    6. Open the following HTML template in an editor:
      /usr/share/openstack-dashboard/openstack_dashboard/templates/_stylesheets.html
    7. Add a line to include your new stylesheet: (I’ve highlighted the new line in bold.)
      ...
       <link href='{{ STATIC_URL }}bootstrap/css/bootstrap.min.css' media='screen' rel='stylesheet' />
       <link href='{{ STATIC_URL }}dashboard/css/{% choose_css %}' media='screen' rel='stylesheet' />
       <link href='{{ STATIC_URL }}dashboard/css/custom.css' media='screen' rel='stylesheet' />
      ...
    8. Restart apache just for good measure:
       sudo service apache2 restart
    9. Reload the dashboard in your browser and fine tune your CSS appropriate.

    You’re done!

    [ezcc]

  • OpenStack Nova Essex MySQL Database Schema Diagram and SQL

    I’m in the process of setting up an OpenStack Essex installation on a small cluster of Ubuntu 12.04 servers using the Canonical-supplied and supported packages. I innocently and unfortunately ended up with a state corruption issue causing nova-compute to crash on startup, and found it necessary to dive into the schema as a troubleshooting aid. I used MySQL workbench to reverse-engineer the schema into an EER diagram as well as raw SQL. I’m sure others will find this useful as the OpenStack community slowly upgrades to Essex. Hope it helps!

    [Download diagram, SQL, and MySQL workbench file]

  • OpenStack Diablo on Ubuntu 11.10 w/MySQL

    If you’re installing the OpenStack cloud computing platform “Diablo” release on Ubuntu 11.10 server using the official guide and are using MySQL, don’t forget to:

    sudo apt-get install python-mysqldb

    The “glance” package (and probably others) are configured to use SQLite by default and do not automatically install this dependency by default. Unfortunately, the November 2011 version of the manual also fails to mention this and the log file (/var/log/glance/registry.log) prints a variety of “INFO [sqlalchemy.engine.base.Engine…” messages but no warnings or errors. I assume a corresponding comment applies to PostgreSQL as well but have not confirmed.

  • Quick FASTQ File Parsing Via Memory Mapping In C/C++

    I recently had a need to speedily parse through 8GiB+ .fastq text files to calculate a simple statistic of genomic data. My initial “pfastqcount” implementation in Ruby worked fine, but with many files to process took longer than I had hoped in addition to consuming an alarming amount of CPU. I ended up reimplementing the pfastqcount command-line program in C, which takes one or more .fastq files, memory maps them, and creates the statistic. Simply dropping my algorithm down to raw C significantly sped up the process and reduced CPU usage, especially coming from an interpreted language. If any of you bioinformaticians find the need to implement a FASTQ data processing algorithm in C, I encourage you to fork the project and use it as a template. The project is Apache 2.0 licensed for your convenience and publicly available on GitHub.

  • Hacking Your Own Genome

    Many thanks to everyone that participated in my Hacking Your Own Genome session at today’s Desert Code Camp 2011.2 event in Chandler, Arizona! I’m very passionate about the topic area, and hope the session was both entertaining and useful. Here are the presentation materials, source code for the “youandme” application, and a few other links you might find useful. Thanks again for the opportunity and don’t hesitate to reach out and stay connected!

    Presentation:

    Additional Links:

    Have fun!