Nuby Ruby / Rails Tales

Cucumber, Capybara, and Webkit testing for Ajaxified actions

2012-01-23T10:04:00.000-08:00

I recently upgraded my testing stack and now use Capybara and Webkit with Cucumber (and RSpec, too). I ran into a number of problems and found this post by Mathew O'Riordan incredibly helpful. He tracks the history of his research and testing through a github project and at various points on the blog entry shows the commit for the changes he just discussed: kudos on that approach!

A couple of gotchas that I ran into along the way:

First, you may need to add the following line to your features/support/env.rb file:

require 'capybara-webkit'

O'Riordan doesn't do this in his project, but I got an error:

"no driver called :webkit was found, available drivers: :rack_test, :selenium (Capybara::DriverNotFoundError)"

and found this discussion which gave me the solution.

Second, although it may be obvious, I missed the fact that you MUST put

@javascript

before each scenario that requires in-browser javascript. I had thought this was just a way of grouping scenarios together, but it actually triggers the use of the webkit driver to run the step.

Solving one IE rendering problem...

2011-01-27T12:29:00.000-08:00

I know this has been posted elsewhere, but since I recently ran into the issue and had to do some digging and coding to work around it, I thought I'd post it for others to find.

While working with a drop-down select box in a form, I wanted to be able to show/hide different sections of the form depending on the value of the selection (I'm currently using the Prototype javascript library for this project). It all worked fine in every browser except IE (surprise... NOT!). IE was not setting the "value" property of the select box.

After some testing, I found that IE DID properly set the selectedIndex attribute, so I could use that to get IE to work like I wanted it to; here's the code:

MHC.select_display = function(select_id, select_value, select_number, toggle_id) {
/***
* NOTICE IE6 properly sets the selectedIndex value but does NOT set the value property; therefore, we have to give both values to this routine
* to handle both browsers.
* alert("Hi - I've been changed! / " + select_id + " / " + select_value + " / " + toggle_id + " / " + $(select_id).selectedIndex + " / " +  $(select_id).options[$(select_id).selectedIndex].value + " / " + $(select_id).value) + " / " + $(select_id).options.value;
**/     
if(Prototype.Browser.IE) {
  if($(select_id).selectedIndex == select_number) {
    $(toggle_id).appear();
    }
  else {
    $(toggle_id).hide();
    }
  }
else {
  if($(select_id).options[$(select_id).selectedIndex].value == select_value) {
    $(toggle_id).appear();
    }
  else {
    $(toggle_id).hide();
    }
  }
};

While I don't know this for a fact, I'm guessing that other problems with IE form fields (such as checkboxes and the "checked" attribute) may be due to the timing of updates to the attributes; you may be able to find alternative attributes that you can interrogate to get things done.

KPackagekit / aptitude woes... solved

2011-01-25T11:52:00.000-08:00

Recently I was trying to make some updates to my Kubuntu 10.04 machine (where I do all my Rails development) and ran into a serious problem: both KPackagekit (KDE package manager) and aptitude were unable to even start.

KPackagekit gives:

"an unknown error has occurred"

"The package list needs to be rebuilt. 
This should have been done by the backend automatically"

aptitude gives:

E: Read error - read (5: Input/output error)
E: The package lists or status file could not be parsed or opened.

Turns out it was a "simple" problem -- Kpackagekit and aptitude are both based on the Debian APT package. That package manager has a built-in limit as to the amount of cache space that it supports. If you exceed that limit (by having a bunch of repositories and packages stored on disk in /var), then APT and its descendents just give up. The error messages above are indicative of this kind of error (and certainly aren't very descriptive).

The problem is that - once you get to this point - you can't do much of anything. Attempts to run either command just fail.

Searching for KPackagekit info yielded nothing useful, but when I searched on aptitude and its error message, I found the solution: increase the cache limit.

Here's the source of this information. The fix is simple; just run this command and then you should be OK.

sudo apt-get update -o APT::Cache-Limit=25165824

If you need, you can increase the Limit even further... it's only 24 megabytes which is nothing on today's systems.

Excel spreadsheets & Ruby => :tricky

2011-01-13T17:05:00.000-08:00

I've been working on an ERP data conversion project, using Ruby to convert the data before it gets dumped into the Oracle database. I only recently got involved to help out, and an earlier decision mandated that we produce our converted data in spreadsheets [I know, I know -- not what I would have decided, but they wanted folks to be able to easily manipulate the data].

Anyway, we're using the Spreadsheet gem which is pretty nice; however, it does have some limitations, which can result in oblique error messages such as the following:

ruby customer_fix_zip.rb
/Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:376:in `validate!': OLE2 signature is invalid (Ole::Storage::FormatError)
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:368:in `initialize'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:110:in `new'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:110:in `load'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:77:in `initialize'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:83:in `new'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/ruby-ole-1.2.11.1/lib/ole/storage/base.rb:83:in `open'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/spreadsheet-0.6.5.0/lib/spreadsheet/excel/reader.rb:1144:in `setup'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/spreadsheet-0.6.5.0/lib/spreadsheet/excel/reader.rb:121:in `read'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/spreadsheet-0.6.5.0/lib/spreadsheet/excel/workbook.rb:32:in `open'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/spreadsheet-0.6.5.0/lib/spreadsheet.rb:62:in `open'
    from /Users/jseidel/.rvm/gems/ruby-1.9.2-head/gems/spreadsheet-0.6.5.0/lib/spreadsheet.rb:68:in `open'
    from customer_fix_zip.rb:12:in `'

Googling for help didn't... there are only a few posts out there about this situation, but not many responses.

This may be caused by a password on the spreadsheet -- at least that was the cause in my case. Once I had removed the password protection, everything worked just fine. Hopefully, this will help other folks out there who are experiencing the same problem.

Custom Form Builders in Rails...

2010-09-25T07:50:00.000-07:00

I've started working on a new Rails application and needed to display chunks of spreadsheet-like text for financial data (e.g., Profit & Loss Statement and Balance Sheet). Everything went fine (well, not so fine: money fields were a bit of challenge, but more about that later) until I started displaying my data in a view. As you can imagine, there are a lot of cells for these displays and after some prototyping, I realized that the default size of 30 for an input field was just too big. Of course, that's easy to solve by just adding ":size => 15" to the text_field definition, but the prospect of adding that to numerous fields and than probably having to change them all left me pretty grumpy. That's when I figured that a Custom Form Builder would be the way to go.

As usual, I headed off to the Rails API but found little there and much of the external documentation on how to use them was either way over-complicated for what I wanted to do, or going in another direction. I did find a few good resources, including the following:

1. Ryan Bates' "Mastering Rails Forms" episode 3 - Great presentation; worth every penny of the $5 cost per episode.
2. APIDoc description of form_for method - APIDoc is your very good friend.
3. Alex Reisner's "Anatomy of a Form Builder" - Best introduction of the details behind Form Builder I found.
4. Ruby Pond's Semantic Form Builder - Lots of complex changes, nicely done
5. Aldenta post with metaprogramming example - Shows how to replace a slew of methods with metaprogramming; compare to Ruby Pond example where they are all hand-crafted.

Because of their often complicated solutions and the fact that there were different ways of doing things, I found it challenging to extract the minimum that I would have to do make a simple change. In the end, after much reading and trial coding, Ryan Bates' "Mastering Rails Forms, episode 3" completed my current understanding of the process.

Here are the key points that I learned which finally helped me get through the morass and make it work for me. In the discussion that follows, CustomFormBuilder is my class name.

To start, this is the code that I wanted to change...
from this:

<td>f.textfield :name, :size => 12</td>

to this:

f.textfield :name

Not a big change at all, but with many lines like this, I thought this would be a nice, easy way to learn about custom form builders.

Here's what I learned that helped me understand how it all works.

1) the basic form builder code looks like this:

class CustomFormBuilder < ActionView::Helpers::FormBuilder
 ...your code...
end

There may be other approaches out there that will work, but this is the most straight-forward way I have found.

You want to be able to reference your custom form builder easily, as in:

form_for :person, :builder => CustomFormBuilder do |f|
...form code...
end

It turns out there are ways to customize even this, so that your code looks like this:

custom_form_for :person do |f|
...form code which automatically uses your CustomFormBuilder
end

but that's for a little further down the page.

2) There is more than one way to implement the logistics of a custom form builder (reminds me of Perl's TMTOWTDI and I am rapidly coming to the conclusion that this applies to Rails everywhere). This was one of the things that tripped me up at first, since different posts give different instructions on how to create your builder. By logistics I mean where do you put the code that will be your form builder so that you can reference it easily?

You can put it in a helper file... either in application_helper.rb or controller_helper.rb. The basic code layout is as described above... The really important thing about including this code in an existing helper file is that the code MUST come after the end statement that terminates the helper code! Failing to do this will result in your form builder not being found (this one cost me a few hours) or your having to reference it as:
```
:builder => ApplicationHelper::CustomFormBuilder
```
You can put it in a separate file in the helpers directory, with a name that matches the class name, i.e., custom_form_builder.rb
You can put it anywhere you like (for example, like Ryan Bates does, in a separate directory that you create called app/form_builders). In this case, however, you must add that directory to Rails' default search path by adding the following to your environment.rb file (just uncomment and modify the skeleton statements that's already there):
```
config.load_paths += %W( #{RAILS_ROOT}/app/form_builders)
```

3) You can make your CustomFormBuilder the default by adding the following to the bottom of your environment.rb file. Doing this means that you can leave off the :builder => option in your form_for:

ActionView::Base.default_form_builder = CustomFormBuilder

However, I think I agree with Ryan Bates that this is not such a great idea -- it hides what's going on and could make it difficult for some one new to your code to figure it out.
[Note]: Other authors recommend doing this in app/helpers/application_helper.rb and one author explicitly stated that the environment.rb approach would not work because helpers don't get initialized until they are actually called/used {could this be a version-dependent issue?]

4) Altlernatively, you can make a helper method that encapsulates the CustomFormBuilder and makes it explicit that you've got some extra magic going on. Do this in your Application Helper file:

def custom_form_for(*args, &block)
  options = args.extract_options!
  options.merge(:builder => CustomFormBuilder)
  form_for(*(args + [options]), &block)
end

Again, Ryan Bates in his "Mastering Rails Forms" episode 3, had the clearest description that I have found but there is an essentially identical representation at http://apidock.com/rails/ActionView/Helpers/FormHelper/form_for and a pretty good description of the whole form builder approach.

5) The code to actually do this very simple change is also very simple:

def text_field(name, options = {})
  options.reverse_merge! (:size => 12)
  @template.content_tag(:td, super)
end

The "reverse_merge" is so that I can override the default size of 12 with another value coded in the form

6) You'll see in some of the examples that the method signature is given with *args instead of option as I have shown it. That's a more general approach, but requires that you know a little more about how Rails can manipulate the arguments; see below. Suffice it to say that it works for my simple example, but if you're going to get serious about making more complicated customm form builders, you should get familiar with the following approaches.

extract_options! is a Rails method that will -- not surprisingly -- pull the options hash out of the method arguments. Very useful for the general method signature which uses *args.

merge! and reverse_merge! are methods that will merge and overwrite the receiving hash with the results. So, for example:

options.merge!(:size => 12)

will add :size => 12 to the options hash, while:

options.reverse_merge!(:size => 12)

will add :size => 12 to the options hash ONLY if there is no :size element already there.

Note the bang (!) on each of these methods -- very important. If you use the non-bang version, you won't update the options hash and things just won't work.

content_tag is a method on the @template object (provided to you by the FormBuilder) that wraps selected text in the given HTML tag. Very useful for building your output string.

There's lots more, but that should be enough to get you started -- hopefully a little easier than I did. Now that I understand some of these basics, looking at the more complicated examples is much easier for me.

Firing up a new Kubuntu machine for Rails development...

2010-08-24T17:57:00.000-07:00

Well... it finally happened.

My Kubuntu machine which I prefer to use for Rails development was getting pretty crochety. I had started out on 7.04, did a distribution upgrade to 8.04, ran into some major problems installing DropBox (LinuxFormat Magazine helped me fix this problem). Finally, I tried a distribution upgrade to 9.10 and it was barely limping along. Several folks suggested a clean install from scratch, so I decided to go with Kubuntu, 10.04, 64-bit. Here's how I went about it.

Prior to the installation, I took a tar backup of my system and also did a copy (cp -a ...) to an external USB drive. Be careful with cp: if you have other media attached (like another network-attached drive I had a /media/wdhd1), the command "cp -a / /" will follow all those links and try to get EVERYTHING onto the USB drive. I wound up simply copying over the directories that I was most interested in gaining quick access to, such as: /home, /etc, /root, /rails-app. If you've got an old system that you're going to replace, pick the directories that you don't want to lose.

BTW, I really like the NDAS combo drives: eSata, Network-Direct Attached, USB. Reasonably-priced and very effective.

First get Kubuntu (or whatever flavor of Linux distro you want). I went to the Kubuntu site at http://www.kubuntu.org/getkubuntu where you can either download directly or get a CD/DVD. I downloaded the 64-bit desktop version here: http://www.kubuntu.org/getkubuntu/download#download-block
Install kubuntu - pretty straight-forward.

Make sure that your machine will boot from a CD
Insert CD, reboot, follow the instructions.
A true linux sysadmin might have gotten fancy with the disk partitioning, but I just went with the default which was everything in a single partition. It’s not like this is a heavily-loaded server...right?
There were a number of prompts for keeping or replacing existing files (such as mysql/mycnt, /etc/apcupsd/apcupsd.conf, /etc/cups/cupsd.conf, etc...) and I just took the default since I had copies of all my previous config files.
At one point, my mouse went inactive (I connect it through a KVM switch) but that cleared up pretty quickly.

Install your favorite browser by using Kubuntu’s built-in Konqueror browser. I chose Chrome -- I’ve been using Firefox for years, but in the past, I’ve run into problems with Firefox on Linux: older versions, conflicts when trying to upgrade... I still use FireFox on Windows/Mac, but Chrome is my preferred browser for Linux.
The Ubuntu community has a very good site on how to do the Rails install: https://help.ubuntu.com/community/RubyOnRails

Make sure you DO NOT do “sudo apt-get install rails”. This gives you an old version and you can’t track your gems easily. If you mistakenly do that, simply type “sudo apt-get remove rails” and go back to the above link
REMEMBER to add the new ruby gems path to your path so that your system can find the stuff you just installed (this is in the . The command is:
“export PATH=/var/lib/gems/1.8/bin:$PATH”
You’ll have to add this to your .bashrc file so that it will always be done when you start up.

BTW, I add the following two aliases to my .bashrc so that I can quickly and easily edit .bashrc and update my environment for any new values

alias brcs='source /home/jseidel/.bashrc'
alias brcvi='vi /home/jseidel/.bashrc'

The ‘brcvi’ alias drops me into vi (replace with your preferred editor) so that I can make changes. Notice that it always points to my home directory so that I can do this regardless of where I am.
the ‘brcs’ alias “sources” the current/updated .bashrc file. I.e., it “Reads and executes commands from .bashrc in the current shell environment”, with the result that the changes you made are reflected in your current bash environment. So to add a new alias, I create the alias in my current bash, test it out, then brcvi to add it to my .bashrc and then brcs to make it active and then test to make sure I didn’ have a typo.

Install git -- you are using git, aren’t you? GitHub has a great tutorial on how to do this. Just go to: http://help.github.com/linux-git-installation/ and follow the instructions. There's also an excellent book, Pro Git, by Scott Chacon, that is available on line at http://progit.org/book/.
Once you’ve installed git, you have to generate an SSH keypair (see http://en.wikipedia.org/wiki/Secure_Shell if you want to learn more about SSH and http://en.wikipedia.org/wiki/Public-key_cryptography if you want to learn more about keypairs) so that you can speak encrypted git. GitHub has another excellent tutorial on how to do this; see http://help.github.com/linux-key-setup/. Just follow the instructions there and you should have no problem. I did install the xclip utility -- makes it easy to copy/paste the information into github.

If you don’t want to key-in your SSH passphrase everytime you use it, GitHub has help here as well. Go to http://help.github.com/working-with-key-passphrases/ and about half-way down the page there’s a section on “autolaunching ssh-agent” when you start your machine so that you only have to enter the passphrase once when you boot your machine.
Turns out I had some permissions problem in my .ssh directory: I kept getting
“Bad owner or permissions on /home/jseidel/.ssh/config”
when testing things with “ssh git@github.com”. Well, GitHub was there again, with their trouble-shooting page at http://help.github.com/troubleshooting-ssh/. Towards the bottom of the page in the “SSH Config” section, the two ‘chmod’ commands did the trick for me.
chmod 700 ~/.ssh chmod 600 ~/.ssh/*
And don’t forget to setup your git config options -- at least these two
git config --global user.name "your name"
git config --global user.email your@email.com

I use RubyMine (http://www.jetbrains.com/ruby/download/index.html#linux for a 30-day free trial) so that was the next thing to install. Replace this section with whatever IDE you prefer: Aptana, NetBeans, ...

To run RubyMine, you have to have the Sun Java (JDK 1.6). I found that at http://www.oracle.com/technetwork/java/javase/downloads/index.html and followed the instructions. It turned out to be completely straight-forward.

Now you’ve got everything ready to do; let’s create your first rails app in your shiny new Linux development machine:
   rails MyApp
   cd MyApp
   git init
   git commit -m "Empty Rails application"
Now create your github repository -- GitHub has the answer again: http://help.github.com/creating-a-repo/

Once this is done, copy in the repository URL

Back to bash and hooking up your new repository with your new rails app

Tell your local git repository about the github repository:
git branch add origin
Push your MyApp rails application up to github:
git push origin master

And you should be good-to-go -- happy rails to you!

RMagick (finally) installed on Kubuntu...

2010-05-26T17:23:00.000-07:00

Start with the very well-done RMagick installation FAQ at: http://rmagick.rubyforge.org/install-faq.html

./configure runs OK, then make throws this error:
"error: ‘png_info’ has no member named ‘trans’"

After much searching and trying, ran into several other errors, and finally realized I had to install the ImageMagick development support package

And there's a huge dependency list:

Went to KPackageKit to get the development version which installed correctly...

But it still failed in RMagick installation because it (RMagick) needed a later version (I had 6.5.4 and Rmagick 12.13.1 wanted 6.5.9)

Went to the ImageMagick home page and downloaded and installed their latest version (6.6.1-7) and that installed correctly...

But now when trying to run the ImageMagick convert program (as a quick test), I got another error message:

"error while loading shared libraries: libMagickCore.so.3: cannot open shared object file: No such file or directory"

Here's the key to getting ImageMagick to finally run... which I finally stumbled upon at http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=15702 after getting that error message. Put the following two lines in your .brashrc (or equivalent) and now it works... Obviously something to do with default paths and where ImageMagic was installed.

export LDFLAGS="-L/usr/local/lib -Wl,-rpath,/usr/local/lib"
export LD_LIBRARY_PATH="/usr/local/lib"

Hopefully, some of these pointers will help someone.

Unbelievable...

Changing erroneous git commit message...

2010-04-29T17:35:00.000-07:00

Just the other day, I was making a few small changes to my new site and noticed -- just before I pushed them to github -- that I had created a bad commit message: somehow I had copied the text from the previous commit. Not Good. I figured that would be easy to change and -- while it turned out to be quite easy -- it wasn't easy to find how to do it. Googling under various phrases kept getting me things like:

"...just use git rebase -i and 'edit' the commit." While that's quite true, unless you really get git, it's like the old Microsoft helicopter joke. (I suppose if I really read through the excellent documentation provided for git, I would have been able to figure it out, but that's a lot of reading for a quick fix.)

I finally had some help from my friend Wolf Arnold and with a little bit of trial and error, I was able to modify my commit message and then push my 3 commits to github. Here's how to do it. NOTE: This procedure only works if you have not pushed your commits. Once pushed, the commit message is inviolable - by design - as far as I can tell.

First, make sure you know which commit you want to change. I used gitk (for Linux; gitx on the Mac), which gives you a visual display of your commits and the status of the remotes you are pushing to. Find the relative position of the commit you want to change. In my case, it was the 2nd on the list -- that is, one down from the top.
In your root project directory (where you normally type your "git status" and other commands), type the following:
```
git rebase -i HEAD~n
```
where 'n' is equal to your commit's position on the list, counting from 1 at the top of the list. Notice that the character between HEAD and n is the "twiddle" character or "tilde".

Once you enter that command, you are presented with a screen (in your favorite editor) that looks something like this (the numbers following 'pick' are the internal ids for your commits):

pick <lots of numbers>
pick <lots of different numbers>
# Rebase 9a122e0..9a122e0 onto 9a122e0
#
# Commands:
#  p, pick = use commit
#  e, edit = use commit, but stop for amending 
#  s, squash = use commit, but meld into previous commit 
# # If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

At this point, you can do one of several things:
1. Edit the file and change the word "pick" to "edit" on the top line of the file. That's the commit that you want to change. Save the file and exit the editor. That will set you up to make the changes in your commit.
2. Delete every line in the file and save/close it. That's equivalent to a No-Op" and will terminate the rebase.
3. There are more options, as you can see from the instructions in the file itself, but we'll leave that for another day.
When the edit finishes, you will receive instructions at the command line. The next step in the process is to run the command:
```
git commit --amend
```
and you will again be presented with an edit screen where you can change the commit message (or other things, but I didn't get into that either). Just change the line with the commit message to what you want it to be.
Once that's done and you've saved/closed the file, run the command:
```
git rebase --continue
```
and it'll be done. You can use gitk to check things out and then push to whatever remote you're using.

Enjoy...

Authlogic login error messages...

2010-04-27T11:42:00.000-07:00

I've been using autthlogic to provide login/signup features for my new web application, and I really like it. It provides lots of great features and has a fair number of plugins and tutorials to add functionality.

A few weeks past, I added a password reset feature. This was quite straight-forward: I just followed the password-reset tutorial and was up & running pretty quickly.

However, when I did some additional testing on the basic authlogic functionality, I noticed that it took a very simplistic approach to providing error messages for login failures: it would report "Invalid email address" or "Invalid password" if either of those fields were in error. This is not good security: an attacker could determine a valid email address and then use it for social engineering to gain further information.

Support to fix this is already in Authlogic -- it's just not the default behavior. You have to add the following method call to your UserSession class:

     class UserSession < Authlogic::Session::Base

       generalize_credentials_error_messages true

     end

More details can be found at: http://railsapi.com/doc/authlogic-v2.1.3/classes/Authlogic/Session/Password/Config.html#M000178

This post is mostly about collecting the various sources of information about Authlogic -- it's such a popular Rails plugin, that there are many search results, some of them dead or not very helpful. I spent almost an hour googling to find the right stuff, and figured I'd post what I found in case it might save someone else some time.

General github/authlogic documentation page: http://rdoc.info/projects/binarylogic/authlogic. The "documentation" link sends you to an RDoc Info page: the main page is an overview, and you get more detail by clicking on the 3 buttons in the upper right-hand corner: "Namespace list", "Method list", and "File list".

More documentation can also be found at: http://railsapi.com/doc/authlogic-v2.1.3/ .

All in all, really good stuff!

Using PostgreSQL

2010-04-14T04:40:00.000-07:00

Now that PostgreSQL is installed and running on my Mac, I need to do something useful with it, like create my database and get it migrated through Rails. The terminology and approach is a tad different from MySQL, but after a bit of googling, I found this PostgreSQL documentation page which got me rolling. This page on creating a database was actually the starting point, because it worked through the various errors which might occur.

The key difference for me was that the PostgreSQL superuser is called 'postgres', not 'root' as in MySQL. Prior to digging out this piece of information, everything I tried got the following error:

createdb: could not connect to database postgres: 
FATAL:  role "jseidel" does not exist

The other thing that was a minor roadblock was getting out of psql (PostgreSQL's interactive SQL command program). The usual MySQL/bash suspects (quit; exit; bye; {Ctrl-C}) didn't work. With psql you either type "\q" or {ctrl-D} and you're out.

Finally, since I'm using MacPorts, the location of the PostgreSQL files are different than documented in the PostgreSQL guide: my files are rooted off /opt/local/lib/postgresql84 instead of /usr/local/pgsql as is indicated in that guide.

Now I'm able to create my database and see if it connects with Rails...

Getting PostgreSQL to run on a Mac...

2010-04-12T18:24:00.000-07:00

So I decided to switch to PostgreSQL from MySQL. First off, it's what's used by default at Heroku, and I'm so far loving Heroku, its great integration with git, and the resulting ease of deployment. Secondly, I'm a tad concerned about what might happen in the future since Oracle now owns MySQL: could be 'interesting'.

Anyway, I'd always heard that installing PostgreSQL was more challenging than installing MySQL: "MySQL put more effort into making things easier for first-time users" is what my friends have told me. But I also heard that PostgreSQL had better and more standard SQL support (for example, support for foreign keys).

Anyway, with a bit of searching, I finally found this RubyOnRails page which seemed to do a great job of laying things out for all 3 platforms. Seemed pretty straight-forward:

Install the server:

sudo port install postgresql83-server

Start the server:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.postgresql83-server.plist

Setup the database:

sudo mkdir -p /opt/local/var/db/postgresql83/defaultdb
sudo chown postgres:postgres /opt/local/var/db/postgresql83/defaultdb
sudo su postgres -c '/opt/local/lib/postgresql83/bin/initdb -D /opt/local/var/db/postgresql83/defaultdb'

Two problems:

It failed installing the database -- complaining that it couldn't find the header file libpq-fe.h.
It installed version 8.3 and version 8.4 is now current.

More searching led me to the MacPorts site which has the required library:

sudo port install libpqxx

However, it also wound up installing the 8.4 version of PostgreSQL, but version 8.3 still is the one that gets launched... seems to work, but certainly not as clean as I would like.

Given the problems I had in the past before I cleaned my machine and installed Ruby & Rails "from scratch" with MacPorts (see my previous post on this issue) as I could easily get PostgreSQL with relatively little hassle.

So far, so good...

PS Check out "Enterprise Rails" (Dan Chak, O'Reilly). He does a really nice job of discussing Rails' scaling and reliability issues and does it all in the context of PostgreSQL. Good book!

MacPorts headaches... solved!

2010-03-22T10:26:00.000-07:00

So I finally found a way to solve all those MacPorts headaches, thanks to Erik Michaels-Ober, a friend of a friend. As I and many folks have learned, Mac OS X is less than standard *Nix when it comes to setting up their directories, and it doesn't take kindly to co-creation of a parallel directory tree. MacPorts puts things in the (*Nix) standard /opt directory, but once that's done, you get path and name clashes.

I've seen bits and pieces of a solution as noted in my previous posts, but they really didn't fix the problem. And of course, there's the issue that's been reported elsewhere that upgrading to Snow Leopard can wreak havoc on a development setup.

So following Eriks' instructions, here's what I did:

I wiped my system clean and installed Snow Leopard from scratch. You may be able to just do an upgrade, but I chose to go with a clean system.
Install the XCode Developer Tools that come with Snow Leopard (or download the latest version from the Apple website. Warning: it’s a big file).
Remove all the pre-installed rubygems from your system. You can do this using the “gem uninstall” command, or brute force. I chose the brute force method, but only after creating a copy before I did the deletion, just in case. Once I knew it worked, I got rid of my copies.rm -rf /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8/*
rm -rf /Library/Ruby/Gems/1.8/*
Install MacPorts (see the MacPorts installation page).
Make sure you’re running MacPorts version 1.8.2 (type: port version). If not, run the command:
sudo port selfupdate
Install the MacPorts version of ruby itself:
sudo port install ruby
Install the MacPorts version of rubygems:
sudo port install rb-rubygems
Install the MacPorts version of git:
sudo port install git-core
Upgrade to the latest version of rubygems:
sudo gem update --system
Make sure you’re running rubygems version 1.3.5 (type: gem -v). If not, run the command: update_rubygems
Install all the gems you want (rails, heroku, etc.). They will be installed into /opt/local/lib/ruby/gems/

I set up my machine about a month ago and it's been working fine ever since.

Enjoy...

MacPorts headaches... follow-up

2009-10-26T14:30:00.000-07:00

Some of these issues apparently have to do with the order in which things are done and may also depend on the fact that some of these packages require native C-language extensions.

I wanted to work with nokogiri to do some XML parsing. Given my previous post, I figured I'd better install this using MacPorts, which I did:

     sudo port install nokogiri

That was pretty straight-forward and worked quickly. However, when I went to irb to check things out, I got the following message:

irb(main):001:0> require 'nokogiri'

HI.  You're using libxml2 version 2.6.16 which is over 4 years old and  has

plenty of bugs.  We suggest that for maximum HTML/XML parsing pleasure,  you

upgrade your version of libxml2 and re-install nokogiri.  If you like  using

libxml2 version 2.6.16, but don't like this warning, please define the  constant

I_KNOW_I_AM_USING_AN_OLD_AND_BUGGY_VERSION_OF_LIBXML2 before requring  nokogiri.

So I quit irb and installed libxml2

     sudo port install libxml2 

which also went just fine. Unfortunately, nokogiri still gave me that long warning message.

What I had to do was re-install nokogiri so that it would recognize the new version of libxml2 (which was now in the /opt/local tree) and then everything worked fine.

Seems like I can get things to work now, but having to remember previous dependencies and re-install packages to pick up those new items isn't going to be fun.

MacPorts headaches...

2009-10-26T12:45:00.000-07:00

Recently I started doing some of my development work on a little MacBook. Pretty nice as I drop down into terminal, have all the power of *nix (FreeBSD-based in this case), and still have Mac niceties available to me, such as MacVim for editing.

Everything is rocking along just fine: I've got Cucumber and RSpec running and working my way thru "The RSpec Book" (pretty useful so far). However, I needed to install FXruby so that I could do some graphical work and figured this would be no problem. Turned out to be a little tricky and I had to remember an issue I ran into a while back on a Linux system.

First off, this won't work:

     gem install fxruby

It'll install that gem alright, but that's not enough to get things running: you have to get Fox installed as well and that doesn't come along with the gem.

Of course, to do that, you have to install MacPorts which is actually relatively painless (see the MacPorts Guide for how to do this).

Once you've done this, then you can install the complete FXRuby collection of stuff (see the FXRuby Book at PragProg.com)...

     sudo port install rb-fxruby

and it's a very large collection indeed: it took over 1 hour on my machine.

The only hitch I ran into was having too old a version of the Mac XTools package, so that was another 800Megs of download and an install. If you do this make sure you get the right version: XTools 3.1 is for OS X 10.5 and XTools 3.2 is for 10.6. Check this first or you'll spend most of an hour installing fxruby only to have it complain about the wrong XTools version and you'll have to start over.

MacPorts updates your ~/.profile or ~/.bash_profile file (note that the MacPorts install docs only mention ~/.profile and I had to dig a bit to find that they had properly modified my ~/.bash_profile file) so that your system can find the added programs...

But that's not quite everything you have to do and here was the gotcha for me. Even after I went through all this, I kept getting strange messages out of my test runs, things like:

  "unable to load file ubygems"

(the missing 'r' is not a typo: that was the exact error message) when executing the "require 'rubygems'" statement.

I tried adding the command:

      export RUBYOPT=rubygems

to my profile so that I wouldn't have to add a require 'rubygems' statement but that didn't solve the problem.

BTW, when I went to the RubyGems Guide, they said to use the -rubygems option on the ruby command line, but that also generated the same error.

After a little more digging, I found the problem: MacPorts installs everything in /opt/local, which is not where the normal gem command installs things (if you do a simple "gem install ' they wind up in /usr/bin). Thus, after MacPorts was installed, ruby got a tad confused and couldn't find everything it needed. Then I remembered what I had run into when I was working on my Linux box: trying to install a LAMP stack: everything got put into /opt/local, but then it was out of the normal stream of things and there were ongoing problems. I finally bit the bullet in that case, removed the LAMP stack, and manually installed everything into their default locations. Life has been pretty good in Linux-land ever since.

The solution? Install rubygems using MacPorts so that my FXRuby could find what it needed. That was a simple:

     sudo port install rb-rubygems

and now my ruby apps would run again, load the gems, and I had a simple FXRuby app running.

Quite frankly, I'm not sure of all the nitty-gritty details of this problem and I'm not even sure that I won't have further problems downstream when I want to use another gem, but I'll face that problem when I come to it. If anyone has a definitive answer as to WHY this happened and a better approach to Mac OS X gem installation, I'm all ears.

A Users Guide is a great developer tool...

2009-09-07T12:27:00.000-07:00

Today I was reminded just how important documentation is, particularly End-User documentation... you know, the instructions for how folks are actually going to use your system.

In my spare time, I've been developing a very small system for a client (unfortunately not in Ruby/Rails, but that's another story). It's designed to replace one labor-intensive portion of their Commission calculation process, which is spreadsheet-intensive. You've seen them before: someone develops a simple spreadsheet to "save time" and eventually, the spreadsheet is a monster and the whole process takes days to complete and is fraught with potential errors. And with the sales reps watching the numbers like a hawk, nothing slips by.

The piece I'm replacing is for maintaining the Work In Process costs and pricing factors for the products developed in-house. It's a tiny database application which outputs an Excel workbook that is dropped into the monster spreadsheet and thereby replaces a very time-consuming manual update process. In fact, the process has been so challenging in the past that cost updates have been few and far between. The new system will allow them to make pricing changes at will.

Over the past few weeks, time permitting, I've developed small pieces of the system and run it by the client to make sure that my understanding is correct, and they've approved the prototype pieces that I've provided to them. This is actually part of my proposal process: try out important pieces of the system, and then let the client approve the results.

Today, I started writing the contract for the actual development. As part of the scope of work, I provided screen shots and a high-level overview of the procedural steps for using the system. As I worked through the steps in the workflow, I suddently realized that I was omitting one piece! I had prototyped that portion of the system early on and it worked OK, but the table I had created was only a dummy table, hand-crafted for testing, and I had not provided the necessary support for a user to maintain that table!

Fortunately I found this out before completing development and installing it for my client. At this stage in the game, it was pretty easy to add the required functionality and include it in the Users Guide.

You can call it what you want: Requirements or Design or Testing. Developing a Users Guide can help in all areas of your application life cycle methodology. In this case, I like to think of the Users Guide as being a form of UI Testing: I used it to test my prototype design (screenshots, data entry forms, reports, and output files) against the user interaction that would be required to actually do something productive with the system.

Does this apply to Rails? You bet! Just watch Jospeh Wilk's Cucumber presentation at Scotland on Rails 2009. He suggests that you "...think about the user interface first " and asks "...how is the business user going to achieve value through the interface?" He also stresses figuring out what's really important to you and to your user (see his example of "Popping the Why Stack"). I particularly like his question: "Have we achieved what the user wants yet?" It's all about a single-minded focus on the user of the system and the value that the system is to provide.

Fortunately my Users Guide test failed and I was able to address the issue today, not later down the road when it would be more time-consuming. And I saved a potentially awkward situation with my client.

So... I'm going to be more careful about writing Users Guides and testing the user workflow in the future...

When do you write your Users Guide?

Snipping the blight... fantastic!

2009-08-28T15:58:00.000-07:00

I've been fighting an elusive bug for the past few weeks: I've developed some major enhancements for my passwords site plus upgrading to more recent versions of Rails and associated plugins (improved security and testability being my primary goal). Things were going along fine until I deployed my app to production: I was able to login without errors, but was then immediately returned to the login screen. Needless to say, the Capistrano deploy:rollback task was just what the doctor ordered (that's why I love Cap: it took a bit to get it setup originally, but it really makes deploying or undeploying applications a snap - if you have an app in production and you don't have Capistrano or something similar, do it now).

Anyway, I setup up a production environment on my development machine and, sure enough, it failed in production mode but ran fine in development mode. I modified the production settings so that I would get error messages [Note to self: put it back like it was!] and got a message saying that a module in the filter chain had returned false... it looked as if it were failing authentication somewhere in the login system.

I started digging into the code and using the debugger to see what was going on: from the symptoms and the error message, it seemed to be a problem with session state. Checking with my hosting tech support and local Rails folks, I learned there were reports of problems with sessions in Rails 2.3.2 and one of the local chaps, Brendon Whateley, reported some serious session issues that he had and chased the code back to Rails initialization and lots of 'magic' aliasing and overriding of ActiveRecord methods (nice stuff when it works). To quote:

"Having spent the best part of a week debugging the sudden failure of sessions recently, I may be able to help. The first thing to figure out is if the session is saving state and not loading, or loading but not saving. In my case the ActiveRecordStore was not serializing the session data, but was loading and saving the session object. This meant that previously set session values still existed, but any changes did not persist across calls. I was able to validate that the logins were successful, but suffered similar issues to you.

"In the end I tracked the problem down to overriding the ActiveRecord::Base.create_or_update method. For some reason it completely broke ActiveSessionStore serialization, even if the redefined method contained the exact same code as the original. I never completely figured out WHY that broke, but if I moved my changes to create and update methods that create_or_update called, I no longer experienced that problem."

Made sense to me, and since I am using some of the ActiveRecord callbacks, it seemed quite reasonable. So off I went, digging thru my code, researching how the callbacks were used, trying to find out if anyone else was using them, more debugging... all of which went nowhere.

Yesterday, I bit the bullet and upgraded to Rails 2.3.3. Very easy; just do

  gem install rails

and try it again in production mode.

You can imagine my surprise when all of a sudden, the app crashed and emitted the following error messge:

ActiveRecord::StatementInvalid (Mysql::Error: Unknown column 
'assets.type_id' in 'on clause': SELECT count(DISTINCT `assets`.id) AS 
count_all FROM `assets`  LEFT OUTER JOIN `types` ON `types`.id = 
`assets`.type_id WHERE ((user_id = 1 and ( entry like '%' or entry 
like '%' ))) ):
  -e:2:in `load'
  -e:2

I had never received any such error message in Rails 2.3.2 (obviously some kind of bug there which caused this to appear as if it were a filter chain termination), but this made it clear what had happened. In the process of upgrading from Rails 1.x to 2.x, I had experimented with using some database constraint logic. It had been a tad problematic and I wanted to focus on the Rails upgrade so I removed those migrations. It was probably at that point that the development database and the production database got out of sync (I haven't yet gone back to my subversion repository to see exactly what/when it happened, but I may do that yet out of curiosity). I had added a new "type_id" field that was required to support the new version of ActiveScaffold and it had never been added to the production database, even though the schemo_info table contained the same information in both databases (I know, because I checked before I started the whole deploy process).

So what did I learn from this?

Even though there were several other folks looking at this, we all missed the obvious: development and production mode are different not only in the code base but also in the database. I suffered from a perceptual set when identifying potential sources of the problem (and Paired Programming isn't always a silver bullet). In the future, I'm going to be a little more diligent when exploring potential solution spaces.
Rails migrations, while very nice, are also fragile. I've had problems with them in the past: simply checking that the schema_info values are the same just isn't always enough. I'm going to add migrations and database state to my checklist of things to review before deployments or when making changes.
Rails migrations, while very nice, are also something of a siren song, luring one into complacency about the database. "When they are good they are very, very good, and when they are bad they are horrid." After a long period of no problems, I had assumed (mistake!) that they were under control: in a way, I thought of them as if they were "source code control" for my database. They are not, and I'm going to pay more attention to that in the future.
Rails upgrades are in the same category: they can be remarkably easy, such as my switch from 2.3.2 to 2.3.3 or they can be remarkably messy, as has been my switch from 1.3.6 to 2.3.3. On the one hand, I want to stick with a known version: there are just too many changes that I view as frivolous in the Rails code base to want to go through all the deprecations and just plan different stuff unless it's really necesary. At the same time, we do need to stay reasonably up-to-date to take advantage of the definite improvements such as better plugins and greater security. This is an individual decision, based on your particular requirements. Regardless, YMMV.

So... one more bug snipped out of my app and that much closer to deployment. Some cleanup and more testing and it should be good to go.

Learning RSpec/Cucumber - Day 1

2009-08-26T08:41:00.000-07:00

I've wanted to improve my testing for some time now and so I'm pairing on the RSpec book with my friend, Allan Miller. We're going to work through the book/examples in tandem and get together regularly to review what we've learned. We just downloaded the RSpec book by Pragmatic and assigned Chapters 1-3 (this is the book that was highly recommended by our Ruby Meetup folks). We'll work through this and then meet in a couple of days to see how things went and move on to the next group of chapters.

For now, I'm running all this on my MacBook, without any IDE... just straight terminal access and MacVIM. Chapters 1 & 2 are introduction; chapter 3 is where you start actually doing stuff.

First problem (minor) that I ran into was that the rspec installation wasn't clean; I got the following errors:

Successfully installed rspec-1.2.8

1 gem installed

Installing ri documentation for rspec-1.2.8...

Installing RDoc documentation for rspec-1.2.8...

Could not find main page README.rdoc

So far, this doesn't seem to have affected anything -- I just ignored it and moved on (so far, I don't use the ri or RDoc information at all).

The cucumber installation was a little more disconcerting:

Installing ri documentation for builder-2.1.2...

ERROR: While generating documentation for builder-2.1.2

... MESSAGE: Unhandled special: Special: type=17, text=""

... RDOC args: --ri --op /Library/Ruby/Gems/1.8/doc/builder-2.1.2/ri --title Builder -- Easy XML Building --main README --line-numbers --quiet lib CHANGES Rakefile README doc/releases/builder-1.2.4.rdoc doc/releases/builder-2.0.0.rdoc doc/releases/builder-2.1.1.rdoc --title builder-2.1.2 Documentation

(continuing with the rest of the installation)

but again, it was only documentation so I moved on from here as well (have to pick your battles, you know).

Everything went just fine, except for a couple of glitches: one was just some missing default snippets and the other was a result of those missing snippets. Here's what happened:

The first attempt at cucumber did NOT display the snippets for the 3 undefined steps (pp 27-28 in my beta copy of the book) as expected. Rather than being able to copy and modify them I had to type them in by hand. [I tried the command twice just to make sure.. sure enough, didn’t display, although a later point where I had only one step undefined, it DID display the snippet template. Go figure.]
Because of #1, I made a transcription error in the regexp for step 3: I omitted the ‘^’ exclusion character. This cost me several minutes (regexp’s are like that!). I have a feeling that there may be more of these... hopefully I’ll be able to get the step snippets more reliably in the future and then just copy them into my step definitions as a starting point.

All in all, a successful beginning... But I do have a concern -- given that we're using regexp's, there are MANY opportunities for small errors that can cost time. Seems like a bit of a contradiction: cucumber is a "higher level" testing language than rspec, but it uses one of the lowest level syntaxes to accomplish this.

I'll just have to wait and see what this leads to... and I'm sure going to copy/paste those snippets whenever I can.

Pair programming really works!

2009-08-26T06:32:00.000-07:00

I've known this in the past and I had another "Aha!" moment the other day: pair programming (or debugging in this case) is great!

I've had a problem for the past week or so in one of my applications. I'm in the middle of a big upgrade (from Rails 1.3.6 to Rails 2.3.2 and lots of other associated changes), when I hit a hard stop; my screen wasn't getting updated correctly after an Ajax call.

I struggled with this for quite some time and then today called Allan Miller, a good friend of mine over to help me debug it. He sat down, I described he problem and demonstrated it. He said "Well, let's take a look and make sure that you're actually getting an XHR request." so we fired up FireBug and entered a couple of trials: the POST request was properly executed and my application did see it as an XHR request. Everything's working as exepcted so far, but no errors and no screen refresh.

Since he had been focusing on the Ajax call, I decided to show him the code on the page, and used FireBug's feature to "click an element to inspect" and pointed out the code [line breaks inserted for readability; this is all one line of code]:

<a class="ltr-locate" onclick="new Ajax.Updater('asset-content', '/asset/get_assets?ltr=F',
{asynchronous:false, evalScripts:true, parameters:'authenticity_token=
' + encodeURIComponent('xxx')}); return false;" href="/asset/get_assets" align="right">F</a>

"See" I said. "There's the AjaxUpdater call to update the... ... that's it!" The first field in the AjaxUpdater call signifies the named area on the page that will be updated as a result of the call. In this case 'asset-content'. "I'll bet that they changed the name in the intervening releases" I said.

Sure enough, that was the case: the name name is 'as_asset-content' and updating the link_to_remote call solved the problem. What a relief!

I've got 3 take-aways from this:

Two heads are at least 10 times better than 1, if you compare 50 total minutes of work (25 minutes times 2 heads) with at least 10 hours of work with my one head. There's just something magical about having to REALLY explain your code to an outside observer that makes you think thru things much more carefully. It's making the thinking process external, rather than internal where your hidden assumptions come into play without your even knowing it.
The AjaxUpdater call gives NO ERROR if the named area doesn't exist; it just quietly closes up shop, assuming that you knew what you were doing.
I was reminded how great FireBug is for this kind of thing. Quick and easy to see that the Ajax call was firing with the right information so that we could move on to the next question.

Extrapolating that out, I would imagine that -- had there beeen two folks in the seat from the beginning -- this whole thing wouldn't have taken so long... and the code would probably be better for it.

Reminds me of the old "Double your pleasure..." chewing gum ad, only in this case it's really true.

Interesting svn "gotcha"... II

2009-08-19T11:48:00.000-07:00

Turns out, there's any easy fix for this situation (see Interesting svn 'gotcha'). All you have to do is:

Checkout your entire project into a different directory using your command line client.
Run the script/plugin install... command on that working copy.
Checkin the updates you just made.
Update the IDE working copy to the latest from repository.

Easy; problem solved...

Interesting svn "gotcha"...

2009-08-17T16:31:00.000-07:00

I just learned that you have to be careful about which version of subversion you are using if you have an IDE and also want to use the command-line client.

Starting about V1.4, there were changes in the internal structure of the subversion directory hierarchy (i.e., the .svn files that it creates to track things) which means that you have to make sure that your versions are in synch.

For example, I'm using Aptana Studio, which is eclipse-based, and which uses the latest subclipse client (subversion for Eclipse). That latest version is V1.6. However, on my Kubuntu development box (V8.04/Hardy Heron), the latest subversion package right now is V1.4. If I try to run a subversion command from the shell, I get this error message:

jseidel@EDP15:~/AptanaStudio/4MyPasswords$ svn status
svn: This client is too old to work with working copy '.'. You need
to get a newer Subversion client, or to downgrade this working copy.
See http://subversion.tigris.org/faq.html#working-copy-format-change
for details.

The help link above will explain that -- due to new features in the later versions -- they had to introduce new structure which is incompatible with the older clients.

Since I don't want to downgrade my working copy, I went looking for V1.6 command-line clients. Sad to say, they aren't packaged (yet) for Kubuntu/Hardy --so far I only found it for the Karmic version of Ubuntu.

There are other svn clients out there, either as wrappers around the command line client, or full-on clients. One that I looked at was kdesvn. This looked to be a nice client and was at V1.6. Unfortunately, the developer only provides a Fedora package, so I would have to build my own from source.

I went to the subversion site, but the referenced Ubuntu binaries only pointed back to the Ubuntu site and the older version, so I tried building it from source. Bad move! As stated on their site, there are a number of dependencies which I started to work through one at a time. When I reached a point of having to figure this out:

jseidel@EDP15:~/downloads/subversion-1.6.4/subversion-1.6.4$ apr/buildconf
buildcheck: checking installation...
buildcheck: autoconf version 2.61 (ok)
buildcheck: autoheader version 2.61 (ok)
buildcheck: libtool version 1.5.26 (ok)
buildcheck: local copy of find_apr.m4 does not match APR's copy.
An updated copy of find_apr.m4 may need to be checked in.
buildcheck: local copy of PrintPath does not match APR's copy.
An updated copy of PrintPath may need to be checked in.
Copying libtool helper files ...
buildconf: Using libtool.m4 at /usr/share/aclocal/libtool.m4.
Creating include/arch/unix/apr_private.h.in ...
Creating configure ...
configure.ac:185: error: possibly undefined macro: AC_PROG_LIBTOOL
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
Generating 'make' outputs ...
apr/buildconf: 91: build/gen-build.py: not found rebuilding rpm spec file
apr/buildconf: 107: build/get-version.sh: not found
cat: ./build/rpm/apr.spec.in: No such file or directory

I quit trying: I want to work on my application, not have to learn THIS much about the internals of the tool I want to use... Argh! For now, I'm sticking with the Aptana support which is working (with some help from the Aptana support folks), although I do miss the ability to do quick svn commands from the shell.

I think Linux needs to come up with a better installation mechanism...

Capistrano now needs a pseudo-tty...

2008-04-16T17:12:00.000-07:00

I've been using Capistrano 2.x for sometime now and really like it but ran into a strange problem a couple of days ago when attempting to deploy a new version of my website.

I had last deployed in March of this year with no problems. Since that time I had made no changes to Capistrano (no upgrades, tweaks, etc.), but on this deploy, it failed miserably. The only indicator were numerous messages saying:

** [err] stdin: is not a tty

Sometimes it caused the command to abort and sometimes it didn't.

The solution was hard to find but easy to implement. Just add the following to your deploy.rb:

default_run_options[:pty] = true

Apparently, versions up until Cap 2.1 would always allocate a pseudo-tty (pty) for every command it ran. However, with the removal of this default in Cap 2.1, "...some commands, in certain environments" fail with no pty [so says my Rails host]. Adding this new option put my deploys back in shape.

I wonder why this got removed in the first place...?

Capistrano on Windows client...

2007-07-19T16:41:00.001-07:00

It was supposed to be easy: just read the Capistrano manual make a little magic with your deploy.rb file, and you're in business. Sounds wonderful, doesn't it? Would that it were so easy.

The documentation (manual and elsewhere) makes it clear that Cap'ping to a Windows server is pretty challenging... Cap is made for *nix machines. However, I didn't find anything about a Windows client, other than the statement that "you must have an svn client installed on your machine." I did -- RapidSVN, TortoiseSVN, and the svn client embedded in the Aptana IDE.

However, once I got to actually attempting to deploy my application, it failed every time with a "no such file or directory" error message and a reference to the repository trunk itself. No amount of hair-pulling (it was a VERY long weekend), searching the web, emailing a few contacts helped... not matter what I did, it continued to fail in exactly the same manner. I even built a new development environment on another machine just to make sure it wasn't some brain-dead Windows issue: no dice.

Tuesday night, I went to the local Ruby on Rails Meetup and got the answer from another Windows-based developer Hob Spillane -- thanks!

So... drum roll please... here's how to get Cap working on your system:

You MUST have the svn binaries installed on your Windows client machine and in your execution path before Cap can deploy successfully -- the GUI clients do not provide what Capistrano needs. The binary you need depends on the version of Apache that your provider is running: 2.0 and 2.2 require different binaries.

For my Apache 2.2, I went to the subversion download page, and clicked on the svn-win32-1.4.4.zip link, installed it in /usr/bin/svn/bin, added that directory to my PATH statement and -- voila! -- Capistrano deploys like a charm! (When I went to the download site, there was only the choice of a zip file, no setup.exe. That may change by the time you get there.)

If your version is Apache 2.0, go to this download page and chose the appropriate setup.exe or .zip file.

Nice...

Of presentations and security...

2007-04-28T15:07:00.000-07:00

So... I had the chance to give a presentation to the East Bay Ruby on Rails Meetup Group (that's the Oakland part of the San Francisco Bay) about what it's like to be a nuby. Bottom line: the first step is an easy one, but after that it's a doozy (see the presentation here).

During the great discussion that followed, one of the group pointed out that production logging is at the :info level and that's not good -- information entered into the site is stored in clear text and therefore not as secure as it should be (people still have to hack the site, but obviously that's not good enough).

So I set about closing this loophole and also getting the sessions out of file storage and into the database where they are protected via password.

This should have been simple and easy, but it turned out to be really tricky: an example of things not being exactly what they seem to be. Here's the deal.

I had previously updated my environment.rb file to customize the logger, having found this code snippet at snippets.com.


config.logger = Logger.new("#{RAILS_ROOT}/log/#{ENV['RAILS_ENV']}.log",2, 131072)

For some reason I couldn't get it to work as I reported in an earlier post. I had finally gotten it to work (I think it may have had to do with restarting my server, but it's unclear) and life was good: I had smaller logs and didn't have to wait so long for it to load when I was doing some testing.

However, what I found was that -- once this new logger was running -- I couldn't change the logging level (normally done with config.log_level in environment.rb or environments/production.rb). I wanted a logging level of :error (instead of :info) so that the operation of the site wasn't so readily stored. I tried a number of things, including the following code in environment.rb:


if ENV['RAILS_ENV'] == 'production'
    config.log_level = :error
  else
    config.log_level = :debug
  end

but that didn't work... things kept getting logged at the :info level in production -- definitely not good.

I also tried setting it with config.logger.log_level = :error but that didn't seem to work either.

I finally -- after much futzing around -- decided simply to remove the new Logger snippet and see if that would do the trick: and it did. Now I can set the logging level in production (in the environments/production.rb file) with a simple config.log_level = :error.

Someday I'll go back and try to understand what happened. For now, the system is operating the way I want it to and I've moved the config.logger snippet to my environments/development.rb file so that I have that capability in development where I really need it.

Fortunately, the sessions question was as easy as 1-2-3:

1. Run the command:
rake db:sessions:create
2. Run the migration:
rake db:migrate
3. Change the production.rb file:
config.action_controller.session_store = :active_record_store

and we're in business!

This is really an analogy for my experience with Rails: some of it is subtly obscure and challenging (like the Logger situation), while some of it is wonderfully intuitive and easy (like the Sessions situation).

Let's hear it for more of the latter!

Applicaton is live!

2007-04-12T16:46:00.000-07:00

Well... I finally got the application out there, up, and running. It obviously still has much work to be done, but the basic secure functionality is there.

4MyPasswords -- a secure well-lighted place to store you passwords and other confidential information (e.g., Frequent Flyer numbers; credit card numbers; ...)

Feedback welcome!

A nice plugin for secure sites...

2007-04-08T11:41:00.000-07:00

I realize that I didn't comment about the ssl_requirement plugin by DHH (David Heinemeier Hansson, the Rails guru) . It's really a very nice approach to requiring that certain actions be accessed only through SSL/https and letting the other actions through. Once I found it, it was quick and easy to install and worked flawlessly.

However, I couldn't use it: the fundamental controller for my application uses AjaxScaffold (AS), which provides all the CRUD/UI stuff that I needed and allowed the customizations that I wanted (albeit with some learning curve; see a previous post). AS uses a veritable galaxy of actions to perform its magic and there was no way (I tried) to identify the right actions individually. What I really needed was a way to identify a controller instead of an action that required SSL. I briefly considered and then as quickly rejected the idea of attempting to enhance the plugin. It's a small plugin and seems to be pretty straight-forward in operation, but I'm just not at that level and I wanted to focus on my basic application. Maybe another time...

I did have one other question regarding the plugin. Reviewing the code, I saw that it works by trapping and then redirecting the actions that you want secured. I wonder if that might not be too much overhead for a site with lots of traffic? I just don't know at this point.