Doug's Home Blog

On OS/X Snow Leopard the password to use for relaying mail from /usr/bin/mail to Gmail is set in /etc/postfix/relay_passwd and activated with sudo postmap relay_passwd. For 2-step authentication, use an application-specific password generated on the Google security settings page.

I had to jump through some hoops to get a development environment set up to work on creating a Firefox 4+ add-on that implement’s Oplop. The difficulties all had to do with virtualenvs.

The Firefox Add-on SDK (aka Jetpack) uses a virtualenv but I want to keep the dependencies for Oplop development in a project-specific virtualenv, not in the firefox-addon-sdk-1.2.1 one. I use virtualenvwrapper so I added postactivate and predeactivate hooks to make it easy to switch between the 2 virtualenvs. The postactivate hook is:

Code:

and the predeactivate just destroys the sdk alias as a cleanup.

For development, Oplop requires Jinja2 and SCons. Jinja2 is easily installed in a virtualenv with pip, but Scons doesn’t play nice with pip and I ran into this issue. Fortunately that’s easily remedied by downloading the scons tarball, unpacking it and installing with python setup.py install (with the project virtualenv activated, of course).

With that all in place I can use workon oplop-firefox-addon to activate the virtualenv where Jinja2 and Scons are available and use Scons to render templates, etc., then use sdk to flip to the firefox-addon-sdk-1.2.1 virtualenv to use cfx to test the add-on.

pip install "matplotlib==1.x" in a virtualenv fails with Python 2.6.6 on OS/X 10.6.8 because it can’t find the ft2build.h header file.

Assuming that freetype2 is installed in /usr/local, the work-around is to download the matplotlib tarball and install from source with the following in setup.cfg:

Code:

Over on the SauceLabs blog Santiago Suarez Ordoñez has a nice description of how to use the attach_file method in Selenium 1, and some of the restrictions that pertain to it. One of those restrictions is that the file that attach_file operates on has to be available from the root level of an HTTP server. To quote Santi:

using the following URL will not work:

http://saucelabs.com/subdirectory/subdirectory2/file.txt
while the following will:

http://saucelabs.com/file.txt

So, you could drop your test files into the root directory of some Apache server you have control over, but Python offers a more flexible solution: run an adhoc server to serve files from your test files directory:

python -m SimpleHTTPServer 8000

Nice, but you still have to make sure that adhoc server is running or you tests will fail. With a little work I was able to come up with a way to make my tests self-contained by starting up the server to handle requests from attach_file in a text class setup method. The key is that the server has to run in its own thread, otherwise it blocks the tests from executing:

Code:

The AttachmentFileRequestHandler is there to override translate_path from SimpleHTTPRequestHandler to that the upload test files are served from a sub-directory of the test suite.

ThreadedTCPServer just uses SocketServer.ThreadingMixIn to enable the server to run in its own thread.

The server is started before the tests run by the setUpClass class method and shutdown when they finish by tearDownClass, another class method. Those methods are courtesy of Michael Foord’s excellent improvements to the unittest module; available from PyPI as unittest2 for Python 2.4, 2.5, and 2.6, and from the standard library as unittest for Python 2.7, and 3.2 and up.

The test I’ve shown here is trivial - just confirming that the value of the input tag is set in the HTML fragment:

Code:

In my real tests I confirm that the uploaded file has been stored as an attachment to a CouchDB object, and then use httplib to do a download request for the attachment and confirm that the response headers are correct.

As also Santi points out in his post, attach_file only works if you run your tests against Firefox.

Selenium 2

The introduction of webdriver in Selenium 2 make things easier. No more need for attach_file and a server to handle its requests - just use the send_keys method to put the absolute path of the test upload file into the input element.

Code:

You can test against a local instance of Firefox as shown in test_firefox_upload_file or use the standalone Selenium server to test against a remote browser. In test_remote_htmlunit_upload_file I’ve used that approach to run the test against the HtmlUnit driver which is blazingly fast because it doesn’t render content to a screen. It’s not available directly in Python, hence the use of Remote.

One caveat: Selenium 2 doesn’t support the file selector input element yet for WebKit (Chrome, Safari, …) browsers.

The code above is based on the selenium2.0b3 Python bindings available from PyPI. The Selenium 1 example and the Remote Selenium 2 test were run against selenium-server-standalone-2.0b3.jar available from the Selenium project repository on code.google.com.

Starting with Python 2.6 packages can be installed on a per-user basis in addition to installing them system-side. Details are described in PEP 370. This is useful in situations where the user does not have system administrator access.

In it’s simplest form:

• Create your own site-packages directory in the default location:
  

  Code:

  $ mkdir -p ~/.local/lib/python2.6/site-packages/

  
  
• Download the tarball of the package you want to install. Let’s call it fizzbang.tar.gz.
• Unpack and install the tarball using the --prefix option of setup.py install:
  

  Code:

  	$ tar xvzf fizzbang-1.0.tar.gz
  	$ cd fizzbang-1.0
  	$ ptyhon26 setup.py install --prefix=~/.local

  
  
• If fizzbang has any executables associated with it they will be installed in ~/.local/bin, so you probably want to add that directory to your PATH.
  

Applying it to the EOS ocean Machines

On the EOS ocean machines I want to use virtualenv so that I can have the additional granuarity of managing package installations on a per-project basis.

• Set the PYTHONUSERBASE environment variable to override the default of ~/.local. Note that the EOS ocean machines use csh as the default user shell, so this goes in ~/.cshrc:
  

  Code:

  	# Override default per-user Python site-packages location
  	setenv PYTHONUSERBASE ~/.local/lib/python2.6/site-packages/

  
  
• Edit ~/.cshrc to set an alias to use Python 2.6 by default, add ~/.local/bin to the path, and set an environment variable to make virtualenv use distribute instead of setuptools:
  

  Code:

  	alias python /usr/local/python26/bin/python
  	set path = ($path /usr/bin/X11 ~/.local/bin $PGI/linux86/6.0/bin /usr/java/j2re1.4.0/bin)
  	# Force virtualenv to use distribute instead of setuptools
  	setenv VIRTUALENV_USE_DISTRIBUTE True

  
  Log out and back in for these changes to take effect.
  
• Create the user site packages directory:
  

  Code:

  mkdir -p ~/.local/lib/python2.6/site-packages/

  
  
• Grab the current release of virtualenv via PyPI and install it in my new, personal package space. At time of writing, the current virtualenv was 1.4.8:
  

  Code:

  	$ cd wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.8.tar.gz#md5=74ded4025a56e538c1c8df6b9825a8b8
  	$ tar xvzf virtualenv-1.4.8
  	$ cd virtualenv-1.4.8
  	$ python setup.py install --prefix=~/.local