Mail Relay from MacBook to Gmail

2012/05/19 | by Doug [mail] | Categories: Computing, sadahome.ca, Mac Stuff

On OS/X Mountain Lion 2 postfix files need to be modified, and a password file needs to be created to enable relaying mail from /usr/bin/mail to Gmail. That relay is used by the rdiff-backup scripts.

/etc/postfix/main.cf needs to have the following lines added at the end:

Code:

# Enable relaying to Gmail
relayhost = [smtp.gmail.com]:587
# Enable SASL authentication in SMTP client
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_password
smtp_sasl_security_options = noanonymous
# Enable Transport Layer Security (TLS); i.e. SSL
smtp_use_tls = yes
smtp_tls_security_level = encrypt

/System/Library/LaunchDaemons/org.postfix.master.plist needs to have an OnDemand key added with the value true:

XML:

<key>OnDemand</key>                                                        
<true/>

/etc/postfix/sasl_password needs to be created with the content:

Code:

[smtp.gmail.com]:587    username@gmail.com:gmail_password

where username and gmail_password are replaced by the appropriate values. For 2-step authentication, use an application-specific password generated on the Google security settings page.

Use:

Code:

sudo postmap /etc/postfix/sasl_password
sudo chmod 600 /etc/postfix/sasl_password
sudo chmod 600 /etc/postfix/sasl_password.db

to activate the Gmail authentication password, and set tight permissions on it.

Use:

Code:

sudo postfix start
sudo postfix reload
sudo postfix stop

to load the new /etc/postfix/main.cf configuration into Postfix. (Not 100% certain that this set is necessary.)

Use:

Code:

sudo launchctl load -w org.postfix.master.plist

to update the Postfix launchd configuration so that Postfix starts on demand when there is mail to relay.

Finally test the relay with:

Code:

date | mail -s test username@gmail.com

mailq is useful to check the progress of the test message while it is being relayed to Gmail.

Permalink

Firefox Add-on SDK Oplop Dev Environment

2011/11/26 | by Doug [mail] | Categories: Computing, Python, Firefox

I had to jump through some hoops to get a development environment set up to work on creating a Firefox 4+ add-on that implement’s Oplop. The difficulties all had to do with virtualenvs.

The Firefox Add-on SDK (aka Jetpack) uses a virtualenv but I want to keep the dependencies for Oplop development in a project-specific virtualenv, not in the firefox-addon-sdk-1.2.1 one. I use virtualenvwrapper so I added postactivate and predeactivate hooks to make it easy to switch between the 2 virtualenvs. The postactivate hook is:

Code:

SDK_DIR=$HOME/Documents/devel/firefox-addon-sdk-1.2.1
ADDON_DIR=$HOME/Documents/devel/firefox/oplop-addon/Firefox/add-on
 
alias sdk='cd $SDK_DIR && source bin/activate && cd $ADDON_DIR'

and the predeactivate just destroys the sdk alias as a cleanup.

For development, Oplop requires Jinja2 and SCons. Jinja2 is easily installed in a virtualenv with pip, but Scons doesn’t play nice with pip and I ran into this issue. Fortunately that’s easily remedied by downloading the scons tarball, unpacking it and installing with python setup.py install (with the project virtualenv activated, of course).

With that all in place I can use workon oplop-firefox-addon to activate the virtualenv where Jinja2 and Scons are available and use Scons to render templates, etc., then use sdk to flip to the firefox-addon-sdk-1.2.1 virtualenv to use cfx to test the add-on.

Permalink

Building Matplotlib in a virtualenv on OS/X 10.6

2011/11/15 | by Doug [mail] | Categories: Computing, SOG Code, Mac Stuff, Python

pip install "matplotlib==1.x" in a virtualenv fails with Python 2.6.6 on OS/X 10.6.8 because it can’t find the ft2build.h header file.

Assuming that freetype2 is installed in /usr/local, the work-around is to download the matplotlib tarball and install from source with the following in setup.cfg:

Code:

[directories]
basedirlist = /usr/X11
Permalink

File Uploads in Selenium 1 and 2

2011/03/27 | by Doug [mail] | Categories: Computing, Python

Over on the SauceLabs blog Santiago Suarez Ordoñez has a nice description of how to use the attach_file method in Selenium 1, and some of the restrictions that pertain to it. One of those restrictions is that the file that attach_file operates on has to be available from the root level of an HTTP server. To quote Santi:

using the following URL will not work:

http://saucelabs.com/subdirectory/subdirectory2/file.txt
while the following will:

http://saucelabs.com/file.txt

So, you could drop your test files into the root directory of some Apache server you have control over, but Python offers a more flexible solution: run an adhoc server to serve files from your test files directory:

python -m SimpleHTTPServer 8000

Nice, but you still have to make sure that adhoc server is running or you tests will fail. With a little work I was able to come up with a way to make my tests self-contained by starting up the server to handle requests from attach_file in a text class setup method. The key is that the server has to run in its own thread, otherwise it blocks the tests from executing:

Code:

import os
import posixpath
from selenium import selenium
from SimpleHTTPServer import SimpleHTTPRequestHandler
import SocketServer
import threading
import urllib
import unittest2 as unittest
 
 
class AttachmentFileRequestHandler(SimpleHTTPRequestHandler):
    """Serve files from the :file:`test_data` sub-directory of the
    current directory and any of its sub-directories.
    """
    def translate_path(self, path):
        """Translate a /-separated PATH to the local filename syntax.
 
        Components that mean special things to the local file system
        (e.g. drive or directory names) are ignored.
        """
        # abandon query parameters
        path = path.split('?',1)[0]
        path = path.split('#',1)[0]
        path = posixpath.normpath(urllib.unquote(path))
        words = path.split('/')
        words = [word for word in words if word]
        path = os.getcwd()
        path = os.path.join(path, 'test_data')
        for word in words:
            drive, word = os.path.splitdrive(word)
            head, word = os.path.split(word)
            if word in (os.curdir, os.pardir): continue
            path = os.path.join(path, word)
        return path
 
 
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    pass
 
 
class TestSeleniumFileUpload(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        httpd = ThreadedTCPServer(('', 8001), AttachmentFileReqestHandler)
        httpd_thread = threading.Thread(target=httpd.serve_forever)
        httpd_thread.daemon = True
        httpd_thread.start()
        cls.attachment_server = httpd
        cls.attachment_server_thread = httpd_thread
 
 
    @classmethod
    def tearDownClass(cls):
        cls.attachment_server.shutdown()
        cls.attachment_server_thread.join()
 
        
    def test_upload_file(self):
        browser = selenium(
            host='localhost', port=4444,
            browserStartCommand='firefox',
            browserURL='http://localhost:8000')
        browser.start()
        browser.open('/')
        browser.attach_file('id=attachment', 'http://localhost:8001/foo.txt')
        self.assertTrue(browser.get_value('id=attachment').endswith('foo.txt'))
        browser.stop()
 
 
if __name__ == '__main__':
    unittest.main()

The AttachmentFileRequestHandler is there to override translate_path from SimpleHTTPRequestHandler to that the upload test files are served from a sub-directory of the test suite.

ThreadedTCPServer just uses SocketServer.ThreadingMixIn to enable the server to run in its own thread.

The server is started before the tests run by the setUpClass class method and shutdown when they finish by tearDownClass, another class method. Those methods are courtesy of Michael Foord’s excellent improvements to the unittest module; available from PyPI as unittest2 for Python 2.4, 2.5, and 2.6, and from the standard library as unittest for Python 2.7, and 3.2 and up.

The test I’ve shown here is trivial - just confirming that the value of the input tag is set in the HTML fragment:

Code:

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>selenium upload example</title>
</head>
 
<body>
<input id="attachment" type="file">
</body>
</html>

In my real tests I confirm that the uploaded file has been stored as an attachment to a CouchDB object, and then use httplib to do a download request for the attachment and confirm that the response headers are correct.

As also Santi points out in his post, attach_file only works if you run your tests against Firefox.

Selenium 2

The introduction of webdriver in Selenium 2 make things easier. No more need for attach_file and a server to handle its requests - just use the send_keys method to put the absolute path of the test upload file into the input element.

Code:

import os
from selenium import webdriver
import unittest2 as unittest
 
 
class TestSeleniumFileUpload(unittest.TestCase):
    def upload_file(self, driver):
        driver.get('http://localhost:8000/')
        element = driver.find_element_by_id('attachment')
        path = os.path.join(os.getcwd(), 'test_data/foo.txt')
        element.send_keys(path)
        self.assertTrue(element.value.endswith('foo.txt'))
 
 
    def test_firefox_upload_file(self):
        driver = webdriver.Firefox()
        self.upload_file(driver)
        driver.close()
 
 
    def test_remote_htmlunit_upload_file(self):
        driver = webdriver.Remote(
            desired_capabilities=webdriver.DesiredCapabilities.HTMLUNIT)
        self.upload_file(driver)
        driver.close()
 
 
if __name__ == '__main__':
    unittest.main()

You can test against a local instance of Firefox as shown in test_firefox_upload_file or use the standalone Selenium server to test against a remote browser. In test_remote_htmlunit_upload_file I’ve used that approach to run the test against the HtmlUnit driver which is blazingly fast because it doesn’t render content to a screen. It’s not available directly in Python, hence the use of Remote.

One caveat: Selenium 2 doesn’t support the file selector input element yet for WebKit (Chrome, Safari, …) browsers.

The code above is based on the selenium2.0b3 Python bindings available from PyPI. The Selenium 1 example and the Remote Selenium 2 test were run against selenium-server-standalone-2.0b3.jar available from the Selenium project repository on code.google.com.

Permalink

Per-User Installation of Python Packages

2010/05/16 | by Doug [mail] | Categories: Computing, SOG Code, Python

Starting with Python 2.6 packages can be installed on a per-user basis in addition to installing them system-side. Details are described in PEP 370. This is useful in situations where the user does not have system administrator access.

In it’s simplest form:

  • Create your own site-packages directory in the default location:

    Code:

    $ mkdir -p ~/.local/lib/python2.6/site-packages/

  • Download the tarball of the package you want to install. Let’s call it fizzbang.tar.gz.
  • Unpack and install the tarball using the --prefix option of setup.py install:

    Code:

    $ tar xvzf fizzbang-1.0.tar.gz
    $ cd fizzbang-1.0
    $ ptyhon26 setup.py install --prefix=~/.local

  • If fizzbang has any executables associated with it they will be installed in ~/.local/bin, so you probably want to add that directory to your PATH.

Applying it to the EOS ocean Machines

On the EOS ocean machines I want to use virtualenv so that I can have the additional granuarity of managing package installations on a per-project basis.

  • Set the PYTHONUSERBASE environment variable to override the default of ~/.local. Note that the EOS ocean machines use csh as the default user shell, so this goes in ~/.cshrc:

    Code:

    # Override default per-user Python site-packages location                      
    setenv PYTHONUSERBASE ~/.local/lib/python2.6/site-packages/

  • Edit ~/.cshrc to set an alias to use Python 2.6 by default, add ~/.local/bin to the path, and set an environment variable to make virtualenv use distribute instead of setuptools:

    Code:

    alias python /usr/local/python26/bin/python
    set path = ($path /usr/bin/X11 ~/.local/bin $PGI/linux86/6.0/bin /usr/java/j2re1.4.0/bin)
    # Force virtualenv to use distribute instead of setuptools                      
    setenv VIRTUALENV_USE_DISTRIBUTE True

    Log out and back in for these changes to take effect.
  • Create the user site packages directory:

    Code:

    mkdir -p ~/.local/lib/python2.6/site-packages/

  • Grab the current release of virtualenv via PyPI and install it in my new, personal package space. At time of writing, the current virtualenv was 1.4.8:

    Code:

    $ cd wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.8.tar.gz#md5=74ded4025a56e538c1c8df6b9825a8b8
    $ tar xvzf virtualenv-1.4.8
    $ cd virtualenv-1.4.8
    $ python setup.py install --prefix=~/.local

Permalink

:: Next Page >>

Doug's Home Blog

A private blog for life, cycling, and my computing fiddlings.

| Next >

April 2020
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      

Misc

XML Feeds

What is RSS?

powered by b2evolution free blog software